A very common question that I see asked on stackoverflow.com, the greasemonkey-users mailing list, and in #greasemonkey on freenode, is how to use third party libraries with a user script.

The common response is to use @require, but these days people want their user scripts to work on Google Chrome too, and unfortunately Google Chrome user scripts do not support a lot of the Greasemonkey metadata block, including @require.. so another method is required.

Cross Browser Solutions

Use script element & setTimeouts

One method that has been mentioned elsewhere is to create a script element and add it to the page, and use setTimeout's to wait until the third party js, in this example jQuery, is loaded. The example in the link I provided is for Greasemonkey, but with little effort it can be made to work with Google Chrome.

Use script element & associated onload event

This method creates script element for jQuery (or some other third party script), and appends it to the body element, and listens to the load event to figure out when the library is loaded and ready to be used.

Google Chrome does not allow user scripts to access javascript objects in the page scope from the user script's scope, so in order to use jQuery at all on Google Chrome you must load jQuery into the page scope and inject your script into the page scope. So this is what my example below does essentially, more specifically it stringifys a callback function which is called when jQuery is done loading.

Example

A common use case of Google Website Optimizer (GWO) is to test (in order to optimize) a page's ability to have a user click an internal link, I like to call this the internal pass thru rate of a page. Meaning the click event of any internal link on a test page will count as a conversion for the test. This is not the opposite of the bounce rate which it is sometimes mistaken for, because the pass thru rate is oblivious to the difference between first time visits and other visits. The pass thru rate is not the same as the click thru rate either, because the pass thru rate relates to visits and the click thru rate relates to impressions. This is usually desired either because you have no other conversion to track -- although time on page is always another option, but the results typically take longer to acquire -- or because you wish to optimize the bounce rate, and this is as close as you can get at the moment.

Google doesn't have any documentation on how to track all internal links on a page, but they do provide a page on how to track an individual link, which any novice JavaScript programmer can follow in order to track all of the internal links, by simply repeating the process for every internal link on the test page(s). The main problem here is that Google's process requires that the onclick attribute of every link be set to "return false", albeit this can be done via JavaScript, it is not desirable. Google Website Optimizer's Technical Lead Engineer, Eric Vasilik, explained Google's method in a blog post from August 2009, called "Tracking Outbound Links -- The Right Way", and I wrote a follow up post in December 2009, called "Tracking Outbound Links - The Really Right Way" in which I describe a better method which is slightly harder to implement but does not require any use of the onclick attribute.

If you are at least a novice JavaScript programmer that wants to measure the pass thru rate of a page, then you can probably implement Google's method or mine with relative ease after reading the blog posts I mentioned.

If you are not a novice JavaScript programmer, or simply want to save some time implementing this type of conversion over and over again, then I would suggest you take a look at the JavaScript click track library that I released in early December '09, because with this javascript library you could simply add the following code to your page:

The above code will tag all non-rel-external links or links with the internal hostname on the page; for just internal hostname links replace "all-internal" with "internal-hostname". I would recommend that you wrap the above into a function though, to be run on the DOM ready event or page loaded event, with jQuery that would look like:

The first line in the code above will setup a function to be executed when the DOM is ready, and all of the page's links have been added. The second line is the first of the function the be executed on DOM ready, and it is adding an array of match objects to the global clickTrackingLib object provided by the click tracking library that I wrote. The following 5 lines define a single match object for the input array, which consists of a preset match function to match internal links (as I said already this preset can be changed, and you can also define a custom match function), and a simple tracking function. When clickTrackingLib.attachTrackingFunctions(null,99); is executed, links are tested against the match functions in the clickTrackingLib object's match object array, and where there is a match the associated trackingFunc function is made in to a onclick event listener for the matched link. As for the two inputs, the null means try all links, and the 99 means use a 99 millisecond delay.

At this point you might be wondering, what if the page has dynamic content, which is changed via ajax, well in that case you can round up the new link(s) in to an array and run the clickTrackingLib.attachTrackingFunctions(links, delay) function again like so:

clickTrackingLib.attachTrackingFunctions(links, delay) will also accept a single link.

Over the last week I've been spending a great deal of time looking over the Greasemonkey and Webmonkey source code, both because I want to understand both code bases more, but also because I'm interested in Firefox extension internals in general. While looking through these two code bases I saw a common file which could be optimized. This file was the convert2RegExp.js file, which looks like it came from Adblock at some point, at least in part. In fact I saw a number of changes that I could try in order to speed up the function, so I decided to time them all.

Test Factors

Factor I: Check for /\.[^\.ld]*t[^\.td]*l[^\.lt]*d/ in pattern string character loop

The first thing I noticed was that the regular expression that checks the pattern string for a ".tld" string was being run on every pattern input, this seemed obviously bad to me since there was a loop prior to the regular expression which runs through the pattern string's characters, so why didn't that loop at least check for the ".tld" substring first? the check could be as simple as making sure all of the required letters exist in the string first, or make sure that the ".tld" substring, exactly, is in the pattern string, or what I found to be the best way was to make sure the regular expression /\.[^\.ld]*t[^\.td]*l[^\.lt]*d/ matched the pattern string via the character loop through the pattern string. The average cases and best cases will go much faster despite the fact that the latter case would be matching some pattern strings that following tld regular expression (that is already in convert2regexp) would not match, thus just adding work in this case, and making the worst possible case even slower.

Factor II: Cache tldStr and tldRegExp

The convert2RegExp( pattern ) function creates the tldRegExp and tldStr from literals on every execution! that might be a faster operation I thought, but I expected it to be slower than simply looking up the value of a variable when I timed it, and even though I haven't figured out how to test the memory consumption yet, I expect creating a large string from a string literal over and over again would increase the peak memory usage, and thus the garbage collector pause time as well. So, caching the tldStr and tldRegExp seemed like it would be a small win.

Factor III: Use array.push()/array.unshift() and array.join() instead of +=

From what I've read about Firefox 3.6 the += operator in loops is optimized to use a single StringBuffer, but there were a few += outside of the loop, and I figured some people will probably use versions of Firefox < 3.5 for some time to come still, and using an array there would certainly improve/decrease the peak memory usage. I didn't know what the affect on performance would be, but this change is commonly said to be the better approach for JavaScript in the past to present, mainly because of the memory issue. Furthermore, the more memory that is used, the more work there is for the garbage collector to deal with, which means your computer is even slower, and that time is hard to measure. I do know that a new feature to Firefox 3.6 is that the garbage collector frees the memory in a new thread, which means that older versions of Firefox did not, and that means that the chances of pauses were even greater. For more reading on these changes I am mentioning in Firefox 3.6 please read this article.

The Tests

Description

In order to test the factors listed above I knew I needed a number of different versions of the convert2RegExp function, but the other piece I needed was a collection of pattern strings to test. I made the different versions of convert2RegExp easily, but when it came to making the pattern set(s) to test I had to do some more thinking.

Test Pattern Sets

I may not have made the best choice, but I decided to use two pattern sets:

1. Pattern Set 0: 3/50 patterns use the magic tld expression, 2/50 do not use the tld expression, but match /\.[^\.ld]*t[^\.td]*l[^\.lt]*d/
2. Pattern Set 1: 2/50 patterns use the magic tld expression

In retrospect I think testing an even worse case might be advantages, although I don't suspect the tld expression is used very often. It's hard to say however, I would like to see an audit of userscripts.org, but even with that it'd hard to know what the average case is for all Greasemonkey users, even when you take the install counts from userscripts.org into account, although all of that perspective would be nice to have.

I am an avid user of Greasemonkey and I write quite a few userscripts, and I find that in practice that there are very few times when I would need to userscript to use the magic tld expression, so my feeling is that the ratio is probably closer to 1 tld pattern per 100 or more.

Test Pages/Versions

I decided to run my tests of the different factors listed above for FF3.5/FF3.6 on WinXP, OSX, and Ubuntu (only FF3.5). The test pages worth pointing out are:

• Original
• Alternate 3: test of only factor I
• Alternate 5: test of only factor II
• Alternate 6: test of factor II + minor change to return a value asap.
• Alternate 11: test of factor I + II, uses array.unshift() and array.join(), and returns a value asap
• Alternate 12: test of factor I + II, uses array.push() and array.join(), and returns a value asap
• Alternate 13: test of factor I + II, and returns a value asap
• Alternate 15: test using array.unshift() and array.join() as only change

Test Results

Here are the tests I ran and the results I record (the links are to published google docs spreadsheets, all values are in seconds):

AMD 2.21Ghz, DDR2, WindowsXP

• FF3.6 Test: http://spreadsheets.google.com/pub?key=ta1SodwhpEDu84DGQFha4IQ&output=html
• FF3.6 Test I: http://spreadsheets.google.com/pub?key=tRfipvzCX5k_t5C6x3kNpbw&output=html
• FF3.5 Test: http://spreadsheets.google.com/pub?key=tSLVJBH7QCS69KYKugQZJEg&output=html
• FF3.5 Test I: http://spreadsheets.google.com/pub?key=tiZ5WBFo3xyINA1UrVKBG8w&output=html

Intel Duo 2.0Ghz, DDR3, Mac OSX

• FF3.6 Test: http://spreadsheets.google.com/pub?key=tjunrmJDRMql9lraJluyNdw&output=html
• FF3.6 Test I: http://spreadsheets.google.com/pub?key=t-L9k5rRF1WrEkXy0HiuJww&output=html

Intel Duo 2.0Ghz, DDR3, Windows XP

• FF3.5 Test: http://spreadsheets.google.com/pub?key=tGdaL4e0NKxWfYewPfKEvvA&output=html
• FF3.5 Test I: http://spreadsheets.google.com/pub?key=tSjVusVuI3B_y-77UXaWNjw&output=html
• FF3.6 Test: http://spreadsheets.google.com/pub?key=tMHASlMvO9MXJeKJgmecKzQ&output=html
• FF3.6 Test I: http://spreadsheets.google.com/pub?key=tIwBb34GBojadWHiy_snxyw&output=html

Intel 2.0Ghz, DDR, Ubuntu

• FF3.5 Test: http://spreadsheets.google.com/pub?key=t-zxUEHHGclQYuSYbhTroFA&output=html
• FF3.5 Test I: http://spreadsheets.google.com/pub?key=tRv04ytir3BkH9oGbUozUqA&output=html

Each test was using a version of convert2RegExp, on a set of 50 patterns, 2500 times.

Results

• In all cases alternate 3 was faster than the original.
• In all cases alternate 5 and 6 were faster than the original.
• In all cases alternate 6 seemed to be slightly faster than 5.
• In all cases alternate 15 is much faster than the original for FF3.6, and slightly slower for FF3.5.
• For FF3.5 alternate 13 > alternate 12 ~= original ~> alternate 11
• For FF3.6 alternate 11 > alternate 13 > alternate 12 > original.

Although for FF3.5 we know that alternate 13 is a bad choice because it uses the += operator which result in O(N^2) characters copied, so we can't use that in my opinion. The final choice has to be between alternate 11 and alternate 12.

This is where I need your help, I have no idea why using array.unshift() is so much faster in Firefox 3.6, and taken that it is, and that the majority of Greasemonkey users will be using the latest version of Firefox, should we use array.unshift() and not array.push()?

Tweet Me!

I recently ran in to this error message when trying to pull or push anything from/to GitHub with TortoiseGit and PuTTY: "Disconnected: No Supported authentication methods available".

It turns out this message was a result of my lack of knowledge of PuTTY, and I simply did not have Pageant running with the ssh key for GitHub loaded.

I did some searching on Google and there wasn't much help out there that I could find, so I wanted to explain this for the next guy and make a reference to the only page that helped me, which was at GitHub: Guides: Addressing authentication problems with SSH