Yet Another Nonsensical Javascript Benchmarking of Mostly Unreleased Browsers

After a couple of too-close-together performance-competition-between-browsers thingies, I decided to make my very own quick & dirty benchmark of current browsers. Webkit-based at least, although soon I'd like to at least try again Firefox; it's been months since I abandoned it for Webkit itself, due to a multitude of small problems, some of them caused by myself (much more than a hundred tabs always open, lots of extensions to lessen the load) but magnified by Firefox itself (extensions failing, slowness even when with much more reasonable numbers of tabs, sluggish Flash performance, problematic Java)...
In fact it's a great moment to make this kind of test, since Safari 4.0.2 has just appeared, Stainless 0.6.5 too, and ... well, WebKit has a recent nightly (r45641). Chrome is the available one right now: 3.0.192 (developer release)
A surprise has been to learn that Opera seems to have abandoned the race some time ago (ten times slower, javascript-wise, than any of the webkit-based browsers???). Will be... interesting to throw that to the next one who claims that Opera is holier than thou, blah, blah.
The benchmarks are Google's V8 (version 4) and Webkit's Sunspider.
v8: (bigger is better)
  • webkit: Run it once: 2160. Open new tab, run it again in the new tab: goes down to less than 2050, which is about 5% less. Next tabs hover just over 2050. Closing tabs and starting anew (but w/o restarting Webkit) goes back to 2160. (later I closed everything and retried, and didn't go over 2100)
  • safari: 2080 -> 1980. Same behaviour.
  • stainless: starts at 2064 and with each tab improves up to 2120!
  • chrome: pretty stable around 2950. Wow.
sunspider: (smaller is better)
  • webkit: starts at 787, worsens to 818 after a couple of new tabs with the same V8.
  • safari: from 795 to 805
  • stainless: again, improves from 695 to 680!
  • chrome: again, pretty stable winning at 650...
Interesting: Sunspider continually loads something between each subtest (something substantial). V8 only loads on, well, the first load; afterwards looks like the cache is enough, since every new tab with V8 doesn't cause (a lot of) traffic. BUT, in Stainless V8 seems to be reloaded in every new tab. Maybe Stainless is not sharing cache between the tabs?
And it looks like tabs are sometimes slow and sometimes fast: open tab, load V8, let it run. Let's say it gets a lowish score, 2050. Make new tab, load V8, let run. Let's say this one gets a somewhat high score, 2150. Now, run both tests again. The "slow" tab keeps being slow, and the "fast" tab keeps being fast... Strange.
(and looks like the first tab created in every window is "slow", and the next are "faster")
And with this, I forbid myself to keep wasting time benchmarking unreleased browsers. (But I sure will keep an eye on Stainless and chrome....if they had session saving, maybe I would switch right now! but at least they will be an interesting debugger...)
The machine used was a 2007 white macbook Core2Duo 2.16 GHz, with 3 GB RAM and about 5% of background CPU load.

No comments

Post a Comment