Safari 4 vs. Chrome 2.x in Peacekeeper
Posted 03-20-2009 at 11:53 AM by benchmarker
Hello and welcome to the Peacekeeper -blog. This blog will serve as an outlet for any Peacekeeper related news and interesting statistics.
Users in the community forums raised suspicions about Safari 4.0 scoring better than Google Chrome in Peacekeeper compared to the established SunSpider JavaScript benchmark, so we decided to investigate differences between the two in the individual Peacekeeper tests using data gathered from the Peacekeeper submits so far.
The chosen method was a fairly ad-hoc pair-wise comparison of scores from users who had ran Peacekeeper on both Safari 4.0 and Chrome 2.x using the “Benchmark another browser”-function. When users had several Safari or Chrome scores the highest score for each browser was chosen. This gives us a reasonable assumption that these scores were achieved under similar conditions apart from the browser.
The scores are illustrated here as scatter plots with the Chrome score on the x-axis and Safari score of the same user on the y-axis.
Please refer to the Peacekeeper FAQ for information on the individual tests. The charts contain a minimum of 635 data points.
Safari indeed currently scores consistently better than Chrome across the whole score spectrum.
Both array tests clearly favor Safari. SunSpider doesn’t have pure tests for array manipulation so Peacekeeper having these tests tips the scales in Safari’s favor.
The DOM tests also, apart from DOMJQueryBasics, favor Safari. SunSpider doesn’t test DOM performance at all, so this is certainly a big reason for Peacekeeper favoring Safari over Chrome more than SunSpider.
The string operation test scores are more equal with Chrome beating Safari in some tests.
The stringFilter –test and stringValidateForm –test to a lesser degree show curious differences in Chrome scoring. This appears to be caused by differences between Chrome 2.x versions.
The render suite represents another facet of browser performance not tested by SunSpider.
Safari appears to be limited to 100 frames per second in rendering while Chrome goes up to 250 fps. This means that Peacekeeper actually favors Chrome over Safari for fast systems in the rendering tests because 100 fps is probably a fast enough frame rate for any practical browser use scenario.
The renderAlpha –test appears to have occasional anomalous scoring on both browsers which signals a problem with the test that we’ll have to address in future Peacekeeper releases.
The Community –suite shows the same issues with the apparent 100 fps rendering cap in Safari. The situation is particularly bad in the community05Scroll test that has a very light workload and the 100fps cap is reached on a majority of the systems while Chrome shows performance differences for a wider range of systems.
So in conclusion, Peacekeeper favors Safari more than SunSpider due to additional types of tests that score better on Safari.
Other interesting results from this analysis were the occasional anomalous scoring in the renderAlpha test indicating problems with the test and the lower fps cap for Safari, which indicates that the workloads for the rendering tests might have to be increased to bring fps rates down.
- Pasi Virtanen
Site Manager
Users in the community forums raised suspicions about Safari 4.0 scoring better than Google Chrome in Peacekeeper compared to the established SunSpider JavaScript benchmark, so we decided to investigate differences between the two in the individual Peacekeeper tests using data gathered from the Peacekeeper submits so far.
The chosen method was a fairly ad-hoc pair-wise comparison of scores from users who had ran Peacekeeper on both Safari 4.0 and Chrome 2.x using the “Benchmark another browser”-function. When users had several Safari or Chrome scores the highest score for each browser was chosen. This gives us a reasonable assumption that these scores were achieved under similar conditions apart from the browser.
The scores are illustrated here as scatter plots with the Chrome score on the x-axis and Safari score of the same user on the y-axis.
Please refer to the Peacekeeper FAQ for information on the individual tests. The charts contain a minimum of 635 data points.
Overall Score
Safari indeed currently scores consistently better than Chrome across the whole score spectrum.
Array Suite
Both array tests clearly favor Safari. SunSpider doesn’t have pure tests for array manipulation so Peacekeeper having these tests tips the scales in Safari’s favor.
DOM Suite
The DOM tests also, apart from DOMJQueryBasics, favor Safari. SunSpider doesn’t test DOM performance at all, so this is certainly a big reason for Peacekeeper favoring Safari over Chrome more than SunSpider.
String Suite
The string operation test scores are more equal with Chrome beating Safari in some tests.
The stringFilter –test and stringValidateForm –test to a lesser degree show curious differences in Chrome scoring. This appears to be caused by differences between Chrome 2.x versions.
Render Suite
The render suite represents another facet of browser performance not tested by SunSpider.
Safari appears to be limited to 100 frames per second in rendering while Chrome goes up to 250 fps. This means that Peacekeeper actually favors Chrome over Safari for fast systems in the rendering tests because 100 fps is probably a fast enough frame rate for any practical browser use scenario.
The renderAlpha –test appears to have occasional anomalous scoring on both browsers which signals a problem with the test that we’ll have to address in future Peacekeeper releases.
Community Suite
The Community –suite shows the same issues with the apparent 100 fps rendering cap in Safari. The situation is particularly bad in the community05Scroll test that has a very light workload and the 100fps cap is reached on a majority of the systems while Chrome shows performance differences for a wider range of systems.
Conclusions
So in conclusion, Peacekeeper favors Safari more than SunSpider due to additional types of tests that score better on Safari.
Other interesting results from this analysis were the occasional anomalous scoring in the renderAlpha test indicating problems with the test and the lower fps cap for Safari, which indicates that the workloads for the rendering tests might have to be increased to bring fps rates down.
- Pasi Virtanen
Site Manager
Total Comments 0
Comments
Recent Blog Entries by benchmarker
- Safari 4.0 out of beta, now even faster (06-09-2009)
- "Fastest Firefox ever" is barely half as fast as Safari (06-03-2009)
- Peacekeeper moves out of beta (05-11-2009)
- Safari 4 vs. Chrome 2.x in Peacekeeper (03-20-2009)