I'm writing a CSS book
JavaScript archives Compatibility CSS coaching Thidrekssaga QuirksBlog Donations Politics Mobile About COH
QuirksMode
PPK sitemapcontact

Reading out the end time in browser speed tests

A few weeks back I did some DOM speed tests on mobile browsers (results forthcoming). The most important result of these tests is not the actual values (although they’re interesting), but the fact that I could finally prove a theory that I’ve had in the back of my mind for at least two years now.

Basically, when setting up a speed test you should be very careful to allow the browser to render the result on screen before you close the test by reading out the second timestamp.

An example will clarify this point. A DOM speed test may look roughly as follows:

function testIt() {
	var startTime = new Date().getTime();

	// actual DOM functionality to be tested goes here

	var endTime = new Date().getTime();
	var result = (endTime-startTime)/1000;
	// print result
}

Although this script might seem fine at first sight, it’s not. The problem here is that the entire test, including the time measurements, are wrapped in one function, and that some browsers only applies the result of the test (i.e. the changes in the DOM you want to test) to the screen after the function has ended entirely. Thus the end timestamp is read before the browser has applied the result to the screen.

So what really happens here is the following:

  1. Get start time
  2. Perform DOM manipulation in browser memory
  3. Get end time and calculate result
  4. The function ends, and only now does the browser start to apply the results of the DOM manipulation to the actual DOM.

Not all browsers wait until the end of the function to apply the results, but some do, among which Safari for the iPhone 2.2. These browsers only report the time it takes them to build the new DOM tree in the browser memory, and disregard the time it takes to show it on the screen.

Any testing methodology must work in all browsers, so our test function must work around this problem.

The correct way of conducting this test is setting a timeout for reading out the end time. The function ends when the in-memory DOM manipulation has been done, which allows the browser to apply the changes.

Because the browser uses its complete capacity for applying the changes to the screen, executing the timeout is deferred until it is done, even if that takes far longer than the formal timeout time.

function testIt() {
	var startTime = new Date().getTime();

	// actual DOM functionality to be tested goes here

	setTimeout(function () {
		var endTime = new Date().getTime();
		var result = (endTime-startTime)/1000;
		// print result
	},10)
}

What happens now is the following:

  1. Get start time
  2. Perform DOM manipulation in browser memory
  3. Define a function to get end time and calculate result and set a timeout
  4. The function ends, and only now does the browser start to apply the results of the DOM manipulation to the actual DOM.
  5. Once the DOM manipulation is finished, the browser is freed up to treat the timeout we set, and now the function that gets the end time and shows the result is executed.

In other words, this allows us to also measure the time the browser needs to show the results on screen.

This is what we really want to know. If the browser is lightning-fast in performing DOM manipulations in its memory, but sluggish in applying them to the screen, the net result for the end user is a relatively sluggish user experience.

This is not a random theoretical musing. I tried both methodologies on Safari iPhone 2.2 in a test that generated 5,000 list items. The first, incorrect, method yields a total time of about 3 seconds, while the second, correct, method yields a total time of about 14 seconds. That’s an 11 second difference.

(I’d be interested in the iPhone 3 results. I deliberately haven’t updated my iPhone yet, because there still are some tests I want to conduct on the 2.2 OS.)

In other words, the iPhone 2.2 takes 3 seconds to perform the necessary calculations in its memory, but subsequently takes 11 seconds to show the changes on screen. It’s slow, in other words. The first method does not reveal that; we need the second one for an accurate reading.

Therefore it’s important that whenever you do speed tests, the end time should be calculated only after the browser has finished applying the results to the screen.

This is the blog of Peter-Paul Koch, web developer, consultant, and trainer. You can also follow him on Twitter or Mastodon.
Atom RSS

If you like this blog, why not donate a little bit of money to help me pay my bills?

Categories:

Comments

Comments are closed.

1 Posted by Ionut Popa on 17 August 2009 | Permalink

Shouldn't you substract the time used in the delay ? (10ms)

2 Posted by Andrew Hedges on 17 August 2009 | Permalink

iPhone 3G (not 3GS), incorrect test took 1.822 seconds. Correct test took 12.205 seconds.

On the iPhone web app I'm currently developing, I noticed I had to set a 10ms delay to get certain DOM operations to "take" as well. Glad to see I'm not crazy!

3 Posted by ppk on 17 August 2009 | Permalink

@Ionut: No. The delay time is wholly used by the browser rendering the changes. That's the whole point of this trick.

4 Posted by Jean-Philippe Martin on 17 August 2009 | Permalink

Hi,
Great test and simple.

5 Posted by Andrea Giammarchi on 17 August 2009 | Permalink

PPK generally speaking you are right but there are a couple of problems here.

Basically timers are rarely fired under 15 milliseconds and, at the same, time we could be interested in both operations: the computation time, and the rendering time.

In few words these DOM tests should be performed returning 2 results. Let me explain with code:

{{{
function testIt() {

var startTime = new Date;

// actual DOM functionality to be tested goes here

// computation time
var endTime = new Date;

setTimeout(function () {

// render time
var renderTime = new Date;

var result = {
comp: (endTime - startTime) / 1000,
render: ((renderTime - endTime) - 15) / 1000
};

// print result
},
15 // browsers do not generally
// fire events with a delay
// less than 15
);
};
}}}

We will probably spot some browser a la Opera with high computation and low render or iPhone 2.2 with low computation and high render.

Do you agree? Regards

6 Posted by Andrea Giammarchi on 17 August 2009 | Permalink

@ppk No. The delay time is wholly used by the browser rendering the changes. That's the whole point of this trick.

What if some browser renders directly before the timeout then?

7 Posted by masklinn on 17 August 2009 | Permalink

@Andrea
> What if some browser renders directly before the timeout then?

The test result will be off by 15ms.

Considering with this test all mobiles are going to be far above 1s, that gives an error of at worst 1.5% (and usually far lower, on ppk's page the fastest listed device is at 5.4s, which gives us a potential error of 0.28%). Not exactly something worth worrying about, I don't think PPK considers these tests scientific, they're here to give people an idea of what to expect rather than hard numbers (I might be proven wrong if ppk whips out bar charts with error intervals and full-blown stat analysis of the results, but I doubt he's going to bother)

8 Posted by Matthew Gast on 17 August 2009 | Permalink

iPhone 3GS w/ 3.0 OS took 0.533ms for the incorrect test, 3.063ms for the correct test.

9 Posted by Andrea Giammarchi on 17 August 2009 | Permalink

@masklinn in any case he should consider both times so the only thing is to remove "- 15" from my example.

This kind of consideration is interesting in any case and for each browser, that is why my suggestion.

Regards

10 Posted by Ionut Popa on 17 August 2009 | Permalink

@ppk This test and the 10 milliseconds are somewhat empirical and not reliable. We should find something better...

11 Posted by Andrea Giammarchi on 17 August 2009 | Permalink

@Ionut maybe 10 ms for DOM tests are not a problem while I was thinking about a general purpose function but being DOM something "a part" I agree we should find a way to perform the layout linearly in the single callback. For ppk tests and purpose I guess this technique is almost perfect though

12 Posted by Dmitry on 18 August 2009 | Permalink

10.8 seconds on my Nokia E90 (webkit).

13 Posted by Ash Searle on 18 August 2009 | Permalink

@ppk I guess there's a reason why you couldn't use a zero-timeout: setTimeout(fn, 0); any chance you can explain why?

Several people seem confused by the arbitrary 10ms delay - and you can't call it "the correct way" without explaining that the computation stage takes hundres of times longer, and that "setTimeout(fn, 0)" is flawed in some way (is it?)

14 Posted by Jake Archibald on 18 August 2009 | Permalink

I've been using a similar method to test rendering times, although I'm not comparing the results between browsers, just comparing different techniques on an individual browser.

Out of curiosity, why 10ms? I've used 0ms in the past to simply place the callback in the queue (after rendering). Have you encountered a browser that would call func setTimeout(func, 0) before redraw has completed?

15 Posted by Jake Archibald on 18 August 2009 | Permalink

Hah, @Ash Searle, snap.

16 Posted by mrm on 18 August 2009 | Permalink

For those of you who don't understand how timers work, the infamous John Resig posted an article on this subject a while ago. http://ejohn.org/blog/how-javascript-timers-work/

17 Posted by BARTdG on 18 August 2009 | Permalink

@mrm: Interesting link. It raises one question however.

If PPK's test is based on the fact that javascript is single-threaded, does that mean that the trick doesn't work in Google Chrome, as Chrome's js engine is multi-threaded?

18 Posted by Joost Diepenmaat on 19 August 2009 | Permalink

@BARTgG: chrome's js engine might be multithreaded, but in a single page/tab, I'm willing to bet you won't actually get multiple threads, since there is no way to do that safely in clientside JS. the whole DOM and most of the BOM relies on a single threaded event model.

On the other hand, I don't think there's any hard /guarantee/ anywhere that the rendering engine will update the whole page just because you're returning from an event handler.

@PPK: doesn't reading an element's offsetWith and/or similar properties force an immediate rendering update? If that's reliable that seems to me to be a better way to doing this.


19 Posted by Michael Johnson on 27 August 2009 | Permalink

@Joost/BARTdG I seem to remember reading in the JavaScript spec that the JavaScript engine should be single threaded, or at least appear to be to the actual JS code, in order to make certain promises (I'm not sure which promises...and don't have time to look it up atm)

20 Posted by Tanny O'Haley on 3 September 2009 | Permalink

Palm Pre webOS 1.1.0 took 1.932 for the incorrect test and 6.604 for the corrected test. It didn't seem to matter if the screen was zoomed or not.

21 Posted by Randall Parker on 3 September 2009 | Permalink

Whether a 10 msec wait time will introduce a substantial error in the measurement depends on 2 factors:
1) The granularity of the OS scheduler. On Windows PCs that's about 16 msec. On some mobiles it might be 1 msec. Other (non-phone) Linux ARM embedded devices I work on have 1 msec OS scheduler frequency.
2) The amount of time for the page rendering. If the page rendering time is very small then the 10 msec timeout will be too large a fraction of the total time. For a 1 second page rendering time the 10 msec will have an insignificant impact.

22 Posted by Dean Edwards on 4 September 2009 | Permalink

// actual DOM functionality to be tested goes here

It depends on the "DOM functionality" you are testing.

If you are causing a reflow then the the reflow calculations are suspended until you exit the current execution context.

If you can force an immediate reflow then you can still check the time immediately after. What will cause an immediate reflow varies from platform to platform. Checking the offsetWidth of an element is usually sufficient, calling getComputedStyle() sometimes works too. It's an art more than it is a science. :)

23 Posted by peet on 4 September 2009 | Permalink

Using setTimeout, you are assuming it has to wait until after the rendering.

At the end of the DOM changes, write the inline JS that marks the end time.

I.e. it is less of an assumption that rendering is consecutive, than rendering and setTimeout will be consecutive.

24 Posted by clebio on 13 September 2009 | Permalink

Palm Pre (webOS 1.1.0):
3 runs: 6.8, 7.6, 6.6 seconds
(second method)

25 Posted by seeth on 23 December 2009 | Permalink

Blackberry Bold 9000 with OS5
Test running ...
It took 4.597 seconds.