Testing mobile browser compatibility — the beginning

About a month ago the software department of Vodafone Internet Services, based in Düsseldorf, Germany, asked for my help in creating mobile widgets according to the W3C Widgets specification. In particular, they’d noticed there are differences between browsers even on mobile phones (imagine my surprise), and decided they needed advice from a specialist (that would be me).

Better still, it quickly turned out that they were willing to pay me for doing serious mobile browser compatibility tests and publishing them on this site. The payment thingy is quite unusual, I can tell you (though not entirely unique).

This is easily the best job offer I’ve gotten in my entire freelance career, so I hurried to accept it. Meanwhile I’ve done mobile tests for five days; enough to offer some guidance for setting up a doctrine for mobile browser testing.

As far as I’m concerned you can look over my shoulder while I’m working, but please PLEASE remember that everything I say may change radically without notice after I’ve tested the same browsers on other devices.

Right now I’ve only done a few tests of functionality that’s basic to the mobile experience, and even these basic tests will likely have to be expanded. Besides, right now getting a general feeling for mobile testing and its manifold problems is more important than running lots and lots of tests.

If you’re interested in helping me you can follow me on Twitter, which is shaping up to be a fantasically valuable aid.

I report oddities and bugs within minutes of finding them, and I’ve received quite a bit of useful feedback already. In fact, it’s likely that these tests won’t ever succeed without cloudsourcing some of the more complex problems.

(Please remember I do most of my tests during European office hours.)

Sheer numbers

The first and most obvious problem in mobile browser testing is the sheer numbers. The Vodafone master list of devices that must support their widgets contains 43 phone types, and those are only the modern ones that have a reasonable chance of containing a modern browser.

For the moment I’m considering every single browser on every single device unique; so if I test five Nokia phones that all have WebKit, I’m testing five different WebKit versions and the resulting compatibility tables will show five entries.

Sure, I assume that these five WebKits will resemble each other, and I expect to be able to eventually merge these five entries into one. I’m not going to hurry that process, though; one of the points of this research is to find out if there are differences between these five devices. There could be, and if there are Vodafone and I (and, in fact, the world at large) have to know about them.

Fortunately, the Vodafone team has quite a few phones, and although it’s absolutely impossible to test everything on every phone, it does allow me the chance to validate my conclusions time and again.

The browsers

How many browsers are there in the mobile space? Right now I think the correct answer is five, but I may be wrong. They are:

There’s also Fennec of the Mozilla Project, but as far as I know it’s in early alpha, and I was even unable to find a formal homepage for it. It’s not yet the default browser on any mobile device; as soon as it is I’ll include it in my tests.

If you know of more mobile browsers, please leave a comment.

Opera and WebKit form the vanguard of the mobile space, IE, NetFront and Blackberry the rearguard. We’ll leave it at that for the moment; I will provide a more detailed breakdown when I have more data.

Test pages

I’m creating new test pages for my mobile tests. The main reason is readability; my current test pages are optimised for screen, and the events tests, especially, require ample amounts of space to see all event reports. Mobile devices just don’t have that space available. Besides, the event test pages refuse to work in Nokia WebKit (two versions tested).

After three or four tests I found that the version of IE Mobile I’m using doesn’t support the event object at all and requires you to use inline events. document.onclick, for instance, is not supported. (Oh, and createTextNode() seems to be missing, too.)

So theoretically, if I want to give IE Mobile a fair chance to compete, I’d have to write tests that use inline event handlers only and don’t depend on the event object.

Right now I’m not giving IE Mobile a fair chance. Obviously, I will validate my conclusions by an IE Mobile test on a completely different phone, but if that IE version has the same problems, I’m going to quietly forget about it. I’m just not going to waste my time on a browser that forces me to break one of the core rules of unobtrusive JavaScript.

By the way, don’t blame the regular IE team for this fantastically incapable browser. IE Mobile is created by a completely different team.

But exactly which IE Mobile was I testing? That turns out to be a tricky question.

Identification

Identifying exactly which browser is running on a certain phone is going to be a major problem. Mobile browsers, being chromeless, are usually impossible to distinguish by sight, so I need a browser detect script to help me out. Unfortunately, writing such a script is not as easy as it seems.

Right now browser vendors are running around like headless chicken strewing identification strings like chaff, making sure they’re totally incompatible with desktop identification strings, other mobile identification strings, and occasionally even other identification strings sent by the same browser.

Opera is rumoured to be pulling its act together, but I haven’t yet personally verified that.

In any case, distrust any market share stats of mobile browsers you see. The detection script is quite likely to be totally wrong.

IE

When I started writing this entry I was convinced that the IE Mobile I was testing was the brand-new IE Mobile 6. Why? Because it identified itself as IE6. Silly me. I’m not going to make that mistake any more.

When I saw this IE Mobile 6 promo video I started to doubt my identification. The IE Mobile I test doesn’t have all these new features, even though it runs on a touchscreen phone. Besides, IE Mobile 6 has not yet been released (I think).

So the mobile browser identifying itself as IE Mobile 6 is in fact the earlier IE Mobile that was branched off from IE4. Still, it supports basic Ajax, something that IE4 didn’t.

Now if the old IE Mobile identifies itself as IE6, how will the new IE Mobile, officially announced as version 6, identify itself? Also as IE6? I wouldn’t put it beyond Microsoft.

To make things worse the old IE Mobile also has IEMobile 7.11 in its identification string. No doubt the new IE Mobile will announce itself as IEMobile 8.01 at the very least.

WebKit

WebKit version numbers are mostly useless. The most common mobile one I encountered is 422, which is distinctly lower than Safari 3.0’s 522. The funny thing is that WebKit on Nokia supports :only-child and Safari 3.0 doesn’t.

Therefore it seems likely that Nokia has branched off its browser from WebKit 422 and then continued to work on it by itself, importing (apparently) certain newer WebKit modules but not updating to an entirely new version. They didn’t bother to update the identification string, either.

Worse, many other mobile device manufacturers are doing the same. WebKit/422 on Nokia is likely to be different from WebKit/422 on Motorola. I’m inches away from declaring the WebKit version number completely worthless.

By the way, the navigator.vendor of all mobile WebKits I’ve seen so far is Apple! This is pure misinformation (except on the iPhone), and I assume it’s a mistake of the WebKit core team who’ve accidentally left this value in place when they received updates from Apple. Please make this property empty; no information is better than false information.

Opera

Although modern Opera’s Mobile or Mini have Mobile or Mini in their identification string, the particular versions I’m using don’t. Fortunately Opera Mobile version numbering follows the Opera desktop one, so the knowledge that I’m testing version 9.5 helps me instead of hindering me.

As to the Mini, it took me and an Opera employee quite some time to identify it as version 2; distinctly antiquated. Besides, when I sent it over to Steve Souders’s UA String Parser, which uses the HTTP User-Agent string instead of JavaScript’s navigator.userAgent, this browser identified itself as NetFront! The string was totally, and I mean totally, different.

You can’t make this up.

New classes of incompatibilities

It seems that Nokia devices refuse to send key codes to the browsers. Although all key events are properly fired, keyCode and friends, which are supposed to contain the key number, stubbornly remain 0. This is not the browsers’ fault; if the device withholds this information there’s nothing they can do.

Until now I’ve seen this Nokia bug only on devices with numerical keyboards. Maybe touch devices or devices with qwerty keyboards will behave properly. One more thing to test.

This is an example of an entirely new class of problems: device compatibility errors. Since I’ve only discovered them just now I’m not yet sure how to treat them, except that we should be very careful to distinguish them from browser compatibility errors.

There’s also a new class of browser compatibility errors: when browsers are unable to make full use of the device’s capabilities. An example is an orientation change from portrait to landscape or vice versa: the device may support this fine, but if the browser doesn’t, nothing happens. On the HTC Diamond Opera supports orientation change; IE Mobile doesn’t.

The beginning

This is only the beginning of our research into the mobile sphere and our discovery of exciting new classes of incompatibilities. Again, if you’re interested in helping me out and/or have experience in mobile browser compatibility problems, follow me on Twitter. I’ll need the help.

This is the blog of Peter-Paul Koch, mobile platform strategist, consultant, and trainer. You can also follow him on Twitter.
Atom RSS

I’m speaking at the following conferences:

(Data from Lanyrd)

Categories:

Monthlies:

Comments

Comments are closed.

1 Posted by Robin on 6 March 2009 | Permalink

Wow… Rather you than me I have to say, but I’m sure the information will prove useful!

2 Posted by Harley on 6 March 2009 | Permalink

Congrats on the job!

I'm happy someone like you landed this gig. This information has been needed for a long time by the industry at large. And since you're so open with your findings, this can only make the job of the web developer (me!) easier.

3 Posted by Shaun Inman on 6 March 2009 | Permalink

Was going to tweet this but the 140 character limit got in the way.

Not sure if this is relevant to your tests but I've observed chromeless Webclips on the iPhone and iPod touch exhibiting behavior that differs from the same page displayed in MobileSafari. The one example that comes to mind is onOrientationChange, the event doesn't fire in a chromeless webclip.

Best of luck in this endeavor!

4 Posted by Wacek on 6 March 2009 | Permalink

Great job!

btw. an quite complete overview of available mobile browsers can be found here: http://en.wikipedia.org/wiki/Mobile_browser

but most of them doesn't support JS, and therefore they are uninteresting in this context ;)

Have fun and good luck! ;o)

5 Posted by Wolf on 6 March 2009 | Permalink

This might be of little use but who knows: the Blackberries before the Storm use the "Blackberry browser" while the Blackberry Storm uses a Webkit-based browser.

6 Posted by David on 6 March 2009 | Permalink

You may also want to consider the Playstation Portable web browser. I believe this is considered a mobile browser.

I've also heard of one called Skyfire, which I thin is the only mobile browser that supports Adobe Flash.

7 Posted by Alberto on 6 March 2009 | Permalink

What is the browser in SonyEricsson phones? Is it also WebKit based?

Are you treating Opera Mini as a full browser? Isn't it more like a thin client connecting to Opera servers where the real page rendering is done?

8 Posted by Joris on 7 March 2009 | Permalink

Great news you got this Job. Can't wait for the next episode is this new browser soap ;-) feels like 1999 all over. Good read, very valuable investigation.

9 Posted by Stephen on 7 March 2009 | Permalink

A few common browsers I ran across when I did similar work a few months ago:

OpenWave (now renamed to Myriad, I think) is a frequently deployed browser on cell phones, such as my Razr.

Sidekicks seem to have their own browsers.

Palm OS runs the Palm Blazer browser.

10 Posted by Sander on 8 March 2009 | Permalink

There's also MicroB, the gecko-powered browser on the Nokia N810 and N800. Not a phone, but definitely a mobile device.

11 Posted by ppk on 8 March 2009 | Permalink

Thanks for all the useful comments!

Shaun re chromeless Safari windows: isn't there a setting you can use to fix a Safari window in one orientation? If that's so, it's not a bug but a feature.

Wolf re Blackberry: Thanks! Useful info.

David: Vodafone doesn't have a Playstation Portable, so I guess I won't test it in the near future. Thanks for the Skyfire tip; it uses the same client-server scheme as Opera Mini. Will have to test it eventually.

Alberto: Phone in SonyEricsson I tested is Opera Mini, but I'll test a modern SE next week, I hope.

Stephen: OpenWave doesn't support HTML, Myriad does, but little product information can be found. Sidekick does Opera Mini-like client-server config with Gecko rendering engine. Palm Blazer seems to use NetFront engine.

Sander: Vodafone doesn't have these Internet Tablets available, so I doubt I'm going to test this browser any time soon.

12 Posted by gday on 8 March 2009 | Permalink

@ppk, have you checked out the new Bolt browser from Bitstream? It's an Opera Mini clone. Not sure what it's running.

@David

"You may also want to consider the Playstation Portable web browser. I believe this is considered a mobile browser."

It's NetFront.

"I've also heard of one called Skyfire, which I thin is the only mobile browser that supports Adobe Flash."

Nope. All browsers that support plugins can support Flash. So it's basically not up to the browser, but rather a question of whether the Flash plugin is installed or not.

Skyfire uses Gecko.

13 Posted by Wade on 9 March 2009 | Permalink

@Alberto: Sony Ericsson have used NetFront for many years on their phones. With more recent high-end phones they tend to put Opera on, though. Maybe they're starting to give up on NetFront.

14 Posted by schwoortz on 9 March 2009 | Permalink

That's a great research you're doing!

BlackBerry browsers (pre Storm): Terrible. Especially as they are shipped with various default settings! The exact same device is pre-configured differently, depending on vendor and branding. With/without support for various things like js, css "table-support".

With IEMobile it seems to me that things also depend alot on the windows version itself (CE, 5, 6, ...) and the actual device. The newer candidates seem to handle things "quite ok".

15 Posted by Wolf on 9 March 2009 | Permalink

I'm building a mobile app at the moment and I'm stuck at the font sizes part.

My current solution is to use percentages, seems to be the safest bet. Most mobile browsers override seem to override your font sizes with their settings; so I'm setting some "good defaults" and make sure my design works properly with a good range of font sizes.

16 Posted by schwoortz on 9 March 2009 | Permalink

- Unfortunately, conditional comments do not work for IEMobile
- same for media handheld (not reliable)
- support for font-sizes differs alot between devices/platforms

17 Posted by Dominique Hazael-Massieux on 9 March 2009 | Permalink

Hi,

I work in W3C where I (among other things) chair a group dedicated to testing mobile browsers:
http://www.w3.org/2005/MWI/Tests/

Any chance we could interest you in joining your efforts to ours?

Dom

18 Posted by Ben Cachel on 10 March 2009 | Permalink

Android based mobile browser does a good job at rendering complex web pages. I had a chance to use one Android powered device last weekend, the only one in Down Under marked so far (pls correct me if I am wrong), great toy! Good on you PPK.

19 Posted by Reese Payton on 13 March 2009 | Permalink

Thanks for letting us watch over your shoulder, it's been an interesting and enlightening read and I thank you for it.
Until The Next Time,
Reese Payton

20 Posted by Rob on 15 March 2009 | Permalink

Just when you think that everyone may be getting their act together with web browsers a whole new pile of crap comes along to sift through.

Good read, thanks.

21 Posted by Tomomi on 17 March 2009 | Permalink

Great work!!!

The other major(?) browsers I can think of are:

- Obigo by Teleca (Samsung)
- Blazer by Palm
- InfraWare (LG, Helio)
- Gecko - Fennec by Mozilla and Skyfire

But I feel that the first three are going away from market, since LG and Samsung started shipping new devices with Webkit (S60) browser.

BTW, as gday mentioned, Bolt is also a webkit-based proxy browser.

22 Posted by Tomomi on 17 March 2009 | Permalink

Oops forget about Blazer. Blazer3+ are based on NetFront engine.

23 Posted by slxception on 24 March 2009 | Permalink

A couple things to add:
PSP uses the NetFront browser, while the Nintendo DS/DSi will use a variation of Opera (but not mobile or mini AFAIK. It seems to use a modified version of Opera 9 http://en.wikipedia.org/wiki/Opera_Web_Browser#Nintendo_DS

As for Safari's CSS settings, in order to style specifically for iPhone, you need to use:

@media only screen and (max-device-width: 480px) {
/* style info */
}
Source: http://tinyurl.com/c756g8