Over the past few weeks I have done some fundamental research into the touch action and its consequences, and it’s time to present my conclusions in the form of the inevitable compatibility table. I have also written an advisory paper that details what browser vendors must do in order to get by in the mobile touchscreen space. Finally, I discuss a few aspects of my research in this article.
Disclosure: This research was ordered and paid for by Vodafone. Nokia, Microsoft, Palm, and RIM have helped it along by donating devices to me.
When a user touches the screen of a touchscreen phone, sufficient events should fire so that web developers know what’s going on and can decide what actions to take. Unfortunately most mobile browsers, especially Opera and Firefox, are severely deficient here.
The touch action is way overloaded, and most browsers have trouble distinguishing between a click action and a scroll action. Properly making this distinction is the only way of creating a truly captivating mobile touchscreen browsing experience.
The iPhone’s touch event model is excellent and should be copied by all
other browsers. In fact, these events are so important that I feel that any browser that does
not support them by the end of 2010 is out of the mobile browser arms race.
one problem with the iPhone model, and it’s relatively easy to fix.
I have created a drag-and-drop script that works on iPhone and Android
as well as the desktop browsers,
a multitouch drag-and-drop script that works only on
the iPhone, and a scrolling layer script that
forms the basis of faking
position: fixed on iPhone and Android, who do not
support that declaration natively.
I will hold a presentation on my research at the DIBI conference, Newcastle upon Tyne, 28th April. It will likely include future discoveries and thoughts.
One of the most serious problems of the current touchscreen interfaces is that the touch action is way way overloaded. When the user touches the screen he may want to start a click action, a scroll action, or a resize action. Distinguishing correctly between these three actions is what sets a good touchscreen interface apart from a bad one.
This is especially important with regard to the click action. Far too often, an intended click on an element does not work because the user moves his finger a tiny little bit during the action, and the operating system concludes he wants to perform a scroll action instead. The page does not really scroll because the user does not really move his finger, but the click action is canceled, and this results in a seemingly unresponsive interface.
This problem is especially severe in the Samsung WebKit that runs in the Samsung H1 and M1 Widget Manager, but it occurs on other operating systems, too.
Only iPhone and Palm have really solved this problem; the other browser vendors are in various stages of catching up.
Events can be divided into three groups:
The touch events are supported only by iPhone and Android, in that order. Even multitouch Androids do not support more than one series of touch events (i.e. more than one finger) at a time.
All browsers support the legacy events, although some don’t support all of them. Still, this is largely irrelevant.
The real chaos is in the interface event group. Most WebKit-based browsers support most events reasonably well, but all others, including Opera and Firefox, don’t support them at all or make a mess of them. That’s annoying, because it’s these events that actually tell us what the user is trying to achieve.
The touch events are touchstart, touchmove, and touchend. As you’d expect the first fires once when the user initially touches the screen, the second continuously while the user is moving his finger, and the third when the user releases the screen.
See Touching and Gesturing on the iPhone for more information on how the touch events work on the iPhone. I do not repeat this background information in any of my articles.
All browsers MUST (in the sense of RFC 2119) support these events at their earliest opportunity. Any browser that does not support them by the end of 2010 is out of the mobile browser race.
The touch events are currently supported only by iPhone and Android. There are a few differences between their models:
touchesinterface, while the Android stores them directly on the event object itself.
There is also the touchcancel event that fires when the user’s touch is “canceled.” I haven’t yet studied it; I feel that it’s useful only in a very few edge cases.
The touchmove and touchend events also fire when the touch action moves out of the element where the touchstart event took place.
If a touch action moves into an element, touchstart generally fires. I’m wondering if we should use touchenter instead, which in turn presupposes the existence of a touchleave event.
I’m also wondering whether we need a touchhold event, and whether the touch events should return an area instead of just a coordinate of a single pixel.
The interface events fire when the user actually takes an action instead of aimlessly touching the screen. These actions include:
Browser compatibility is a bloody mess:
Current websites are created exclusively for desktop, and they use the legacy events extensively. Since mobile browsers want to give their users access to the “fixed” web, they have to support these legacy events.
Still, the touchscreen environment is not the same as the desktop environment. Most notably, a mouse action on the desktop is not equivalent to a touch action on a phone. The user of a touchscreen phone needs to touch the screen for pretty much any action (zoom, scroll, click), something that is not true for a mouse.
This is how the legacy events work currently:
:hoverstyles, if any, are applied to the element.
:hoverstyles are removed.
In addition to the official test page I created the following tests:
position: fixedto work on the iPhone and Android. Essentially, if you make this script vertical instead of horizontal it’ll create a scrollable layer between “fixed” panels. There are various tricky bits involved, though, and I’ll have to return to this problem later. Still, we now have the basic solution.
I suspect that on the iPhone the actual clickable area is slightly shifted downwards with regard to the visible HTML element. That is, a click just below the element may also be counted as a click on the element itself. This does not go for the area just above the element.
To try it for yourself, use the multitouch drag-and-drop. Try it in normal vertical orientation first, and try to touch the element as low as possible. Then turn the device 180 degrees and try again. You’ll find that you have to touch the HTML element distinctly higher now.
I’m still trying to figure out how to officially prove that Apple does this; maybe I’ve misinterpreted the test results.
Finally, during my research I noticed time and again how unbelievably lousy Apple’s Safari iPhone documentation site is. I advise developers to use my pages instead — as usual.
Part of the problem is the content; once you get to the correct page you will find a terse summary of the situation that is correct as far as it goes but mostly leaves out stuff that doesn’t fall exactly in the page’s topic, even if it’s closely related.
Worse, the invention of the cross-reference has taken Apple completely by surprise. I wish I could say it’s scrambling to catch up, but it isn’t. No cross-references whatsoever. Anywhere.
Try it yourself. Go to the TouchEvent page. It contains absolutely zero reference to the crucial TouchList interface that exposes useful information about the touch events. There is in fact a workable page about TouchList. Try to find it from the TouchEvent page. You won’t.
Alternatively, try finding the same information from the Safari Reference Library home page. Search works — if you know what to search for. The official navigation is absolutely useless.
Apple’s pseudo-frames or something also drive me crazy. The keyboard focus does not snap to the actual documentation page, which means I initially can’t scroll. (Firefox)
Apple, please fix.
Upcoming speaking gigs:
Comments are closed.