iPhone events

Yesterday I walked into the local phone store because the “Temporarily Unavailable” sign had been removed from their “Get your iPhone here” poster. To my utter surprise they had six (6!) entire iPhones for sale, and no, there was no waiting list. I walked back home with a shiny new gadget, impatient to start testing it.

Meanwhile I’ve done some tests; now it’s time for a report.

Before we continue, let’s get the bad CSS news out of the way: Safari on the iPhone does not support position: fixed. Certain Other Browsers were ridiculed for this lack; Safari won’t be.

I’ve updated the CSS Table, the Core Table and the Events Table. In this entry I’m going to talk about JavaScript events on the iPhone. They’re — interesting.

Documentation: none

The first reason they’re interesting is that they behave rather different than the Safari Web Content Guide for iPhone suggests. Not to mince words; this “documentation” is woefully incomplete and far too often plain wrong.

For instance, take a look at this test page. It proves that the iPhone Guide’s explanation of the event order when the user taps the screen is incorrect. The mousedown, mouseup and click events always fire when the user taps the screen, and not just when the content hasn’t changed after the mousemove (which is a totally whacko criterion, anyway).

All this is nothing new. Yet another excellent browser with shitty documentation. My main reason for creating this site was the total inadequacy of browser vendor documentation sites. So I have to study a new one. Big deal.

Let’s get started. And let’s completely ignore the documentation, just as it ignores us.

Moving your fingers

The iPhone uses a subtly different event model than traditional browsers. The user uses his fingers for all actions, and although fingers can be seen as a mouse (sort of) and a tap as a click, this comparison is not correct.

The problem is that several gestures, notably moving your finger and double-tapping, are reserved for system functions. Besides, the user can also put his finger somewhere else on the screen without any sort of pointer crossing the intermediate space—something that’s unthinkable in traditional mouse interaction.

The mouse is a continuous pointing device; the finger is discontinuous. That’s a profound difference that I wish I were able to clearly understand and explain.

is not the same as moving the mouse

In any case, it’s a difference that spells disaster for the mousemove, mouseover and mouseout events, as well as good old :hover.

All of the above depend on reading out and interpreting the mouse movement, and mouse movement has until now been one of those topics that were so taken for granted that nobody ever really thought about it.

But what if the mouse just disappears and reappears at a random place? What if there’s no movement but only a series of discrete click events?

With the coming of the iPhone the mouse model has lost its inescapable logic. Mousemove, mouseover and mouseout (and even poor old :hover) have been downgraded to device-specific events that may not survive in the long run.

But — and here lies the problem — these events are used in countless web sites and applications for a variety of purposes, and Apple simply cannot afford these sites not working on the iPhone.

That places the creators of Safari for iPhone in the same quandary as the speech browser vendors: they have to support an event model that doesn’t make sense in their interface but is used all over on the Web.

They have to redesign the triggers for these events in terms that make sense to their interface, even if that means that the event names don’t make sense any more. (They also have to document this tricky step properly.)

User focus

On the iPhone, mousemove, mouseover, mouseout, mousedown, mouseup and click and :hover depend on user focus.

  1. When the user focuses on an element by single-tapping it, the :hover styles are applied and the mouseover, mousemove, mousedown, mouseup and click events fire (always; and in that order).
  2. When the user focuses on another element, the mouseout event of the old element fires and the :hover styles are removed from it.

Initially, only links and form fields are clickable on the iPhone. However, you can make any element clickable by registering an event handler for mouseover, mouseout, mousedown, mouseup, mousemove or click on it.

Even better, once you’ve registered the event handler you can safely remove it. The element will remain clickable despite the event being gone. This is very useful to know — and the documentation is completely silent on this important point. You read it first here.

Play with this test page to get the idea.

The consequences

I haven’t yet used the iPhone to test a real application that depends on mouseover/out or mousemove, but I think the style of interaction changes quite a lot. (Comments are welcome; I’m looking for feedback on this issue.)

I am beginning to feel a secret hope that heavy mouse interaction (as in “You need a mouse to use our wonderful interface”) is on the way out, and click-based interfaces are the future.

A click is a sure sign of the user wanting to focus on a certain element, and that’s the information you need. What he does or does not do with a pointing device that may or may not be there has ceased to matter.

I don’t think it’s a coincidence that most of today’s cutting-edge applications are profoundly click-based.

In order to make your application iPhone-friendly you have to think in terms of user focus, and not of mouse movement. (And no, I have no idea what that means either. But no doubt someone will figure it out—or maybe someone already has figured it out.)

As a bonus, this helps accessibility in general. A user focus-based interface is far more adaptable to different devices and user needs than a continuous-pointing device-based interface that punishes all users of other devices.

The finger as mouse button

Now let’s consider the finger tap, which generally functions as a click. It even behaves a bit like a traditional click: the iPhone follows a link or opens a form field for editing when the user releases the screen.

Now the tap interaction looks suspiciously like the mousedown — mouseup — click trinity. User touches screen equals mousedown, user releases screen equals mouseup, and they are followed by a click event.

However, that’s not quite the way it works on the iPhone.

With a mouse, you can depress the button on one element, move the mouse and release the button on another element. Mousedown and mouseup events are duly fired, but because their targets are different elements no click event is fired.

But we already saw the iPhone doesn’t do mouse movement, and if you can’t move the mouse between the mousedown and mouseup events, the difference between mouseup and click becomes meaningless.

Mousedown, in theory, still has a valid function as a separate event that’s fired when the user touches the screen—regardless of what happens later. Unfortunately for theory, that’s not the way it works on the iPhone.

When you touch an element and leave your finger in place for a while, you get the option of de-focusing it (i.e. you’re allowed to state you don’t want to follow this link after all). In JavaScript event terms, I’d expected a mousedown event to fire, but not a click (I was uncertain of mouseup). Instead, no event fires.

Thus, the only way of firing mousedown and mouseup events is clicking on something. The three events refer to the same interaction; they have merged fully into one event with three names. (Of course, the reality of the Web requires all three names to remain supported forever.)

I’m not sure I’m happy with this situation. Although mouseup is pointless in the absence of mouse movement, I’m not so sure that the independent mousedown event should be abolished, too. Someone might be able to come up with interesting interactions that hinge on users pressing their fingers to the screen.

As to the double tap and right tap; the one is reserved for system functions (zoom) and the other doesn’t exist. There go the dblclick and contextmenu events, as well as the horribly mangled button property. Good riddance.

Conclusion

All in all, the iPhone offers a fascinating event model properly designed for a totally new interface, but grafted on the old mouse-and-keyboard event model we’ve been using for ten years now. We don’t yet have the faintest idea what we can do with it, but figuring it out promises to be a marvellous journey.

I have more things to say about the iPhone events, but they’ll have to wait for another time. None of the other topics I wanted to discuss are quite as intriguing as the mouse events, and I’m getting tired.

More iPhone news later.

This is the blog of Peter-Paul Koch, mobile platform strategist, consultant, and trainer. You can also follow him on Twitter.
Atom RSS

I’m speaking at the following conferences:

(Data from Lanyrd)

Categories:

Monthlies:

Comments

Comments are closed.

1 Posted by Alex Robinson on 1 August 2008 | Permalink

Interesting stuff, but I wonder how different the iPhone's handling of mouseout and mousemove is to how regular Safari (or indeed any other browser) handles input from a grahics tablet?

2 Posted by Sander Aarts on 1 August 2008 | Permalink

I don't really see how using a finger on a touch screen is very different from using a pen on a tablet. You can lift your pen from the tablet as well and put it somewhere else, making the cursor jump. I've never experienced any problems with that.

3 Posted by John McKerrell on 1 August 2008 | Permalink

Have you tried the touch and gesture events yet? I was playing with these yesterday, I found this blog post to be quite helpful: http://www.sitepen.com/blog/2008/07/10/touching-and-gesturing-on-the-iphone/

4 Posted by Steven on 15 August 2008 | Permalink

As other commenters have posted - how does this compare to how other "touch" devices work, such as tablets and PDAs?

I see hover events as a way to help users aim the cursor - It's a lot easier to tell exactly what object your mouse is pointing at when the object itself changes.

However, the iPhone leaves out the clumsy intermediary pointing device - instead of using your hand to control a mouse to move a cursor to touch an object, you just touch the object. So, really, its a net gain in usability.

5 Posted by Nathan on 26 August 2008 | Permalink

In regards to the first couple of commenters, graphics tablet users have the ability to make the cursor jump, yes, but you can also glide just like you would with a normal mouse. The pen actually touching the tablet is akin to a click (so drawing on the tablet is a click and drag) - but the cursors position is picked up when you hover the pen over the tablet (within about an inch distance from the surface).

6 Posted by Kyle Meyer on 26 August 2008 | Permalink

If you look at http://kittelson.com both in a normal browser and then an iPhone, you'll notice the drop down menus work in mobileSafari, simply by touching for slightly longer than a typical "click". The click event never fires, allowing you to drill through the submenus.

These menus are built entirely in CSS using :hover, and every element is a link.

7 Posted by Daveed V. on 26 August 2008 | Permalink

@Sander & Steven: Unlike (I think) the browsers on TabletPCs, MobileSafari uses (i.e., reserves) tap-and-slide for scrolling the view (there are no scrollbars). There are therefore no corresponding for capture by the browser's content.

8 Posted by Jason Short on 26 August 2008 | Permalink

I have always assumed this is a big real reason why Flash has not been available on Safari.

9 Posted by Kontra on 26 August 2008 | Permalink

@Jason: I've explored various UI ramification regarding that in:

The new UI wars: Why there’s no Flash on iPhone 2.0
http://counternotions.com/2008/06/17/flash-iphone/

10 Posted by Andy on 26 August 2008 | Permalink

Try using any web application that relies on Google Maps API, such as Wikimapia http://wikimapia.org and you'll quickly see why the iPhone hijacks any Safari request to maps.google.com and redirects you swiftly to the Maps iPhone app.

There is no way to zoom the map of any Google Maps API application, there is only pan (the zoom function of the iPhone is actually disrupted as well). Additionally, map overlay/marker click events are disrupted, rendering the application basically useless.

11 Posted by ppk on 26 August 2008 | Permalink

Hi guys,

Lots of useful stuff is coming in. Did a large site link to this article?

Anyway, please keep the comments flowing. They'll be really useful to me once I get back to this topic.

Thanks.

12 Posted by Sam Kington on 26 August 2008 | Permalink

Daring Fireball (http://daringfireball.net/linked/2008/08/26/mouse-events) linked here just now.

13 Posted by Ryan Cannon on 27 August 2008 | Permalink

Are the weirdness of legacy events due largely to the entirely new event model that Apple built? I haven't had time to experiment with them, but I imagine touchstart/end and gesturestart/end will fill the role you envision should be filled my mousedown/up.

14 Posted by P T Withington on 28 August 2008 | Permalink

I don't think this is exactly right, because I played with my phone for a bit last night and it definitely _does_ allow you to make a dragging gesture. If you tap (without dragging), that gives you mousedown/mouseup/click; but if you touch/hold/drag/remove, that gives you mousedown/mouseout...mouseup, and no click.

I also discovered an interesting thing about the gestures the OS intercepts: Dragging is intercepted to mean scrolling, but if you touch and hold before dragging it locks your scrolling to be just vertical or horizontal (whichever axis you start dragging in), sort of like holding the shift key when dragging with your mouse. I actually think it would have been more useful to be the opposite -- restrict drag to one axis _unless_ you hold first, since _usually_ you are using drag to scroll text up or down and it is annoying when you also scroll it a little to the side...