While researching about recognizing gestures, I was reminded how much I love designing user interfaces. Designing user interfaces is a fertile area to uncover new ways to conceptualize ideas and create a more natural communication link between users and machines. The naïve approach of user-interface design focuses on when the context and communication method between the user and machine are ideal. Designing good user interfaces, especially ones that involve innovative new approaches, can involve taking care of a lot of (near) invisible details, such as how to handle ambiguous and invalid inputs and handle undesirable and non-ideal world conditions.

I think there is always room for improvement in any user interface, and much of that improvement is again, mostly invisible to the end user in the way that expert foley (sound effects) is invisible to a movie viewer, until the foley is either missing or implemented poorly. Unlike foley, which is static once the movie is completed, the near invisible improvements to a user interface should continue to undergo incremental improvements.

The challenge is that most people never notice the improvements, and that makes it hard to justify the design and maintenance resources to effect such improvements. For the increasing number of end products that support software updates, the need to continue to improve the inner and invisible workings of a user interface can spell the difference between sustainable success and losing out to a competitor.

Some examples to watch over the next year or two:

The iPhone supports using two fingers to pinch the display to zoom in and zoom out of an image, and the map function in the iPhone supports this capability. It also supports finger flicking, or dragging, that signals the system to scroll the image in the direction of finger motion. The way the software was implemented on the device I had, more than 30% of the time, when I pinched the map, the system would perform the zoom function and also interpret a finger flick when I lifted my fingers from the display, and this would throw or scroll the map in a seemingly random direction and distance away from the zoom point.

Implementing a temporal deadband between interpreting multiple commands from the same finger contact is one approach that might resolve this undesirable behavior. Adjusting the width of the deadband based on whether the user affects a correction back to the original zoom point after a flick, could be another refinement. In any case, many types of refinements would involve improvement in the system's inference engine.

Interestingly, Apple offered a different approach that avoided this problem—one that I found after reading the handbook that came with it. Double tapping with a single finger zooms in while double tapping with two fingers zooms out. While this solution is adequate, it requires the user to explicitly adjust to the context to avoid what additional software design could accommodate in a nearly invisible fashion.

I also experienced a subtler problem that could have benefited from improvements in the system's inference and error communication engine. I had the iPhone for a few days, and the cellular coverage where I was at was on the edge (pun intended) of non-existent. My daughter was especially interested in the IM (instant message) capability. The system would send an IM once in a while if you held it just the right way in exactly the right spot and when the moon was in the azimuth of the third house. My daughter got frustrated with the system after about an hour.

What I did not know was that there was an unsent IM left in the system, and I would periodically get these messages that would say, "The last transaction cost $0.00." This would happen even when I put the system into standby. I would look at the system and ask what transaction would that be, but it could not tell me and/or I did not know the proper way to ask. This totally drained the battery in less than a day, even though the system was in standby. I eventually found the unsent IM and removed it. The lesson learned here is, give the user more and possibly increasing information if you are going to keep unsuccessfully repeating the same task over and over. Again, the elegant ways to avoid this kind of problem can (should?) be near invisible to the end user.

Another example product involves my new television service. This version 1.0 product also has plenty of room for improvement. I recently commented on the remote control for that system. Recently, the company sent someone to my house to show me how to access and use the features of the new service. When I asked how to list only the channels I can view, the answer was, you could manually build a list of channels you want to see. I pointed out that the system knew better than me which channels I paid to see and wouldn't it be a tremendous value-add if there were an option to exclude those channels with a single option select? Once again, it is a near-invisible detail that would make this product better for a portion of the user population.

Another version 1.0 product example: I have heard people complain about the Wii remote: "It does not perform an accurate re-creation of swinging a bat or golf club." "The games are not very good." I think the underlying cause for these complaints is partially due to how well the basic interface maps with how people do the things they do, and this amplifies the disconnect between user expectations and designer experience working with the interface. I expect the interface to improve in the realm of near-invisible features with software updates, and these in turn will enable more realistic recreations of motions and better games that have never existed before on a computing system that cost significantly less than many tens of thousands of dollars (think aerospace, military, and medical).