If you think about mobile sensor API, your first reaction might be that this is old news. This feature has been introduced to the world when iPhone shipped with accelerometers, further refined when next generation included gyroscope as well, there is a bunch of applications that are controlled by motion sensors both in AppStore or Android Market thus it must be easy to develop such apps, users are happy and everything is alright.
Except none of this is completely true. Accelerometers arrived earlier than iPhone, even with all the years of improvement their input is still unreliable, API usually deals with very raw data, there is no high-level library to detect common motion gestures (famous "shake" gesture being the only exception) or normalized, useful data and as a result the applications using sensors to control them are still limited in scope and hard to use. All of this has been mentioned in the video above. It is really perplexing to see that little effort happened to provide effective, easy-to-use solution ready for mobile developers. This explains the second part of the title. Hope this situation changes, and we'll see true motion control revolution.
Let's have a look at these points one by one :
1) Accelerometers are available long before iPhone ever existed
My first personal encounter with mobile phone with motion sensor arrived in 2004 in form of Nokia 3220 with Xpress-on Fun Shell add-on, where accelerometer was installed. You can see the 3220 phone here. Rather impractical, you had to buy quite expensive accessory, slide it on the device before it was usable...definitely not for the masses. However it was a great platform for experiments and I ported the famous "Homerun" game : it was a lots of fun when controlled by motion sensor. As Nokia pretty much abandoned the technology and never made it integral part of the mainstream phones (at least until iPhone meteor hit the Earth) we put the entire project on hold, but I have soft spot for sensor controlled applications ever since.
2) Sensors are typically dipped in thick layer of mobile marketing sauce
Again, mentioned in the video, it is fascinating to see how many different marketing names the sensors can get really confusing. What 6-axis sensor means really ? Well, we can probably live with one more example of marketing-newspeak, unfortunately it means that inclusion of some of the sensors is still the part of the marketing "shall we add this $1 chip to charge $50 more for this particular device" game. Sad. Motion sensors needs to be absolute standard across all mobile devices. Hope we are getting there, albeit slowly.
3) Raw data are unreliable and hard to work with
It is nice to have access to raw data if you need them, but it is not so nice that quite often this is the only data from sensors that are available, and they need quite a bit of processing before they are any useful. Even if you manage to filter out noise, some sensor data needs to be combined with data from the other sensor to offset errors, and this is exactly what Sensor Fusion is all about. Love that term and hope this will become an integral part of every major mobile OS API. Android platform seems to be heading in the opposite direction. The Orientation Sensor has been deprecated in Froyo, citing :
heavy processing that is involved, the accuracy and precision of the orientation sensor is diminished (specifically, this sensor is only reliable when the roll component is 0)as the reason for the move. Now you need to do a bit of computation on your behalf. There is arbitrary Rotation Matrix available and it is possible to get orientation relative to such a matrix. However, it is not specified, if the values from various sensors are corrected in any way...seems they are not. Anyone has any experience with this ? Seems it is still quite easy to get data, that are totally wrong, especially from the magnetometer.
On the iOS, the data from the sensors seems to be available in processed form, thanks to "Core Motion" framework, that can provide not only separate data from accelerometer, magnetometer and gyroscope, as well as processed data of both accelerometer and gyroscope fused together. There is CMDeviceMotion class that has CMAttitude property which holds information about device motion and attitude, available either in quaternion, rotation matrix or Euler angles form. Seems that iOS currently lacks any support for dedicated linear acceleration sensor, thus such information needs to be extracted from accelerometer and might be inaccurate.
4) Total lack of standardized motion gestures
Already mentioned that shake gesture is probably the only exception. The lack of other, high-level gestures that can be recognized by the phone OS and intercepted by the application developer in easy-to-use way, really limits what might be possible to achieve with sensor controlled applications. We need this to see new stream of applications that can be used even without direct eye interaction.
On iOS, only Shake gesture is currently recognized, an it is mapped to the UIEvent (same event type as other events such as multitouch events), which is rather strange. On Android platform, there is no support for any motion gesture recognition at all.
Edit: Added link to the Samsung proprietary kinetic gestures
Conclusion
Let's hope the set of sensors available on even most basic phones will only grow, and they will become a true standard, such as (multi)touch screen is one now. At the same time, let's hope that quality of the data provided by the sensors will improve, using the techniques mentioned in the video above.
We believe that the last point is the most pressing problem : the lack of set of motion gestures, that are understood by the users on one hand (i.e. the UX across applications and perhaps across platforms is consistent) and they are easy to recognize for the application developers (ideally by calling one line of code).
To facilitate this lengthy and tedious process, we would like to take an active approach. First, we looking for other people, both designers, developers or users, who see the importance of the common motion gesture library - please get in touch. Second, we would like to announce start of active development in this area...there will be a tool, that will help to explain, record and collect motion gestures in order to understand them better and eventually start a web site that will serve as a collection of such gestures. Watch this space for further announcements.
