Text input for future computing devices (SHARK shorthand and ATOMIK)
*This page is outdated. Click here for updated information.
Client computing is rapidly moving beyond the desktop computers. Smart phones, PDAs, and next generation tablet computers will play a more central role in people’s information life. To make these devices truly useful, the computer industry must find an effective way to let the user write text efficiently without a desktop keyboard.
There are many mobile text entry methods. Each has critical shortcomings. Inventing and developing an effective text entry method is a non-trivial user interface research challenge. It requires creativity, human performance insights, research on skill acquisition, and technology development.
Per-Ola
Kristensson, Barton Smith,
Shumin Zhai and other
researchers at the
SHARK Shorthand
|
|
Click here to download a full size video demo (30MB) |
SHARK (shorthand aided rapid keyboarding) takes advantage of the expressive jotting capabilities in pen-based computing. It enables the user to write text in “sokgraphs” ― a form of shorthand defined on a stylus keyboard as a graph.
A sokgraph is the trajectory pattern of a word defined on a stylus keyboard layout, preferably ATOMIK. Unlike traditional shorthand systems which require a large amount of training to begin using them, the user of SHARK Shorthand will first trace the letters of a word on the stylus keyboard. The trajectory of the trace is displayed to the user by transient digital ink and morphed to the ideal sokgraph. Each trial of tracing is also a trial of learning its sokgraph. Over time, the pattern of the sokgraph builds up in the user’s memory so the production of the trace becomes partly visually guided and partly memory recall driven. As the contribution of pattern recall, or open-loop action increases, the user’s dependence on visual guidance will decrease. Eventually a user may completely remember the sokgraph and gesture it based primarily on memory recall.
The stylus keyboard in SHARK works as a “training wheel” towards sokgraph gesturing. It is also a mnemonic device that helps the user remember the sokgraphs.
SHARK Shorthand uses novel pattern recognition technologies to match the user’s input to a large lexicon of words. Special algorithms, feedback mechanisms, and interface techniques are developed to support users’ gradual transition from visually guided tracing on keyboard to recall driven gesturing.
You can download a free trial version of SHARK Shorthand at IBM alphaWorks
See the following reviews and press report on SHARK Shorthand:
·
jkOntheRun
Top Ten Tech Blog
The published research papers on SHARK Shorthand to date are:
· Zhai, S., Kristensson, P-O., “Shorthand Writing on Stylus Keyboard” in Proc of CHI 2003 - ACM Conference on Human Factors in Computing Systems. pp 97-104.
· Zhai, S., Kristensson, P-O, Smith, B.A., In search of effective text interfaces for off the desktop computing, Interacting with Computers, Vol.16, in press.
· Kristensson, P-O., Zhai, S., SHARK2: A Large Vocabulary Shorthand Writing System for Pen-based Computers, Proc. ACM Symposium on User Interface Software and Technology (UIST 2004), Oct 24-27, Santa Fe, New Mexico, forthcoming.
The following is a paper is on a method called ESK (elastic stylus keyboard) that is half way between SHARK shorthand and ATOMIK:
· Kristensson, P-O. and Zhai, S. Relaxing Stylus Typing Precision by Geometric Pattern Matching. Proc. IUI 2005—ACM Conference on Intelligent User Interfaces, 2005, pp. 151–158, ACM Press.
See another
SHARK shorthand home page maintained by
my Ph.D. student and collaborator Per-Ola
Kristensson
ATOMIK
One method of mobile text input is the stylus virtual keyboards. Unfortunately the traditional QWERTY layout, invented by Christopher Sholes, Carlos Glidden, and Samuel W. Soule in 1868 for the purpose of minimizing mechanical jamming, is a poor choice for stylus keyboards. This is because the QWERTY layout was so arranged that many adjacent letter pairs (digraphs) appear on the opposite sides of the keyboard. Contrary to popular belief, this design in fact is good for two handed typing, because it also facilitates the frequent alternation of the left and right hand. When used as a stylus keyboard, however, it means the stylus has to travel back and forth more frequently and over longer distance than necessary.
ATOMIK (Alphabetically Tuned and Optimized Mobile Interface Keyboard) was designed to address this problem. It has the following three features.
First, ATOMIK has higher movement efficiency for stylus typing than any other existing stylus keyboards. This was achieved by a Metropolis optimization algorithm in which the keyboard was treated as a "molecule" and each key as an "atom". The "atomic" interactions among all of the keys drove the movement efficiency – defined by the summation of all Fitts' law movement times between every pair of keys, weighted by the statistical frequency of the corresponding pair of letters in English - towards the minimum.
Second, the layout was alphabetically tuned. There is a general tendency that letters from A to Z run from the upper left corner to the lower right corner of the keyboard. This gives novice users a cue to look for letters that are no yet memorized.
Third, we maximized, without sacrificing the first two features, the letter connectivity of the most common words. Many common words or comment fragments of words, such as "the" and "ing" are totally connected.
More detailed technical information, together with literature review on the topic, can be found in the following papers:
1. Zhai, S., Hunter, M., Smith, B.A., Performance Optimization of Virtual Keyboards, Human-Computer Interaction, Vol 17 (2&3). pp 229-269.
2. Smith, B.A., Zhai, S. Optimised Virtual Keyboards with and without Alphabetical Ordering - A Novice User Study, in Proc. of INTERACT 2001: Eight IFIP Conference On Human-Computer Interaction, Tokyo, Japan, July 9-13, 2001. pp 92-99. (this is a report of the experiment on the effect of alphabetical tuning)
3. Lee, P., & Zhai, S. (2004). Top-down learning strategies: can they facilitate stylus keyboard learning? International Journal of Human-Computer Studies, Vol. 60 (5&6) pp. 585-598.
4. Zhai, S., Smith. B. A. Alphabetically Biased Virtual Keyboards Are Easier to Use - Layout Does Matter, in Extended Abstracts of CHI 2001, ACM Conference on Human Factors in Computing Systems, Seattle, Washington, 31 March-5 April 2001 (Short Talk), p321-322. (this is an earlier, shorter, less complete version of 2.)
5.
Zhai, S., Hunter, M., Smith, B.A., The
Metropolis Keyboard -- An Exploration of Quantitative Techniques for
Virtual Keyboard Design, in the Proceeding of the 13th Annual ACM
Symposium on User Interface Software and Technology (UIST 2000), November 5-8,
2000, San Diego, California. pp 119-128 (this is an earlier, shorter, less
complete version of 1.)
Return to Shumin Zhai's home page