A Hidden Feature of Google Handwriting

Google Handwriting recognition lets you write search queries on your mobile phone or tablet without using the keyboard. It can recognize shapes as well.

Google recently added handwriting recognition capabilities to their web search interface thus giving users an option to scribble search queries without opening the keyboard. Once you turn on the Handwriting mode, the entire Google page turns into a scratch pad – you can write anywhere on the screen and Google will instantly convert your freehand drawing into digital text.

The results are accurate and though the conversion happens on Google’s servers, you won’t notice the delay. Google suggests using block letters but cursive writing works as well.

You can use Google’s handwriting search on most touch-screen mobile phones and tablets but if you are on the desktop, you may change the user agent of your browser to iPad (or Android) and Google will then offer the Handwriting option on your desktop as well.

How Google Handwriting works?

When you draw a shape on the Google page, the path is captured in an array of X, Y points which is then sent to Google’s IME (Input Method Editor) API as a POST Request. It returns a list of possible suggestions (for instance, if you draw a round shape, IME may interpret that as “o, 0, O” while a straight line can be interpreted as “l, I, i, 1″) and one of them is added to the Google search box.

The Undocumented Feature

The JavaScript file associated with Google’s Handwriting feature reveal a hidden feature – other than letters and digits, you can also draw shapes and Google can convert some of them into words. For instance, if you draw a “heart” shape on the search page, Google will render that as the word “heart.”

Watch this YouTube video for a quick demo.

1 2 3 4 5 6 7 8

a = a.replace(/\u2661/g, "heart"), a = a.replace(/\u263A/g, "smiley face"), a = a.replace(/\u2639/g, "frowny face"), a = a.replace(/\u00A9/g, "copyright"), a = a.replace(/\u00AE/g, "registered trademark"), a = a.replace(/\u221E/g, "infinity"), a = a.replace(/\u2713/g, "check mark"), a = a.replace(/\uD834\uDD1E\u0020/g, "g clef"));

When you draw a heart shape, Google’s IME returns the Unicode equivalent of the heart character and Google, at the client side, converts that symbol into a word.

Google Handwriting Recognition includes similar logic for emoticons, check marks, copyright and trademark symbols but they aren’t working at this time. It does however offer a strong hint that Google Handwriting could be coming to other Google products – like Gmail and chat – because one rarely write smileys and frowning  faces in the search box.