{"id":6306,"date":"2016-03-18T03:00:50","date_gmt":"2016-03-18T03:00:50","guid":{"rendered":"http:\/\/www.kurzweilai.net\/?p=276132"},"modified":"2016-03-18T03:03:07","modified_gmt":"2016-03-18T03:03:07","slug":"youll-interact-with-smartphones-and-smartwatches-by-writinggesturing-on-any-surface-using-sonar-signals","status":"publish","type":"post","link":"https:\/\/hoo.central12.com\/fugic\/2016\/03\/18\/youll-interact-with-smartphones-and-smartwatches-by-writinggesturing-on-any-surface-using-sonar-signals\/","title":{"rendered":"You&rsquo;ll interact with smartphones and smartwatches by writing\/gesturing on any surface, using sonar signals"},"content":{"rendered":"<div id=\"attachment_276215\" class=\"wp-caption aligncenter\" style=\"width: 517px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;\"><a href=\"http:\/\/www.kurzweilai.net\/youll-interact-with-smartphones-and-smartwatches-by-writinggesturing-on-any-surface-using-sonar-signals\/fingerio_phone\" rel=\"attachment wp-att-276215\"><img class=\" wp-image-276215\" title=\"FingerIO_phone\" src=\"http:\/\/www.kurzweilai.net\/images\/FingerIO_phone.jpg\" alt=\"\" width=\"507\" height=\"368\" \/><\/a><p style=' padding: 0 4px 5px; margin: 0;'  class=\"wp-caption-text\">FingerIO lets you interact with mobile devices by writing or gesturing on any nearby surface, turning a smartphone or smartwatch into an active sonar device (credit: Dennis Wise, University of Washington)<\/p><\/div>\n<p>A new sonar technology called <a href=\"http:\/\/fingerio.cs.washington.edu\/\" >FingerIO<\/a> will make it easier to interact with screens on smartwatches and smartphones by simply writing or gesturing on any nearby surface. It&#8217;s is an active sonar system using the device\u2019s own microphones and speakers to track fine-grained finger movements (to within 8mm).<\/p>\n<p>Because sound waves travel through fabric and do not require line of sight, users can even interact with these devices (including writing text) inside a front pocket or a smartwatch hidden under a sweater sleeve.<\/p>\n<p><iframe frameborder=\"0\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/PpVUCEZvNjI?rel=0\" width=\"640\"><\/iframe><br \/>\n<em>University of Washington Computer Science &amp; Engineering | FingerIO<\/em><\/p>\n<p>Developed by University of Washington computer scientists and electrical engineers, FingerIO uses the device\u2019s own speaker to emit an inaudible ultrasonic wave. That signal bounces off the finger, and those \u201cechoes\u201d are recorded by the device\u2019s microphones and used to calculate the finger\u2019s location in space.<\/p>\n<p>Using sound waves to track finger motion offers several advantages over cameras &#8212; which don\u2019t work without line-of-sight or when the device is hidden by fabric or another obstructions &#8212; and other technologies like radar that require both custom sensor hardware and greater computing power, said senior author and UW assistant professor of computer science and engineering\u00a0<a href=\"https:\/\/homes.cs.washington.edu\/~gshyam\/\" >Shyam Gollakota<\/a>.<\/p>\n<p>But standard sonar echoes are weak and typically not accurate enough to track finger motion at high resolution. Errors of a few centimeters would make it impossible to differentiate between writing individual letters or subtle hand gestures.<\/p>\n<p>So the UW researchers used &#8220;<a href=\"https:\/\/en.wikipedia.org\/wiki\/Orthogonal_frequency-division_multiplexing\" >Orthogonal Frequency Division Multiplexing<\/a>&#8221; (used in cellular telecommunications and WiFi), allowing for tracking phase changes in the echoes and correcting for any errors in the finger location.<\/p>\n<div id=\"attachment_276335\" class=\"wp-caption aligncenter\" style=\"width: 650px;  border: 1px solid #dddddd; background-color: #f3f3f3; padding-top: 4px; margin: 10px; text-align:center; display: block; margin-right: auto; margin-left: auto;\"><img class=\" wp-image-276335\" title=\"fingerIO options\" src=\"http:\/\/www.kurzweilai.net\/images\/fingerIO-options.jpg\" alt=\"\" width=\"640\" height=\"133\" \/><p style=' padding: 0 4px 5px; margin: 0;'  class=\"wp-caption-text\">Applications of fingerIO. a) Transform any surface into a writing interface; b) provide a new interface for smartwatch form factor devices; c) enable gesture interaction with a phone in a pocket; d) work even when the watch is occluded. (credit: R. Nandakumar et al.)<\/p><\/div>\n<p>Two microphones are needed to track finger motion in two dimensions, and three for three dimensions. So this system may work (when available commercially*) with some smartphones (it was tested with a Samsung Galaxy S4), but today\u2019s smartwatches typically only have one microphone.<\/p>\n<p>Next steps for the research team include demonstrating how FingerIO can be used to track multiple fingers moving at the same time, and extending its tracking abilities into three dimensions by adding additional microphones to the devices.<\/p>\n<p>The research was funded by the National Science Foundation and Google and will be described in a\u00a0paper to be presented in May at the Association for Computing Machinery\u2019s\u00a0<a href=\"https:\/\/chi2016.acm.org\/wp\/\" >CHI 2016 conference<\/a>\u00a0in San Jose, California.<\/p>\n<p><em>* Hint: Microsoft Research principal researcher <a href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/desney\/\">Desney Tan<\/a> is a co-author.<\/em><\/p>\n<hr \/>\n<p><strong>editor&#8217;s comments:<\/strong> This tech will be great for students and journalists taking notes and for controlling music and videos. It could also help prevent robberies. How would you use it?<\/p>\n<hr \/>\n<p><strong>Abstract of\u00a0<em>FingerIO: Using Active Sonar for Fine-Grained Finger Tracking<\/em><\/strong><\/p>\n<p>We present fingerIO, a novel fine-grained finger tracking solution for around-device interaction. FingerIO does not require instrumenting the finger with sensors and works even in the presence of occlusions between the finger and the device. We achieve this by transforming the device into an active sonar system that transmits inaudible sound signals and tracks the echoes of the finger at its microphones. To achieve subcentimeter level tracking accuracies, we present an innovative approach that use a modulation technique commonly used in wireless communication called Orthogonal Frequency Division Multiplexing (OFDM). Our evaluation shows that fingerIO can achieve 2-D finger tracking with an average accuracy of 8 mm using the in-built microphones and speaker of a Samsung Galaxy S4. It also tracks subtle finger motion around the device, even when the phone is inside a pocket. Finally, we prototype a smart watch form-factor fingerIO device and show that it can extend the interaction space to a 0.5\u00d70.25 m2 region on either side of the device and work even when it is fully occluded from the finger.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A new sonar technology called FingerIO will make it easier to interact with screens on smartwatches and smartphones by simply writing or gesturing on any nearby surface. It&rsquo;s is an active sonar system using the device&rsquo;s own microphones and speakers to track fine-grained finger movements (to within 8mm). Because sound waves travel through fabric and [&#8230;]<\/p>\n","protected":false},"author":13,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[47,48,53,43],"tags":[],"class_list":["post-6306","post","type-post","status-publish","format-standard","hentry","category-computersinfotechui","category-electronics","category-internetcloudtelecom","category-news"],"_links":{"self":[{"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/posts\/6306"}],"collection":[{"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/comments?post=6306"}],"version-history":[{"count":1,"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/posts\/6306\/revisions"}],"predecessor-version":[{"id":6307,"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/posts\/6306\/revisions\/6307"}],"wp:attachment":[{"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/media?parent=6306"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/categories?post=6306"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hoo.central12.com\/fugic\/wp-json\/wp\/v2\/tags?post=6306"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}