See with your ears! The vOICe for Android maps live camera views to soundscapes, offering augmented reality for the totally blind through sensory substitution and synthetic vision. Also includes a talking color identifier, talking compass and a talking GPS locator.
Is it an augmented reality game or a serious tool? It can be both, depending on what you want it to be! The ultimate goal is to provide a form of vision to the blind, but sighted users can simply have fun playing the game of sight-without-eyesight. You can also help us by blogging about how *you* learn to see with sound.
How does it work? The vOICe uses pitch for height and loudness for brightness in one-second left to right scans of any view: a rising bright line sounds as a rising tone, a bright spot as a beep, a bright filled rectangle as a noise burst, a vertical grid as a rhythm. Best used with stereo headphones for the most immersive experience and most detailed auditory resolution.
Just experiment with simple visual patterns first, because real-life imagery is complex. Randomly drop a bright item such as a DUPLO brick on a dark table top, and learn to reach for it through sound alone (closing your eyes if you have eyesight). Next try and explore your own safe home environment, learning to associate the complex sound patterns with what you already know is there.
For serious users: learning to see with sound is like learning a foreign language or learning to play a musical instrument, really challenging your perseverance and brain plasticity. It may well be the ultimate brain training system, bridging the senses through artificial synesthesia.
Please report bugs to feedback@seeingwithsound.com, and visit the web page seeingwithsound.com/android.htm for detailed description and disclaimers.
Thank you!
Recent changes:
Support for Google Goggles added: tapping the right edge of the screen launches Google Goggles, if installed, or else visits the Android Market to let you install Google Goggles.
