The vOICe for Android

The vOICe for Android


See with your ears! The vOICe for Android maps live camera views to soundscapes, offering augmented reality and unprecedented visual detail for the totally blind through sensory substitution and computer vision. Also includes a talking color identifier, talking compass, talking face detector and a talking GPS locator, while CamFind visual search and Google Goggles can be launched from The vOICe for Android by tapping the left or right screen edge.


Is it an augmented reality game or a serious tool? It can be both, depending on what you want it to be! The ultimate goal is to provide a form of synthetic vision to the blind, but sighted users can simply have fun playing the game of sight-without-eyesight. Visually impaired users with severe tunnel vision can try if the auditory feedback helps them notice changes in the visual periphery. You can help us by blogging and tweeting about your experiences, use cases, and how *you* learn to see with sound.

How does it work? The vOICe uses pitch for height and loudness for brightness in one-second left to right scans of any view: a rising bright line sounds as a rising tone, a bright spot as a beep, a bright filled rectangle as a noise burst, a vertical grid as a rhythm. Best used with stereo headphones for the most immersive experience and most detailed auditory resolution.

Just experiment with simple visual patterns first, because real-life imagery is complex. Randomly drop a bright item such as a DUPLO brick on a dark table top, and learn to reach for it through sound alone (closing your eyes if you have eyesight). Next try and explore your own safe home environment, learning to associate the complex sound patterns with what you already know is there. Sighted users can also use the app with Google Cardboard compatible devices through a swipe-down on the main screen to toggle the binocular view.

For serious users: learning to see with sound is like learning a foreign language or learning to play a musical instrument, really challenging your perseverance and brain plasticity. It may well be the ultimate brain training system, bridging the senses through artificial synesthesia. The vOICe for Android runs on Google Glass as a side-loaded app, using the tiny camera in these glasses to generate a live sonic augmented reality overlay, hands-free! You will want to use an external battery connected via USB cable to keep the small Google Glass battery from draining too quickly.

Why is it free? Because our foremost goal is to make a real change by lowering barriers to use as much as we can. You will find that competing technologies cost upwards of $15,000 and yet have lower specs. The perceptual resolution offered by The vOICe is unmatched even by $100,000 "bionic eye" retinal implants (PLoS ONE 7(3): e33136).

Please report bugs to [email protected], and visit the web page seeingwithsound.com/android.htm for detailed description and disclaimers. We are on Twitter at @seeingwithsound.

Thank you!

Recent changes:
v1.87: Experimental virtual reality mode for use with Google Cardboard and compatible devices (for sighted users); swipe down on main screen to toggle.

v1.86: Accessibility design change: mute toggle no longer mutes screen reader.

v1.85: USB mouse support added (e.g. for oDroid and wireless mini-keyboard with integrated trackpad), and accessibility fix for scrolling Options menu with TalkBack active.
Add to list
Free
69
3.5
User ratings
76561
Installs
100,000+
Concerns
1
File size
1624 kb
Screenshots
Video of The vOICe for Android Screenshot of The vOICe for Android Screenshot of The vOICe for Android Screenshot of The vOICe for Android Screenshot of The vOICe for Android Screenshot of The vOICe for Android Screenshot of The vOICe for Android Screenshot of The vOICe for Android
About The vOICe for Android
See with your ears! The vOICe for Android maps live camera views to soundscapes, offering augmented reality and unprecedented visual detail for the totally blind through sensory substitution and computer vision. Also includes a talking color identifier, talking compass, talking face detector and a talking GPS locator, while CamFind visual search and Google Goggles can be launched from The vOICe for Android by tapping the left or right screen edge.


Is it an augmented reality game or a serious tool? It can be both, depending on what you want it to be! The ultimate goal is to provide a form of synthetic vision to the blind, but sighted users can simply have fun playing the game of sight-without-eyesight. Visually impaired users with severe tunnel vision can try if the auditory feedback helps them notice changes in the visual periphery. You can help us by blogging and tweeting about your experiences, use cases, and how *you* learn to see with sound.

How does it work? The vOICe uses pitch for height and loudness for brightness in one-second left to right scans of any view: a rising bright line sounds as a rising tone, a bright spot as a beep, a bright filled rectangle as a noise burst, a vertical grid as a rhythm. Best used with stereo headphones for the most immersive experience and most detailed auditory resolution.

Just experiment with simple visual patterns first, because real-life imagery is complex. Randomly drop a bright item such as a DUPLO brick on a dark table top, and learn to reach for it through sound alone (closing your eyes if you have eyesight). Next try and explore your own safe home environment, learning to associate the complex sound patterns with what you already know is there. Sighted users can also use the app with Google Cardboard compatible devices through a swipe-down on the main screen to toggle the binocular view.

For serious users: learning to see with sound is like learning a foreign language or learning to play a musical instrument, really challenging your perseverance and brain plasticity. It may well be the ultimate brain training system, bridging the senses through artificial synesthesia. The vOICe for Android runs on Google Glass as a side-loaded app, using the tiny camera in these glasses to generate a live sonic augmented reality overlay, hands-free! You will want to use an external battery connected via USB cable to keep the small Google Glass battery from draining too quickly.

Why is it free? Because our foremost goal is to make a real change by lowering barriers to use as much as we can. You will find that competing technologies cost upwards of $15,000 and yet have lower specs. The perceptual resolution offered by The vOICe is unmatched even by $100,000 "bionic eye" retinal implants (PLoS ONE 7(3): e33136).

Please report bugs to [email protected], and visit the web page seeingwithsound.com/android.htm for detailed description and disclaimers. We are on Twitter at @seeingwithsound.

Thank you!

Recent changes:
v1.87: Experimental virtual reality mode for use with Google Cardboard and compatible devices (for sighted users); swipe down on main screen to toggle.

v1.86: Accessibility design change: mute toggle no longer mutes screen reader.

v1.85: USB mouse support added (e.g. for oDroid and wireless mini-keyboard with integrated trackpad), and accessibility fix for scrolling Options menu with TalkBack active.

User reviews of The vOICe for Android
This is truly one of the most amazing things I have seen/heard. This will vastly improve the quality of life for the sightless. Bless you, Devs!
May 12, 2011
Android Market Comments
A Google User
Mar 28, 2015
Use of sighted person This app is very nice, glad to have it for free Just wondering how sighted person will see using this app?
A Google User
Jan 24, 2015
Side by Side Support Excellent app, excellent developer communication. This is a must-have app!
A Google User
Jan 3, 2015
Side by Side Support A Side by Side display mode that would make this compatible with Google Cardboard/other HMDs would make this app pure gold. The benefit would be the ability to see the 2D video field in your entire field of view. Though not 3D, you could still see how the colors translate into sounds. You could actually walk around wearing this application for extended periods of time. Coming from a VR enthusiast, this would be an incredible feature. Similar to how Gear VR has a camera feed of 2D video as your whole FOV.
A Google User
Oct 22, 2014
Nice Haven't tried it but sounds cool
A Google User
Sep 12, 2014
Neat A cool concept that was fun to play with. I'm sighted so I don't really need it. It hooks the android Mediaserver and doesn't release it when the app is closed, so it sucks battery like crazy.
Version 1000000
Version 1000000