Feed
Index

SONIVIVI






Sonivivi is a mobile application that helps the partially blind, especially those who have recently lost their eye sight, to become more confident in their everyday life. It does this by improving their sense of hearing and by encouraging them to be self-sufficient.


Since it is intended for use by the visually impaired, Sonivivi has an interface that does not rely on sight. Using gestural input and sound as output, the application allows the partially blind to train their listening skills.

Sonivivi is a simple and playful training game that incorporates 3-D stereo sound to position sounds in space. It offers an simpler and less labour intensive alternative to more traditional methods such as learning to play an instrument. While it is primarily designed for use by the partially blind, Sonivivi can also be of use to the elderly or anyone who wants to train their hearing.





This project is work in progress. Daim is interested in universal design for the disabled or the elderly who could use the benefits of technology to enhance their lives.


From training game,
make them to enhance listening skills.

Sonivivi is a mobile application to improve a sense of hearing for the partially blind





Experience Prototyping:





Walkthrough:




DESIGN CHALLENGE:

How might we provide a new way of mobile system to enrich life by developing a different abilities of senses in a intuitive and subtle way?

_Concept : Training game on mobile application for the partially blind
_Target : Those who have recently lost their eye sight, and need to be more having self-confident for their everyday life.
_Quote :
“ We can find the way where I want to go through music, all sounds can be composed to music” ( Frietz, music teacher at IBOS, Denmark)
" I didn't want to get a training in a music class due to my depression a lot, but I am now musician. However still I am afraid of going out alone"(Iviza, completely blind, Denmark)
" If I can have confidence to go outside by myself without who helps me, I will be happy even more than now." ( Katcha, partially blind at IBOS, Denmark)
"If I imagine my future about losing eye sight, that is awful for finding a way, I am so scared, I just try to be more optimistic, I am still young and can see the world." ( Girvert, partially blind since 2002, South Korea)
_Problems : Many dangerous and unexpected situations from outside, it makes them to be afraid of journey / The existing device is not considering about their dignity in their real life. ( loud voice information, too many devices around them.boring, difficult to use..)
/ They don't want to be regarded as a completely blind, they need to get used to their given fate.
_Solution : Prompting development about a sense of hearing to navigate from indoor to outdoor with step by step. / Improving their self-confidence to adopt at the various listening skill.

_Impact :
The application has been presented to a partially blind girl and her teacher. She was able to play all the levels of the application fluently by herself after quick introduction and greatly enjoyed the experience. Even though she never used a touch screen device before, the assisted interface made it possible. That breaks the misconception that the partially blind can’t use a touch screen device because of it’s dependency on graphical user interface.
Sonivivi provides an opportunity for self-training so the partially blind can develop their listening skill without help of the teacher. As they complete next levels and their skills improve it gives them sense of achievement and boosts self-confidence.
_Value :
For Teacher : it’s a great extension of their tool-set. It establishes a common ground to compare results between students and helps better personalize their learning program. Because of it’s playfulness it motivates the student, which is especially important to engage those that have just lost their sight. It also offers affordable and more flexible alternative to existing single-purpose devices.

For the parents : it’s a chance to empathize with their partially-blind child and keep them practicing outside of the school hours.

Another potential group of users are elderly who start to lose their sight or people who would like to test their listening skills.

Tool or skill set : Tangible User Interface design, User research, Experience prototyping, Sound design, Graphic User Interface, Programming language.


HOW IT WORKS:

People can train with headset with a smartphone. _iOS Application
Written in Objective C language
Compiled and build using XCode on OSX
- Sound :
3DMixer object from the built-in Core Audio library.
It has more options than OpenAL that using simple panning to give you 3d effect.
3DMixer has more advance algorithms for spatiality like Head Related Transfer Functions (HRTF) that we are using.
- iOS built-in voice synthesis (normally used for Voice Over but I used it for custom text e.g.the game level, introduction of each levels of game, distance and direction)
Links,
About HRTF
http://interface.idav.ucdavis.edu/sound/tutorial/


CONTEXT:


As touch screen devices are becoming more and more popular because of their interface flexibility and gestural controls their usage among partially blind is still limited. Even though newest devices have very good assistive features like voice-over. In this project I wanted to explore potential of using existing technologies and platforms to facilitate new ways of learning.

What’s the existing state of the art (how they learn and practice nowadays)

The current education for the partially blind has a range of training exercises for navigation, a sense of direction or distance to be intended by teacher or trainer.
However, most of those methods don’t combine spontaneity and technology. There is a need for more quantifiable and systematic gauge to aid them develop their listening skill that takes aspects of human sense of hearing into account.


What if we produce the binaural sound effect for improving listening skill to navigation in a enjoyable way?

In this project 3D stereo sound has been used as a tool for enhancing the ability of hearing for two main purposes: (1) to train the partially blind hazy navigation of distance and direction, and (2) to give them direct and easy to understand audio feedback of their actions

Sonivivi is a mobile application that combines gestural interface and voice-over to make training simple but enjoyable. The training guides the user step by step through levels of increasing complexity creating an engaging experience.

Early prototyping of sound navigation:





Constraints:

Sonivivi was developed through iterative prototyping. I started with a simple application to understand how people recognize the distance and direction of the sound. It proved that more sophisticated method of generating 3d sound was needed and that led me to using binaural sound which greatly improved people’s perception of the space. During the process I involved partially blind, completely blind, their caretakers and teachers.

From this prototyping I found many potentials to develop this idea and tried more sound and navigation through use mobile device.
However, there has had many things to consider more:
_Many unexpected situations. Still scary to go outside.
_It Needs more realistic and clear sound to navigate.
_Different reaction depends on how I much I can hear and act upon that sounds.


Co-creation and brain storming with partially blind and teacher:

"Simple motivation to open their ears can eventually give big enjoyment to them"
(Frietz, Music teacher for the blind)


Earlier paper prototyping for interface of mobile application:


The second prototyping for interface:





Why mobile phone based on touch screen?


"Smart phone is an amazing invention for the blind, I don't have any devices any more"
( Grilvert, Partially blind, South Korea)
"One smart device is better than many different devices, they don't want to learn instructions for each device"
( Birgit, Technologist in the institute)
Sonivivi mobile needed to be as effortless as possible for the users to be able browse through non-graphical interface with voice information and drop into the the gesture experience with ease.
The real goal was to make the content not only more accessible but more immerse.
Through use the easiness of use of swipe gesture, the visual impaired can be a part of society and community but also they will learn how to use of gesture in this era of omnipresent touch-based technology.

At the beginning gestures were used only for moving sounds around the space. In the final application we used them also for navigation which was inspired by real world metaphors like flipping page of a book that the partially blind understand.If they can use smart phone also get application, there might be not use many devices around them in economic and that will have more opportunity to enhance their life.

Use existing technology, people can change perception of usage of device and behavior. I opened predictable potentials that the disabled can enjoy touch based screen, the focus on graphic user interface and I proved that we can see the potentials soon from learning by doing.

Graphical user interface, although not essential, evolved from simple graphics to polished but high-contrast design that was easily readable by the partially blind.

Voice-over was initially used only for instructions and distance feedback. In the final prototypes it has proven to be essential also for navigation within the application and encouraging users to explore possible options.


*You can see my stream of project, what is the inspiration, quotation, related project also my history of notes about final thesis.
the stream of blind


*This final thesis is designed by daim yoon at CIID.