A project with Artlink Edinburgh, looking at using technology to improve the quality of life for differently able people. In this case we were working with Andy, who has very limited mobility, and little vision, but a really strong sense of music and rhythm. Working with Alison Stirling and Steve Hollingsworth we created a customised music player adapted to his needs.
Writeup[edit]
Since andy's main relationship with the world is auditory, we decided to work with sound, so see how we could enrich his life. This was difficult proposition, because Andy seems very happy with life - so long as he can listen to Radio Forth, everything is good!
We explored a few ideas - creating virtual instruments suited to his particular mobility, finding ways to bring the soundworlds of other places into his room - so he could hear the weather and birds outside, or the people in his mum's house. In the end, since sound is so important to Andy, it seemed like we shouldn't interfere with his perception of the world, but look at helping him take control of the sound and music he listens to.
Steve had been working with Andy, using a modified radio which allowed Andy to change radio stations, which Andy seemed to be reacting to well. Building on this, we decided that a customised MP3 player would provide Andy with access to all sorts of sounds, so long as we could design an interface which worked for him.
The concept behind the interface is that there are two actions to carry out while listening to music - asking for something different, and asking for something similar. So, these actions are mapped to two large buttons which Andy can control. The first button selects between different genres in Andy's music collection and the radio. The second button either skips the track which is currently playing, or scans for the next radio station, depending on which is happening at the moment. Audible menus can be used to give Andy feedback about what is happening when he presses buttons, if that is found to help.
Another part of the project is the idea of an album of memories - in the same way that a sighted person would have photos of objects which they relate to, we can build a set of sound file of things that Andy remembers - his mother reading stories and playing the cello, musical boxes that were given to him etc. By having this as one of the "genres" in his music player, he can browse through old memories when he feels like it.
On the technical side, the operation of the player has been prototyped in Max/MSP, using a modified USB dancemat to function as a controller; this allows us to try various modes of operation with Andy, to see what he responds to. The final build will use a standard MP3 player, with modified firmware, based on Rockbox (www.rockbox.org), which will be made available for other similar projects. The MP3 player will work as normal, so that anyone can load new music onto it using a computer, but it will plug into a special box attached to Andy's chair with the custom buttons on.
If anyone wants the modified rockbox files, or advice about modifying mp3 players, email me.