Utilizing Immersive Visual Cues in Virtual Reality
Visual cues are an important tool in music therapy to help reinforce positive outcomes. By utilizing visual cues in synchronous ways that would be difficult in reality, music therapy augmented with virtual reality offers unique ways to cue the user visually that also supports simultaneous musical cues.
Visual cues offer a way to reinforce an event or another cue, like a musical cue. In music, we have many different kinds of visual cues. We use notation to cue what notes to play and how to play them. Conductors give visual cues to let musicians or singers know when to begin and stop, the tempo of a piece, or the dynamics.
Music therapists also use visual cues for very similar reasons. Need to get a large group of people to play and stop together? We give them a visual cue. Need to help an individual or group play a specific melody? Provide color-coded notes and point to them rhythmically. Visual cues are a staple of proper music therapy practice, but there are limitations.
For instance, one person can only do so much. There is a limit to the number of visual cues one person can give in one moment. The actual visual cues are also restricted to what is physically available (i.e. hand motions, facial expressions, hand-made materials, or similar visual objects. We are also typically predisposed to such visual cues through evolution and cultural norms.
Virtual reality allows for an additional set of visual cues that either would be unavailable otherwise or difficult to achieve simultaneously with music.
The prototype above, “Beat Machine”, provides an example of how visual cues can be used in a VR medium to support a therapeutic goal. Each grid of cubes changes color slightly according to the beat or note value of that grid. Faster notes are seen changing faster and vice versa for slower notes. This gives a visual cue in 3D space much like a music therapist pointing to notes rhythtmically. However, these cues run in perfect timing to the tempo and all at once.
When a person touches a cube, it turns ON, shown by visually lighting up the cube. When a “live” cube is triggered, it glows while the sound is also triggered. These visual cues let the person know when a cube may trigger, if a cube will trigger, and exactly when a cube triggers which also reinforces the sound played.
The end result is a visual effect that compliments the musical effect the person created. Even the positioning of the cubes, with lower cubes providing lower pitches and higher cubes providing higher pitches, reinforces the musical feedback of the melodies the person creates.
In the end, I want to utilize as much of the brain as possible during the experience in order to reinforce its effects. By utilizing visual cues in VR with music therapy, we are able to activate the musical, visual, spatial, and motor centers of the brain.