Creating a Music System in VR
One of the most important aspects of music therapy is that it is reactive to the individual.
Live music is of the moment. It is an art form that exists at any given point in time and only in that point in time before it changes and becomes something else. This fluidity lends itself to change and can be impacted socially with others.
It's what makes music fun!
In music therapy, we stress the importance of engaging with an individual in the moment. Improvising with someone, reacting to their musical choices as they, in turn, react to yours can create a beautiful, completely musical communication of sound.
For individuals who may normally have difficulty expressing themselves verbally or are unable to speak entirely, this form of self-expression can prove invaluable for building of self-esteem, exercising creativity, or simply finding enjoyment.
As music therapists, we must be skilled in reacting musically in the moment with an individual. Likewise, any tools or technology we use must also be adaptable in the moment or it risks becoming a burden more than an asset.
Adaptability in VR
One of the first things I needed to establish during development for Music Therapy Augmented was the ability to adapt the music system within the program itself, live, during a session.
This required a system to translate the necessary musical changes (key, scale, tempo, octave transposition) to values and variables the program could understand and then either process within the application or send via MIDI to an external DAW.
I prefer the versatility and power of an external DAW and samples and decided to work within the MIDI system already established.
What I came up with as an initial prototype was a simple pentatonic chime instrument that could:
transposed to any key
moved up and down multiple octaves
and changed between major and minor pentatonic scales
Each chime was simply controlled via key press and assigned an initial MIDI value corresponding to a C Major Pentatonic scale. The note value itself was assigned as a variable that was then able to be manipulated through a quick change menu.
A more elegant solution would be to preset a full collection of scales that would act as basic ratios (Major Scale = 0,2,4,5,7,9,11,12). Then add an integer value based on desired key (+0 being C) and add/subtract 12 for octave. This would like start around middle C and be adjusted from there.
Given that in MIDI, notes are purely mathematical values, the manipulation of them in a musical way really just requires translating music therapy concepts into mathematical adjustments.
For the prototype, the values are adjusted through a menu, but these adjustments could be done using any input, including a keyboard or MIDI keyboard/controller. This would allow the music therapist to alter the variables of key, scale, octave live with the client or patient and could be done in a non-obtrusive and musical way.
The next step for this would be to flesh out the system with more scales and notes, while creating an in-depth menu system to adjust what inputs are accepted to alter the variables and to make the output compatible for any instrument blueprints being developed.
- Julian Brill, MT-BC