This short video demonstrates the ability to control any parameter in Ableton Live using an Xbox Kinect, Synapse and Live using just body movement.
In this simple experiment the movement of my right hand on the Y axis is controlling the duration of a reverb and the X axis is controlling the dry/wet parameter of the effect. I ran a couple of loops through it to demonstrate the sound change in conjunction with my right hand movements.
Once Synapse is running, it is relatively easy using Max patches to link any limb or body movement(s) to any parameter, as in the above example this can be utilised to control effects parameters, but can also trigger audio loops, MIDI note and tuning data, along with virtually any other parameter that can be MIDI mapped. This can be utilised for instance, by moving a hand in one direction or the other to control a slider, although you can go into further granular detail and create patches for example which utilises two hands swiping out to start music playing, whilst swiping inwards will stop it.
The idea is to integrate the sound and image together using this technique and the previously discussed technique of controlling Quartz Composer parameters. This is a highly effective technique that allows both audio and visual manipulation in real-time with simple body movements.
There are a couple of key disadvantages to using Synapse that I have come across during the course of these recent experiments. 1) As far as I am aware it can only track one individual at a time 2) To enable tracking the user must first stand in front of the camera with their hands up until the camera recognises your body. These two issues may be problematic in an exhibition scenario as it will require some basic instruction and willing participants. This may add an additional layer of complication to the piece, removing the simplicity I am aiming for and also put people off from interacting with the piece.
- Cycling74, 2014: Soundflower Available at: [Accessed 7 May 2014]
- St. Jean, J. (2013). Kinect Hacks: Tips & Tools for Motion and Pattern Detection. Cambridge, O’Reilly Media.
- Synapse, 2014: Kinect and Ableton Live Available at: <http://synapsekinect.tumblr.com/post/6307739137/ableton-live> [Accessed 7 May 2014]