media collage creative
Screen Shot 2018-11-28 at 1.43.32 PM.png

Climber Synth [WIP]

As an artist, I love how sonic feedback changes my flow.

Playing music that has already notated is valuable and rewarding, but for me it can never compare to the satisfaction of exploring new possibilities on an instrument.

The concept for Climber Synth came out of a conversation with Sid Fels about how a rock climber wearing an EEG headset, might perform a piece for an audience below.

Until I can get my hands on some EEG gear and jump into the heavy data science work of making sense of brainwaves, I decided to take a go at capturing the body articulation of a climber on a tread-wall in order to design a real-time synth engine for immediate sonic feedback.

 My capture system consists of a Microsoft Kinect v2 and Max patch to record OSC data streaming from the Kinect.  For development purposes, climber skeleton articulation data was captured to a text file. This replay-able articulation data functions as a mock climber input source to facilitate synth engine development without a climber present.  Once the synth engine is developed, Climber Synth will be able to offer real-time sonic feedback to climbers performing on the tread wall or on  any  climbing route.   Next steps:   1. Develop synth engine based on skeleton articulation data   Pertinent questions for research:   1. How could sonic feedback facilitate performance on the climbing wall? What kind of performative movement will be inspired?  2. How could sonic feedback provide real-time “coaching” for a climber’s technical form?

My capture system consists of a Microsoft Kinect v2 and Max patch to record OSC data streaming from the Kinect.

For development purposes, climber skeleton articulation data was captured to a text file. This replay-able articulation data functions as a mock climber input source to facilitate synth engine development without a climber present.

Once the synth engine is developed, Climber Synth will be able to offer real-time sonic feedback to climbers performing on the tread wall or on any climbing route.

Next steps:

1. Develop synth engine based on skeleton articulation data

Pertinent questions for research:

1. How could sonic feedback facilitate performance on the climbing wall? What kind of performative movement will be inspired?

2. How could sonic feedback provide real-time “coaching” for a climber’s technical form?