Mais où ? is a live audiovisual performance developed in TouchDesigner, where visuals are not pre-rendered or triggered, but performed in real time alongside a pianist.
The system establishes a bidirectional relationship between sound, gesture, and image. Rather than mapping audio to visuals in a reactive way, the structure is built as a continuous negotiation: intensity, timing, and articulation from the piano influence a generative visual field that remains unstable and non-repeatable.
The piece premiered at the Video Mapping Festival (Lille), in the atrium of the Palais des Beaux-Arts, across a series of live performances.
Technical Overview
- Real-time generative system built in TouchDesigner (no pre-rendered sequences)
- Hand tracking using Torin Blankensmith’s MediaPipe–TouchDesigner integration→ captures detailed finger positions over the keyboard in real time
- Custom interpretation layer combining:
- finger position relative to piano keys
- temporal movement of the hands
- inferred intensity (from motion + audio)
- intentional use of non-tracking / loss of signal as part of the system
- Rather than mapping notes directly, the system reconstructs an interpretation of gesture and intensity:→ no MIDI, no discrete note triggering→ continuous, probabilistic reading of performance
- Visual generation driven by this interpretation:
- gesture → spatial influence
- intensity → deformation, scale, and energy of the system
- instability (tracking gaps, ambiguity) feeds into visual variation
- Designed for live performance constraints:
- variable lighting and occlusion
- non-deterministic input
- need for continuous, resilient output
The result is a system that does not “react” to notes, but translates the physical act of playing into a shifting visual field.
Artistic Approach
This project was initiated within a writing residency, where the conceptual framework was developed, followed by three months of prototyping leading into two nights of performances (five per evening).
It marks a shift from building interactive systems for audiences to performing the system itself.
The visual layer is treated as an instrument:not illustrating the music, but existing in tension with it.
The work explores:
- instability as structure
- loss of control within a constrained system
- co-authorship between performer and machine
Each performance produces a different visual trajectory, shaped by the pianist’s interpretation.
Credits
Visual system & direction: Ruby-Maude Rioux
Piano: Frédéric Volanti
Project page : https://www.rubymaude.com/interactive/mais-ou