BasicPitch2MIDI for TouchDesigner For Beginners & Intermediate

Built a single TOX that wraps Spotify's Basic Pitch polyphonic-transcription model and exposes it as proper TD-native I/O:

• One CHOP input — any audio source
• outActive CHOP: 128 channels (ch1n0..ch1n127), one per MIDI pitch with current velocity. Wire to instancing, GLSL, DMX, midiout — anything.
• outAudio CHOP: built-in 8-tone polyphonic synth (sine / triangle / saw / square / pluck / bell / organ / soft pad) re-rendering the detected MIDI
• Live piano-roll visualization rendered inside the TOX preview itself, synced to the source playback position
• One-click pip-installer for the model + deps (Mac CoreML / Win ONNX)
• ~2.3 s latency in Stream mode (model needs 2 s of audio context — can't go below that floor); File mode is sample-locked

Threshold tuning was the biggest lesson. Defaults are tuned for slow piano; for dense music they throw away 80% of the notes. Camel by Camel went from 344 detected notes → 2,893 just by dropping Onset/Frame thresholds and min-note-length. Per-source tuning is the entire game.

Apache 2.0. Mac + Win. Patreon link below.

Use case ideas (for caption/post variations)

Visuals

1. Audio-reactive instancing — pipe the 128-channel velocity CHOP into a geometry COMP's instancing translate-Y to make 128 cubes dance to the actual NOTES, not just amplitude bands
2. Pitch-driven particles — route active note pitches to particle emitter positions so each instrument lives in a spatial region
3. GLSL piano roll shaders — feed the pre-rendered piano roll PNG to your own shaders for stylized music-vis backgrounds
4. Real-time score display — show a scrolling music staff that follows what's actually playing, not just a generic spectrum
5. DMX lighting from any audio — send MIDI to a lighting cue engine; cymbal hits trigger strobes, bass triggers floor pads

Music production

1. Cover any song as MIDI — drop in a Spotify rip, get a MIDI file you can re-orchestrate in Ableton
2. Re-perform live audio — sing or play guitar, get a MIDI feed your synth can sustain forever
3. Loop sampling — auto-transcribe drum-machine sample packs to MIDI for sequencing
4. Mashup tooling — detect chord changes in any track and align visuals/automation to the bar boundaries
5. Pitch-correct vocals visually — see exactly which pitch the singer hit vs the intended note

Live performance

1. Solo musician → A/V show — your guitar runs the visuals AND the synth pad accompaniment
2. DJ visual rig — extract melodic content from the track you're playing and let it drive the geometry
3. Generative jam — feed your own playing back into a generative system that responds to the melody, not just the beat
4. Streamer reactive overlay — game music or background tracks drive an overlay that moves with the music

Education / analysis

1. Music theory teaching — load any recording, see the chord progression as a piano roll, students follow along
2. Vocal coaching — visualize a student's pitch in real time vs the target
3. Transcription assistance — first-pass MIDI for songs that don't have published charts

Research / generative

1. Train-data prep — bulk-transcribe a music library to MIDI for ML training
2. Audio-aligned generative video — Stable-Diffusion / Sora prompts tied to detected note events for temporally-aligned generation
3. Spatial audio mapping — pitch → 3D position, generative environments that respond to harmonic content