Essentia CHOP Suite for TouchDesigner
Real-time and offline audio analysis as native C++ CHOPs. Windows (x64) and macOS (Apple Silicon).
The Essentia CHOP Suite brings professional audio analysis into TouchDesigner as five native C++ operators, powered by Essentia, the open-source audio analysis library from the Music Technology Group at UPF. Spectrum and mel bands, MFCCs, pitch and key detection, onset and BPM tracking, and EBU R128 loudness, all running inside TD in both real-time (per-frame) and batch (full-file) modes.
v2.0 in one line: connect audio, get analysis. Every analyzer takes raw audio directly and runs its own FFT internally. There is no intermediate Spectrum CHOP to wire up and no per-network FFT bookkeeping to manage. This is a breaking change from v1.x, so if you are upgrading an existing network, read the migration notes in the repository.
Get it on GitHub: github.com/DarienBrito/EssentiaTD
Interactive guide (parameters, use cases, examples): darienbrito.github.io/EssentiaTD
Overview
The five operators
The four analyzers share a Mode switch between Realtime (per-frame) and Batch (full-file) analysis. Essentia Spectrum is an output operator that exposes the raw FFT.
- Essentia Spectral: MFCCs, spectral centroid, flux, rolloff, contrast, HFC, complexity, mel bands, and optional PCA. The workhorse for timbre and brightness.
- Essentia Tonal: pitch (YinFFT), HPCP chroma, key and scale, dissonance, and inharmonicity.
- Essentia Rhythm: onset detection, BPM estimation, and beat phase and confidence.
- Essentia Loudness: EBU R128 loudness, RMS energy, and zero-crossing rate.
- Essentia Spectrum: FFT magnitude and phase as a raw sample buffer, for visualization, GLSL, resynthesis, or custom feature work. The analyzers do not depend on it.
Realtime vs Batch
- Realtime (default): per-frame analysis at TD's cook rate. Audio accumulates in an internal ring buffer and each cook analyzes the latest window. Output is one sample per channel, ready to drive visuals live.
- Batch: full-file offline analysis on a background thread, triggered by a Compute pulse or an Autocompute toggle. Output is one sample per analysis frame across the whole file.
Flipping Mode never requires rewiring. Both modes read the same audio input.
What changed in v2.0, and why
In v1.x, the Spectral, Tonal, and Rhythm operators consumed a shared Essentia Spectrum CHOP in realtime. v2.0 removes that middle step: each operator now takes audio and runs its own FFT.
The reason is that a shared spectrum forces a single FFT size onto operators with conflicting needs. Tonal needs a large window (4096) to resolve adjacent semitones, while Rhythm needs a short window to keep onsets sharp. Forcing a 4096 spectrum onto Rhythm cut onset-detection F1 from 0.47 to 0.13 on a measured reference, because the longer window blunts the very transients onset detection relies on. Giving each operator its own FFT resolves the conflict, and the converted Rhythm reproduces the previous onset accuracy exactly at its own default window.
Good to know
- Tonal auto-picks its FFT size. Key and chroma need fine frequency resolution, so Tonal defaults to auto (4096 at 44.1/48 kHz, 8192 at higher rates). A too-coarse setting can report a confident but wrong key, so trust the default unless you have a reason not to.
- Keep Rhythm's default window. Realtime reads the latest window once per cook, so 1024 at 60 fps or higher is the sweet spot. The operator warns you if the window is too small for your frame rate, or too long for tight onset timing.
- Mono by design. The suite analyzes a single channel to keep outputs manageable (stereo would double every channel name). For stereo sources, sum to mono with a Math CHOP set to average before analysis.
- Batch BPM is sample-rate correct. It uses RhythmExtractor2013 with internal resampling to 44.1 kHz, so BPM is accurate at any input rate. This fixes a scaling bug that affected non-44.1 kHz files in v1.1.8.
- Essentia Spectrum is raw data, not a display. It outputs linear-bin magnitude and phase for GLSL, resynthesis, or custom analysis. For a perceptually scaled on-screen spectrum, TouchDesigner's built-in Audio Spectrum CHOP is the better choice.
Install
The simplest route is the installer for your platform, available from the Releases page:
- Windows: run EssentiaTD-Setup.exe. It installs to Documents\Derivative\Plugins\Essentia for the current user, no admin rights needed. Close TouchDesigner first.
- macOS: open EssentiaTD.pkg. It installs to ~/Library/Application Support/Derivative/TouchDesigner099/Plugins/Essentia, no admin password needed.
Restart TouchDesigner afterwards to load the operators. The plugins are unsigned, so on first run you may need to allow them past Windows SmartScreen (More info, then Run anyway) or macOS Gatekeeper (System Settings, Privacy & Security, Open Anyway). You can also copy the plugins manually from the zip, or build from source. Full instructions are in the repository README.
License and source
The Essentia CHOP Suite is open source under AGPL-3.0-or-later. Source code, full documentation, and the issue tracker are on GitHub: github.com/DarienBrito/EssentiaTD.
