Mediapipe Finger Pinch Dial Controller For Beginners & Intermediate

A hand tracking "dial" that maps the angle between your thumb and index finger to a -1 to +1 CHOP output, with a visible colored line overlay on your webcam feed. Both hands work independently. Useful for controlling any parameter in real-time with just your fingers.

This tutorial uses the MediaPipe TouchDesigner plugin by Torin Blankensmith and Dom Scott. Grab the latest release from their GitHub. Huge thanks to Torin for building and maintaining this plugin — it's the backbone of hand tracking in TD.

Hold a pinch between your thumb and index finger. When the line between them points straight up (index above thumb), the output is 0. Rotate your index finger clockwise and the output ramps to +1. Counter-clockwise goes to -1. There's a 30° dead zone around vertical so small movements don't trigger anything. The line overlay changes color — white at center, green at +1, red at -1. Both hands output independently.

You need MediaPipe set up in your project. Drop mediapipe.tox and hand_tracking.tox into your project from the plugin download. Set your webcam in the MediaPipe component and turn off all models except hand tracking to save processing. In the Common tab, enable "Enable External TOXs" and point to your local toxs/ folder so the project file doesn't balloon in size.

Start with a videodeviceinTOP (cam1) pointed at your webcam. Add a flipTOP (flip1) connected to it with flipx enabled — this mirrors the webcam so your hands move in the direction you expect. This flipped output becomes your base video feed. Drop a nullTOP (rawvideoinput) after the flip so you have a clean reference point.

Your hand_tracking2 component (the hand tracking tox) should already be in the project. It outputs two important CHOPs: normalized_data which has all 21 landmark positions per hand in normalized 0-1 coordinates, and helpers which has computed values like hand_active flags. We'll reference these throughout.

Add a selectCHOP (normalized) that pulls from hand_tracking2/normalized_data and a selectCHOP (helpers) that pulls from hand_tracking2/helpers. These give you clean top-level references to the tracking data without reaching into the component every time.

You need the thumb tip and index finger tip positions for each hand, plus the hand_active flags.
Create three selectCHOP nodes:

1. sel_h1_tips — select channels: h1:thumb_tip:x h1:thumb_tip:y h1:index_finger_tip:x h1:index_finger_tip:y from the normalized CHOP.
2. sel_h2_tips — same pattern but with h2: prefix for the second hand.
3. sel_active — select channels: h1:hand_active h2:hand_active from the helpers CHOP.

Merge all three into a single mergeCHOP (merge_tips). This gives you one CHOP with all the raw data the dial calculation needs — 10 channels total.

Create a scriptCHOP (dial_calc) and wire merge_tips into its input. Create a textDAT (dial_calc_callbacks1) and point the scriptCHOP's callbacks parameter to it.

def onCook(scriptOp):
    scriptOp.clear()

    inp = scriptOp.inputs[0]
    if inp is None or inp.numChans == 0:
        scriptOp.appendChan('h1_dial')[0] = 0
        scriptOp.appendChan('h2_dial')[0] = 0
        scriptOp.appendChan('h1_angle_deg')[0] = 0
        scriptOp.appendChan('h2_angle_deg')[0] = 0
        scriptOp.appendChan('h1_distance')[0] = 0
        scriptOp.appendChan('h2_distance')[0] = 0
        return

    DEAD_ZONE = 30.0
    MAX_ANGLE = 90.0
    RAMP_RANGE = MAX_ANGLE - DEAD_ZONE

    for hand in ['h1', 'h2']:
        active_name = f'{hand}:hand_active'
        active = 0
        try:
            active = inp[active_name].eval()
        except:
            active = 0

        if active = MAX_ANGLE:
            dial = 1.0 if angle_deg > 0 else -1.0
        else:
            t = (abs_angle - DEAD_ZONE) / RAMP_RANGE
            dial = t if angle_deg > 0 else -t

        distance = math.sqrt(dx*dx + dy*dy)

        scriptOp.appendChan(f'{hand}_dial')[0] = dial
        scriptOp.appendChan(f'{hand}_angle_deg')[0] = angle_deg
        scriptOp.appendChan(f'{hand}_distance')[0] = distance

The angle math uses atan2(dx, dy) — note the argument order. This gives 0° when the index finger is directly above the thumb (the "12 o'clock" position). Positive angles go clockwise (index moves right), negative goes counter-clockwise. The dead zone kills output within ±30° of vertical so you can hold a natural pinch without triggering anything. From 30° to 90° the value ramps linearly from 0 to ±1. Beyond 90° it clamps at ±1.

The distance channel is the Euclidean distance between thumb and index tip in normalized coordinates. It's useful for controlling things like effect intensity based on how far apart your fingers are.

Add a lagCHOP (lag_dial) after dial_calc with a small lag value (around 0.05-0.1 seconds on both lag and overshoot) to smooth out jitter. Then a nullCHOP (null_dial) at the end as your clean output reference. The null outputs 6 channels: h1_dial, h2_dial, h1_angle_deg, h2_angle_deg, h1_distance, h2_distance.

This is where it gets tricky. You need to draw a visible line between the thumb and index finger on top of the webcam feed. The approach is to use SOPs rendered with an orthographic camera, composited over the video.

The Line SOPs

Create two lineSOP nodes — line_h1 for hand 1 and line_h2 for hand 2. Each line has two endpoints (point A and point B) driven by expressions.

For line_h1:

• pax (point A x): op('normalized')['h1:thumb_tip:x']
• pay (point A y): op('normalized')['h1:thumb_tip:y'] * (720.0/1280.0)
• pbx (point B x): op('normalized')['h1:index_finger_tip:x']
• pby (point B y): op('normalized')['h1:index_finger_tip:y'] * (720.0/1280.0)

For line_h2, same pattern with h2: prefix channels.

The * (720.0/1280.0) on the Y coordinates is aspect ratio correction. The ortho camera will have a width of 1.0, which maps directly to the 0-1 X range. But the Y range needs to be squished by the aspect ratio to avoid stretching.

Critical coordinate mapping note: Use the raw X and raw Y values from MediaPipe. Do not invert Y (no 1-y). Do not invert X (no 1-x). MediaPipe's hand_tracking2 component outputs normalized data that already matches the flipped webcam's coordinate space. Y=0 is at the bottom in both systems. This took several iterations to get right — trust the raw values.

Thickening the Lines

A raw lineSOP is infinitely thin and won't render visibly. Pipe each line through a wireframeSOP (thick_h1, thick_h2) to give them thickness. Set the radius to something small like 0.002 for a clean thin line.

Geometry Containers

Each thickened line needs a geometryCOMP to hold it. Create geo_h1 and geo_h2. Inside each geo, drop an objectmergeSOP and set its merge0sop parameter to the path of the corresponding thick line (e.g., /project1/thick_h1). The objectmerge pulls the external SOP geometry into the geo container for rendering. Make sure the objectmerge has display and render flags enabled.

Materials — Color from Dial Value

Create two constantMAT nodes — mat_h1 and mat_h2. These drive the line color based on the dial output. Set the color channel expressions:

For mat_h1:

1. Red: 1.0 - max(0, op('null_dial')['h1_dial'])
2. Green: 1.0 + min(0, op('null_dial')['h1_dial'])
3. Blue: 1.0 - abs(op('null_dial')['h1_dial'])

Same pattern for mat_h2 using h2_dial.

This gives you white at 0 (R=1, G=1, B=1), pure red at -1 (R=1, G=0, B=0), and pure green at +1 (R=0, G=1, B=0). The transitions are smooth and linear across the dial range.

Assign mat_h1 as the material on geo_h1, and mat_h2 on geo_h2.

Create a cameraCOMP (cam_line) and set it to orthographic projection (projection = ortho). Set orthowidth to 1. Position it at tx = 0.5, ty = 0.28125, tz = 1. The tx of 0.5 centers the camera on the 0-1 X range. The ty of 0.28125 comes from (720/1280) / 2 = 0.5625 / 2 — this centers the aspect-corrected Y range in the camera's view.

Create a renderTOP (render_line) at 1280x720 (match your webcam resolution). Set the background to fully transparent — bgcolorr = 0, bgcolorg = 0, bgcolorb = 0, bgcolora = 0. Point the camera parameter at cam_line. Set geometry to * so it picks up both geo containers. You may need a lightCOMP in the scene too — a default one with ambient light works fine since you're using constantMAT which doesn't need lighting, but TD sometimes wants one present.

Enable transparency = sortedblending on the render so the transparent background works properly.

Create an overTOP (over_line). Wire render_line into input 0 and rawvideoinput (your flipped webcam) into input 1. The over operator composites the transparent-background line render on top of the webcam feed. The lines should now appear directly on the thumb and index finger tips.

Finish with a nullTOP (null_out) after the over for a clean final output.

Do NOT add a flipTOP after the render. The line positions already match the flipped webcam space because MediaPipe outputs coordinates in that same space. Adding another flip would reverse everything.

Always add a performCHOP (fps_monitor) at the end of your network. It gives you real-time FPS, cook time, and GPU stats. Hand tracking is computationally expensive and you want to know immediately if your network is dropping frames. Keep an eye on it as you build.

Network Layout

Organize everything left to right with adequate spacing between operators. The signal flow should be clear at a glance:

Input stage (left): cam1 → flip1 → rawvideoinput

Tracking stage: hand_tracking2 → normalized / helpers

CHOP pipeline: sel_h1_tips + sel_h2_tips + sel_active → merge_tips → dial_calc → lag_dial → null_dial

SOP pipeline (parallel): line_h1 → thick_h1 → geo_h1 (with objectmerge inside), same for h2

Materials: mat_h1, mat_h2 (positioned near their respective geos)

Render stage (right): cam_line + light1 + geos → render_line → over_line (+ rawvideoinput) → null_out

Monitoring: fps_monitor at the far right

Tuning and Tips

The dead zone of 30° works well for most people but you can adjust DEAD_ZONE in the script if you want more or less sensitivity. Same for MAX_ANGLE — 90° means you hit full deflection at a right angle, which feels natural. Setting it to 60° would make the dial more sensitive.

The lag CHOP values control smoothing. Lower values (0.02) give snappier response but more jitter. Higher values (0.15) smooth things out but add latency. Start at 0.05 and adjust to taste.

If your lines don't align with your fingers, double check that you're not inverting any coordinates. The most common mistake is adding (1-y) or (1-x) transforms. The MediaPipe hand tracking component already outputs data in the same coordinate space as the flipped webcam — use the raw values.

The distance channel (h1_distance, h2_distance) is in normalized 0-1 space. A typical pinch distance when fingers are touching is around 0.02-0.05. Fingers spread wide might be 0.15-0.25 depending on how close you are to the camera. Use a mathCHOP or expressions to remap this range to whatever you need.

Using the Dial Output

The null_dial CHOP is your interface to the rest of your network. Reference it from anywhere with expressions like op('null_dial')['h1_dial'] to drive parameters on other operators. Some ideas: control hue shift on a hsvadjustTOP, drive the speed of a noiseTOP animation, map it to audio volume, use it as a crossfader between two video sources with a switchTOP, or wire it to the index of a switchTOP to flip between effects.

The distance channel works great as an intensity control — pinch tight for subtle, spread fingers for full effect. Combine dial angle for effect selection with distance for effect strength and you get a surprisingly expressive two-parameter controller from a single pinch gesture.