Kinect CHOP

From Derivative
Jump to navigation Jump to search

Summary
[edit]

NOTE

OS: This operator is only supported under the Microsoft Windows operating system.
Hardware: This CHOP only supports the Kinect for Windows hardware and the Kinect 2, not Kinect for Xbox hardware.

The Kinect CHOP reads positional and skeletal tracking data from the Kinect and Kinect2 sensors. Up to 6 people's full skeletons can be tracked (2 on the original Kinect), and the center position of an addition 4 people in the camera view is tracked as well. Multiple Kinect devices can be used by using multiple Kinect CHOPs and changing the Camera parameter.

It also supports face tracking.

To use a Kinect 2 device you need to install the Kinect 2 SDK or runtime from here.

To use the original Kinect 1 you will need to install the Kinect Runtime 1.8 for builds greater than 12000, or Kinect Runtime 1.7 for builds below 12000. Multiple Kinects: The Kinect 2 SDK does not allow for multiple kinects to be used on one system at a time. There is no way to connect to different devices through the SDK. The Kinect 1 can do this, but not the 2.

See also Kinect TOP, Kinect and Kinect1.

PythonIcon.pngkinectCHOP_Class


Parameters - Kinect Page

Active active - When 'On' data is captured from the Kinect sensor.

Hardware Version hwversion - - Choose between Kinect v1 or Kinect v2 sensors.

  • Version 1 version1 -
  • Version 2 version2 -

Sensor sensor - Selects which Kinect sensor to use. Only available when using Kinect v1.

Skeleton skeleton - - Selects options for skeletal tracking.

  • Full full - Track full skeleton.
  • Seated seated - Track seated skeleton.
  • Off off - Do not track skeleton.

Max Players maxplayers - Limits how many players to track.

Interactions interactions - Enables interactions, which returns interaction data such as "grip" and "press". The additional channels output are prefixed with p[1-2]/hand_l_* and p[1-2]/hand_r_*.<

  • For the x and y values of the interaction hand, 0.0 corresponds to left edge of interaction region and 1.0 corresponds to right edge of interaction region, but values could be outside of this range.
  • For the z value of the interaction hand, 0.0 means that hand is close to shoulder and 1.0 represents a fully extended arm, but values could be outside of this range.

NOTE: Interaction mode requires a depth resolution of 640x480. As such, a Kinect TOP displaying a depth image for the same sensor will output an image with a resolution of 640x480 and the Near Depth Mode setting will be set in the Kinect CHOP.

For guidelines in creating applications with Kinect Interaction, download the Kinect for Windows Human Interface Guidelines.

Relative Bone Rotations relbonerotations - Returns rx, ry, and rz relative rotation channels for each bone. The rotation is relative to the previous joint.

Absolute Bone Rotations absbonerotations - Returns rx, ry, and rz absolute rotation channels for each bone.

Bone Lengths bonelengths - Returns the length for each bone.

Unroll Bone Values unrollbones - Return the rx, ry, and rz values in a form that includes no discontinuities (or wrap arounds), so that they can be safely blended, filtered, lagged etc. For example, a rotation value running from 0 to 360 continuously, unrolled would start at 0 and increase in value indefinitely.

World Space Positions worldspace - Returns the tracked positions in world space coordinates. For each tracked point, a tx, ty, tz triplet of channels will be output.

Color Space Positions colorspace - Returns the tracked positions in image space coordinates relative to the color camera. For each tracked point, a u, v, tz triplet of channels will be output. Note this is only available on Kinectv2.

Depth Space Positions depthspace - Returns the tracked positions in image space coordinates relative to the depth camera. For each tracked point, a depthu, depthv, tz triplet of channels will be output. Works on both Kinectv1 and Kinectv2.

Face Tracking facetracking - Returns face tracking channels for detected faces.

Status Channels statuschans - A number of additional status channels are reported when this is turned on. A group of channels will report if a joint is currently being tracked, and another group of channels reports if part of the player is clipped outside the bounds of the Kinect sensor. The additional channels output are suffixed with tracked or clipped.

Near Depth Mode neardepthmode - Enables near mode for skeleton tracking, which allows the depth camera to see objects as close as 40cm to the camera (instead of the default 80cm).

Flip Skeleton U Direction flipskelu - Flips the u-axis for skeleton channels, helpful when using a mirror image from the camera.

Flip Face U Direction flipfaceu - Flips the u-axis for face channels, helpful when using a mirror image from the camera.


Parameters - Smoothing Page

Joint Smoothing jointsmoothing - Activates Kinect's smoothing algorithm for joint tracking and enables the smoothing parameters below.

Smoothing smoothing - Increasing the smoothing parameter value leads to more highly-smoothed skeleton position values being returned. It is the nature of smoothing that, as the smoothing value is increased, responsiveness to the raw data decreases. Thus, increased smoothing leads to increased latency in the returned skeleton values. Values must be in the range 0 through 1.0. Passing 0 causes the raw data to be returned.

Correction correction - Lower values are slower to correct towards the raw data and appear smoother, while higher values will correct toward the raw data more quickly. Values must be in the range 0 through 1.0.

Prediction prediction - The number of frames to predict into the future. Values must be greater than or equal to zero. Values greater than 0.5 will likely lead to overshooting when moving quickly. This effect can be damped by using small values of Max Deviation Radius.

Jitter Radius jitterrad - The radius in meters for jitter reduction. Any jitter beyond this radius is clamped to the radius.

Max Deviation Radius maxdevrad - The maximum radius in meters that filtered positions are allowed to deviate from raw data. Filtered values that would be more than this radius from the raw data are clamped at this distance, in the direction of the filtered value.

Rotation Smoothing rotationsmoothing - Activates Kinect's smoothing algorithm for rotations. As with joint smoothing above, higher levels of smoothing will introduce more latency.


Parameters - Common Page

Time Slice timeslice - Turning this on forces the channels to be "Time Sliced". A Time Slice is the time between the last cook frame and the current cook frame.

Scope scope - To determine which channels get affected, some CHOPs use a Scope string on the Common page.

Sample Rate Match srselect - - Handle cases where multiple input CHOPs' sample rates are different. When Resampling occurs, the curves are interpolated according to the Interpolation Method Option, or "Linear" if the Interpolate Options are not available.

  • Resample At First Input's Rate first - Use rate of first input to resample others.
  • Resample At Maximum Rate max - Resample to the highest sample rate.
  • Resample At Minimum Rate min - Resample to the lowest sample rate.
  • Error If Rates Differ err - Doesn't accept conflicting sample rates.

Export Method exportmethod - - This will determine how to connect the CHOP channel to the parameter. Refer to the Export article for more information.

  • DAT Table by Index datindex - Uses the docked DAT table and references the channel via the index of the channel in the CHOP.
  • DAT Table by Name datname - Uses the docked DAT table and references the channel via the name of the channel in the CHOP.
  • Channel Name is Path:Parameter autoname - The channel is the full destination of where to export to, such has geo1/transform1:tx.

Export Root autoexportroot - This path points to the root node where all of the paths that exporting by Channel Name is Path:Parameter are relative to.

Export Table exporttable - The DAT used to hold the export information when using the DAT Table Export Methods (See above).


Info CHOP Channels

Extra Information for the Kinect CHOP can be accessed via an Info CHOP.

Common CHOP Info Channels

  • start - Start of the CHOP interval in samples.
  • length - Number of samples in the CHOP.
  • sample_rate - The samplerate of the channels in frames per second.
  • num_channels - Number of channels in the CHOP.
  • time_slice - 1 if CHOP is Time Slice enabled, 0 otherwise.
  • export_sernum - A count of how often the export connections have been updated.

Common Operator Info Channels

  • total_cooks - Number of times the operator has cooked since the process started.
  • cook_time - Duration of the last cook in milliseconds.
  • cook_frame - Frame number when this operator was last cooked relative to the component timeline.
  • cook_abs_frame - Frame number when this operator was last cooked relative to the absolute time.
  • cook_start_time - Time in milliseconds at which the operator started cooking in the frame it was cooked.
  • cook_end_time - Time in milliseconds at which the operator finished cooking in the frame it was cooked.
  • cooked_this_frame - 1 if operator was cooked this frame.
  • warnings - Number of warnings in this operator if any.
  • errors - Number of errors in this operator if any.


TouchDesigner Build:

CHOPs
Ableton Link • Analyze • Angle • Attribute • Audio Band EQ • Audio Binaural • Audio Device In • Audio Device Out • Audio Dynamics • Audio File In • Audio File Out • Audio Filter • Audio Movie • Audio NDI • Audio Oscillator • Audio Para EQ • Audio Play • Audio Render • Audio Spectrum • Audio Stream In • Audio Stream Out • Audio VST • Audio Web Render • Beat • Bind • BlackTrax • Blend • Blob Track • Body Track • Bullet Solver • Clip Blender • Clip • Clock • Composite • Constant • Copy • Count • CPlusPlus • Cross • Cycle • DAT to • Delay • Delete • DMX In • DMX Out • Envelope • EtherDream • Event • Expression • Extend • Face Track • Fan • Feedback • File In • File Out • Filter • FreeD In • FreeD Out • Function • Gesture • Handle • Helios DAC • Hog • Hokuyo • Hold • Import Select • In • Info • Interpolate • Introduction To s Vid • Inverse Curve • Inverse Kin • Join • Joystick • Keyboard In • Keyframe • Kinect Azure • Kinect • Lag • Laser • Laser Device • Leap Motion • Leuze ROD4 • LFO • Limit • Logic • Lookup • LTC In • LTC Out • Math • Merge • MIDI In • MIDI In Map • MIDI Out • MoSys • Mouse In • Mouse Out • NatNet In • Ncam • Noise • Null • OAK Device • OAK Select • Object • Oculus Audio • Oculus Rift • OpenVR • OSC In • OSC Out • Out • Override • Panel • Pangolin • Parameter • Pattern • Perform • Phaser • Pipe In • Pipe Out • PosiStageNet • Pulse • RealSense • Record • Rename • Render Pick • RenderStream In • Reorder • Replace • Resample • S Curve • Scan • Script • Select • Sequencer • Serial • Shared Mem In • Shared Mem Out • Shift • Shuffle • Slope • SOP to • Sort • Speed • Splice • Spring • Stretch • Stype In • Stype Out • Switch • Sync In • Sync Out • Tablet • Time Slice • Timecode • Timeline • Timer • TOP to • Touch In • Touch Out • Trail • Transform • Transform XYZ • Trigger • Experimental:Trigger • Trim • Warp • Wave • WrnchAI • ZED