VolvoxLabs employed TouchDesigner as the central brain of the lounge, using it to control data flow and the automation of many of the elements within the four engagement pieces. TouchDesigner also serves as the content management system, and is used for video mapping and playback, generative design, robotic element controls, and lighting control.
The technology objective in this meticulously crafted and responsive space is to awaken the senses, enabling human interaction to become "a medium for art that explores and celebrates the evolution of ideas that in turn shape technology and inspire creativity beyond the Microsoft Theatre."
The four permanent interactive installations at the Microsoft Theatre Lounge are all controlled and run entirely on TouchDesigner. VVOX founder and director Kamil Nawratil expands: "Aside from a very few rendered scenes everything is real-time graphics. We also enabled the theatre managers and executives with a custom interface to control each of the installations via a tablet. TouchDesigner was especially useful for us since we didn't have to use Crestron or similar devices to enable on-off states on each server and the five Barco projectors. A Surface tablet controls the whole gallery lounge space all through TouchDesigner, using python, magic packets and JSON."
MOMENTUM | A Kinetic Sculpture
Inspired by the Microsoft logo, Momentum is made up of 160 wooden tiles, 80 Ultramotion motors, and 10 micro-controllers (Arduino boards). The magnificent facade undulates and illuminates with different colors in response to human motion using the Kinect sensor. The effect is mesmerizing!
Approximately 1000 linear sq ft of ash wood was sourced and CNC-milled (computer numerical control) inhouse to create 160 wooden tiles. All of the technology driving the movement and light are embedded directly into the 30' x 9' facade.
In addition we built communication between all computers and projectors in the space to the tablet. We enabled magic packet control to turn the machines on/off sending JSON information to the projectors. TouchDesigner was fantastic here: simply load up libraries and suddenly we have our own Crestron system!
Kamil Nawratil: We're particularly proud of the 'Momentum robotic wall at the entrance to the lounge. It's the first thing you see while going down the escalator and it's a beast!
In Momentum, we've embedded 80 motors, interactive LEDs and custom PCBs inside of the moving facade. It all runs extremely smoothly thanks to serial communication to Arduino and DMX Out from TouchDesigner.
Inside our TouchDesigner network, everything starts with a texture, a simple grid of 20 x 4 (the same as the motor layout of the wall) that drives the values of the motor actuation distances. The same grid is also converted to channels and RGB values, and are sent out to LEDs. This very minimal setup lets us quickly iterate over different kinetic motions.
FORM | A Projection Mapping Experience
This digitally fabricated wall is used as a video-mapping surface with content mapped and projected onto it using high lumen projectors. The Form wall also uses Kinect sensors to capture user movement and invite interaction with generative content that spontaneously animates based on user motion.
KN: The Form projection-mapped diamond wall acts as a continuation of the kinetic wall pattern. We built a custom CMS for the staff with an interface built in TouchDesigner showing a representation of the mapped wall with drag-and-drop clip playback. This allows for dynamic file assignment and enables staff to change out content for the entire wall, manipulate content to go from big and/or small diamonds, and so on. It's functional, intuitive and gives the staff full content control.
We used Vert Pusher (thanks Dev Harlan) to map these elements as it really works well with a lot of geometry points. In addition to allowing for new content, we've also generated real-time scenes wrapping around the geometry. We placed two Kinect sensors right in front of the piece to get extra motion data to control generative visuals.
DAVINCI | A Photo Booth Experience
The third experience is an autonomous photo booth where visitors can overlay their "selfies" onto pre-selected background scenes of their choosing. The 4-monitor video wall is equipped with a Kinect sensor, and a touchscreen interface guides users through the steps to taking photos and then to emailing or texting these to themselves or a friend.
KN: DaVinci is the 'shared documentation' part of the experience. We call it the DaVinci because it can do so many things, like display content and videos, take pictures and act like a digital green screen. The added trickery is that the RGB image from Kinect does not perform well in low light so we built functionality for converging the lens perspective with a DSLR.
In essence, we take a high-res photo with a DSLR and match it perfectly with the Kinect depth to get the digital green screen effect. The calibration is done outside TouchDesigner and produces intrinsic data used to process stereo images through GLSL. Big thanks to Vincent Houze for stepping in with his OpenCV knowledge.
Mosaic Mural | An Evolving Digital Canvas
KN: The Mosaic Mural installation displays a playback of the most recent photos taken in the photo booth. As new photos are received a "reshuffling of the tiles" animates. Inside the network we're using a Texture 3D TOP to load up all the files and display them as textures on rectangle instances. There are about 1700 images uploaded and displayed in various grids.
TouchDesigner is our central brain controlling data flow and the automation of many of the elements within the lounge. For this particular project we're using it as CMS, for video mapping and playback, generative design, robotic element controls, and lighting control. Touch is so extensible that there seems to be no task it can't handle!
VolvoxLabs was founded in 2012 by Kamil Nawratil. The studio first began creating full service content solutions. VVOX soon went on to create responsive systems in live performances and custom built kinetic sculptures.
Having a multi-disciplinary process has enabled the studio to work on a wide-range of projects. VVOX has it's production facility in Brooklyn. The shop is equipped to handle fabrication of all types, including large interactive fixtures, mechanical and electronic elements, and kinetically engineered systems.
VVOX continues to work on many projects in the US and with clients around the world with the focus of giving more movement, life and action to all things.
Client: Microsoft / AEG / Microsoft Theater
Select photos owned by Microsoft.
Creative Director: Kamil Nawratil
Technical Direction/TouchDesigner Development: Kamil Nawratil
TouchDesigner Development and Kinect Integration: Javier Cruz
Technical Drawings for Momentum the Kinetic Sculpture: Miron Nawratil
Lead Physical Computing and Fabrication: Zyia Zhang
Electrical Engineering: Bill Bowen
Physical Computing and Fabrication Assistant: Jason Rabie
Fabrication: Hannah Fink
TouchDesigner Effects on Digital Mosaic Mural: Vincent Houze
TouchDesigner Effects on Form Projection: Shuvashis Das
Physical Computing Assistant: Danara Sarioglu
Project Management: Pa Her
Momentum Documentary Video: NORTH + NOMAD