Housed in a 70-foot video dome that also functioned as a fully immersive 360-degree VR theatre, The Lab (designed and curated by META.is, sponsored by HP and presented by The Verge) created a space where experiential technologists were given equal billing to their musical counterparts - and as some even put in "stole the show".
Of the seven interactive installations three were designed and produced by NYC TouchDesigner artists: VolvoxLabs produced Origins, projection-mapped visuals spanning the entire dome's facade, Gabriel Pulecio installed Infinity Wall, a very trippy mirror and LED infinity experience, and Dave & Gabe provided silk cocoon, user-driven sonic experiences with Hyper Thread.
The work is tremendous and we could not be prouder or more impressed by these three teams. We found a few minutes to debrief (which was not easy as they don't sit still) and share with the community a little more background on these installations. Enjoy!
VolvoxLabs | Origins
Produced by VolvoxLabs, the façade encompasses the entire exterior of The Lab serving to heighten the visceral experience for attendees with ultra-high-definition video projection-mapped onto the surface of The LAB to create "a new visual journey throughout the festival".
We've written about the prolific Brooklyn-based VVOX twice in the last few years: the first article looked spellbound at co-founder Kamil Nawratil's graduate work at the School of Visual Arts in New York, The Perception of Consequence and the second time about VVOX's dd:OUTPOST _ Personalized Environment "a fully integrated, personalized visual system built from the ground up, including projection mapping on geometrically faceted bar, programmed user-interactive LED wall, an interface that pulls in real-time Twitter and Instagram feeds, that users can control interactively. The whole system is situated within a functional and modular interior design."
These examples illustrate the studio's prowess not only with creating dynamic motion graphics used in responsive, immersive environments that frankly seem to be 'alive', but also in fabrication and architectural staging. Also of note, VVOX is responsible for launching and maintaining TouchIn NYC, a very active meetup held regularly in Brooklyn for the TouchDesigner community.
Kamil Nawratil: For the inaugural Panorama Festival in NYC, we used projection mapping techniques to transform the façade of The Museum of Reinvention aka The Lab building into a luminous, living and breathing portal.
Titled, Origins, we created a visual journey, employing a montage of growth mechanisms, microorganisms, data-driven generative content and living systems to evoke a sense of wonder about hybrid connections.
We created high-definition video, live action visualizations of natural organic forms projection-mapped onto the surface of the LAB facade, creating the illusion of being contained within its structure. To encapsulate and create a live input from the festival itself, we included generative data-driven 'moments', which over time created a genome of the Panorama experience.
The plan was to distinctively pair 3D animation with generative sequences to create a narrative focused on the tension and symbiosis between the organic systems and technology, resulting in their evolution of something altogether different.
The narrative unfolded in five segments:
- SEED - PARTICLE GROWTH MICRO-ORGANISMS
- EMERGENCE - FACADE GEOMETRY VORONOI GROWTHS
- AWAKEN - LIVE ACTION MOMENTS OF LIVE TISSUES
- FLUID DYNAMICS - UNDULATING FLOW
- GENERATIVE - PANOROMA GENOME DATA DRIVEN INTERACTION
TouchDesigner was used to create a robust cueing system for playback of 4k content throughout the festival. Acting as a hub for inputs and outputs we also used TouchDesigner to create the Genome and Weather informed generative elements of the mapping.
Within the Genome ecosystem a free flowing experience is triggered from social media data and continually collects relevant interactions. Over time and as participants interact with social media networks, we collect the information to 'grow' the visual flow of the Genome. We used geolocation data to store the information and translated it to visual representation via the GLSL programming language.
In addition to Genome we involved weather data into the experience. Using the weather API we created a forest of generative coral flow stemming from The Lab facade. Using wind speed and temperature information the coral flow changed and reacted to the Panorama weather location.
TouchDesigner was fantastic here again, we were able to pack in 70GB of Houdini renders and mix them up with the generative work seamlessly. We also heavily used Dev Harlan's Vert Pusher component for mapping.
Creative Director: Kamil Nawratil
Design: Kamil Nawratil, Pa Her
Producer: Pa Her
Technologist: Javier Cruz
Visual Artists: Kamil Nawratil, Shuvashis Das, Conor Statuz
Prototype Fabricator: Zyia Zhang, Hannah Fink
Gabriel Pulecio | Infinity (Tiles of Virtual Space)
Infinity, created by Gabriel Pulecio, is an interactive installation consisting of mirrors, lights, and sounds that are responsive to an individual's presence. Equipped with a motion sensor, the sculpture can capture the position of one or more festivalgoers and trigger a ripple in time resulting in an ever changing visual and auditory environment.
Gabriel Pulecio Studio: Founded in 2015 the studio operates in the intersection between art, technology and motion design. For the past two years, Gabriel has been focusing his digital expertise on creating interactive sculptures and non-linear installations that push the boundaries between digital and physical mediums. His studio not only serves as a laboratory for spatial research and prototyping, but also as an interactive showcase where Gabriel's original works are installed and exhibited. Located in Brooklyn, NY.
Gabriel Pulecio: Developed as a part of ongoing research into light and space, Infinity (Tiles of Virtual Space) is an interactive immersive installation that transforms a finite space into a vast, endless expansion using light and sound. Infinity was premiered at Panorama Festival at The Lab.
Infinity is intended as a space for reflection and self awareness. It is founded in an idea of a technology that is not distracting us from where we are but quite the opposite. Infinity reminds us that we are here, in the present moment, and that every step taken in any direction creates new ripples of light in time, building infinite structures of light that only can be seen by looking a bit further away than our own reflection.
When entering the installation the tiles beneath trigger animations expanding from the individual that move up the wall. By walking in one creates trails of light that get reflected several times in the mirrored four wall structure. It never ends...!
How TouchDesigner Was Used
The interactivity of Infinity was accomplished with 2 Kinect for Windows v1 which were chosen for their near depth mode capability - a must in such tight space. The left and right side depth images were merged into one large image using image stitching techniques and fed into the Blob Tracking TOP. The Blob of each person was analyzed and the channels used to control interactive animation parameters.
We isolated areas of the depth image to use as animation triggers when the rate of change in their average color passed a certain point.These animations were then sampled based on the LED locations and the sample data fed into Serial Out CHOPs to Teensies running OCTOWS2811 and fastLED.
The tiles are precisely fabricated with laser-cut double mirror panels that are lined by LED strips, to give an optical illusion of infinite space. The installation detects the position of human activity which triggers mapped tiles to simultaneously light up and play a sound that ripples dynamically over time, expanding the user to an immersive environment of virtual three dimensional space.
Infitiny is equipped with 4 active monitor speakers mounted on the ceiling and one subwoofer. The audio was created by Jeff Dodson. Several tracks were created for this project, and the same logic that randomizes the animations also randomizes the sounds being triggered. The sound helps to create "the expanded universe within".
In the active mode of the installation the audio is triggered at the same time as the animations, but when the installation is in idle mode (OR people are passive, e.g. lying down) it is the audio that drives the visuals. Jeff''s distinctive sounds added a great deal of depth and character to the experience.
Concept, Design, Direction: Gabriel Pulecio
Lead Technologist: Daniel Schaeffer
Programing: Daniel Schaeffer, Lucas Morgan, Gabriel Pulecio
Sound Design / Composer: Jeff Dodson
Fabrication | Structure: Youngbuk
Fabrication | CNC Milling: Gamma NYC
Fabrication | LED Panels: Gabriel Pulecio Studio
Special Thanks: Marcus Metzger, Grace Jeon
Dave & Gabe | Hyperthread
Designed by Brooklyn-based creative team Gabe Liberti and Dave Rife of Dave and Gabe, Hyper Thread is a large-scale interactive sonic experience that invites the audience to explore, manipulate, and create sounds in real time through tactile engagement. Via a series of silk cocoons inside of a larger undulating silk envelope, tension and movement of the audience are translated into musical components of the 3D soundscape and immersive lighting environment.
Dave & Gabe is an interactive installation studio that unites sound, light, and tactility into immersive experiences. They believe environments should be expressive, encouraging collaboration and amplifying the energy within. Working with artists and brands, they explore new directions in real-time animation, generative 3D sound, and physical design.
For Hyper Thread, we wanted to encase people in an individual, isolated world while simultaneously providing a sense of togetherness with the others inside the piece. By hanging seven cloth cocoons from a structure draped with silk fabric, we created a coherent environment that felt both entirely transportive and embracing. The swinging of each cocoon controlled the sound and light of the environment.
For the sonic component, we were lucky to work with six composers who created unique compositions with the ability to be modulated by the movements of the pods. Then, by leveraging a spatial audio system comprised of 24 loudspeakers, we could set unique trajectories for each of these sources within a 3D acoustic field. The lighting, diffused by the swaddled cloth, was also amplifying the swinging movements, uniting the entire environment.
To control our array of DMX fixtures, we used TouchDesigner to drive the light animations via our incoming interaction data. We used the oscillations from our wireless IMU sensors as well as the midi notes being generated in the Max For Live objects to trigger these changes. We also assigned each composition it's own palette of colors so that as the music would transition, so too would the look and feel of the entire space.
Because the entire installation was contained inside of the silk covering, it created a sense of mystery upon approaching. You could hear the different components of the music changing and see the brightness of the lights swelling inside the fabric, but what was happing in there? Once the curtain was drawn back and you stepped in to sit in one of the pods, that feeling of being held in suspension was tranquil and invigorating -- a departure from the intensities of being at a large music festival.
Jon Baken, Composer
Nick Bratton, Composer
Shane Myrbeck, Composer
Yotam Mann, Composer
Yuka C. Honda, Composer
Kae Burke, Fabrics
Jeff Sturdivant, Fabrics
Marcus Swagger - Gamma NYC, Digital Fabrication
Adrian Martinez, Build Lead
Daniel Stewart, Systems Wiring
Alcy Sivyer, Build
Rose McKee, Build
Harrison Lockhart, Build
Thea Rae, Build
Adam R. Brown, Director/Editor (video)
Kyle I. Kelley, Cinematography (video)
BullMoose Pictures, Production (video)