By the fall of that year it was clear that the content and creative timelines of the project could not follow a more traditional production process and would likely require significant flexibility right up to opening. After an internal review it was felt that proceeding with the in-house tools would not likely support the end-game flexibility that was likely required in the project and alternatives should be considered.
At that time, TouchDesigner was brought forward as an evolving product that might meet the PWL and Rockheim needs. I joined Parallel World Labs in April 2010 to work said project, which was scheduled to open on august 5th 2010. This left us with about 4 months to refine the existing exhibits and, in addition, take certain exhibits from prototype to finalized state. All in all we finished 14 Touch Designer based installations. Note that over 40 TouchDesigner systems were used for this project!
The TouchDesigner team for that period consisted of Jesse Scott, Chris Quine and myself (Achim Kern) and was led by creative director and CEO Stacey Spiegel and CTO Rodney Hoinkes. Jesse Scott and Chris Quine were relatively new to Touch Designer experience, but they both picked up touch quickly and created some of the amazing interactive experiences you can see below in a timeframe that is only possible with a tool like TouchDesigner.
Please note that this article only covers the actual TouchDesigner implementations, for all exhibits a complex system of content compilers, servers, sensor systems, lighting controls, etc. was written by CTO Rodney Hoinkes and many others.
Tribute
(Final TouchDesigner Implementation: Achim Kern, Jesse Scott – Design: Stacey Spiegel)
Tribute is the first exhibit visitors see when they enter the main floor of the museum. It’s a series of 6 concave screens (all representing different decades in Norwegian music). Via motion based interaction users ‘clear’ off the foreground image, which shatters to thousands of small pieces that fall to the ground. Then a video appears underneath and plays.
2000's
(Final TouchDesigner Implementation: Achim Kern – Design: Stacey Spiegel, Rodney Hoinkes)
The Column is another 3 screen exhibit. Using a laser pen, visitors can interact with 3 separate columns of media. The center column features videos and sound, while the two bordering columns contain images. Picking images by pointing the laser pen rotates the column to the selected image and enlarges it. If the selected image corresponds to a video, the system plays back the music video. Clicking on the images in the outer edge of the column will rotate/transform the column to a new section without enlarging the content, enabling quick navigation. By clicking on the colored icons (e.g. the orange ”INDIE” in above pictures) you can reorganize the content based on the selected tag.
90's black metal
(Final TouchDesigner Implementation: Achim Kern – Design: Stacey Spiegel)
In this exhibit visitors select an actual cassette tape and put it in an old cassette deck to pick a band. Music videos of the chosen band are shown on a 5×4 matrix and certain areas of the room are lit with red light, indicating that interaction can take place at those locations. If the visitors approach one of these areas, new content is displayed on the big screen and the area/object that triggered the new content is lit with red strobes to give the user additional feedback that they interacted with the exhibit.
The new content appears with various origami style transitions and always leaves parts of the background video visible creating interesting collage-like compositions of images and videos. On the fourth trigger, the content that was activated on the first trigger is unfolded to ensure the screen is never too cluttered... The last trigger also always becomes inactive until another trigger type is activated and each trigger is associated with a specific type of content, i.e. if you go to the instruments you trigger live footage, if you go to the posters you get band art, …
90's Cafe
(Final TouchDesigner Implementation: Jesse Scott – Design: Stacey Spiegel)
This exhibit is designed to look like an espresso bar from the 90s, and has a six-screen LCD matrix on the wall, as well as a touchscreen with an icon-based menu to select various bands/videos.
80's
(Final TouchDesigner Implementation: Achim Kern – Design: Stacey Spiegel, Christopher Quine)
80s is a 3 screen exhibit, where users pick a band on a touchscreen and then physically play with the displayed content using the sliders on an analogue mixing desk. LEDs on the mixing desk indicate if content is associated with a slider, and also whether it is a movie or an image. Bringing up a movie will take over the ambient sound and you can hear that movie’s sound as long it is the topmost video. Certain content was also equipped with a “bubble” overlay providing additional info about the band or that piece of content. You can also pick a virtual representation of a famous Norwegian recording studio, and then remix a song using the sliders on the mixing desk.
70's Band Bus
(Final TouchDesigner Implementation: Jesse Scott – Design: Stacey Spiegel)
This is a surround video environment designed to feel like you are inside a tour bus. The interaction derives from a separate application, which runs alongside TouchDesigner on each machine, allowing you to browse through a series of virtual rock magazines, with stories on different genres and artists; when you touch a picture, it animates and moves to the edge of the screen, while TouchDesigner selects an accompanying video and animates it overtop of the idle video on the “window” screens.
60's
(Final TouchDesigner Implementation: Achim Kern and Christopher Quine – Design: Christopher Quine)
The 60s room is styled like a typical Norwegian living room from said era. Visitors can load new content on 3 screens through gestures and motion. With each new trigger, the previous content “shrinks”, while the new content is grouped around it. This keeps the entire story and all its chapters, which are associated with the current band, visible at all times. In addition, an old TV displays additional info about the current band/chapter. After 7 chapters, a curtain goes down on all 3 screens and you’re again in a living room with a 60s wallpaper. The next trigger then loads a new band.
50's
(Final TouchDesigner Implementation: Jesse Scott – Design: Stacey Spiegel)
This was a fun project, mainly because of the environment that it was set in—including an original 1957 T-Bird! – and the fact that we got to use an old 'Rock-Ola' jukebox as the control mechanism. Its buttons were used to trigger videos across the 3 screens.
The interaction in the 50s poster part of that exhibit was very casual; the motion triggers were not visibly marked, so as people would go and leaf through old children's magazines or turn the dial on an analogue radio, the image would switch. All the images were related to the 'artifacts' in the exhibit, so you slowly got more and more context...
Artifact Wall
(Final TouchDesigner Implementation: Jesse Scott – Design: Stacey Spiegel)
Visually, this exhibit is a series of industrial shelves filled with various famous instruments, amplifiers, et cetera, as well as a series of screens installed on the shelves. The interaction is done with a laser pen; the screens show content connected with the instrument just triggered.
Rocking With Ronnie
(Final TouchDesigner Implementation: Jesse Scott – Design: Stacey Spiegel)
This is a touchscreen-driven project where Ronnie Le Tekrø, guitarist for TNT, teaches you a few licks, then rates your playback alongside his.
Hip Hop room: Break-dance Touchscreen
(Final TouchDesigner Implementation: Achim Kern – Design: Stacey Spiegel)
This is a small exhibit using a touchscreen with a slick UI to navigate through different categories of break-dance styles and then allows you to select various dance moves and watch them in slow-motion.
Hip Hop room: TouchDesigner <> L.A.S.E.R. Tag
(Final TouchDesigner Implementation: Jesse Scott – Design: Stacey Spiegel)
In the same room, Jesse Scott create a TouchDesigner based control interface for the original L.A.S.E.R. Tag software, which allows visitors to “paint” on the wall using a laser pen and adjust the drawing style on a touchscreen.
Map
(Final TouchDesigner Implementation: Jesse Scott – Design: Stacey Spiegel)
This is a six-story map of Norway with triggers installed in place of every major city in the country. Using a laser pen, visitors can trigger playback of video content specific to the selected city.
By Achim Kern
Photos provided by Rockheim/Benedikte Skarvik/Geir Mogen and Jesse Scott
We had a few questions for Achim after this marathon project...
Derivative: What things gave you the biggest rush when you saw them manifested?
AK: For me that definitely was seeing the 2000s exhibit up and working. When I first heard about it, I wasn’t sure that a laser pen based interaction would work, but once I saw a few people flipping through videos and images and seeing the endless reflections of their interactions in the mirrors, I was definitely thrilled.
D: Did you have to go through many versions/attempts (of UIs) to get to what worked best?
AK: It differed from exhibition to exhibition. Most of them had a lot of pre-planning and thought put into them, so it worked out just fine when using them for the first time on site. With others, especially the 80s- where you had to interact with the exhibit via an analogue mixing desk- it turned out that due to the absence of motorized faders, the initial concept didn’t work out perfectly. You could end up in situations where the position of the faders and the position of their related content did not match up (e.g. after switching to a different band), and moving the faders then caused the content to jump to the new fader position. With the initial design of that exhibit, these jumps were visually too strong, so we had to come up with a modification that would work better with these side effects.
I think the most demanding exhibit in this regard was the 60s room. During the development, it became crucial to the client to retain the previously displayed content when a new content was triggered. So Chris came up with an amazing concept that kept nearly all content history for the current band displayed on the screen, giving the tour guides the possibility to tell the complete story of a band while always having the possibility to reference information that was triggered minutes ago.
D: For all the exhibits visitors had to learn the UI on the spot (immediately). What refinements did you make to enable this?
AK: All exhibits were designed with these requirements in mind, so we only had to refine smaller user interface issues. For example, after using the laser pen based interaction on the 2000s, it turned out we needed an additional method to allow users to navigate around the column in larger steps. However, some exhibits were explicitly designed to work in a way that the user accidentally discovers that he can interact with the exhibit. For example, in the 50s poster section, the visitors can look at (and touch) various artifacts from that period, and while they reach towards them, they trigger new content on one of the screens above them. No hints were given, but this concept only worked for this specific room where the content was more or less secondary to the artifacts. In other exhibits, adding simple hints like colored light to indicate an existing trigger and light-strobes indicating that something was triggered worked very well.
D: Achim - what did you learn here that is new for you?
AK: As always, you learn lots of new stuff on every project, this time most notably how to track a laser pen on top of an actual projection. That knowledge came in handy for a TouchDesigner based version of laser tag, a project that Jesse and I are currently working on. In addition, I learned some new techniques on how to overcome TouchDesigner’s lack of asynchronous loading of content while keeping within GPU and CPU memory limits.
While not a new technique, one major lesson learned from working on such a huge project – where toe files were created and consequently modified by many different people – is that it takes a lot of pre-planning and careful component design to ensure that things are readable and therefore can be (re-) used and modified easily by other people. And, as always, that you definitely should work things out on paper first, it gives you a much better idea of possible issues and at the same time helps other people to understand your networks when they take over the project.
