Company Post

Tom Sepe Built 2 Robots for Meow Wolf's OmegaMart and They Almost Killed Him

Tom Sepe has a rich history of making kinetic art starting with building flaming art cars for Burning Man, then creating more permanent installations for Obscura Digital involving the design, fabrication, installation and maintenance of some complex projects like transforming a Tesla Model S into a mobile projections vehicle  for the documentary film Racing Extinction. He's built a Sweeperbot, a Slapping Machine even an Insult Bot but also tells us that he would never advise anyone to ever make anything that moves; with zero margin for error it’s not work for the faint of heart, the outcome can be potentially dangerous in fact, but when you get it right “making things move is magic”. So when Meow Wolf came knocking looking for two humanoid robots for their Las Vegas attraction OmegaMart he signed on. Here is the fascinating account of how Tom spent 7 months designing, building and programming Herman and the Receptionist, two robots that are completely controlled via TouchDesigner.

Derivative: Can you tell us about your background and your work?

Tom Sepe: I grew up in the theatre, acting and helping behind the scenes from age 5 till I was 18. As a kid I used to run the sound and lighting board for the grown-ups' productions. However, I was really good at math and loved science, I remember learning to program in BASIC on computers at school and I eventually had a Macintosh and then an Apple IIe. So naturally I went to college as a physics major at UCSB. Eventually I switched back to the arts and enrolled in the College of Creative Studies at UCSB, a multi-disciplinary program that gave me the freedom to basically create my own major... which included sculpture, printing, drawing, video editing, photography, computer graphics, dance, theater and even performance art. But my love for the technical never ceased and when I moved to the Bay Area after college I quickly found work as a graphic designer, web designer and video editor.

During that time in the late 1990's and 2000's I was also involved with various artists and groups, such as FiveTonCrane building flaming Art-Cars for the Burning Man festival, or assisting on giant steel kinetic sculptures. The list of crazy projects is endless, I was even teaching people to walk on stilts and breathe fire! Then I started working as a Technical Director for Obscura Digital in 2011. Obscura had just begun breaking out of an events-only paradigm and was creating more permanent or semi-permanent installations, so I was often involved in the design, fabrication, installation, and ongoing maintenance of projects that could be anything from a huge projection show in a 100-year-old dome in a mall, to making a massive touch-screen kitchen table for YouTube's Brand Lab. Though I was mostly focused on the hardware aspects, during this time period I began learning TouchDesigner.

As amazing as it was to be working for Obscura I had zero time or energy for my own projects or artistic growth, so I chose to go back to being freelance, and that freed me up for more professional development and to be able to seek out funding to produce my own projects.


Derivative: How has TouchDesigner been useful to you as a tool?

I first learned TouchDesigner as a glorified media player, powerful and precise, able to do projection mapping, network and sync outputs from multiple computers, timing with audio, etc. Then I started understanding how with a little bit of programming knowledge it can be so much more. It is such a swiss army knife and so customizable. My first big freelance project I created a real-time show controller and complex media cueing system that allowed me to on-the-fly seamlessly provide the audio and visuals for a presenter on stage at a huge conference.

Then I started using TouchDesigner to control actual hardware, motors, servos, actuators, LEDs, lighting, and other special effects.

I've been able to create custom control interfaces for my projects, and even interface with different software over wifi, using phones or tablets or even hardware controllers of my own design as inputs. My most recent project at Meow Wolf's OmegaMart in Las Vegas is two humanoid robots that are completely controlled via TouchDesigner.

Derivative: What was the design brief for the Meow Wolf project/robots and can you tell us a bit about what these robots do at OmegaMart?

Tom Sepe: My original idea submission to Meow Wolf was for a kind of fortune teller or oracle robot. The user would interact with the robot, ask it questions via a keyboard and monitor and it would respond by speaking and moving. I had been researching different chatbot software, and was excited to make something that could contextually respond, perhaps with some humor, or a deep thought every now and then, something that would seem alive. Meow Wolf has a whole back-story or theme that goes with each of their exhibits, a non-linear narrative that allows the user to endlessly explore and discover different facts about the story-universe. Clues can be found that lead to more stories or tidbits and more clues leading the user through the whole installation, all in a very tactile and interactive way. So the folks at Meow Wolf took over the chatbot side of things in order to write in bits of the narrative into the robot's responses. Oh and they asked me to make two robots! So both of the robots became characters in their story.

Derivative: How did you actually build the robots and how did you use TouchDesigner to control them?

Tom Sepe: Both robots are fully controlled by TouchDesigner. For the robot movements, there is an instance of TD that is pulling animation data from an FBX file created in Blender3D. There is also a GUI that allows for direct control of the robots through a series of sliders, this was mostly for dev and testing purposes, as well as troubleshooting. A second instance of TD handles the screen animations as well as all the OSC communication with Meow Wolf's Show Controller. ( using TouchDesigner as well )
This instance also handles the keyboard input. The input gets sent via OSC to a chatbot server called Chatscript, that runs on another machine on the network. Chatscript analyzes the input and generates a response message that includes the text that will get converted to speech as well as a movement command. The OSC message response gets sent to a Text to Speech package, and then gets run through Max MSP for added effects, before coming out of the speaker built into the robot. The movements are triggered and played to correspond with the length of the audio signal. There is a "library" of movements that get called up and triggered, for the most part pre-assigned to a semi-scripted and semi-generated response from the chatscript. To make the actual movements, TD takes the animation data and sends out data over a SerialDAT to the motor controller boards and finally to the smart servos. There are features built into the TD network that allow for setup of the servos, as well as fine tuning the limits and other parameters of each servo's movement profile. In addition there are some "watchdog" features in the TD network that check on the status of the servos and can automatically shut down any movements and do error reporting to the Show Controller if there is a major communication or hardware failure internally. It took me about 7 months of almost full-time work to design, build and program both the robots.
Using TouchDesigner to control an entire robot body is not ideal for numerous reasons, and previously I went down a whole development route using ROS and hardware level controllers, but I ran into some complications that I just didn't have the time or budget to solve. Furthermore Meow Wolf is using TouchDesigner in numerous places in their exhibit, so it made sense to develop something that the client would be able to understand and possibly even modify or upgrade at a later date. The good thing about it was that with my limited Python skills and a decent understanding of TouchDesigner I was able to get these things off the ground.

Derivative: How many people were involved in the project?

Tom Sepe: I had a very tight budget for this ambitious project, so I wasn't able to hire as much help as I would have liked but there were some super key players that helped me get it done. Primarily it was just me working for about 8 months straight, 6 days a week. The talented Eleanor Buell was my weekly shop assistant. She would come in about 1 day a week and bust out a bunch of tasks. At the end of the day we would do a full proper cleanup of the whole shop, which was so much more fun and easier with two people! Clay McCabe, who is an accomplished builder and artist in his own right, was my pinch hitter, and would show up here and there whenever I was in a bind. I couldn't afford a project manager, however Rosy Wolfe and I would meet for about an hour once a week, and she and I would just go through my list and she would help me prioritize and not get overwhelmed by all the details. She also acted as a committed listener and confidant, allowing me to vent or celebrate various moments in the process. I can't emphasize enough how crucial having her in that role was! It made a huge difference in my sanity. Wendy Fitch helped me with some finish work, and I hired Stephen Brawner to help with some of the initial hardcore programming. Finally one of my best friends Dallas Swindle was always available to dive into the technical details of whatever issue I was facing, and just help my brain think better.

The process for both robots started with a written description and a few lousy sketches I made. Then I assembled dozens of photographs of other robots from anime to live-action, and created a stylistic "mood board" for each robot. One was going to be more polished and refined, kind of like C-3PO, and the other was going to be more industrial and cartoony, like a Wall-E or Johnny-5. SInce my illustration skills are poor, I asked Meow Wolf if they could draw up some sketches based on the mood boards and the description. We had a couple of rounds of back and forth and settled on a concept drawing for each robot. This was an important stage as it gave me a clear guideline to work from, and it gave Meow Wolf a clear expectation of what would be delivered.

Next was a prototyping stage where I tested out various motors and control systems. My biggest concern here was going to be reliability and longevity. I ended up using some "smart-servos" because each motor has a built-in micro controller that will monitor temperature, torque output etc. and the motor will electronically shut off before it physically breaks. The downside to this is it meant a lot more programming and complex interfacing needed to be figured out.

Once I had decided upon and ordered the hardware, I started building the main structural elements which would be all aluminum extrusion and aluminum plate. This erector-set like frame allowed me to go back and make some changes if I needed to, and it also provided a system for attaching external parts. Aluminum is lightweight and easy to drill and tap too. Amazingly I didn't do any welding on the entire project!

The exterior of the two robots differed in that the one was 3D printed, and the other was made with EVA foam. If I could go back in time I would have made them both out of the foam. It required zero CAD modeling, you can cut the foam with a razor blade and glue it with rubber cement. It required a lot less finishing to get the final look as well. It was just a lot of fun to work with.

Derivative: What did you learn from this process?

Tom Sepe: Every single aspect of this project was a chance to improve my skills or try out something new. It was clear to me from the get-go that this was going to be a heavy lift, and that I was intentionally putting a mountain in front of me so that I could climb it. I upped my programming skills and my TouchDesigner skills tremendously on this project. I also have a desktop CNC router that I had previously only used for wood, so I spent a few weeks learning how to machine aluminum on it. I had been using Blender3D for years, but I really had to stretch in order to create the shapes I wanted. It was gruelling but I did it. I'm glad I can do simple CAD, but in the future I'd much rather someone else did that part! In addition I was 3D printing a lot of these external parts, which then had to be glued together to make larger panels, as well as be removable for any servicing. Finally, my airbrushing, weathering and finishing abilities grew as well. At times it was a little crazy, because I was the designer, the CAD modeler, the 3D printer, the machinist, the finisher, the electronics engineer, the project manager and the accountant all at once.

Derivative: How did you get into building robots and what is the attraction there for you?

Tom Sepe: In 2007 I built an electric motorcycle from scratch in the Steampunk style,  and that is about when I started getting more into electronics, motors, controllers, and micro controllers, as well as learning about CNC fabrication. Michael Christian, one of the artists I was working with, purchased a used CNC plasma table, so I spent a couple of months repairing it and learning how to operate it so we could cut very large and sometimes heavy pieces of steel. Here is the one we made that summer:

One of the coolest projects I got to work on with Obscura was transforming a Tesla Model S into a mobile projections vehicle for the documentary film Racing Extinction. I even got to ride around with Elon Musk in it!  You might actually call that the first time I built an art robot. But the CNC plasma table was the first industrial robot project I got my hands dirty with. Right after that I built a whole series of robots for the Roadkill Saloon at the Oregon Eclipse Festival. This was an immersive art experience created in collaboration with Tucker Teutsch where I built a pneumatically operated Mechanical Can-Can, a Sweeperbot, a Slapping Machine and an Insult Bot!

Kinetic art is a pain in the ass. I would never advise anyone to ever make anything that moves. After building art cars for years people would approach me and say, "Hey Tom, I'm thinking about building an art car, can I get some advice from you?" and I'd say, "Yeah... DON'T!" There is so little room for error when it comes to mechanical devices. If you are making a painting and the green is a little too blue, big deal, the painting doesn't cease to be a painting. But if you get your torque calculations wrong, or you don't use the right kind of bearings, or the proper thickness of material, your kinetic art will break, and someone could even get hurt. But making things move is magic, pure and simple. When things move they are alive. And as much as I love large-scale visuals (which can also be incredibly magical), actual 3-dimensional, real-world objects have a certain je ne sais quoi. But the goal for me is always to make people say "OMG" or "WOW!"

I've always been a huge fan of the droids in star wars, and all of the different robots and androids throughout TV & Film history, purely from a tactile "that looks cool" viewpoint, but on a conceptual level there is also a lot to explore... In today's day and age, we are finally starting to see the humanoid robots of science fiction come to life... And we also have more writers that are exploring these concepts of artificial intelligence, cyborg bodies, neural network consciousness etc.

I feel that robots in literature as well as in real-life are mirrors, reflections of ourselves and our psyches, and eventually I want to be able to explore and reveal that in my art.

Derivative: Any other projects you would like to talk about?

Tom Sepe: Right now I am installing a permanent 4-room audiovisual projection installation at the Bosque Redondo Memorial Museum in New Mexico. The subject matter is very somber and subdued telling the tragic stories of the Native Americans in the Southwest, and there is an interactive element where the participant can add a bit of their own story to the exhibit. I'm using TouchDesigner to control everything, from the floating text on the walls, to LEDs that highlight certain artifacts in time with the audio, as well as using motion tracking as a control input.

Derivative: What’s next?!

Tom Sepe: I'm really curious about autonomous vehicles, computer vision, and machine learning, and I just acquired a 4-wheel robotic vehicle the size of a small ATV that I'd like to use to create some mobile art with. It is very much in the gestation stages and I'm looking for collaborators and funding to facilitate the development. I've been knocking on some doors in the Special Effects for film industry... and starting to reach out to some other companies as well... I currently don't have a proper shop, and I really crave teamwork, collaboration, and larger scale projects, so if anyone reading this thinks they might have an opportunity for me, please let me know!


Follow Tom Sepe Web | Twitter | Facebook | Instagram