Derivative: Please tell us a bit about yourself and what inspires you, what you do and how you came to work in this field.
Jay Borgwardt: I fell in love with interactive art in college when I took a class on creative coding in Processing. The TA for that class recommended I try out TouchDesigner, and I was hooked! After that I was lucky enough to get an internship with Harvey Moon at the precursor to Spectra Studio, where I’ve been working ever since.
I’m always interested in projects that use physical objects to interact with digital content. One big inspiration for me was an installation by Design I/O called “Connected Worlds”. In it, children could use large foam logs to change the course of a projected river, and send the water to different virtual habitats. Another was Neil Mendoza’s project “Mechanical Masterpieces” for the Pittsburgh Children's Museum, in which guests could push levers, turn cranks, and pull ropes to interact with digital copies of famous paintings.
I strive to create projects that use novel methods of interaction, not just as a gimmick but as a genuine, unique way of engaging with art and technology.
Derivative: Your recent work the Vinyl Music Station has been a huge hit with the community, for audiophiles and well beyond. Can you explain where the idea originated and what the precedents were for you.
Jay Borgwardt: I love vinyl records. There’s something so magical about being able to see and hold your favorite songs in your hands. I used to work in a record store and was constantly fielding questions about what was currently playing in the store. I designed a simple informational display that would show the title and artist of whatever album was currently playing, and the project grew from there. I actually added in the record end sensor after finishing the first prototype of the Vinyl Music Station; I started listening to records more when I had company over, but I would have to drop everything to flip the record when the side was over to avoid dead air. So I decided to add a sensor to detect when the record side was over, and I used Spotify integration to find and play similar music. This ended up being a lovely mix of analog music and digital music, blending the strengths of both.
Derivative: Please explain how you built the VMS and how TouchDesigner was used.
Jay Borgwardt: As with all of my projects, TouchDesigner drives everything! There are four basic parts: The album database, vinyl recognition, audio visualizer, and end sensor playback. Vinyl recognition now uses computer vision, but the prototype used in the video used RFID tags as well. Once the album is identified, most of the rest of the work is data wrangling and information display. Currently I use containers and text COMPs for the visual elements, but in future versions I’m thinking about switching to rendered Geos, as I hear it can provide a lot more freedom.
The speakers I use have an audio out port which is a really easy way of getting the audio feed from the record player to the computer. From there it’s only a few CHOPs and some instancing to get an audio visualizer. I think I used Kaiho Nishimura's tutorial as inspiration.
The “similar music” player was one of the more difficult parts of the installation, mostly because of Spotify API integration issues. The Spotify API uses OAuth 2.0 authentication, which was a bit of a headache to figure out. After I got access, I could find the 5 most popular songs on the current album and query the API for 100 similar songs, then add those 100 songs to the queue and start playing until the end sensor detected that the needle was no longer at the end of the record.
Above image caption: Finding similar songs on Spotify. The album name is fed into a Spotify API request to get the Spotify tracklist, which is then sorted by popularity. The top five songs on the album are then sent as a request to find similar songs, which are then added to the Spotify queue to be played once the record side is complete.
Derivative: It's really ingenious and useful too. Were you surprised at the response and do you have any plans to release it in some form for public use?
Jay Borgwardt: Thanks so much! I was definitely surprised at the response from the public, I was expecting a few engineers and music enthusiasts would enjoy the Vinyl Music Station but I was blown away by how much of a wider audience it reached. It was really refreshing to see the project through other people's eyes, especially after spending so long tinkering with it. I feet very lucky to get so much positive feedback and constructive criticism on a project that means so much to me.
I'm making a project breakdown for people who are interested in how it works, and I also plan on building and installing custom Vinyl Music Stations for people and businesses who want one. I really want to bring the station to record stores and vinyl bars. The project came from working at a record store so I’d love to bring it full circle. There’s still a lot of polishing to be done, but I’m excited to continue adding to it!
Derivative: We'd love to know a bit about your work with the team at Spectra Studio, how you collaborate and it would be nice to hear about a project - research or client - that is exciting to you right now.
Jay Borgwardt: Working at Spectra Studio is a dream! I get to work with some incredibly talented artists and engineers on crazy projects. We’re all interested in each other's work and have weekly team sessions where we share new techniques, theories, and practices to help us grow both individually and as a studio.
One exciting project we’re working on right now is Harry Potter Visions of Magic, which is currently in Brussels, Belgium. Guests can actually use wands to interact with lights and projection, and it feels really responsive. I had a blast prototyping the Visions of Magic interaction because it ties in so well with my interests: physical objects interacting with digital content.
Derivative: What's next?!
Jay Borgwardt: I have a couple more fun engineering projects that I've been working on, and I also want to get back to my art practice. There are also a few exciting things at Spectra Studio coming up next year as well. I wish I could go into more detail, but for now that's a much as I can say!
Follow Jay on Instagram
