Close
Community Post

From GLSL to T3D: How Josef Pelz Simplified 3D Textures for All

Josef Pelz, a generative artist and creative coder with a background in mathematical visualization and computer science, has created T3D, a groundbreaking toolkit for the TouchDesigner community. Building on his expertise in GLSL and years of working with 3D textures, Pelz developed T3D to eliminate the steep learning curve associated with writing GLSL for 3D texture manipulation. Inspired by his work on exhibitions and a pivotal exchange with the community during the TouchDesigner Berlin Event, T3D empowers creators to explore surfaces, volumetrics, real-time fluid simulations and abstract data with ease, opening new realms of creative potential in real-time visual design.

Derivative: We have talked a bit in the past about your background and education in mathematics and computer science but could please share that with us here?

Josef Pelz: I started studying mathematics when I didn't know what to do after highschool. Being a student in Berlin comes with a lot of advantages, e.g. cheap public transport. I was too late to apply for most programs though. But with math, you can pretty much just enroll yourself. Once I went to the first lectures, I was hooked and kept going. I focused on geometry and in my masters on mathematical visualization. Through those lectures I got to know Houdini and was captivated by the creative applications of all the theoretical things I learned about math and computer science. Later, I wrote my master thesis on real-time smoke simulation with Schrödinger’s Smoke in TouchDesigner.

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

A post shared by Josef Pelz (@josefpelz)

 

Derivative: What was your trajectory in studying math and becoming an exhibited artist?

Josef Pelz: Strange coincidences led me to an internship at the former Pfadfinderei (now PFA-Studios) where I met VanTa who introduced me to TouchDesigner and GLSL. I spend the bigger part of the next months exploring GLSL. I used pretty much nothing but the GLSL TOP for a long time, before learning about the rest of TouchDesigner. I posted some of the results online and came in contact with the amazing TD community. First collaborations and commissions came along, through friends in the community and via social media. 

Derivative: How is a software like TouchDesigner useful in the context of your work and what was the first project where you used it?

Josef Pelz: GLSL is the medium of my work. I could develop GLSL in other environments too, but to me, TouchDesigner is the most intuitive and playful one. And as a visual thinker, I really like the node-based interface. Sketching out an idea or going all the way in with a concept is seamless. TouchDesigner also allows me to create new workflows and solutions for problems I face. Over the last six years I built a small library of solutions and tools that I rely on now. This animation, which became my Icon, is one of my very first. 

Derivative: What inspired you to start developing tools for TouchDesigner? Is T3D rooted in your personal work, perhaps from exhibits like the Nobel Prize Museum or Artechouse? Can you share the story behind T3D and how the project evolved?

Josef Pelz: Seeing other people’s creative applications of my tools is super rewarding, that’s probably the biggest driver for me to build tools. But after all, I build them anyway for my own work and it’s quite satisfying to polish them a bit and make them so user friendly that others can use them. I definitely profit from that myself. And the TD community is such an integral part of it. I learned so much from other people's work and I’m happy whenever I can give something back.

 

 

I have worked with 3D textures since I started working on my master's thesis (3D smoke simulations). I had to write every small operation in compute shaders (e.g. adding two 3D textures) and I never really packaged any of that into reusable components. But I used them more and more frequently, especially because I was intrigued by volumetric raymarching and fluid simulations (inspired by the work of Vincent Houzé).

In the beginning of 2024 I was working on an exhibition at LOAD gallery, which relies heavily on 3D textures.

The Final inspiration to build T3D came during the TouchDesigner Berlin Event, where meeting the community and discussing projects reminded me that many users might be interested in 3D textures but find writing GLSL to be a significant barrier."

At the same time, Torin Blankensmith and I were playing around with the UI of TD and I realized that I might be able to integrate such a toolkit into the OP create dialog. A couple of sleepless nights later I had the first version of T3D!

Derivative: For those unfamiliar, how would you describe T3D and its core functionality within the TouchDesigner ecosystem?

Josef Pelz: T3D is a set of operators built around 3D textures. 3D textures are basically like regular textures but extruded in a third dimension. You can think of them as a box in space where at each position of that box there’s a value (could be color or anything that we can represent with four floats). TouchDesigner generally supports 3D textures, but there are very limited operators to create or manipulate them, there used to be only the texture3DTOP and the glslTOP.

Previously, working with 3D textures required expertise in GLSL, ideally with compute shaders for better performance. T3D now makes 3D textures accessible to all TouchDesigner users.

Most of the T3D operators are built in analogy to respective TOPs. There’s an addT3D, a noiseT3D, a blurT3D, a displaceT3D etc. And their parameters are familiar too.

 

 

Derivative: How does T3D integrate with or expand upon TouchDesigner’s existing 3D tools and workflows?

Josef Pelz: With 3D textures we can do some interesting things! 3D textures can represent surfaces, if we say that every point in the 3D texture with a value higher than a certain threshold is inside of a surface. And T3D has a built-in geoCOMP for rendering 3D textures as surfaces, the surfaceT3D. 3D textures can also represent volumetrics, so for example a cloud, where every value in the 3D texture defines the density of the cloud at that position. There is the volumeT3D to render such an object. But 3D texture can also represent more abstract data, like color in space, a velocity field or a distance field. Just like TOPs, e.g. the slopeTOP. There’s even a fluidT3D that allows you to do real-time fluid simulations with a few clicks. Compositing all these objects with simple operations brings countless possibilities!

Derivative: Can you talk about the technical underpinnings of T3D? What were some key considerations in its design and implementation?

Josef Pelz: Most T3D are really just a single glslTOP that executes a compute shader behind the scenes. Some are more complicated, for example the volumeT3D or the fluidT3D, but since I created most of these for my own work before, the biggest challenge was to seamlessly integrate them into TouchDesigner.

I spent days understanding and manipulating the mechanics behind the OP create dialog that you use to add a new operator to your network.

The goal was to make T3D as intuitive as possible and to make it look like just another operator family. 

Derivative: What has the response been from the community so far and have you seen it used publicly out in the wild and encountered any unexpected results? (One of the beautiful things of TouchDesigner is that tools are used in a way that they were not necessarily intended...)

Josef Pelz: The adoption of T3D in the community has been incredible for me. I’m still amazed every time someone experiments with it or uses it for their project. Torin Blankensmith and Tommy Etkin almost immediately built interfaces to the Shader Park plugin and RayTK respectively, allowing people to convert their results from these amazing tools to T3D. Mini UV definitely is a great inspiration to push T3D further as he’s creating marvelous projects with T3D and building on top of it.

T3D has been used for installations around the world and I’m grateful for all the users tagging me on their social media posts, it’s a joy to see all the creations!

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

A post shared by Mini UV (@_mini_uv)

 

Derivative: Wondering too if you drew inspiration to develop T3D from other developments like Tommy Etkin's RayTK or if there were other tools that may have been motivating? 

Josef Pelz: I didn’t look into how Tommy built RayTK, but seeing RayTK probably made me believe that such a toolkit/ addon could be built!

Derivative: What specific challenges did you face while developing T3D, and how did TouchDesigner’s capabilities help you overcome them? Or perhaps the question is if there are possible features in TouchDesigner that might simplify the development and deployment of such toolkits.

Josef Pelz: The most surprising challenges were those connected to the OP Create Dialog. I delved deep into /ui/dialogs/menu_op (you can copy this path in your TouchDesigner panebar to get there) and had to manipulate a lot of the processes there (a lot of which are still defined in ancient T-Script). All that is done behind the scenes when you drop the T3D.tox into your network editor. One thing that took extra work is the behavior when right-clicking the input of a T3D. You’d expect the OP create dialog to open with the T3D family selected, because you most likely want to connect another T3D to the input. But I couldn’t find a way to detect that the right-click was performed on a T3D. So what’s actually happening is that the mouse position is compared to all operators on the current network level and their input positions. Based on that I decide if a T3D input was right-clicked…

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

A post shared by Hayk Zakoyan (@zakoyanhayk)

 

Derivative: What advice would you give to developers or artists looking to create their own custom tools or extensions for TouchDesigner?

Josef Pelz: Go for it, it’s fun! And btw., the T3D COMP doesn’t really delete itself if you drop it into the network editor, it just moves itself to the root level of your project (/) and hides. You can right-click on the background of the network editor on the root level and select Display/Expose all. Beside the ui and sys components you’ll also find the T3D COMP. You can look into it and see how I set it up! But be aware that this isn’t built to be understood. There’s no reasonable commented code etc.

Derivative: Looking forward, what are your plans for T3D? Are there any exciting updates or new features in the pipeline?

Josef Pelz: I’m really happy with the current state of T3D. It’s still missing some specific operators like optical flow etc., but it’s quite powerful as it is. In the upcoming months I want to focus on making tutorials!

 

 

Derivative: Beyond T3D, what are some other tools or systems you've developed for TouchDesigner? How do they enhance the creative workflow?

Josef Pelz: I developed and released a cloth simulation tool some years ago, but I would change a lot about it nowadays, especially with respect to usability. I’m currently working on a more general physics simulation COMP that I built in POPs. It’s super fun to play with physics in real-time and I’m trying to make it as easy as possible to do that.

Derivative: Is there an opportunity for the community to contribute to the development and if so how?

Josef Pelz: Feedback, bug-reports and feature requests are integral to making T3D better. Thanks a ton to all those who are filing them!

Derivative: And to end with a super nerdy question, what interesting new math research have you come across in the recent months? 

Josef Pelz: I was super excited to read about Vertex Block Decent a novel approach to simulating physics that was published earlier this year. I’m currently building my newest POP simulator on XPBD, but looking forward to implementing VBD eventually!

Derivative: A big thank you to Josef for creating these invaluable tools for the community! For our readers interested in exploring T3D, you can access the latest version on Josef's Patreon.

 

Follow Josef: Website | Instagram | Patreon | Github