Since we're all stuck indoors for a month or two I figured now is probably a good time to get started on SimpleMixer 2. I've started a new repository so anyone can follow along with the development and make suggestions here:
https://github.com/Richard-Burns/SimpleMixer-2
To begin development I'm going to tackle two main issues with SimpleMixer 1.
- Hangs happen when loading clips
- We are limited to two layers and only one set of post effects for all layers.
- The parameter interface should be a set of mappable UI elements, not a standard parameter window.
Lets start with our first two issues. A good way to alleviate the layering and post effects system is to go for a Matrix grid launcher system similar in style to what you might find in Resolume where each cell is a tox or movie clip and each cell can have post effects applied to it.
I looked into this a few months back with the idea being to use a List COMP as that would be the most efficient however there was a bug with background TOPs where they would freeze when the List COMP wasn't being interacted with. That bug is now fixed and so began my task of creating a grid UI system that gives us the functionality we need.
Building a matrix grid launcher
I settled on the following structure for the grid launcher. The yellow rectangles are the List COMPs cells. Each column will in theory run in it's own Engine component.
As you can see above we have multiple cells for one clip yet our Layer section has a whole bunch of elements in one List COMP cell. The List COMP doesn't allow us to split cells this way so we need to find a workaround for the alpha fader etc in that cell. The logic must be programmed into that cell using its u and v callbacks and then a background TOP must be the visualizer for those components. I hate replicators because they're very good at causing trouble but this is a fairly simple use case so we'll go for that and generate those UI elements into a single TOP we can then use as the background image.
A single table decides which cells are active. Clicking on the column triggers that entire column to be active and de-activates the other cells. Clicking on a clips label cell then triggers that clip. Clicking the top left cell de-activates all clips. This is the basic functionality required for now. We want to use an engine component for each column. This would allow us to load clips into a column, wait for them to load and then fade through to that column when all our clips are ready. This is where our engine comp comes into play.
An overview of the Engine Component
As most people have probably seen the Engine Component has been in quite a few TouchDesigner builds for a long while however only in the last couple of builds would I say it was "fairly" stable and therefore ready for use in production.
In the past when building systems that require multiple scenes split into tox files there is always the issue of TouchDesigner hanging up because everything is running in the main process. The Engine COMP seeks to alleviate this by spinning up a separate process and doing all the working in there thus leaving your main process to handle your UI and timeline etc.
The Engine COMP takes inputs in the form of custom parameters and input operators and outputs via output operators so if you have a visual in the form of a tox file like we would use in SimpleMixer then we could put that visual in an Engine COMP and it'd load up just fine.
Where things get tricky
We want to have each column in it's own engine. This becomes tricky though as we may have multiple tox files in a single column. If we have 5 layers then we need to load up to 5 tox files inside that engine. Furthermore we also need to access our tox files parameters.
The engine COMP currently doesn't support DAT input as far as my research shows and it also doesn't support promoting parameters to the top level. It does however support the networking in/out DATs. So we can use Touch Ins and Touch Outs on localhost to communicate with our tox files.
So the theory becomes this:
This is just a little bit crazy...but right now it's the only way I can think of getting dynamic Tox support into the Engine COMP.
The UI tells the engine COMP which toxes to load. The Engine COMP then loads those toxes and gets their parameters via a parameter DAT. This then feeds back the UI and the UI then creates it's parameter interface. The interface then sends those parameters back to the engine to control each tox in the column.
Generating Parameters
Widgets has an Auto-UI component for automatically generating parameters from components so this could be utilized to automatically create our controls. The problem is when it auto-generates these components it hangs the system...and the entire point of us doing this is to stop the system hanging. There is a widget cache component that may help with this but currently I'm either using it completely wrong or it's broken.
Join me next time for the fun of compositing all of this together, switching cells around and generally dealing with all the fun quirks of building an interface.
Feel free to download the repo to see how the grid launcher UI will be.