saving gpu ram!

most of our shows require multiple scenes, but that can be a problem if you’ve only got 1gb of gpu ram, or if your scene has a lot of textures and/or TOP usage.

according to the derivative team, the only way to reclaim GPU ram (for the time being!) is to lower the resolution of a TOP. i thought i’d share a couple of scripts that can be used to recursively set the resolution of every TOP in a network to 4x4. it also saves the original resolutions in a table, which are then referenced again when you want to re-enable that particular part of the network.

STORE_RESOLUTIONS:

if (`opexists("resolutions")`==0) then
	opadd -n DAT:table resolutions
endif
tabdelete resolutions a
cc ../`opname("..")`
set path = ""
foreach o (`execute("lc -R")`)
	set fpo = `strcat($path,$o)`
	if (`strmatch("*:",$o)`==1) then
		set path = `strcat(substr($o,0,strlen($o)-1),"/")`
	else if (`strmatch("TOP*",optype($fpo))`==1) then
		set r1 = `strcat($fpo,"/resolution1")`
		set r2 = `strcat($fpo,"/resolution2")`
		tabinsert resolutions r $fpo `par($r1)` `par($r2)`
		opparm $fpo resolution1 4
		opparm $fpo resolution2 4
	endif
end

(i would also run this afterwards to stop cooking as well!)

opset -c off .

RESTORE_RESOLUTIONS:

for i=0 to `tabnr("resolutions")-1` do
	opparm `tab("resolutions",$i,0)` resolution1 `tab("resolutions",$i,1)`
	opparm `tab("resolutions",$i,0)` resolution2 `tab("resolutions",$i,2)`
end

interesting, but can you explain a bit more (to newbies in coding and first-time-using DATs) - how to import the code in touch and to use it?

This is great - thanks. Scripting stuff from scratch is not my strong point, but with this as a starting point I was able to bodge together something that monitors the total cook time for all operators inside a container.

It’s still pretty rough, but with a bit more work could be very handy to have in the pallette.

The script currently throws bad path arguments as it is processing some stuff that doesn’t have a “cook_time” property. This doesn’t seem to affect the output though, try laying down a hog chop and changing it’s delay to test it out.
performance_analyser.tox (1.58 KB)

I’d love to see an example of this in a .toe file…or better yet implemented in python as well…its something Id love to be able to do!

I’ve been working through getting back some GPU ram using this idea.

Here’s a sample network of what I’m doing with python scripts.
bypass all ops.toe (5.26 KB)

Thanks to some great feedback from Matt Daly, Keith Lostracco, and Richard Burns I made a few changes to this example. In this version:

Bypass On -
Sets the resolution of any render TOPs to 1x1.
Force cooks all render TOPs once.
Turns on the bypass flag for all TOPs (render TOPs included)

Bypass Off -
Sets the resolution of any render TOP to 1280 x 720 (it’d be better to pull this from storage, or a table, it’s just hard coded for the sake of the example)
Turns off the bypass flag for all TOPs.
bypass all TOPs.5.toe (7.47 KB)

Last time… updated this to pull the resolution from the storage of the containing COMP.
bypass all TOPs.6.toe (7.52 KB)

Heaven help me… I can’t stop my self in this quest for optimization. Looking at a messy implementation I wrote some python functions to help make this easier.

Hope this helps other folks who are moving to python in their scripting:

def deactivateCue(TOPs, renderTOPs):

	for i in renderTOPs:
			exec("op('" + str(i) + "').par.resolution1 = 1")
			exec("op('" + str(i) + "').par.resolution2 = 1")
			exec("op('" + str(i) + "').cook()")

	for i in TOPs:
		exec("op('"+ str(i) +"').bypass = True")

	return

def activateCue(TOPs, renderTOPs, runRezX, runRezY):

	for i in TOPs:
		exec("op('"+ str(i) +"').bypass = False")
	
	for i in renderTOPs:
		exec("op('" + str(i) + "').par.resolution1 = " + str(runRezX))
		exec("op('" + str(i) + "').par.resolution2 = " + str(runRezY))

	return	


# for deactivateCue
TOPs = op('base1').findChildren(type=TOP)
renderTOPs = op('base1').findChildren(type=renderTOP)

# for activateCue
runRezX = me.fetch('renderRezX')
runRezY = me.fetch('renderRezY')
renderTOPs = op('base1').findChildren(type=renderTOP)
TOPs = op('base1').findChildren(type=TOP)```

[bypass all TOPs.1.toe|attachment](upload://8bNCoyxHzPHq0psmznTab0v8wkd.toe) (7.53 KB)
1 Like

Hi @raganmd
Just stumbling upon this thread now. Looks like a good approach to dynamically bypass sections of the network not in use.

Nowadays would you still use this approach? Or have you discovered something better in the last 9 years?

Looking back, I would not use this technique again.

At the time it was really useful, but it’s a very difficult way to work and makes managing your network much more challenging.

These days I instead use movie players that let me re-use moviefilein TOPs, and then I try to think about how my scenes have presets or looks that can be recalled rather than building whole new networks.

Is there something in particular you’re trying to optimize around?

I see… thanks for the feedback.

I instead use movie players that let me re-use moviefilein TOPs

This is exactly what I am doing with my movies.

Is there something in particular you’re trying to optimize around?

I have quite a lot of other “scenes” in my project, that are not movies but unique generative content generators. My system plays only 1 scene at a time (max 2 when cross-fading from one to the next). Standard scene switcher stuff… but each one is quite dynamic with a lot of Chops (LFOs, selects, etc) and parent param references (MIDI controller & audio-reactivity data) which cause a lot of cooking even when the scenes are not used. Since Base comps don’t have a bypass mechanism, I thought I would maintain an “active” param on my scene Bases and toggle it off when the scene is switched, which would recursively bypass any children Ops causing cooking (making this determination based on Op types or a tagging convention or something).

I’ve posted other details here: Base Comp - Cooking Optimization

For more context…
I’ve basically built a sort of VJ-system driven by Ableton/TDAbleton using a Push controller for performing live. Now I’m trying to optimize is so that resource usage doesn’t increase proportionally with the number of scene Bases I add.
With TD being a pull-based system, from what I hear, I should be able to theoretically add dozens of these scenes without any overhead. However I don’t see how any reasonably complex performance system could do so without implementing something like you mentioned in your original posts… due to the way Param refs and some Chops always cause cooking.

One feature that’s underused IMO is locking operators. Locking them holds their content, but will prevent down-stream cooking. This allows the op to be initialized and grab any necessary memory, but will prevent changes.

One direction to consider would be to us a combination of tags and locking to help control your unwatned cooking. For example, you can locate ops inside of a base by using findChildren() which accepts a tags argument. You could tag the ops that you know need to be stopped from cooking, and lock them when the scene isn’t playing, then unlock them before you transition into the scene.

This will have a similar behavior to bypassing and un-bypassing, or disabling and enabling ops but will ensure that you don’t have to re-allocate memory and get unexpected behavior.

The other big challenge is that some ops have to always listen for changes - for example, the MIDI device in CHOPs or OSC in CHOPs. Since they’re always cooking that can have unexpected / undesirable results down stream. Ways I’ve solved this problem would be to address this in your routing. For example, you might use a constant CHOP in your network that represents your default values for a scene, and a select CHOP that’s pulling your midi data.

You can then gate the stream of your midi data by using a switch CHOP that’s watching a custom par like “Live” which you can use for transitioning to the live values.

That’s similar to that locking idea, but reduces some of your controls to a single flag to change.

In the past when I’ve built more complex systems I’ve used this “Live” par idea as a way to ensure that I run any changes to prep a scene to be live, and for any shut-down or locking mechanics that need to happen when navigating away from the scene.

These days I’d use some combination there to ensure that I stopped cooking for scene after I left it for another one in my set.

One other piece you should also consider for your projects would be to force-cook all of your ops at start-up. There’s a non-zero chance that some op will not have cooked when you transition into a scene and will need to cook for the first time. Force cooking all of your ops at startup will give you a better sense of your actual resource utilization, and ensure that you’re not cold transitioning into any of your scenes.

The other idea you might also explore would be to limit your render networks - similar to moviefile ops these each hold memory, and even if not cooking it’s hard to get memory back from them when you’re not using them. If you instead use an A / B style system you can limit some of the GPU overhead you might experience by having lots of bases with many render networks in them.

Thanks for the feedback!

Yeah that was exactly the strategy I had in mind, but with bypass instead of locking. Locking sounds promising… I’ll give that a shot.

I’ve already started doing the default + switch trick and it’s working well. I just wish there was a simpler way instead of having this boilerplate stuff everywhere.

That’s an interesting idea, if I’m following correctly. So you mean using only 2 A/B Render TOPs and swapping out the Geo, Camera, etc on scene change? Also sounds promising.

Yep - here’s an old example -
container_optimized_render_system.tox (8.8 KB)

Scenes are located here container_optimized_render_system/base_realtime/base_scenes

The script that changes scenes is here:
container_optimized_render_system/container_ui/button_master1 - which is replicated in each button:

I’d probably make some changes these days - this example is several years old now - but the idea still holds. You don’t need a render TOP in your network to build a scene, which is very handy. The hard part with this idea is that if you’re doing post process work that’s specific to a scene then you need mechanics to that render correctly. Similar to the other idea, I’d move that away from being in the scene, and instead treat that as a reusable resource that you could swap content in or out of depending on what your scene wanted / needed.

1 Like

My main template for building projects these days is based off of Jarrett’s SceneChanger in the palette, but I’ve hacked in proper callbacks, and am doing some optimization nonsense to get around the problem of these forced cooks in my scenes.

@Jarrett’s scenechanger optimizes things by only requesting one scene at a time, so that the pull based system can handle not cooking things that aren’t being referenced anywhere. This is great, but it completely falls apart when you need any kind of external animated data in your network (midi, osc, lfos, tempo information etc).

My method reverses what he’s doing on the output side with his sceneGate, by creating a dataGate at the beginning of each scene. Essentially, any parameter that would be affected by animated data gets published to the parent, then pulled into the scene with a parChop, and termindated by null_data, which is what all of my internal scene operators reference. onExit callback, null_data gets written to a table which is then dattochop’d back into channels, with a switch that gets turned off on exit, and back on agian when the scene is called. This is so that none of my references break when I exit my scene, but instead of always cooking animated data in the parent, they only see static placeholder channels at their most recent value when the scene was exited, so cooks aren’t forced in the scene.

Once the desired scenePars are published to the parent, I can animate them with global LFOs and beatCHOPs, osc and MIDI and even though the values stay animating in the parent when the scene is exited, that signal never gets to the operators inside to make them work when they’re not supposed to. Sometimes I have to do tricky things like creating switches to route around hungry timesliced operators like lags or analyze ops that want to always cook, but this optimization allows for really large projects without having to worry too much about bypassing or resizing anything

Following this paradigm, I’ve had projects with dozens of scenes, some of which are quite heavy, but I can still cruise at a reasonable framerate.

I showed this off a bit at the TD Round Table in Berlin last year, you can hear me yammer a bit at the end of this video. I’ve made a lot of improvements since then, and am hoping for a more public release sometime next year. https://www.youtube.com/watch?v=N2ZzG1yz2K4