Houdini point wrangle

Hey everyone,

I’ve been entrenched in the Houdini world for the last year, and I’m just getting back to Touch. I’m also now realizing I want to integrate some of my Houdini methods into Touch. The main thing that I’ve been missing in the Point Wrangle.

For anyone not familiar with Houdini, it’s a little snippet of super fast code that works on all the points of a SOP on the GPU. Which means it’s very fast and efficient.

Unless I’m mistaken, the closest thing that exists in Touch in GLSL, and it has a similar method of working on each pixel concurrently.

So this is a way to easily add a snippet of GLSL code where you only have to pay attention to the parts you care about.

At the moment it works with any float, vec2, vec3, or vec4 attribute. Just prepend the attribute name with ‘@’ in the code

Some example code:

// This will make your SOP deform in a wave
@P.y += sin(@P.x)

// If you set up some variables, you can now have it animated with control of speed and amplitude
@P.y += sin(@P.x + absFrame * speed) * amp;

// Color your model based on it's position
@Cd = @P;

The easiest way to figure out what’s going on is to download the attached toe file.

If you are setting it up from scratch, then do the following

  1. Add code to a Text DAT

@P.y = sin(@P.x*10 + absFrame * speed) * amp; @N = vec3(1,0,0)

  1. Add any variables you want to use to a Constant CHOP (like absFrame, speed, amp, etc.) There is a VARIABLES chop that is included in the toe file

  2. After changing variables, go to the Wrangle BASE parameters. There will be a Wrangle tab. Click on the Setup Variables button.

Ongoing problems
The main thing I’m unsure of is the efficiency of the method. You feed it a SOP and a text DAT with your code, and it will do a SOP → CHOP → TOP (with GLSL pixel shader applied) → CHOP → SOP

Is this a viable method? I’m sure there’s some GPU → CPU conversions that I’m unaware of that will slow the whole thing down.

Take a look at the example files, and let me know if there’s anything big reason this won’t work, or any suggestions you have.

UPDATE:
The wrangle will now automatically split incoming geo into sections to get around a resolution limit of the GLSL TOP. Thanks to flowb for the starting point of splitting the geo up

Inside the tox, you can find the logic for applying the GLSL code at wrangle / splitGeo0 / applyGLSL

UPDATE 13-03-18:
Added variables
The tox will also now output your final code as well as any errors that may be reported.

UPDATE 15-03-18
Super useful update.
This should now work with any vector3 attribute

UPDATE 22-03-18
Now works with float, vec2, vec3, vec4 attributes
Fixed major bug where an attribute could not be calculated bsaed on another attribute

I’ve attached a toe and a tox file.
I highly recommend using the toe file to see how everything is set up

Enjoy,
Adam
point_wrangle-v1.5.tox (15.7 KB)
wrangle.toe (40.3 KB)

I’m learning Houdini these days… Well, I am actually learning programing and doing it in H… and I absolutely LOVE the wrangler! I have been dreaming of a way to manipulate geometry like this inside of TD, without needing to learn yet another programming language like GLSL… and here you are :wink:
I couldn’t be happier for this work that you have started! And if this was somehow officially implemented by Derivative one day I would be even happier, because then we could start creating geometry with these wranglers as well… A nice cozy and friendly wrapper around GLSL :stuck_out_tongue:

I am also curious hearing about potential bottle-necks in your set-up…

Keep it up! I am looking forward to see your progress!

Glad you like the idea, but it’s still super basic.

First off, the code is still GLSL, so no way around learning that. I’ve just separated away all the code that you need to have in the final shader, and made a simpler way to add some code there.

As for bottlenecks, I see a couple major ones. First, it’s all hard-coded to position right now. I’m working on finding a way that will let you use arbitrary attributes, but I’m definitely not clear on exactly how that will work yet.

Second, I am in no way certain that this method can actually scale. There’s a good chance the progression from SOP to GLSL and back to SOP is still getting bottlenecked with the CPU but I haven’t started performance testing it to know for sure.

I’m sure someone here who know Touch deeper than me could fill me in on where I’m going to hit CPU limits.

Also, there is a limit of 32,768 vertices, which I assume is coming from the limit of the GLSL TOP.

So in any case, there’s a lot of things to test with, but hopefully it morphs into something usable

That’s crafty :slight_smile:

I guess my main concern about performance is the TOPtoCHOP and CHOPtoSOP chain. I’d expect it to be more optimized if you wrote a GLSLMAT and used the vertex shader instead of converting your frag data to geometry through a series of CPU operations.

I really like the idea of dynamically generated GLSL code though, I think that concept has some really interesting possibilities. Great stuff

Thanks flowb, that’s kinda what I figured would be the bottleneck.

Regarding the GLSL mat, that’s the first pathway I went down. But as far as I can tell, if you assign a GLSL mat to a geometry node, it updates the geometry, but you can’t get the data back out of it as a SOP again.

Do you know of a way to convert a Geometry node with a vertex shader back into a regular SOP?

Right.

Now that you mention I don’t think I thought that through enough.

Even with the new Alembic and C++ SOPs in the experimental branch, you can’t freely mix CPU and GPU based operators. So if the intent is to be able to use CPU based SOPs freely in the chain then what you’re doing there makes a lot more sense.

I added a camera, lighting and render setup to your project, and the OP conversion actually seems like a negligible hit. at least at these low vertex counts.

I built a setup to slice SOPtoCHOP and chopTOtop rows into larger textures a while ago when i needed to do more than 16384 vertices on another project. I want to dig it up and try to work it into your setup to see how it behaves at higher vertex counts.

ok, good to know I’m on the right track.

I’d be interested to see your slicing method. That seems like it would be very helpful for larger point counts.

It’s just some math and replicators that trims through the chop data. sadly no special sauce.

in order for it work here, I was trying to make a complementary comp do reassemble the geometry after the glsl stuff. I seem to be having some issues getting the points back together right, but this at least adds support for more vertices.

I’m getting about 30fps at 130000 vertices. Looks like the Chop to Sops are causing most of the performance problems.
wrangle.12.toe (31.7 KB)

Cool, thanks. I’ll take a look and see if I get any luck with reassembling the geometry.

When you first made this, did the reassembly work? Is it just the GLSL stuff that’s messing that up?

the previous project used the slicing to get more input vertices. This was feeding a GLSL particle system, so there was no need to re-assemble the sliced up texture data into SOPs.

ok, that makes sense. I have an idea of how to reassemble then, I’ll give it a go

Good news!

I’ve gotten the splitting to work by embedding some of your nodes into the wrangle tox.

Basically, I use your row/column logic to drive a delete node inside a replicator. That deletes sections of the points of the incoming geo, which then have the GLSL applied to them individually. Then the resulting CHOPs are joined back together and used in the CHOP to SOP node.

This doesn’t really improve performance at all, but at least the polys are all coming out correctly now.

awesome.

if you post it i’d love to see the results. :slight_smile:

Yep, I updated the link on the first post of this thread

Another update with files on the first post:

I’ve now added variables that you can use and pass in to the GLSL code inside the wrangle.
The wrangle now accepts a CHOP input which can be filled with variable names / values. These names will get passed to your GLSL and you can use them straight in your wrangle code.

The tox will also now output your final code as well as any errors that may be reported.

Really important update. I’ve got this working now for any vector 3 attribute, not just position like before.

Take a look at the updated files on the first post of this thread to see how it all goes together.

I think every update I make to this is going to feel important.

Anyway, the tox has now been updated with 2 very important changes.

First, it works with any attribute that is a float, vec2, vec3, or vec4
Second, it fixes a major bug where attributes couldn’t depend on each other (like @Cd = @P)

Thank you so much for making this happen! I am also a transitional touch user from years of houdini heavy vex user.
However, anytime I try to changed WRANGLE_CODE in the text DAT, my touch crashed. I am wondering if you experienced the same issue before.

Much appreciated.

Hmm, that shouldn’t happen. But actually I haven’t tested this with the newest release, so maybe something has broken. I’m away from my computer for the next few weeks, but when I get back I’ll open this up and try to fix it.

What version of Touch are you using?

Can you send the crash .dmp files to support and we’ll find out why its crashing, it should not crash even if the project needs to be updated to work correctly.

Also, which specific build are you using, I will try to reproduce your crash.