Optical Flow

Shared .tox Components
26

Hello,

Trying out this, it looks like it broke again at some point, glsl1 is looking for a camera_out TOP that does not exist.

But else if I want to do motion blur on 3d objects, I’ll guess I’ll get better results if I try to output a velocity pass from the render as well?

I guess there have been some refinements since the classic stupid opengl tricks pdf (2003!), but it’s probably a good starting point.
nvidia.com/docs/IO/8230/GDC2 … Tricks.pdf

27

The camera_out has always been missing from the .tox, even in the original. The fact it’s not using a Feedback TOP and has a loop in it’s network is an issue also possibly.

28

I’ll dig the optical flow thing more later.

Fore now I was able to get the one from the pdf working (which is also in latest nvidia sdk docs.nvidia.com/gameworks/index. … sample.htm)

I get nice results but since it’s using normals to stretch meshes it doesn’t work with flat or hard edge geometry, so I’m trying to get the advanced sample to work now.
docs.nvidia.com/gameworks/index. … sample.htm

29

Are there any updates on this working in 088?
thanks

30

I wrote a research paper on hardware accelerated motion blur in 2003 for 3D using the vertex data. There is a pdf and video on my website.

clementshimizu.com/hardware- … eneration/

This full image space optical flow is very exciting.

Clement

31

Hey,

just spend a little time with this and fixed it up to work (or so I believe) in the experimental branch of 088.

All Parameters are now on the component as custom parameters.
I think this is what it’s suppose to do - feedback welcome.

cheers
Markus
opticalFlow.jpg
opticalFlowFx.tox (4.01 KB)

32

Hi Markus,

Thanks, just having a very quick look, I don’t think it’s supposed to be blue, is it?

edit : I guess I was still half asleep it’s actually working as it should, just the parameters saved with the tox make it blue.

33

Hey guys,

Sorry to revive an old thread.

With this example, is it possible to pull out the vector information into chop channels.

I saw this implementation here: vimeo.com/114542945 and would love to be able to use the vector data for driving other animations / visualizations.

Cheers,

O.

34

Hey, yes you can always extract the vectors information using a TOPtochop, make sure to reduce image resolution and use the next frame mode, otherwise it can be pretty slow.

35

Vinz, I would love to see an example of how you might do that. I’ve been scratching my head all afternoon trying to work it out.

36

Hey Jason,

Sure I’ll post the setup in the next couple days.

37

Yes! indeed vinz , seeing an example of how to execute that would be great!

38

10 years later, here you go!

Cheers,
Vincent
optical_vectors_display.toe (6.96 KB)

39

Thank you, thank you, thank you! Will test it out when I’m back in the studio :smiley:

40

Hi Vinz

Is this meant to be the vectors of pixel movement like optical flow or something else? Maybe I just don’t get it, but if I connect a more simple source to this setup to see it more clearly(a circle top for instance, moving left to right) it’s showing vectors going in almost all directions, when I would expecct it to be only the axis of movement. Am I missing something or is the shader not actually designed to show this?

41

Hi, keep in mind the original optical flow shader by Andrew Benson which is used here is fast but a rough approximation, compared to more sophisticated but much slower algorithm found in opencv.

It doesn’t do too well on hard edges, and you’ll get better results if you blur your circle beforehand. Even if vectors are a bit off the overall direction is correct.

Cheers

42

Funny timing, just saw it was added to the palette, same algorithm though. (though the palette one needs its format forced to 16/32bit float or it will clamp results to positive values)

Build 2017.7180 - Jun 05, 2017
New Palette
opticalFlow added to Tools folder in Palette.

43

Thanks for pointing this out! Will be 32bit float texture in next release.

Best
Markus

44

Could somebody be so kind as to provide me an example of practical usage of this technique? Its inclusion in the palette suggests that this is of value, yet my previous comments about inaccuracy have led me to pigeon-hole it as an inaccurate, purely aesthetic technique. Is this correct?

45

Ha :wink: depends what you mean by “practical”, I’m mostly in this business to make pretty useless things to look at and play with, this is a recent example with this optical flow technique : instagram.com/p/BVSaPBonvKN … cent.houze

Also this one uses the same optical flow technique to push the fluid around : vimeo.com/218695680