Optical Flow

Hi Markus,

Thanks, just having a very quick look, I don’t think it’s supposed to be blue, is it?

edit : I guess I was still half asleep it’s actually working as it should, just the parameters saved with the tox make it blue.

Hey guys,

Sorry to revive an old thread.

With this example, is it possible to pull out the vector information into chop channels.

I saw this implementation here: vimeo.com/114542945 and would love to be able to use the vector data for driving other animations / visualizations.

Cheers,

O.

Hey, yes you can always extract the vectors information using a TOPtochop, make sure to reduce image resolution and use the next frame mode, otherwise it can be pretty slow.

Vinz, I would love to see an example of how you might do that. I’ve been scratching my head all afternoon trying to work it out.

Hey Jason,

Sure I’ll post the setup in the next couple days.

Yes! indeed vinz , seeing an example of how to execute that would be great!

10 years later, here you go!

Cheers,
Vincent
optical_vectors_display.toe (6.96 KB)

Thank you, thank you, thank you! Will test it out when I’m back in the studio :smiley:

Hi Vinz

Is this meant to be the vectors of pixel movement like optical flow or something else? Maybe I just don’t get it, but if I connect a more simple source to this setup to see it more clearly(a circle top for instance, moving left to right) it’s showing vectors going in almost all directions, when I would expecct it to be only the axis of movement. Am I missing something or is the shader not actually designed to show this?

Hi, keep in mind the original optical flow shader by Andrew Benson which is used here is fast but a rough approximation, compared to more sophisticated but much slower algorithm found in opencv.

It doesn’t do too well on hard edges, and you’ll get better results if you blur your circle beforehand. Even if vectors are a bit off the overall direction is correct.

Cheers

Funny timing, just saw it was added to the palette, same algorithm though. (though the palette one needs its format forced to 16/32bit float or it will clamp results to positive values)

Build 2017.7180 - Jun 05, 2017
New Palette
opticalFlow added to Tools folder in Palette.

Thanks for pointing this out! Will be 32bit float texture in next release.

Best
Markus

Could somebody be so kind as to provide me an example of practical usage of this technique? Its inclusion in the palette suggests that this is of value, yet my previous comments about inaccuracy have led me to pigeon-hole it as an inaccurate, purely aesthetic technique. Is this correct?

Ha :wink: depends what you mean by “practical”, I’m mostly in this business to make pretty useless things to look at and play with, this is a recent example with this optical flow technique : instagram.com/p/BVSaPBonvKN … cent.houze

Also this one uses the same optical flow technique to push the fluid around : vimeo.com/218695680

love it - especially the ‘practical’ bit.

Hey Vincent - is there a reason you used texture with a calculated UV offset rather than textureOffset?

edit - forgot my new sig

thanks in advance. real money slots.

Thanks! I think historical reason for not using textureOffset, like this was probably adapted from an old shader.

Claude Monet and Optical flow:

vimeo.com/196743416

Oh man edge detection :bulb: ! Snap what a timely link. Was literally just flailing with a sig proc chain for all flow + particle physics and completely forgot about edge detection.

Sorry for waking up that thread, but I have a question regarding the Optical Flow example in 099 (and the toe/toxes in this thread). They surely generate some vectors, but the result is only the difference between frames and their horizontal/vertical gradients…

So it is not an actual Optical Flow / Motion Estimation detection like in Nuke or After Effects, that can potentially be used to slow down a movie. Has anyone tried to implement this technique in Touch so Frame Blending can be replaced with (realtime) Optical Flow Timestretching / Frame Interpolation?

This looks like a proper OpenCV implementation:
docs.opencv.org/2.4.5/modules/g … video.html

Hey,

that’s true that this glsl optical flow is a bit of a shortcut/hacky, though it gives pretty nice results for interaction.

I actually went ahead and tried the opencv gpu algorithms in a c++ TOP at the time I was doing research, but they were still much slower, so end up not using them.