Optical Flow

Yes! indeed vinz , seeing an example of how to execute that would be great!

10 years later, here you go!

Cheers,
Vincent
optical_vectors_display.toe (6.96 KB)

Thank you, thank you, thank you! Will test it out when I’m back in the studio :smiley:

Hi Vinz

Is this meant to be the vectors of pixel movement like optical flow or something else? Maybe I just don’t get it, but if I connect a more simple source to this setup to see it more clearly(a circle top for instance, moving left to right) it’s showing vectors going in almost all directions, when I would expecct it to be only the axis of movement. Am I missing something or is the shader not actually designed to show this?

Hi, keep in mind the original optical flow shader by Andrew Benson which is used here is fast but a rough approximation, compared to more sophisticated but much slower algorithm found in opencv.

It doesn’t do too well on hard edges, and you’ll get better results if you blur your circle beforehand. Even if vectors are a bit off the overall direction is correct.

Cheers

Funny timing, just saw it was added to the palette, same algorithm though. (though the palette one needs its format forced to 16/32bit float or it will clamp results to positive values)

Build 2017.7180 - Jun 05, 2017
New Palette
opticalFlow added to Tools folder in Palette.

Thanks for pointing this out! Will be 32bit float texture in next release.

Best
Markus

Could somebody be so kind as to provide me an example of practical usage of this technique? Its inclusion in the palette suggests that this is of value, yet my previous comments about inaccuracy have led me to pigeon-hole it as an inaccurate, purely aesthetic technique. Is this correct?

Ha :wink: depends what you mean by “practical”, I’m mostly in this business to make pretty useless things to look at and play with, this is a recent example with this optical flow technique : instagram.com/p/BVSaPBonvKN … cent.houze

Also this one uses the same optical flow technique to push the fluid around : vimeo.com/218695680

love it - especially the ‘practical’ bit.

Hey Vincent - is there a reason you used texture with a calculated UV offset rather than textureOffset?

edit - forgot my new sig

thanks in advance. real money slots.

Thanks! I think historical reason for not using textureOffset, like this was probably adapted from an old shader.

Claude Monet and Optical flow:

vimeo.com/196743416

Oh man edge detection :bulb: ! Snap what a timely link. Was literally just flailing with a sig proc chain for all flow + particle physics and completely forgot about edge detection.

Sorry for waking up that thread, but I have a question regarding the Optical Flow example in 099 (and the toe/toxes in this thread). They surely generate some vectors, but the result is only the difference between frames and their horizontal/vertical gradients…

So it is not an actual Optical Flow / Motion Estimation detection like in Nuke or After Effects, that can potentially be used to slow down a movie. Has anyone tried to implement this technique in Touch so Frame Blending can be replaced with (realtime) Optical Flow Timestretching / Frame Interpolation?

This looks like a proper OpenCV implementation:
docs.opencv.org/2.4.5/modules/g … video.html

Hey,

that’s true that this glsl optical flow is a bit of a shortcut/hacky, though it gives pretty nice results for interaction.

I actually went ahead and tried the opencv gpu algorithms in a c++ TOP at the time I was doing research, but they were still much slower, so end up not using them.

Thanks Vincent!

True, probably they would need to run in low-res… But since Touch supports OpenCV natively (if I understand correctly) I hoped for an easy solution. Will you be in Berlin as well?

Best, Robert

Hey,

Sorry for the slow response! I missed Berlin, hopefully next year!

Regarding your statement of native openCv in TD it’s not entirely true.
It’s supported in python in TD, but there’s no easy/fast way to share a TOP to python at the moment, so you can’t do much for a movie or live video input.
I think some people managed it in a pretty hacky way using spout for python, haven’t tried.
youtube.com/watch?v=9g6SBwNlw7E

And then I’m not sure python opencv in TD is built with the gpu accelerated modules in all case, so it would probably be pretty slow.

Cheers

Thank you for pointing out the spout example, looks interesting and very experimental :wink:

What’s up optical flowers? :laughing:

I’ve posted a port of Thomas Diewald’s PixelFlow Optical Flow component ([url]https://github.com/diwi/PixelFlow/[/url]) to my GitHub repo
[url]https://github.com/DBraun/TouchDesigner_Shared/tree/master/TOPs[/url]

Video here:
[url]https://vimeo.com/285731980[/url]

Thanks David!