Jardin d'Ete Effect

i am very inspired by the effect achieved here:
quayola.com/landscape-paintings/

does anyone have advise on how to try and recreate the look? my first instinct is to experiment with blurring and chroma key TOPs based on the process video found on the site. Are there any other techniques to look into or perhaps some articles on Computer Vision that explain the underlying mathematical concepts that inform this process?

Thanks!

Start with optical flow:
github.com/DBraun/TouchDesigner … alFlow.tox

Use the vector field as an input into a fluid simulation:
github.com/kamindustries/touchFluid
or
github.com/ancillarymagnet/touchfluids

In the color advection of the fluid simulation, tie the opacity of the color impulse to the magnitude of the optical flow velocity. Experiment with additional compositing/feedback in the color advection.

Hi,

it’s been a long time I saw people exploring this particular approach, but still no project file around. Can somebody be so kind to clear this out for non-programmers. What does it mean to “tie the opacity of the color impulse to the magnitude of the optical flow velocity”?

Thank you!

It basically means the strength (or amount) of the optical flow movement in the image is tied to controlling the amount of transparency in the color. Optical Flow is a technique used to calculate the motion in an image. There is an Optical Flow component in the Palette under the Tools folder.

Hi Niraah,

In the fluid simulator from Kurt Kaminski, the 3d input is color input. Simply connect your movie file in (or any other top) and play with the “color diffusion” parameter. (and connect the optical flow from David or from the Palette to the velocity input)

github.com/kamindustries/touchFluid (you only need the tox file)

Finally got around to playing with this effect on the Kinect

Thanks to David Braun and Kurt Kaminski for their advice and for sharing their techniques!!

Wanted to upload a file that demo-s a method of using the depth data from the Kinect as the input to an optical flow and fluid shader effect chain.

The attached file should unzip and run on a computer with a decent GPU - the file is commented, but still a bit convoluted - so if anyone finds it interesting, feel free to DM me about it with questions.
Optical_Flow_Kinect.zip (4.89 MB)

Hi everyone,
I’ve been playing with this technique lately but I think something isn’t working properly on my system.

I’m using the .tox from Kamindustries, and whenever I plug a video file in (1920x1080), it seems that most of the simulation is happening on a corner (see attached screenshot)

I’ve tried multiple videos and it’s always doing this.

If someone could help me I would appreciate it !

The OpticalFlow tox probably has a reset button on it. I think the Feedback TOPs inside are retaining an old resolution.

Hi DavidBraun !

I’ve tried the reset button and still the same result.
I’ve tried to look inside the tox, I check all nodes and they are all setup to use input resolution. Even the info does get my input resulotion correctly.
I’m still going to investigate, I’ll let you know if I found a solution.

What’s the output resolution of opticalFlow? Does it match the movie file in? Can you go through the TOPs inside fluid1 one by one and see where the resolution might be changing? Are you using non-commercial TouchDesigner which is limited to 1280x720?

The output resolution of the opticalflow is matching the input video (1920x1080 in my case).
I check all the TOPs inside, all of them are set on “use input resolution”
I’m using the commercial version of Touchdesigner.

I noticed that the problem appears when I hit the reset button (not the init one). When I drag the tox in my comp and plug my video it works well. But as soon as I start to tweak settings and press reset it goes like on the screenshot I sent.

Hi there,

I’m having the same issue with the simulation happening in the bottom corner. Did anyone ever find a solve for this?