Page 1 of 1

How to create Morph cut transitions between images?

PostPosted: Sun May 26, 2019 4:56 pm
by garyglitter
Hi all,

I've been wondering if there was a way to create an interpolation-esque transition between 2 or more sources similar to that of Davinci Resolve's "smooth cut" video transition? (See something like this: https://www.instagram.com/p/Bxv2HcgHJ3_/ )

Essentially I'm trying to run a series of images I created with a GAN and use Touchdesigner to sequence them with a morph effect between each of the images.

Can anyone nudge me in the right direction?

:^)

Re: How to create Morph cut transitions between images?

PostPosted: Mon May 27, 2019 7:36 am
by greenpattern
+1 for the morphing!

Re: How to create Morph cut transitions between images?

PostPosted: Tue May 28, 2019 12:19 pm
by greenpattern
Apparently its not easy thing to do , even if you do it in after effects it take some time to tell the computer stretch that thing towards this direction, I already asked question before if you search the forum.

But as a consolation I'd have a play with that tool in the palette called 'pixelrelocator' in image filter section. you can animate the noise...maybe if you fade between pictures feed it in pixel relocator? I know its miles away but still something.

Re: How to create Morph cut transitions between images?

PostPosted: Tue May 28, 2019 1:50 pm
by matthewwachter
You can kinda fake it using GLSL. Here are a couple methods:

Morph1
morph1.gif
morph1.gif (535.39 KiB) Viewed 1924 times

Code: Select all
uniform float progress;
uniform float morph_strength;

vec4 getFromColor(vec2 uv){ return texture(sTD2DInputs[0], uv); }
vec4 getToColor(vec2 uv){ return texture(sTD2DInputs[1], uv); }

vec4 Morph1(vec2 uv)
{
   vec4 ca = getFromColor(vUV.st);
   vec4 cb = getToColor(vUV.st);

   vec2 oa = (((ca.rg+ca.b)*0.5)*2.0-1.0);
   vec2 ob = (((cb.rg+cb.b)*0.5)*2.0-1.0);
   vec2 oc = mix(oa,ob,0.5)*morph_strength;

   float w0 = progress;
   float w1 = 1.0-w0;

   return mix(getFromColor(vUV.st+oc*w0), getToColor(vUV.st-oc*w1), progress);
}

layout(location = 0) out vec4 fragColor;
void main() {
   vec4 o = Morph1(vUV.st);
   fragColor = TDOutputSwizzle(o);
}



Morph2
morph2.gif
morph2.gif (547.46 KiB) Viewed 1924 times

Code: Select all
uniform float progress;

vec4 getFromColor(vec2 uv){ return texture(sTD2DInputs[0], uv); }
vec4 getToColor(vec2 uv){ return texture(sTD2DInputs[1], uv); }

vec4 getFromColorBias(vec2 uv, float bias){ return texture(sTD2DInputs[0], uv, bias); }
vec4 getToColorBias(vec2 uv, float bias){ return texture(sTD2DInputs[1], uv, bias); }

vec4 Morph2(vec2 uv)
{
   float rad = mix((1.0 - (1.0-progress)), progress, progress);
   rad = mix( rad, progress, abs(2.1*progress - 1.0) - 0.1 );
   float bias = 5.0;
   float scale_x = 0.02;
   float scale_y = 0.02;
   float stretch = 0.02;

   vec2 perturb;
   vec2 slope;
   vec4 color;

   vec4 pd_bl_a = getFromColorBias( vec2( -scale_x, -scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_bl_b = getToColorBias( vec2( -scale_x, -scale_y ) + uv, bias ) * progress;
   vec4 pd_tr_a = getFromColorBias( vec2( scale_x, scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_tr_b = getToColorBias( vec2( scale_x, scale_y ) + uv, bias ) * progress;
   vec4 pd_tl_a = getFromColorBias( vec2( -scale_x, scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_tl_b = getToColorBias( vec2( -scale_x, scale_y ) + uv, bias ) * progress;
   vec4 pd_br_a = getFromColorBias( vec2( scale_x, -scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_br_b = getToColorBias( vec2( scale_x, -scale_y ) + uv, bias ) * progress;

   vec4 from = getFromColorBias( uv, bias ) * (1.0-progress);
   vec4 to = getToColorBias( uv, bias ) * progress;

   vec4 d = (pd_tl_a + pd_tr_a + pd_bl_a + pd_br_a + from * 2.0
   -(pd_tl_b + pd_tr_b + pd_bl_b + pd_br_b + to * 2.0 ));
   float diff = d.r + d.g + d.b;
   vec4 sx = ((pd_tl_a + pd_tl_b + pd_bl_a + pd_bl_b)
   - (pd_tr_a + pd_tr_b + pd_br_a + pd_br_b));
   vec4 sy = ((pd_bl_a + pd_bl_b + pd_br_a + pd_br_b)
   - (pd_tl_a + pd_tl_b + pd_tr_a + pd_tr_b));
   slope.x = sx.r + sx.g + sx.b;
   slope.y = sy.r + sy.g + sy.b;

   float p_len = dot( slope, slope ) + 1.0;
   perturb = vec2( slope.x / p_len, slope.y / p_len ) * diff;

   vec2 pert_to = perturb * (1.0-rad) * stretch;
   perturb *= rad * stretch;

   vec4 col1 = getFromColor( uv + perturb ) * (1.0-progress);
   vec4 col2 = getToColor( uv - pert_to ) * progress;
   return col1 + col2; //mix(col1, col2, progress);
}

layout(location = 0) out vec4 fragColor;
void main() {
   vec4 o = Morph2(vUV.st);
   fragColor = TDOutputSwizzle(o);
}

Re: How to create Morph cut transitions between images?

PostPosted: Wed May 29, 2019 8:19 am
by greenpattern
nice one Mat!

Re: How to create Morph cut transitions between images?

PostPosted: Wed Jun 05, 2019 6:07 pm
by garyglitter
Forgive my ignorance, but how would I get this to actually morph? I can feed the GLSL TOP 2 inputs but I'm not sure how to get the animation to actualize.

EDIT: got it, I think? I had to add "progress" in the vectors 1 GLSL page.


Wondering how I would make this work for what I had in mind though... Essentially I have 100+ stills from images generated via a GAN and want to animate it so that each frame is sequenced through the GLSL morph cut. Any idea how I could do this for more than 2 inputs?


matthewwachter wrote:You can kinda fake it using GLSL. Here are a couple methods:

Morph1
morph1.gif

Code: Select all
uniform float progress;
uniform float morph_strength;

vec4 getFromColor(vec2 uv){ return texture(sTD2DInputs[0], uv); }
vec4 getToColor(vec2 uv){ return texture(sTD2DInputs[1], uv); }

vec4 Morph1(vec2 uv)
{
   vec4 ca = getFromColor(vUV.st);
   vec4 cb = getToColor(vUV.st);

   vec2 oa = (((ca.rg+ca.b)*0.5)*2.0-1.0);
   vec2 ob = (((cb.rg+cb.b)*0.5)*2.0-1.0);
   vec2 oc = mix(oa,ob,0.5)*morph_strength;

   float w0 = progress;
   float w1 = 1.0-w0;

   return mix(getFromColor(vUV.st+oc*w0), getToColor(vUV.st-oc*w1), progress);
}

layout(location = 0) out vec4 fragColor;
void main() {
   vec4 o = Morph1(vUV.st);
   fragColor = TDOutputSwizzle(o);
}



Morph2
morph2.gif

Code: Select all
uniform float progress;

vec4 getFromColor(vec2 uv){ return texture(sTD2DInputs[0], uv); }
vec4 getToColor(vec2 uv){ return texture(sTD2DInputs[1], uv); }

vec4 getFromColorBias(vec2 uv, float bias){ return texture(sTD2DInputs[0], uv, bias); }
vec4 getToColorBias(vec2 uv, float bias){ return texture(sTD2DInputs[1], uv, bias); }

vec4 Morph2(vec2 uv)
{
   float rad = mix((1.0 - (1.0-progress)), progress, progress);
   rad = mix( rad, progress, abs(2.1*progress - 1.0) - 0.1 );
   float bias = 5.0;
   float scale_x = 0.02;
   float scale_y = 0.02;
   float stretch = 0.02;

   vec2 perturb;
   vec2 slope;
   vec4 color;

   vec4 pd_bl_a = getFromColorBias( vec2( -scale_x, -scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_bl_b = getToColorBias( vec2( -scale_x, -scale_y ) + uv, bias ) * progress;
   vec4 pd_tr_a = getFromColorBias( vec2( scale_x, scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_tr_b = getToColorBias( vec2( scale_x, scale_y ) + uv, bias ) * progress;
   vec4 pd_tl_a = getFromColorBias( vec2( -scale_x, scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_tl_b = getToColorBias( vec2( -scale_x, scale_y ) + uv, bias ) * progress;
   vec4 pd_br_a = getFromColorBias( vec2( scale_x, -scale_y ) + uv, bias ) * (1.0-progress);
   vec4 pd_br_b = getToColorBias( vec2( scale_x, -scale_y ) + uv, bias ) * progress;

   vec4 from = getFromColorBias( uv, bias ) * (1.0-progress);
   vec4 to = getToColorBias( uv, bias ) * progress;

   vec4 d = (pd_tl_a + pd_tr_a + pd_bl_a + pd_br_a + from * 2.0
   -(pd_tl_b + pd_tr_b + pd_bl_b + pd_br_b + to * 2.0 ));
   float diff = d.r + d.g + d.b;
   vec4 sx = ((pd_tl_a + pd_tl_b + pd_bl_a + pd_bl_b)
   - (pd_tr_a + pd_tr_b + pd_br_a + pd_br_b));
   vec4 sy = ((pd_bl_a + pd_bl_b + pd_br_a + pd_br_b)
   - (pd_tl_a + pd_tl_b + pd_tr_a + pd_tr_b));
   slope.x = sx.r + sx.g + sx.b;
   slope.y = sy.r + sy.g + sy.b;

   float p_len = dot( slope, slope ) + 1.0;
   perturb = vec2( slope.x / p_len, slope.y / p_len ) * diff;

   vec2 pert_to = perturb * (1.0-rad) * stretch;
   perturb *= rad * stretch;

   vec4 col1 = getFromColor( uv + perturb ) * (1.0-progress);
   vec4 col2 = getToColor( uv - pert_to ) * progress;
   return col1 + col2; //mix(col1, col2, progress);
}

layout(location = 0) out vec4 fragColor;
void main() {
   vec4 o = Morph2(vUV.st);
   fragColor = TDOutputSwizzle(o);
}

Re: How to create Morph cut transitions between images?

PostPosted: Fri Jun 07, 2019 1:00 pm
by matthewwachter
Yep exactly, just add the progress vector. The Morph1 shader takes a morph_strength uniform as well. Attached is an example scene.

As for switching more than 2 sources, the best method is to load a new file into the "slot" that is not seen after a transition is completed. Kinda like how a DJ might perform a set with 2 turntables.

Usually I create a component with an extension class method that handles the switching. The component stores a "state" value that switches from 0 to 1 or 1 to 0 and a Timer CHOP is used for the transition progress. The component would either invert the output of the timer chop every other time (0-1 to 1-0) using a math chop or actually swap the top inputs feeding the glsl component.

I have a nice little component that does this and am planning on releasing it publicly after a bit more testing. If you PM me your email I'll send you the beta.

Re: How to create Morph cut transitions between images?

PostPosted: Tue Jul 16, 2019 8:51 am
by flashmove
Simple question is it okay to use this glsl in a project of mine (a musical video clip) that has no commercial intent?
cheers great work as usual