hello my name is matt!
one day a friend showed me this gizmo and today I will explain how it works :D
It is similar to colour smear that you can find here: http://www.nukepedia.com/written-tutorials/colour-smear-for-nuke/
I really recommend looking at the colour smear tutorial to better understand how it works
the idea is the same but instead of using blur and unpremult to make our extend edge we will use IDistort..
So let's do it!
1: Input your ALPHA in your target
2: Add a Blur with size 6
3: Add two transforms x = 1 and x = -1 and merge(from) them respective (A-B)
4: Add more two transforms y = 1 and y = -1 and merge(from) them respective (A-B)
5: Use shufflecopy to output those results In Red and Green Channels respective
X = red
Y = green
6: Use a copy node to translate what you created to the Forward Channel
Red to Forward.u
Green to Forward.v
7: Add a IDstort with 6 in UVscale and then erode(filter) value of 2 the matte
8: Invert your matte and add a Blur with size 2
9: Premult :P
10: Merge over your target
11: Add a Mask Input
12: see if its working lol
The nodes that you would want to tweak for getting the effect that you want are:
The blur in the step 1, For tweaking the UV slope
The IDistort in the step 7 for tweaking how much you want to extend your edge
The erode in the step 7 for tweaking how much you want to over in your target
The blur in the step 8 for softing the matte edge
The mask in the step 11 for masking where you want
Create a gizmo with this knobs linked in them and you are in good way to go hehe :D
That's it, Have fun!!