LensKernelFFT_v01 v1.1


 
This location is for Registered Users Only.
Perhaps you need to login or register.
Contributor: Bob Roesler
Apply optical 'glare' to cg images based on a photographic HDR image. Version 1.1 is identical to v1.0, but saved with a .gizmo extension.
Requirements:
5.2 or later
13 Jul 2010
5064

LensKernelFFT:
This was put together in an effort to put some optical realism into cg imagery.
It employs the FFT tool suite (FFT, FFTMultiply, and InvertFFT), which allows for fast image based convolution. For small image kernels, the Convolve node works well, but as the kernel image gets larger, the Convolve node gets impractically slow. In an FFT convolution, most of the overhead seems to be taken up in the conversion to and from frequency space, and the convolution itself is relatively quick.

The idea here was to make some extremely high dynamic range images, with a still camera, in a dark environment, of a backlit pinhole. The resulting image(s) would thereby record the "signature" of hoh light is scattered within the optical system. By using the resulting image as a convolution kernel, a real optical quality could be applied to cg imagery. 

I have so far been able to make three kernels of useable quality:

  1. 35mmf14_normalized_v01.exr
  2. 35mmf28_normalized_v01.exr
  3. 35mmf56_normalized_v01.exr
LensKernelFFT_02 
These three images are available here and will be required for using the gizmo, please download these along with the gizmo file. (Technically, this is a Group, not a Gizmo...). These three images are HDR photos of the same pinhole, shot with a Canon 35mm f1.4 L series lens at three different f-stops (f1.4, f2.8, f5.6).

If you open these 'kernel' images in Nuke, you'll notice several bright pixels in the center of a very dark frame. However, if you raise the brightness of the image, you'll see that the "black" areas of the image actually have non-zero values, and as you bring the brightness up considerably, the values extend into  a fairly large area of the frame.
LensKernelFFT_01 

What this means, in terms of using the images as convolution kernels, is that a tiny amount of the input pixel's energy is dispersed around a rather large portion of the frame. Under the circumstances of normal image values, the result is a slight softening of the image. However, in the case of very bright specular reflections, the apparent "spread" is larger; and as the input pixel values become greater and greater, the apparent spread gets larger, as well.
To use it, you'll have to:

  • put the three kernel images in an accessible directory, and create a LensKernelFFT_Group_v01 node in Nuke.
  • Open the group node's properties, and go to the "Kernels" tab. There will be three filename knobs whose names correspond to the kernel images (f1.4, f2.8, f5.6).
  • Load the kernel images into the filename knobs, and save the Group wherever you store your Groups & Gizmos.

To use, attach the LensKernelFFT group to an image you want to convolve. It works best on images that use a high dynamic range. In the real world, small specular reflections of the sun or light sources are generally much brighter than other, non-specular areas, and in cg imagery these will b represented by floating point pixel values much greater than 1.0 (values well above 50 might be encountered) .

LensKernelFFT_03
The "output" pulldown should generally be set to "convolution result" (there's an "input kernel" option, that just passes the input kernel image to the output, this is mostly for troubleshooting purposes). 
The "channels" pulldown allows you to choose "RGB" or "RGBA", when the alpha is convolved, the luminance of the rgb kernel is used as the alpha kernel.
The "kernel" pulldown allows you to choose betwen the three kernels. The kernels yield very different results. The f2.8 & f5.6 kernels have a pronounced eight pointed star pattern that corresponds to the diaphram blades. The f1.4 kernel displays none of this, as f1.4 is the wide open setting of the lens. 
One of the byproducts of FFT convolution is "image wrap", that is, the convolution of a pixel on the edge of the screen shows up on the opposite side. This is normally not an issue unless very bright pixels are at the edge of the image. The 'pad frame' parameter adds a number of pixels to each side of the image before the FFT tools are applied, and is meant to be a work around for that byproduct. The larger the '"pad frame" value, of course, the more time consuming the FFT conversion and convolution process.
When making photographic images, everything, dark and light gets scattered through the lens, and, in concept, the full spectrum cg image should be processed through this node. In practice however, I feel that applying the convolution to the entire image makes the dark reas of image too soft. So you might (probably) want to key out the bright areas (a soft key works best, try a blackpoint of 0.1 and a whitepoint of 1.0), and convolve those brightest areas. To preserve overall image energy, you'll probably want to subtract or stencil out the keyed areas from the base image, and add the convolved result to that.
Because this node isn't terribly fast (although, compared to the Convolve node, it's very fast), you may find it best to precomp the result, once you've picked a kernel and key level you're happy with.

There's nothing magical about the kernel images, they are just extremely high dynmic range images (exposure times range from 1/1000 sec to one hour). If anyone is interested in making their own kernel images, they should feel free to do so. My experience is that are much better saved as full float, rather than 16 bit half float.
-Bob Roesler

Please login in order to download these files.

Comments   

 
0 # Ocsai Laszlo 2010-08-09 08:10
Hello

This is a very useful gizmo, thanks.
I found an error, the gizmo is kill the input format pixel aspect to 1.
 
 
0 # Oliver Markowski 2011-10-25 15:56
I like that very much!!! Any idea how we can incorporate obstacle maps into that gizmo?
 
 
0 # jon parker 2012-08-26 12:40
I have a naive question about this if anyone can answer... when normalizing a RGB kernel for convolution, is it more correct to normalize based on the max of R,G,B or to normalize each channel separately? When testing my own images I get slightly different results depending on which method I use.
 
 
0 # Bob Roesler 2012-08-26 13:09
If I understand the question, normalizing the kernel ensures that the total energy (the r,g, & b value ) of each input pixel is conserved, so that the output image is not noticeably darker or brighter than the input image. If you think of it in terms of the "Convolve" node (which will give the same results, but at a time cost for larger kernel images), the rgb values of the kernel image are scaled so that their sum equals 1.0.
 
 
0 # jon parker 2012-08-26 13:41
Thanks for pointing out the convolve node, since it is easier for me to mentally visualize what it does. I was unsure whether normalizing the image meant, for example:
R,G,B = R / normalize(R+G+B ), G / normalize(R+G+B ), B / normalize(R+G+B)
versus:
R,G,B = normalize(R), normalize(G), normalize(B)

Using the convolve example I can see that the right way is the second method, correct?
 
 
0 # Bob Roesler 2012-08-26 14:34
Yes, it would be normalizing on a per channel basis, so the second method is correct.
 
 
0 # dave H 2014-02-12 05:33
images seem to be missing
 
 
+1 # Frank Rueter 2014-02-12 05:54
oops, sorry about that. They should be available here now: http://www.nukepedia.com/footage/footage
 
 
0 # Brian Houlihan 2014-02-12 18:35
the 35mm lens footage is missing from all three links :(
 
 
0 # Frank Rueter 2014-02-12 20:48
the links work for me. I just downloaded all three. what error are you seeing?
 
 
0 # Diana Reynolds 2014-02-12 21:52
Nothing in the "footage" repository (the link Frank just gave), except a Nukepedio logo.
I would love to be able to use these images, if possible, thanks!
 
 
0 # Frank Rueter 2014-02-12 22:06
the link works for me. what happens when you go there via the menu "Downloads/Foot age"?
 
 
0 # Diana Reynolds 2014-02-12 22:47
Oh, I found it! thanks! A-convolving I will go!
 
 
0 # Edison Lau 2015-09-03 14:30
Mmmm... I would like to download as well, but the downloads/foota ge page is showing a psd image with nukepedia logo on it... :( I can't find the links
 
 
+2 # Frank Rueter 2015-09-03 16:12
It works fine for me, have you tried clearing your cache? When I go to Downloads/Foota ge, then click on "Lens Kernels", I get the expected links. I will paste them below for reference:
http://www.nukepedia.com/footageFiles/LensKernels/35mmf14_normalized_v01.exr
http://www.nukepedia.com/footageFiles/LensKernels/35mmf28_normalized_v01.exr
http://www.nukepedia.com/footageFiles/LensKernels/35mmf56_normalized_v01.exr
 
 
0 # Edison Lau 2015-09-03 17:27
Ahhh! I finally found it, I actually thought that menu bar is supposed to be some kind of tagging menu. Thanks!!
 
 
0 # Rion . 2019-03-06 14:35
hey mate
can u please update one more time

tnks
 
 
0 # Hamed Bank 2019-04-15 11:58
It seems to be gone again, I cannot find the
 
 
0 # Artur Margiv 2020-10-28 07:58
Hi Frank,
the links are not working anymore, any other place where they can be found?
Thanks in advance.
 
 
+1 # Chris Berry 2021-08-19 13:25
I found them by googling the filenames Frank linked to, and they appeared on Github where they can be downloaded:
https://github.com/raiscui/nuke7plugin/tree/master/res/Lens%20Kernels

35mmf14_normalized_v01.exr
35mmf28_normalized_v01.exr
35mmf56_normalized_v01.exr
 
 
0 # Chris Berry 2021-08-19 13:30
I've just seen that these are linked to by Mitchell Kehn, who in 2019 created a 'cheap' version of this FTT filter, so I figured he should get a link too:
http://www.nukepedia.com/gizmos/filter/cheaplenskernel
 
 
0 # Gavin Greenwalt 2018-04-02 20:14
Any reason why the kernel images are so large? It's computationally a lot more efficient if you cropped the kernel down to the region where there is in fact data.
 
 
0 # Oliver Banasiak 2019-10-20 13:07
I love this little gizmo, but in Nuke 12 it's broken. Can we please have an updated version? That would be amazing! :) Cheers!
 
 
0 # Oliver Banasiak 2019-12-03 22:45
Is there any love for this little gem out there? It doesn't seem to work with Nuke 12!
 
 
0 # Gregoire Verweijen 2020-01-22 22:14
Any tips on creating your own kernels? or getting more of these?
 
 
0 # Frank Rueter 2020-01-22 22:57
I was working with Bob at the time that he created those, so I happen to know a little bit about it.
He built a little black box that had an opening on one side to let the lens through, and a pin hole on t he opposing side where he allowed the light to enter. He then exposed up to an hour or so (which obviously depends on your camera and settings).
Do this with different lenses and maybe different apertures to get variety.
 
 
0 # Samuel Jorgensen 2020-03-10 18:25
Quoting Frank Rueter:
I was working with Bob at the time that he created those, so I happen to know a little bit about it.
He built a little black box that had an opening on one side to let the lens through, and a pin hole on t he opposing side where he allowed the light to enter. He then exposed up to an hour or so (which obviously depends on your camera and settings).
Do this with different lenses and maybe different apertures to get variety.


Was Bob then using some other software to combine the exposures into a 32bit float image? I'd love to do this with some old Soviet lenses I have. I can share the images when done.
 
 
0 # Frank Rueter 2020-03-10 20:36
I don't recall if he did shoot several slices tbh. The shot kernel was only visible in the centre of frame and you'd have to crank the exposure dramatically to see the light contamination at the edges, so I don' think he shot them in slices as there wasn't a need to high dynamic range in the kernel (I may not be wrong though, it's been a long time).
 

You have no rights to post comments

We have 3125 guests and 67 members online