Noise Removal in Photos with Median Stacks (GIMP/G'MIC & Imagemagick)

In my recent experiments and playing around with even more image averaging in Imagemagick, I decided to have a look at some other methods for calculating pixel values. This time I focused on stacks of images of the same thing.

Why the same thing?


f/8 @ 1100 sec., ISO 25,600 (mouseover to compare straight out of the camera)

Because the (uninspired) above image was shot at ISO 25,600. Go look at it again.

To be fair, I cheated a little bit... The relatively noise-free image you see above shot at ISO 25,600 is actually combined from 10 identical shots.


The trick is that I combined all of the shots using a median evaluation in Imagemagick. This basically means that for every pixel in the image, there were 10 pixel values to choose from. Imagemagick automatically picked the pixel value that was closest to the middle.

So if we had a list of 7 values, for example, that looked like this:

120, 120, 125, 137, 150, 151, 160

Then the median value is just the number in the middle of the list:

120, 120, 125, 137, 150, 151, 160

If there were an even number of values, the median is calculated as the average of the two numbers in the middle (usually).

This is different from the averaging I had done previously, where the pixel value was an average of all the values at a given pixel location (though both methods could be used to reduce noise in the final result).

Here, let's look at a couple of 100% crops side by side between my median stack example and a base image:





Pretty neat, right? (Seriously, look at that last image - you can't even make out "Fast Focused Fearless" in the single shot)

There's a few obvious caveats, though. This is only really effective for static scenes (we'll see why shortly), and the images must be aligned very well for it to work without smearing any fine details. So it should be shot from a tripod or very stable place to minimize movement between shots, or the shots should be aligned in post (using Hugin's align_image_stack - see here for an example of how to do it).

How to do it

There are two ways you can go to achieve this. The fast way, and the slow way.

First make sure your images are well aligned - shoot on a tripod if at all possible, and/or align the images with software as needed. Take as many as you feel like waiting on.

The fast way is with Imagemagick. Once you've aligned all your images (or they are spot on already), put them into a directory. Then in that directory:

convert *.jpg -evaluate-sequence median OUT.jpg

The slow way is using G'MIC in GIMP. Load up each of your aligned images as a layer in GIMP, then run G'MIC.


Then go to Layers → Blend [median] and make sure the Input Layers are All visibles, and optionally output to New layer(s).

Be prepared to wait, I've found the G'MIC version of this blend is 10x+ longer than doing it through Imagemagick. (Though I did poke David about this - so the G'MIC team might optimize this blend mode later).

Removing People

Using a Median filter to combine multiple images has been used to remove spurious objects from a scene. A common example is if you desire to get a shot of a famous landmark or tourist area, where there tends to be a lot of people around.

You can just shoot multiple shots (try for an odd number), and combine them with a median filter to remove the spurious objects throughout the scene.

For instance, here's my well-used space marine traversing my desktop:



As you can see, the final frame is after combining all of the images with a median evaluation. Voila! No more space marine!

In Conclusion

This is a really neat method to use, once you realize it's available to you, and where it can be best utilized.

If you're shooting static scenes (stock photography, still life, etc.), there's really no reason not to go ahead and set your camera to fire off a handful of frames quickly (burst mode!). Setup on a tripod, burst away, and you can significantly reduce the noise in your images with little effort.

If you want to capture scenes with many people/objects moving about, and want a clean image of an environment, just setup on a tripod again, and snap away. With enough frames, it'll make combining the images and removing those objects very easy to do.

[Addendum]
Just a couple of quick things that a few people have pointed out:

1. Can't I just use a lower ISO and longer exposure?
Yes - but what happens when you've pushed your ISO down as far as it can go on your camera? With this method, you can remove even more noise and make images that are cleaner than is possible with a single shot with your camera.

2. This is not far off from what astrophotographers will also do to clean up signalnoise in their images - stack multiple images to clean up the noise. In fact - there is specialized software just for this type of image stacking that I should go try out! :)

Help support the site! Or don’t!
I’m not supporting my (growing) family or anything from this website. Seriously.
There is only one reason I am writing these tutorials and posts:
I love doing it.
Technically there is a second reason: to give back to the community. Others before me were instrumental in helping me learn things when I first got started, and I’m hoping to pay it forward here.

If you want to visit an ad, or make a donation, or even link/share my content, I would be absolutely grateful (and tickled pink). If you don’t it’s not going to affect me writing and posting here one bit.

I’ll keep writing, and I’ll keep it free.
If you get any use out of this site, I only ask that you do one thing:
pay it forward.


18 comments:

  1. You might be interested in my experiments with motion-compensated median filtering. It uses AVISynth (and it's plugins) to calculated the movement of each block of the image and then lines each block up with the corresponding part in a reference image. Finally a median filter is applied to the allighed stack.

    http://iainisbald.wordpress.com/2011/06/28/tech-motion-compensated-denoise-for-multiple-images/#more-571

    ReplyDelete
    Replies
    1. Thanks for this! It's a neat idea to use Avisynth to do this, but I wonder, why? Isn't there a possible loss of fidelity with the conversion to YUV?

      Also, is it just moving macroblocks around? I wonder if it's faster or better than using something like Hugin's align_image_stack to align the images?

      I may have to head off and play with it a bit :) Thanks for pointing it out!

      Delete
    2. Some of the Avisynth pulgins only work in certain colour spaces, so we are stuck with what the authors provided us with.

      I'm not familiar with Hugin's alignment, but the advantage of using Avisynth is that movement in the scene and movement of the camera can be accounted for. For example, I have took two photos, one after the other, of two young women. In the second photo, one young woman turns her head slightly and I moved the camera slightly as well.

      Honestly though, I hardly ever use this method, because it is a bit cumbersome(my Avisynth scripting skills aren't very good, so that doesn't help), and most noise reduction solutions do a good enough job with a single image. I'm trying to write a G'MIC filter to do something similar because then at least I don't have to leave GIMP to do it. I'm not sure it is possible to do as well or as fast as with Avisynth though.

      Delete
    3. I'd have a look at Hugin (where you can find the command line align_image_stack) for auto-aligning the images, it's quite good and very fast. I had previously used it with good results for focus stacking macro photos:

      http://blog.patdavid.net/2013/01/focus-stacking-macro-photos-enfuse.html

      Piping that into IM before GIMP is pretty straightforward, so it might be worth a look at least...

      David is working on optimizing the median blending in G'MIC, so it should be updated soon I hope. When it's done I'll give it another go with G'MIC as well.

      Delete
  2. I'm curious what the results would be if you used a conventional denoise filter on the images before getting the median. I've had very good results using darktable's profiled denoise, as well as its non-local denoise.

    ReplyDelete
    Replies
    1. That's a good question, and might be worth experimenting with. Another nice thing about this method is that you're almost guaranteed to not destroy edges at all, no matter how many images comprise a stack (or how few). It will also tend to keep the image detail, even if they are small - which some de-noise algorithms would throw away (the median filter with an odd number of images in the stack guarantees that the pixel value will come from at least one of the images).

      The results might be better doing this first, then trying another de-noise algorithm. What's nice is that this can be done in high bit-depth with both G'MIC and Imagemagick, so it preserves a lossless workflow from RAW if you wanted.

      Delete
    2. Doing the median stack first is better. If you have enough images you can resolve details that are well below the noise level in one image, but even a sophisticated denoising algorithm cannot recover much detail if it is swamped with noise.

      Delete
    3. The follow image shows how powerful this technique can be at extracting details that are completely covered by noise.

      https://dl.dropboxusercontent.com/u/10782279/median%20example.jpg

      Unfortunatley, the law of diminishing returns kicks in and to get significantly more improvment on this image I would need at least 64 images, probaly 128. IIRC every time you double the amount of images you get a half stop improvement.

      Delete
  3. First time here....
    Your trick for noise reducing using G'MIC on odd number of photo is quite effective...
    But if we have only one shot ?
    I wonder if i duplicate the photo a odd time if vary the position of each a few pixels and apply a blended(median) on all the layers could reduce the noise?


    ReplyDelete
    Replies
    1. Thanks, but it's not my trick ;)

      Only one shot? You'll have to rely more on other NR algorithms to help you. If you offset the image, you will begin blurring sharp edges (and one of the benefits of this method is retaining all the fine edge details).

      Delete
  4. This comment has been removed by a blog administrator.

    ReplyDelete
  5. Pat, in a comment earlier, you mentioned a lossless (or at least as much as it can be) RAW workflow. I wonder how such a workflow could look like. I guess I would have to do several steps before the median computation, i.e. demosaicing, but maybe basic color/level adjustment such as base curve or white balance. Could you (or anyone) elaborate on such a RAW workflow, the tools to use, useful settings and how to get the result back into $rawconverter (preserving the metadata used to correct lens distortion etc.)?

    BTW: Many thanks for the great resources you provide here, with pixls.us and at all the other places.

    ReplyDelete
  6. Have you heard of HDRmerge? It lets you merge several raw files into a DNG with masked optimal exposure in each area. (If you export in an HDR format, you can tone-map it in, say, Luminance HDR.)

    It also makes a convenient workflow because its results can also be edited in my RAW processor. I've been wondering if the same can be done for median stacking.

    https://github.com/Beep6581/RawTherapee/issues/2887

    (Median or geometric mean due to the linear nature of RAW files.)

    The only trouble so far is finding a way to apply demoisaicing and channel balancing, etc. on these pseudo-RAWs.

    ReplyDelete
  7. The median is not "the number in the middle of the list", it's the most frequent number. In your example, the median would be 120.
    Choosing "the number in the middle" would not help you in removing foreground objects and it would be equivalent to taking solely one picture.
    (The average could also be used but it would work well only when there are not many moving foreground objects and it would require more pictures.)

    ReplyDelete
    Replies
    1. I think maybe you are using a different definition of "median" than... everyone else?

      Delete
    2. *Median* is the center value of a sorted list.

      *Mode* is the most frequently encountered value within the list.

      Delete
  8. it works!
    i used hugin and gimp+gim'c.
    I let hugin to do all the alignment automatically, and gim'c to blend

    ReplyDelete
  9. 2 scenarios.
    1) taking 10 photos, low exposure, iso 800 and than add them
    2) taking 10 photos, correctly exposed, high iso and then noise reduced through median.

    Is those 2 scenarios comparable in result?

    ReplyDelete