Mean Averaging Music Videos [Imagemagick/GIMP]

Well, I apparently just can’t get enough of averaging things.


While playing around on the last post for averaging long exposure images, I stumbled across something interesting. Apparently Imagemagick can open video files. At that point I just had to have a go at averaging some music videos.

The results were … interesting to say the least.

These are really worth seeing large, just click the image to embiggen.
The links for each video name below will open the video in the page, in case you wanted to watch.

Some of them were really cool though. For instance here’s BeyoncĂ©’s “All the Single Ladies”:

Beyonce All the Single Ladies blend average mean
BeyoncĂ© - “All the Single Ladies” (5955 frames)


Or kicking it a little old school produces some greats.

Here's A-ha’s “Take On Me”:

A-Ha Take On Me blend average mean
A-Ha - “Take On Me” (6807 frames)


“Tonight, Tonight” from The Smashing Pumpkins:

Smashing Pumpkins Tonight Tonight blend average mean
The Smashing Pumpkins - “Tonight, Tonight” (6475 frames)


I couldn't help but mess around and try some more recent videos as well, of course.

Not surprisingly, Gotye’s “Somebody That I Used to Know’ looks exactly like you'd expect:

Gotye Somebody That I Used to Know mean average blend
Gotye - “Somebody That I Used to Know” (6099 frames)


Of course, I had to do Robin Thicke’s “Blurred Lines” video, for (ahem ) science:

Robin Thicke Blurred Lines mean average blend
Robin Thicke - “Blurred Lines” (6235 glorious NSFW frames)


Pink

Billboards Woman of the Year, Pink, yielded some interesting results:

Pink Blow Me mean average blend
Pink - “Blow Me (One Last Kiss)” (5535 frames)


Pink Try average mean blend
Pink - “Try” (5972 frames)


Pink Just Give Me a Reason blend average mean
Pink - “Just Give Me a Reason” (5814 frames)


Pink True Love mean average blend
Pink - “True Love” (5803 frames)


I find it interesting that if you’re familiar at all with these videos, the blends immediately evoke the sense of what you’ll see watching them.

Die Antwoord

Almost completely on the other side of the musical spectrum, we have Die Antwoord. I want to send a special “Thank You!” to Boing Boinger Xeni Jardin for turning me onto these guys (I consider it telling that I’ve lost count of the cool things that Boing Boing has led me to over the years).

If you haven’t had a chance to give them a listen, do it.

die antwoord enter the ninja mean average blend
Die Antwoord - “Enter the Ninja” (7805 frames)


mean average blend die antwoord rich bitch
Die Antwoord - “Rich Bitch” (4402 frames)


mean average blend die antwoord evil boy
Die Antwoord - “Evil Boy” (6961 frames)


mean average blend die antwoord i fink you freeky
Die Antwoord - “I Fink You Freeky” (5865 frames)


mean average blend die antwoord fatty boom boom
Die Antwoord - “Fatty Boom Boom” (8554 frames)


I like “Fatty Boom Boom” because it really looks like the video sounds. Hectic.

mean average blend die antwoord cookie thumper
Die Antwoord - “Cookie Thumper” (8459 frames)


Creating These

I did mention back at the beginning that it was cool I could load video files directly into Imagemagick. Just one problem: IM likes to load up the entire set of files before doing further operations against them. So trying to load up a full HD video in memory didn't work so well. SD videos loaded, but took forever to process.

To speed it up I just dumped the videos to files first, then operated on them in batches.

You'll need FFmpeg and Imagemagick, of course. I also happen to be using Cygwin, because I'm more comfortable doing things in a bash shell vs. windows command line.

Dumping the Video

I dumped the video using ffmpeg:

ffmpeg -i INFILE -y -f image2 output_%05d.png

This is fairly self-explanatory. The %05d tells ffmpeg to name each file with 5 digits sequentially.

So now I have a directory of files called output_00001.png, output_00002.png, etc.

Batch Averaging

As I said, IM will choke up if you try to load it with too much data (thousands of 1080p frames for instance). So I did a first pass of averaging by only generating the averages for 100 files at a time:

ls output_*.png | xargs -n 100 sh -c 'convert "$0" "$@" -evaluate-sequence mean outdir/"$0" '

Don't worry, it's not so as it looks. Let's break it down:

ls output_*.png | …
Generate a list of files in the directory name output_*.png. Then pipe that list into...

xargs -n 100 sh -c
xargs is used to build a command up. The -n 100 switch tells it to use 100 items from the piped list (all the filenames from the ls command) at a time.

Spawn a shell, and run this command:

convert "$0" "$@" -evaluate-sequence mean outdir/"$0"
This just runs the previous imagemagick command to mean average the images. $0 is the first in the list, $@ are the rest.

Then output the results into the directory “outdir”, with the file named the first in the list.

In my “outdir” directory, there are now a bunch of files with names like “output_00000.png, output_00100.png, output_00200.png” etc. These are averages of 100 frames from my video at a time.

In that directory, it's now trivial to get the final average:
convert output_*.png -evaluate-sequence mean -channel RGB -normalize final.png

This will generate the final mean average from all of the images we output previously. The inclusion of “-channel RGB -normalize” is to automatically normalize the results on a per-channel basis.

Batch Averaging (Windows Command Line)

Reader Belzecue has noted below a means for doing the first run batch averaging with just the windows command line:

FOR /L %i in (0,1,8) DO convert c:\inputpath\output_0%i*.png -evaluate-sequence mean -channel RGB -normalize c:\outputpath\final_0%i.png

The magic part is the options in the FOR command (0,1,8). Basically it means, start counting at 0, increment by 1, and stop at 8. Couple that with the globbing for filename “output_0%i*.png”, and that command will batch up 9 images, averaged across 1000 items at a time.

(The result to the command line will be output_00*.png, output_01*.png, output_02*.png, etc…)

Thanks for the tip!

Conclusion

Of course, I had to leave you all with one last video. The question is, can you guess which video it is?


Guess the video. Here’s a little hint:
“You wouldn't get this from any other guy”

18 comments:

  1. I thought quite a few of these were very beautiful, in particular the Smashing Pumpkins and "I Fink you Freeky"

    ReplyDelete
    Replies
    1. Man am I glad to hear that. I love them, but feelings seem mixed at best from others... :) At least there's one other person that likes them!

      Delete
  2. I'm glad you posted this because yesterday I tried a averaging a 3 minute HD video and it didn't go so well, of course. Pretty cool stuff.

    ReplyDelete
    Replies
    1. BTW, I usually use graphicsmagick instead of imagemagick, but GM does not have the -evaluate option. but it does have -average. imagemagick also has -average. i just did some quick testing and -average seems to do the same thing as -evaluate-sequence mean -alpha off. And the GM average and image magick average option also seems to produce the same output. the output file sizes are much smaller with GM so i guess there are some differences - might just be the default compression level.

      anyway, GM claims to be faster and more efficient. do you think it would handle these large video inputs in a way that would let you do all of the images at once without doing several partial runs and then averaging the partials?

      Delete
    2. I thought the -average option was deprecated a while ago (I could be wrong).

      It might handle it fine, but things will start thrashing unless you've stuffed your machine full of RAM.

      For comparison, "True Love" is a 3:53 song (5803 frames), the total size of the png outputs for that video is 12GB of data...

      Delete
  3. Pat, with Windows processing, it's easy to batch in ImageMagick by running a command-line batch file containing this line:

    FOR /L %i in (0,1,8) DO convert c:\inputpath\output_0%i*.png -evaluate-sequence mean -channel RGB -normalize c:\outputpath\final_0%i.png

    *Note, if doing as a command line instead of batch file, double the "%" character.

    In the example above, I'm running on a file-set numbered "output_00001.png" through "output_08843.png" where the end-counter "8" in "(0,1,8)" is the final "thousand" set. This gives me 9 masters to finally average with the command:

    convert c:\outputpath\final_*.png -evaluate-sequence mean -channel RGB -normalize c:\outputpath\final.png

    So, if your last extracted frame was "output_12843.png" then you'd set the end counter to 12 in the batch file. Hope that makes sense.

    ReplyDelete
    Replies
    1. Nice, thank you for pointing this out. I'll update the post with this and credit - thank you very much for stopping in to share!

      Delete
  4. You just rickrolled us all?

    Rick Astley – Never Gonna Give You Up

    ReplyDelete
    Replies
    1. Haha, yep - nailed it! :) I figured "We're no strangers to love" was too much of a dead giveaway.

      Delete
    2. Enjoyed the humo(u)r :-)

      Oh, another thing: at first glance the command line

      ls output_*.png | xargs -n 100 sh -c 'convert "$0" "$@" -evaluate-sequence mean outdir/"$0"

      is missing a final '.



      Delete
    3. Thank you for pointing that out! :) Fixed.

      Delete
    4. Thank you for pointing that out! :) Fixed.

      Delete
  5. Thanks for this. I did it with a few videos. Tool's AEnema produced a result where the only discernible object in the frame looked like a dildo. I don't know what else I was expecting.

    ReplyDelete
    Replies
    1. Nice. Somehow that doesn't surprise me one bit. :)

      Delete
  6. Great stuff Pat! You may want to refer to some related work — Jim Campbell's "Illuminated Average" series, for example. He averaged e.g. Psycho into one frame, in 2000: http://www.jimcampbell.tv/portfolio/still_image_works/illuminated_averages/index.html . I think it's really interesting to compare his results to yours.

    Really enjoying your blog!

    ReplyDelete
    Replies
    1. Thanks for that link, I'll definitely keep it in mind! In my previous posts about averaging I do reference others like Jason Salavon. This is a new one to me, and a neat find (it helps that I love Psycho).

      Delete
  7. In your Youtube post on patching the old photo you refer to using a prebuilt alternative Gimp distro which includes a bunch of the plug-ins pre configured, but I could not understand the site name you mention? And i don't see a reference to it on site? Love what you are doing and I want to wean myself off PS using your helpful tutorials so far have just done minor stuff in Gimp, keep falling back to PS when I have issues..

    ReplyDelete
    Replies
    1. Sorry about that! The site is partha's website:

      http://www.partha.com

      He has pre-built packages for win and osx that include many handy plugins/scripts!

      Delete