Drawing with GoogLeNet

Drawing with GoogLeNet

In my previous post, I showed how you can use deep neural networks to generate image examples of the classes it’s been trained to classify. Since we’ve already started using deep neural networks in ways they were never intended to be used, let’s abuse them some more.

There’s nothing constraining us to generate image examples of one class at a time. Let’s see what happens if we try to generate two class visualizations close to each other, such as for instance a gorilla and a french horn

Gorilla playing an odd-looking french horn

Well, it kind of looks like a gorilla playing the french horn. Or let’s try dressing up a gibbon via “mixing” the gibbon class with some of the clothing classes:

A gibbon in a poncho (left) and an ET-looking gibbon in a labcoat (right)

Or what about making some scenic nature drawings, such as some foxes underneath an erupting volcano:

Foxes beneath an erupting volcano

Or a ballpoint pen drawing a castle:

A vague ballpoint pen drawing a castle

These mixes of classes kind of work out, though it should be noted that these are the best selections from a number of mixes I tried. It’s also tempting to create mixes of animal classes to generate some new kind of monster breeds, but most of the time this doesn’t work so well. Here’s some I tried though, a mix of a scotch terrier and a tarantula, and a mix of a bee and a gibbon:

A slightly spidery looking scotch terrier (left) and a slightly gibbon-looking bee (right)

Another fun thing we can do when generatinge images is to do the gradient ascent randomly along paths instead of on a single point. This of course takes a bit longer time, but it allows us to “draw” with the output, such as for instance drawing a mountain range of alps:

or a line of jellyfish:

or a circle of junco birds:

If we try to fill a larger region with visualizations of a class, we can also apply clipping masks, i.e. forcing the pixels to zero in some pattern during gradient ascent. So we can for instance use letters as clipping masks and try to create the alphabet with animals:

An A of apesA B of bearsAnd a C of cobras

Alright, that’s enough abuse of our deep neural network for today. I’ve just scratched the surface here, but there are several fun ways to use deep neural networks for creative visual work with a bit of experimentation (and lots of patience). I’m going to put the ipython notebooks I used to make these examples in the deepdraw repository as soon as I’ve cleaned up the code, so stay tuned via twitter.