In the last section I talked about visualizing neural network classes by synthesizing images to maximally activate class labels. This time, we're going to do the same thing with convolutional filters rather than class labels. In addition, we'll play with a few other ways to look at the receptive fields of convolutional filters. All the figured here are generated from a single self-contained Jupyter notebook.
There are a few sections to this series:
Synthesizing images that maximally activate convolutional filters / artificial neurons¶
We can use the same technique as above to synthesize images that maximally activate any neuron in the neural network. This will give us some idea about what that neuron is doing. Like the human visual stream, early neurons are responsive to simple textures and colors, while later neurons respond most strongly to complex patterns. As you move further into the network, the patterns have spatial structure that spans a larger range of the image. Below are a sample of a few images synthesized to maximally activate neurons at each layer of VGG 16. Note that for convolutional layers, these activations are maximized for each convolutional filter across the entire image, not just individual artifical neurons (we'll take a look at individual receptive fields in below).
Image synthesized to maximally activate filters from lower to higher layers in VGG16.
Synthesizing images that maximally activate receptive fields of individual neurons¶
The previous example shows the maximum activation for a convolutional filter across a whole image. We could also pinpoint a single neuron (filter) and find an image that maximally activates that neuron. Because single neurons in CNNs are only receptive to a small region of the image (the receptive field), the synthesized image should only end up changing that region of the image. Here, we'll maximally activate single neurons to take a look at their receptive field. Deeper into the network receptive fields get larger, and more complex.
Synthesized image to maximally activate a single artificial neuron with a receptive field in the center of the image.
It might also be useful to take a look at those receptive fields, zoomed in.
Synthesized image to maximally activate a single artificial neuron with a receptive field in the center of the image. Zoomed into recpetive field area.
Visualizing activation maps from neurons in different layers of a network¶
Another common way of visualizing features of a neural network is to take a look at what neurons are activated across an image for each filter type in a layer. Early in the network, there will be neurons that are specifically interested in colors or edges. Later, images respond to more complex features, sparsely present throughout the image.
Activation maps for convolutional filters in response to an image of Astronaut Eileen Collins.