CS 180 Project 2 - Fun with Filters and Frequencies

For this project, my task was to color these photographs originally captured by Prokudin-Gorskii. My strategy for this was to take the same photograph with blue, green, and red color filters and then stack the photos on top of each other to create a fully-colored image.

Part 1 - Fun with Filters

1.1 - Finite Difference Operator

The gradient magnitude operation tells us generally how steep our gradient in at every pixel (in this case, how drastically our pixels change in brightness). For this operation, we take the partial derivatives of the image in the x and y directions, and we obtain their overall magnitude by taking the square root of the sum of their squares. The result of this operation is a generally accurate contour over the edges of the photograph.

partial derivative dx

partial derivative dy

gradient magnitude

gradient magnitude binarized

1.2 - Derivative of Gaussian (DoG) Filter

We show in this section that we can blur an image by convolving it with a 2D Gaussian filter.

blurred partial derivative dx

blurred partial derivative dy

blurred gradient magnitude

blurred gradient magnitude binarized

Alternatively, we can blur the image by taking the partial derivatives of the Gaussian as well. We do this by convolving our Gaussian filter with our dx and dy operators. When we take the gradient magnitude of the resulting two filters and convolve our image with it, we get a practically identical result.

partial derivative of gaussian (x)

partial derivative of gaussian (y)

same image!

Part 2 - Fun with Frequencies!

2.1 - Image "Sharpening"

For this section, I was tasked with writing code that could "sharpen" an image. The process for sharpening an image involves isolating the image's high frequencies and adding them back to the original image, scaled by some multiplier. The isolation of an image's high frequencies is done by first isolating the image's low frequencies (through blurring, or convolving with a 2D Gaussian filter) and then subtracting the result from the original image. I did this first for an image of the Taj Mahal:

original

alpha = 1

alpha = 5

I also explored how this sharpening procedure would perform on this very blurry picture of an eagle.

original

alpha = 1

alpha = 5

2.2 - Hybrid Images

For this section, I worked on hybridizing images. A hybrid image is effectively one that displays one object when viewed from far away and a different object when viewed up close. This is achieved by isolating the low frequencies (through blurring) of the image that we want to perceive from far away and the high frequencies (through techniques similar to the previous part). First, I tried creating a hybrid of Derek and Nutmeg. This image looks like Derek from far away but Nutmeg up close. We can tell that we maintained the low frequencies of Derek (hence we can see his silhouette from far away) and the high frequencies of Nutmeg (we can see the intricate whiskers and fur patterns when we stare up close).

derek

nutmeg

#dermeg

Shown below are the frequency analyses of the original Derek, original Nutmeg, low frequency Derek, high frequency Nutmeg, and the hybrid image.

derek frequencies

nutmeg frequencies

derek low frequencies

nutmeg high frequencies

dermeg frequencies

I got really ambitious and tried to create a hybrid image of two very influential figures in my life (with very similar names): Stephen Curry and Kirby. As evidenced by the images below, I was not able to do right by my ambition. If I had to guess, the failure probably comes from the fact that Kirby is so wide that you can see him even when looking up close. You can still see Stephen Curry very well up-close too (and not as much from far away).

the goat of basketball

the goat of video games

paralysis demon

In the spirit of running out of ideas at 11PM, I also tried creating a hybrid image of Professor Efros and Professor Klein:

klein

efros

professor hybrid

Multi-resolution Blending and the Oraple journey

This section revolved around carrying out multi-resolution blending, or smoothly blending two images together across a boundary.

2.3 - Gaussian and Laplacian Stacks

For this section specifically, I implemented functions for constructing Gaussian and Laplacian stacks, which will aid in multiresolution blending in the next section. A Gaussian stack consists of increasingly blurred versions of the image (in other words, consecutively stricter low-pass filters), and the Laplacian stack is constructed by subtracting consecutive layers of the Gaussian stack from each other, effectively representing a set of bandpass filters.

Gaussian Stacks:

gaussian stack of apples

gaussian stack of oranges

Laplacian Stacks:

laplacian stack of apples

laplacian stack of oranges

2.4 - Multiresolution Blending (a.k.a. the oraple!)

In this section, I utilized the Gaussian and Laplacian stacks to blend images together, using multiresolution blending. To make sure my blending was as smooth as possible, I took the liberty of making my Gaussian stack of the mask longer and then only using the blurriest images in the stack. I first utilized multiresolution blending to create the oraple, a blend between the orange and the apple.

apple

orange

oraple

apple spline

orange spline

I then tried this on two other examples. The first was a landscape of Kuala Lumpur, the capital of Malaysia, transitioning from day to night. I really like how this one turned out. We can see the transition from sunlight to city light at the boundary.

city day

city night

very cool

I then tried creating an image of the earth falling into a black hole. This required using a circular mask to pull off.

earth

black hole

end of the world