Project 3:
Face Morphing

---

---

In this project, I will morph my face with the face of a celebrity, compute a mean over the population of faces and use the population mean to create a caricature of myself.

Part 1: Defining Correspondances

I first selected 2 images, one from me and one from a celebrity to be morphed, then I had to crop the images, so that they will look similar. Before we can morph two faces together, we need to define a set of corresponding points on each face. For this I used a tool called Correspondence Tool which allows me to easily define corresponding points. I used this tool to define about 60 corresponding points on my face and Taylor Swift's face.

Now, it is time to provide a triangulation of these points to be used for morphing. I used the Delaunay triangulation, which does not produce triangles with very small angles. To calculate the Delaunay triangulation, I used the average of the 2 point sets to avoid distortions during the morphing process.

The following image shows the triangulation of the points on my and taytay's face:

---

Part 2: Computing the midway face

To compute the midway face we need to do three things: 1. Compute the average shape of the two faces.
2. Warp both faces into the average shape.
3. Average the colors of the triangles together.
We already have the average shape from the previous step. We did that by calculating it using the following formula for each of the points:
(X_average, Y_average) = (X_me + X_taytay) / 2, (Y_average, Y_taytay) / 2
Now we need to warp the two faces into the average shape.

To warp the faces into the average shape, we need to compute the affine transformation that maps the average shape to the original shape. We have to do this for every triangle that we got using Dellunay. Based on A*p = q, using the three points of each original triangle p and each point of the transformed triangle q, we can compute the affine transformation matrix A. If p is from our original image and q is from our average shape, we can compute the transformation matrix A by solving the mentioned equation. Using A-1 (inverse of A), we can then compute which pixels of the original image should be mapped to the average shape.

Solving for a,b,c,d,e,f yields us our 3x3 homogeneous transformation matrix of the form:
[a b c]
[d e f]
[0 0 1]

We have to decide which pixel color to use, if we inverse transform our pixel to fill and get a location that is between pixels in the original image. I used nearest neighbor interpolation, this means that, for the midway picture, I use the color of the pixel that is closest to the computed pixel location in the original image. Then I blend the colors of the two images together by taking the average of the two colors.

Here we can see my original image on the left and the midface image warped from my image on the right

Here we can see Taylors's original image on the left and the midface image warped from Taylors's image on the right

Now I only have to average the colors of the two midway images together and I get:

---

Part 3: The morph sequence

For the morph sequence we are writing the following function:
morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac)

I choose a frame rate of 30 frames per second and created 45 frames between 0 and 1, our result is the following animation. Use the slider to see the morphing process.

---

Part 4: The "mean" face of the population

Here we want to compute the average face of a population of faces. For this, used a face dataset of 37 danish people provided by the DTU. Since the facial keypoints were alread annotated, I could directly use them to compute the average face. To calculate the average shape, I just added all the points individual points together and divided them by the number of faces. Then I warped all the faces into the average shape and averaged the colors of the triangles together.

Here are some of the danish faces, which I have morphed into the average danish face shape of the database. On the left slide you can see the original image and on the right side the warped image:

Here we can see the "average" face of the danish "population":

I have also warped my face into the shape of the average danish face and warped the average danish face into my face shape and got the following somewhat disturbing result:

Part 5: Caricatures: Extrapolating from the Mean

We can produce a caricature of my face by extrapolating from the mean face, what we calculated in the previous part. To do this, we need to choose alphas that are outside of our [0,1] range. Here I choose some alphas larger than 1, to increase the feature prominence of the population mean face on my face.

Bells and Whistles 1: Gender Change

Here I changed the gender of my face. I'm showing the morphing of just the shape, just the appearance and both: