Assignment 2: Video Morphing
Due: Wednesday Apr 24, 2019
The goal of this assignment is to automatically create morphing video transitions between faces, similar to these.
Record several sequences of similar head motion across different people. The head should take up most of the frame, and the background should be a uniform color.
This is a programming assignment, which requires a few packages to be installed. While each computer is different, below we provide installation resources you might find useful. You are not required to follow these — install in a way that makes sense for your setup. If you are unable to sort out installation issues on your own, please contact us on Piazza.
You will need the following:
Mac: can install via brew by typing
brew install python3in a console window. Note that the system comes with an older version of python, so make sure you are using the correct one.
Windows: Available via Anaconda
- OpenCV with python binding
Mac: can be installed via brew (or pip). Also, there are many online tutorials such as this one.
pip install opencv-python.
- Numpy and Scipy
Mac: see OpenCV tutorial.
Windows: Available via Anaconda or pip.
brew install dlibor by following these instructions
Windows: Instructions or via Anaconda
Using the Jupyter notebook
Clone the assignment repository.
The bulk of this assignment (including further instructions) is inside
a Jupyter notebook. If you haven’t used one before, this is a useful
tool that allows markdown, code and results to coexist in an
interactive environment. To start the notebook, navigate to the
assignment folder and type
jupyter notebook assignment2.ipynb.
Note that the Jupyter notebook references code which is implemented in other files. You will need to edit these using your favorite text editor or IDE (e.g. PyCharm is free for students). You can run each individual file independently of the Jupiter notebook, which is useful for development and debugging. Once your external code is working, don’t forget to make sure it also works in the notebook.
Important: changes in external files are not automatically loaded by the notebook, so you will need to click
Restart & Run
All in your notebook to use updated files.
Tips and tricks
Numpy arrays require a bit of getting used to, but it is well worth your time. Performing array manipulation “the numpy way” will often lead not only to shorter and more readable code, but also to faster code because the built-in array manipulation functions are highly optimized (e.g. using AVX). In many cases, instead of using
for loops, you can find a one-liner numpy function that performs the exact same operation. Specifically for this assignment, you can solve it all (except the very last section) without adding any
Take the time to understand the concept of array slicing, both basic and advanced. You will never look back!
Some useful function to know are
This list is by no means comprehensive.
Your final submission should include:
- Raw video footage.
- Video morphing results.
- A working implementation. Running the Jupyter notebook should generate all results with no errors.
- Coding tasks:
- Image warping: 10%
- Face landmark detection: 15%
- Vector field extrapolation: 15%
- Face morphing (image to image): 20%
- Face morphing (video to video): 20%
- Overall code readability, elegance and correctness: 10%
- Quality of raw video footage and video morphing results: 10%
To submit your assignment, prepare a ZIP file containing your deliverables with the filename: