CS5955 PPCP - Final Project

Design Document: Due 11/19/2010 at Midnight

Project: Due 12/10/2010 at Midnight

This project is to be done individually. Do not work together on specific code. You may discuss between yourselves about the ideas of the class and main points, but each submission must be unique and of your own design. You will be given a lot of freedom to implement the way you want to.

Project Description

Your task is to create an application that will take a target image that has multiple faces and replace them with other faces. The target image will have face regions identified by rectangles around each face. Each input image will also identify the face region(s) within it. You will calculate the average color of each face region and its standard deviation (for each color component red, green, blue). With this data, you can now find the closest matching input image to one of the target face regions using an arbitrary heuristic. Finally, you will draw the input face onto its matched target face to replace it.

3-Credit Student

Additionally, you will be required to only use the oval region that can be implied by the rectangles specified. This oval region is all that should be used to calculate the average/standard deviation statistics. Furthermore, you must only draw the pixels in the oval from the input image to the region on the destination image.

Design Document

Create a design document of your final project

1. Specify the overall algorithm. You may use pseudo code, diagrams, flow charts, etc.
2. Indicate what architecture patterns you plan to use as discussed in class: e.g. Producer, consumer, worklist, etc.
3. Identify the areas in which you plan to use parallelism and concurrency. Talk about what constructs you’d like to use for different parts and any limitations on parallelism you expect.
4. Indicate shared resources you plan to utilize.
5. The example design document can be found on the course website at the following link:http://www.eng.utah.edu/~cs5955/homework/finalProject/DesignDocExample.pptx

Handin Instructions

This document should be turned in as a pdf with the filename: design_doc.pdf

        > handin cs5955 finalProject design_doc.pdf

Source Images Specifications

You are required to search and accept image files with the following extensions: jpg/png/gif. These image types are already accepted file types by .net’s Image class.

Each image has a face regions specification file:

1. Named after the image file (w/o the file extension) but with a .txt extension.

1. e.g. Image file: Face.JoeMayo.jpg
   Region filename: Face.JoeMayo.txt

2. Each line specifies the rectangle of the face within the image
3. Format: x, y, width, height
   equivalent to the output from:
     var typeConverter = TypeDescriptor.GetConverter(typeof(Rectangle));
    File.WriteAllText(filePath, typeConverter.ConvertToString(myRect));
   You can use the following to help you parse each line:
     (Rectangle)typeConverter.ConvertFromString(line)
4. Region IDs (or indexes) are implied by the line index number. For simplicity, the first line has ID = 0.

Finding Best Fit

Find the average color and standard deviation (for each component red, green, blue) of each of the regions in the source and target image regions.

Using these metrics, compute a rank between each of the source and target regions. Once they have been computed, determine the best matching region to place each face.

Note, it should be possible that you may have more input regions than available in the target image (and vice versa). So your algorithm should be robust when handling these scenarios. In all cases, only the input image that most closely fits the target region should be used.

The following is an algorithm we suggest you implement. If you come up with your own heuristic you need to specify it in your final report.

Let T => Target region, S => Source region

For each color component, k in {R,G,B}

        Compute average k

        Compute standard deviation of k

        rankK = (|stdDevT - stdDevS| + |avgT - avgS|)

/ (stdDevT+1)

Then sum up the ranks for each color component:

rankR + rankG + rankB

This algorithm will produce a rank where the smaller it is, the better the match of the source to the target regions.

Command Line Program Specification

You should create a C# console application project with the name “FaceReplace” (capitalization is whatever you want).

Input parameters:

1. Source images folder path
2. Source region files folder path (the directory to search for .txt files of same name as each source image filename)
3. Target image file path
4. Target image region file path
5. Output image file path

Any extra parameters that you implement should be optional

e.g.

faceReplace myFaceImages myFaceImages\regions1 targets\supremeCourt1.jpg targets\supremeCourt1-regions1.txt out\result1.jpg

Note: in this case, the ‘out’ directory should already exist; you are not required to create the output folder path.

The input folders for images and regions are separate arguments to allow for pointing to a different folder composed of region files that may specify a different region but still meet the requirement that the region file must match the image file name except with a .txt file extension.

Required Output

The console app should output the matched faces in the following format:

source image filename [region id] -> Target region ID

e.g.

        MyFace.jpg [0] -> 4

        Face2.jpg [0] -> 5

        SomeFaces.png [0] -> 1

        SomeFaces.png [1] -> 3

Also, your code should not throw any exceptions that are not caught. Odds are, we’ll supply a path that doesn’t exist or a folder that doesn’t contain any images. Display any errors to the console with a pretty message. Feel free to write errors and exceptions to the Debug window using:

catch(Exception ex){

  System.Diagnostics.Debug.WriteLine(“Exception caught: {0}”, ex);

        This will display in Visual Studio’s Output window (View -> Output).

Your turned in program should not print out anything but the above text, unless invalid input or a friendly error message.

Composing The Final Image

In general though, there shouldn’t be any need for an alpha layer in the output background. But you may want to use an Alpha channel when you draw your input image onto the final. This will become especially important for those that are supposed to measure and paste only the oval of the input image onto the final image.

Google and the MSDN library (F1 from VS 2010) are your best source of information on how to draw using .net.

Here are some important APIs that may help you get a start:

1. Graphics.FromImage - http://msdn.microsoft.com/en-us/library/system.drawing.graphics.fromimage.aspx
2. Graphics.DrawImage -http://msdn.microsoft.com/en-us/library/ms142040.aspx

Extra Credit: Blurred Edges

When composing the final image, blur the edges (of the rectangle or oval as required) by a specific percentage (of the height/width of the original image). Expose the percentage as an optional commandline option (e.g. -blurPerc:5 could indicate to blur by 5%, and -blurPerc:0 would do no bluring, but be the default if not specified). If you implement this feature, you must include instructions in the instructions file so we know to look for it.

Materials & Help

- Be sure to dispose of all instances of Image, Bitmap, FastBitmap, and Graphics objects after they’re used.

Sample Images

        http://www.eng.utah.edu/~cs5955/homework/finalProject/images.zip

New FastBitmap Class

You may use the FastBitmap provided by the ParallelExtensionsExtras libraries or use our own implementation that allows you to lock just the region of the image you want to read/write. You can download class file and samples from the the project url:

        http://www.eng.utah.edu/~cs5955/homework/finalProject/FastBitmap.cs

        http://www.eng.utah.edu/~cs5955/homework/finalProject/FastBitmapExamples.cs

GDI+ Graphics

While you can always work on a pixel by pixel level, it may be easier for you to learn how to use .net’s GDI+ APIs. The main class that one uses to draw on an image is the System.Drawing.Graphics class. There’s one method in particular that should help you to draw a rectangle image onto another:

        Graphics.DrawImage(Image, Rectangle)

See: http://msdn.microsoft.com/en-us/library/yws82c40.aspx

Warning: I don’t believe this method is thread-safe, but it should be possible to create a task that’s iterating thru a BlockingCollection to grab the next available region to process.

Here are some more important API Links:

1. http://msdn.microsoft.com/en-us/library/system.drawing.graphics.fromimage.aspx

Miscellaneous Requirements

1. No third party software except for the files that we provide here, ParallelExtensionsExtras.dll, Alpaca, and .NET 4.0 framework libraries.
2. You must use all images in a given folder that have a matching region text file in the region files folder path. Do not hard code the images that we give you.
3. Your code should be able to handle less faces to replace with than the amount of faces to be replaced. Only the best matches should be replaced.
4. Your code should also be able to handle more faces to replace with than the amount of faces to be replaced.
5. We will give you text files that specify the location and size of the face(s) in the image. Feel free to use your own images and create their region files too.

Project Report

1. Specify your final algorithm (i.e. updated from your design document)
2. Create a Happens Before graph of your algorithm.
3. Identify the heuristic(s) you used for creating a rank when comparing image regions.
4. Demonstrate the stages from your algorithm in the form of performance test snapshots that show the parallelization utilized in your program for these stages. You should be able to see multiple threads performing the same task but on a different piece of work/data.
5. Show example input and outputs.
6. Include things you learned, challenges, hangups
7. (Optional) Include any comments (good or bad) about this course. How can we make it better? What did we do right? How were the course materials?

Grading Rubric

This project is worth 50% of the grade for the course.

1. Design Document (20pts)
2. Command-line driver program (20pts)

1. correct argument handling (even with invalid inputs)
2. correct output text
3. correct output image file
4. instructions file (or via /? cmd line option).

3. Read input and target files and associated region text files (5pts)
4. Computing average color and std. dev. of each color component for each input and target region. (10pts)
5. Bipartide matching using the average color and standard deviation. (10pts)
6. Composing final image, resizing input image region to the same size as the target region, and saving to target file (10pts)
7. Use of concurrency and parallelization constructs. (30pts)
8. Final Report (20pts)
9. For 3-credit students: (20pts)

1. Heuristics data computed using only the pixels in the oval specified by each input/target region.
2. Composing the final image by only copying over the pixels in the input oval.

10. Extra Credit: Blurred Edges (10pts)

Deliverables

1. Design Document (should already be turned in)
2. Source Code in a zip/tar file
3. Project Report - as a file called: report.pdf
4. Instructions document: instructions.txt/doc/pdf

1. Include any instructions regarding how to do any extra tasks.
2. If you make the instructions part of a /? commandline argument, this document need only state so.
3. Indicate how to control blur variables.

Handin Instructions

        handin cs5955 finalProject report.pdf myCode.zip instructions.txt