Stanford University CS 223B: Introduction to Computer Vision

CS 223B Course Project

Two Project Choices -- Pick One of The Two:

1. The "Find Mii" Challenge: You will make a computer vision algorithm to play a Wii game for finding target of interest.

2. Open Research Project: You need to propose an original research topic, or replicate an existing paper. Need instructor's approval.

Important Dates

All dates and restrictions (if not specified otherwise) apply to the open project as well as the Find Mii challenge.

Jan 28 , Finalizing team members : Maximum team size: 2. Send us an email with your team name and team members.

Jan 28 , Proposal submission (open project only) : Submit a 0.5 page course project proposal.

Mar 16 , Final program submission : No late days allowed. Specific time of the day to be annouced.

Mar 17 , Final writeup submission : No late days allowed. Specific time of the day to be annouced.

Mar 19 , Course project presentation and winner demos : We look forward to seeing your great work!

We don't set milestones for the project. But make sure to get it started as early as possible.

Write-up Format

We have provided the template for your final write-up. For open project, your proposal should follow the same template, and should be no more than 1 page. Your proposal should describe as clearly as possible the following,

•What is the computer vision problem that you will be investigating? Why is it interesting?

•What image or video data will you use? If you are collecting new datasets, how do you plan to collect them?

•What method or algorithm are you proposing? If there are existing implementations, will you use them and how? How do you plan to improve or modify such implementations?

•Which reading will you examine to provide context and background?

•How will you evaluate your results? Qualitatively, what kind of results do you expect (e.g. plots or figures)? Quantitatively, what kind of analysis will you use to evaluate and/or compare your results (e.g. what performance metrics or statistical tests)?

Given the template, your final write-up should be between 8 - 10 pages. The following is a suggested structure for your report,

•Title, author

•Abstract: It should not be more than 300 words;

•Introduction: this section introduces your problem, and the overall plan for approaching your problem;

•Background/Related Work: This section discusses relevant literature for your project;

•Approach: This section details the framework of your project. Be specific, which means you might want to include equations, figures, plots, etc;

•Experiment: This section begins with what kind of experiments you're doing, what kind of dataset(s) you're using, and what is the way you measure or evaluate your results. It then shows in details the results of your experiments. By details, I mean both quantitative evaluations (show numbers, figures, tables, etc) as well as qualitative results (show images, example results, etc).

•Conclusion: What have you learned? Suggest future ideas.

•References: This is absolutely necessary. Reports without references will not receive a score higher than 20 points (total is 40 points).

•Supplementary materials: This is NOT counted toward your 8-10 page limit. Please submit your codes as supplementary materials.

Honor Code

You may consult any papers, books, online references, or publicly available implementations (such as SIFT) for ideas and code that you may want to incorporate into your strategy or algorithm, so long as you clearly cite your sources in your code and your writeup. However, under no circumstances may you look at another group’s code or incorporate their code into your project.

The "Find Mii" Challenge

Find Mii on Wii

"Find Mii" is a game on Nintendo Wii Play. It basically involves identifying certian avatars (Miis) from a bunch of them, standing still or moving around in various styles. If you are not yet proficient in this game, let Elvis show you how to play it in the videos below.

As we know computers are designed to work in certain areas of human endeavor that are not terribly challenging to human intelligence but sometimes beyond human patience. In this course project you will be programming the computer to play the game as good as Elvis did.

You don't have to worry about any "interface" issue (as some of you asked in class). Actually you will only be dealing with images instead of programming the Wii game controller.

Project Mission

In the course project, you will be focusing on 4 tasks in "Find Mii" as described below.

Task 1 : Find this Mii! ... You will be given a reference picture of one Mii. Identify that one in a crowd.

Task 2 : Find 2 look-alikes! ... Pay attention to the faces (and hair styles?) as they might be wearing different sweaters.

Task 3 : Find n odd Miis out! ... Some Miis are odd in their styles of shaking heads or footsteps. Find them out.

Task 4 : Find the fastest Mii! ... Someone is running (or swimming) fast. Catch that one.

For each task you have 3 levels: easy, medium, and hard.

For each task & level we will give you a video file from gamplay (that would be 12 in total), and you need to identify a given number of Miis.

Minimum requirement: 3 different tasks, 1 level for each.

You will be handling different tasks & levels separately (of course you can have functionalities shared among them, just follow the comments in the infrastructure code). You don't have to do all the 12 task & levels, the minimum requirement is that you complete at least three different tasks, one level for each. However you are strongly encouraged to do all 4 tasks on higher levels to earn more points for the final challenge! (See the scoring rules for detail.)

Scoring

The performance of your program (your overall score in the challenge) will be calculated according to the following rules.

Score in a given task and level: A correct click on frame 1 is worth 1 point, each frame thereafter is discounted by 0.99. And you can NOT access (even read into memory) any frame after the one you made your final click for a task. Score is averaged over multiple clicks if applicable. For example, if a task requires two clicks, you return 1st click at frame 5, and 2nd click at frame 20 (both of them are correct), your score would be (0.99^4 + 0.99^19)/2 = 0.8934. In this case, you can NOT access any frame after frame 20.

Score in a given task: Your score in a given task is the highest score you achieve on the 3 levels. When comparing the scores on different levels, scores on level 2 are multiplied by 1.5, and scores on level 3 are multiplied by 2. For example, for task 1, your score is 1.0 on level 1, and 0.77 on level 2, and you are not doing level 3. Your final score for task 1 would be max(1*1, 0.77*1.5, 0*2) = 1.155

Overall score in the challenge: Your overall score in the challenge is calculated by summing up your scores in all 4 tasks.

Important Notice: Your programs will be scored on videos slightly different from the ones we gave you. So you should make sure that your programs work properly on different gameplays for the same task and the same difficulty level. In other words, do NOT incorporate into your program any highly biased assumptions that you get from watching the videos yourself, because that's not what computer vision is about. An easy way to verify this issue is to apply your algorithm to the same video but starting from different frames, and your method should be working properly in this situation.

Your project will be scored by its overall quality, including but not limited to the performance of your program, the quality of your code and writeup, and how innovative you are. Your writeup should include your general ideas of accomplishing the tasks, the high-level structure of your code, interesting experiments you did to validate any part of your method (with beautiful plots), and any other things that you think would interest us.

Data and Code

The data and infrastructure code for this project can be found in the AFS path: /afs/ir/class/cs223b/findmii

If you use OpenCV ...

Because we need to compile and run all your programs in 1~2 days for the final competition. Please connect your program with our Matlab infrastructure code (by following the comments therein), and make sure that your C++ program compiles and runs correctly on the 'corn' clusters with:

the header file path /afs/ir/class/cs223b/opencv/include

and the lib file path /afs/ir/class/cs223b/opencv/lib

Please follow the example given by 'Makefile' and 'opencvexample.cpp' in the project AFS path. Windows executables or Visual Studio projects will NOT work. . We apologize for the inconvenience, but this is the only feasible way to have all your programs automatically scored in a short time.

Hints for Winning the Challenge

1. Take a look at the data and infrastructure code as early as possible.

2. Be as innovative as you can.