Last Modiﬁed: May 14, 2017

CS 175: Project in AI (in Minecraft): Spring 2017

Project Status

Sameer Singh

http://sameersingh.org/courses/aiproj/sp17/

Like we discussed in our individual meetings, the status report consists of a prototype, working implementation

of your proposed idea. By working, I mean that the environment may be limited in size and difﬁculty, but the

system should be generally achieving the goal you proposed. The main idea is to consider your status report as a

complete version of the ﬁnal report, but one that is solving a simpler problem. If you are not clear about what the

scaled down setup for your particular project should be, get in touch with me (if you cannot ﬁnd a time, email me).

The status submissions add up to a total of

100 points

(which makes up 30% of your ﬁnal grade), 80% of

which are given to the group as a whole, and 20% speciﬁc to individuals. The breakdown of the points is as below:

Clean up Website 20

Technical Description Report 30

Video Summary 30

Peer Grading 20

Total 100

1 Cleaning up the Website (20 points)

Given that you are now progressing on your project, it is time to update the website to look better.

◦ Project/Team Name:

Most of the teams picked very unimaginative titles for their teams and repos, like

“CS175-project”, “Minecraft-project”, or even the placeholder “PROJECT”. Now, pick a short name that is

more interesting, funny, or random, and if it’s descriptive of your project, perfect! Some examples of good

names from the proposals include “Torchlight”, “LARS” (for Lava Avoiding Reward Searcher), “Malrio”, and

“Sandhog”.

◦ Rename the Repository:

Using the above team name to rename the repository you are using for your code.

The owner of the repository should go to Github, click on “Settings” (of the repo), change the “Repository

name” to the one you chose, and click “Rename” and conﬁrm the change. Your project website should now

be hosted on the new URL (the old name will still work temporarily for pushing and pulling). Submit the

new URL on Canvas.

◦ Change the conﬁg ﬁle:

Also update the project title and the subtitle in the ﬁle

config.yml

in the

/docs

folder. This will change all the pages to have the correct title as the header.

◦ Improving the First Page:

Most of you did not change the ﬁrst page,

index.md

, at all (understandably,

since it was not required). Well, now it is. Change the main page of the project to be something more

interesting, by including things like images/screenshots that are appropriate to your project, links to you

source code repository, links to pages, videos, or other resources you found relevant, and/or a summary of

your project.

2 Technical Description (30 points)

The remaining submissions for the status report should be included on the website by adding a ﬁlename called

status.md in your repo’s /docs folder. This ﬁle should start exactly with the following lines:

1 ---

2 layout: default

3 title: Status

4 ---

Apart from the video, this page should contain the following

create level-2 headers for these by starting the lines with ##

Project Status UC Irvine 1/ 3

CS 175: Project in AI (in Minecraft) Spring 2017

◦ Project Summary:

Since things may have changed since proposal (even if they haven’t), write a short

paragraph summarizing the goals of the project (updated/improved version from the proposal).

◦ Approach:

Give a detailed description of your approach, in a few paragraphs. You should summarize the

main algorithm you are using, such as by writing out the update equation (even if it is off-the-shelf). You

should also give details about the approach as it applies to your scenario. For example, if you are using

reinforcement learning for a given scenario, describe the MDP in detail, i.e. how many states/actions you

have, what does the reward function look like. A good guideline is to incorporate sufﬁcient details so that

most of your approach is reproducible by a reader. I encourage you to use ﬁgures, as appropriate, for this,

as I provided in the writeup for the ﬁrst assignment (available here:

http://sameersingh.org/courses/

aiproj/sp17/assignments.html#assignment1). I recommend at least 2-3 paragraphs.

◦ Evaluation:

An important aspect of your project, as we mentioned in the beginning, is evaluating your

project. Be clear and precise about describing the evaluation setup, for both quantitative and qualitative

results. Present the results to convince the reader that you have a working implementation. Use plots, charts,

tables, screenshots, ﬁgures, etc. as needed. I expect you will need at least a few paragraphs to describe each

type of evaluation that you perform.

◦ Remaining Goals and Challenges:

In a few paragraphs, describe your goals for the next 2-3 weeks, when

the ﬁnal report is due. At the very least, describe how you consider your prototype to be limited, and what

you want to add to make it a complete contribution. Note that if you think your algorithm is quite good,

but have not performed sufﬁcient evaluation, doing them can also be a reasonable goal. Similarly, you may

propose some baselines (such as a hand-coded policy) that you did not get a chance to implement, but

want to compare against for the ﬁnal submission. Finally, given your experience so far, describe some of the

challenges you anticipate facing by the time your ﬁnal report is due, how crippling you think it might be,

and what you might do to solve them.

You might need to include some math to describe what you are doing. To enable

math mode, you will need to update your

/docs/

layouts/default.html

either copying over the latest version from

https://github.com/sameersingh/

gh-skeleton/blob/master/docs/

layouts/default.html

, or just copy the few

lines from

https://github.com/sameersingh/gh-skeleton/commit/283c

. This

change also gives an example of how to include math in your

.md

ﬁles, i.e. by

putting L

X formatted equations between pairs of $$ .

3 Video Summary (30 points)

One of the submissions for the status is to create a video summarizing your progress so far. The video is limited

three minutes

. The video should contain a brief problem description (using images, screenshots, or screen

captures), an example capture in Malmo of how a simple baseline performs (such as at the beginning of training),

and an example capture of a run that is working. You are free to include more details, such as summary of how

you did it, some of the failure cases, or what you plan to do in the remaining weeks, but it is not needed.

You should embed the the video on the status report page on the website. For example, if you upload the

video on YouTube, add the embed video code provided by YouTube to the top of

status.md

. The video should be

of reasonably high quality, i.e. a minimum resolution of 1200

720 (i.e. 720p), and speech, if any, should be

comprehensible. I will create a thread on Piazza to discuss video logistics (please contribute as well as ask!).

4 Peer Grading (20 points)

In order to better familiarize yourselves with the other projects, and to train you to read and evaluate other

people’s work, we will be asking each of you to grade the progress report for

four

other projects. The peer review

assignments will be assigned individually, and thus the 20 points for completion are individual-speciﬁc, not group

speciﬁc. Your grading assignments for the projects will show up on Canvas, and has to be completed by

June 2nd,

2017

. Note that the peer grades will not be the ﬁnal grade for the project, but they will deﬁnitely be taken into

account.

Project Status UC Irvine 2/ 3

CS 175: Project in AI (in Minecraft) Spring 2017

4.1 Grading the Website

Given the project assigned to you, check out the website and the video summary. You should grade the website

on the requirements above: Is the team name unique to the project? Is the ﬁrst page not the default one, and

is somewhat interesting? Does the team page look cleanly formatted? Overall, does it look like a website with

polished content that can shared publicly? Give the appropriate score, and write a paragraph to justify it. You are

not supposed to evaluate the technical content of the website when doing this.

4.2 Grading the Video

Now, ﬁnd the video on the status page, watch it, and evaluate it for the requirements from the video. Primarily,

was the video easy to understand and follow? Was the problem setup clear from the video? Was the general

approach evident? What did you think of the implementation, does it seem like a reasonable prototype of the

goals of the project? As before, provide a grade, and justify it in a paragraph, as the peer evaluator.

4.3 Grading the Report

Finally, read the status report, available on the

status.html

page, and evaluate it for the criteria described above.

Was the summary adequate for you to understand the problem? Is the approach described in the appropriate

level of detail? Were the experiments evaluating all the aspects of the project you think are important? Are there

obvious baselines the group did not take into account? What did you think of the results? Do you think the plan

for the next few weeks look reasonable? When providing the grade, justify it by using as many paragraphs as

needed (I expect at least one each for the approach and for the evaluation).

Note:

See

https://community.canvaslms.com/docs/DOC-10651

for some in-

structions on how to perform peer reviews in Canvas. A “peer review” rubric

is available for you to use when grading the above three parts of each project.

Project Status UC Irvine 3/ 3