Last Modified: May 14, 2017
CS 175: Project in AI (in Minecraft): Spring 2017
Project Status
Sameer Singh
Like we discussed in our individual meetings, the status report consists of a prototype, working implementation
of your proposed idea. By working, I mean that the environment may be limited in size and difficulty, but the
system should be generally achieving the goal you proposed. The main idea is to consider your status report as a
complete version of the final report, but one that is solving a simpler problem. If you are not clear about what the
scaled down setup for your particular project should be, get in touch with me (if you cannot find a time, email me).
The status submissions add up to a total of
100 points
(which makes up 30% of your final grade), 80% of
which are given to the group as a whole, and 20% specific to individuals. The breakdown of the points is as below:
Clean up Website 20
Technical Description Report 30
Video Summary 30
Peer Grading 20
Total 100
1 Cleaning up the Website (20 points)
Given that you are now progressing on your project, it is time to update the website to look better.
Project/Team Name:
Most of the teams picked very unimaginative titles for their teams and repos, like
“CS175-project”, “Minecraft-project”, or even the placeholder “PROJECT”. Now, pick a short name that is
more interesting, funny, or random, and if it’s descriptive of your project, perfect! Some examples of good
names from the proposals include “Torchlight”, “LARS” (for Lava Avoiding Reward Searcher), “Malrio”, and
Rename the Repository:
Using the above team name to rename the repository you are using for your code.
The owner of the repository should go to Github, click on “Settings” (of the repo), change the “Repository
name” to the one you chose, and click “Rename” and confirm the change. Your project website should now
be hosted on the new URL (the old name will still work temporarily for pushing and pulling). Submit the
new URL on Canvas.
Change the config file:
Also update the project title and the subtitle in the file
in the
folder. This will change all the pages to have the correct title as the header.
Improving the First Page:
Most of you did not change the first page,
, at all (understandably,
since it was not required). Well, now it is. Change the main page of the project to be something more
interesting, by including things like images/screenshots that are appropriate to your project, links to you
source code repository, links to pages, videos, or other resources you found relevant, and/or a summary of
your project.
2 Technical Description (30 points)
The remaining submissions for the status report should be included on the website by adding a filename called in your repo’s /docs folder. This file should start exactly with the following lines:
1 ---
2 layout: default
3 title: Status
4 ---
Apart from the video, this page should contain the following
create level-2 headers for these by starting the lines with ##
Project Status UC Irvine 1/ 3
CS 175: Project in AI (in Minecraft) Spring 2017
Project Summary:
Since things may have changed since proposal (even if they haven’t), write a short
paragraph summarizing the goals of the project (updated/improved version from the proposal).
Give a detailed description of your approach, in a few paragraphs. You should summarize the
main algorithm you are using, such as by writing out the update equation (even if it is off-the-shelf). You
should also give details about the approach as it applies to your scenario. For example, if you are using
reinforcement learning for a given scenario, describe the MDP in detail, i.e. how many states/actions you
have, what does the reward function look like. A good guideline is to incorporate sufficient details so that
most of your approach is reproducible by a reader. I encourage you to use figures, as appropriate, for this,
as I provided in the writeup for the first assignment (available here:
aiproj/sp17/assignments.html#assignment1). I recommend at least 2-3 paragraphs.
An important aspect of your project, as we mentioned in the beginning, is evaluating your
project. Be clear and precise about describing the evaluation setup, for both quantitative and qualitative
results. Present the results to convince the reader that you have a working implementation. Use plots, charts,
tables, screenshots, figures, etc. as needed. I expect you will need at least a few paragraphs to describe each
type of evaluation that you perform.
Remaining Goals and Challenges:
In a few paragraphs, describe your goals for the next 2-3 weeks, when
the final report is due. At the very least, describe how you consider your prototype to be limited, and what
you want to add to make it a complete contribution. Note that if you think your algorithm is quite good,
but have not performed sufficient evaluation, doing them can also be a reasonable goal. Similarly, you may
propose some baselines (such as a hand-coded policy) that you did not get a chance to implement, but
want to compare against for the final submission. Finally, given your experience so far, describe some of the
challenges you anticipate facing by the time your final report is due, how crippling you think it might be,
and what you might do to solve them.
You might need to include some math to describe what you are doing. To enable
math mode, you will need to update your
either copying over the latest version from
, or just copy the few
lines from
. This
change also gives an example of how to include math in your
files, i.e. by
putting L
X formatted equations between pairs of $$ .
3 Video Summary (30 points)
One of the submissions for the status is to create a video summarizing your progress so far. The video is limited
three minutes
. The video should contain a brief problem description (using images, screenshots, or screen
captures), an example capture in Malmo of how a simple baseline performs (such as at the beginning of training),
and an example capture of a run that is working. You are free to include more details, such as summary of how
you did it, some of the failure cases, or what you plan to do in the remaining weeks, but it is not needed.
You should embed the the video on the status report page on the website. For example, if you upload the
video on YouTube, add the embed video code provided by YouTube to the top of
. The video should be
of reasonably high quality, i.e. a minimum resolution of 1200
720 (i.e. 720p), and speech, if any, should be
comprehensible. I will create a thread on Piazza to discuss video logistics (please contribute as well as ask!).
4 Peer Grading (20 points)
In order to better familiarize yourselves with the other projects, and to train you to read and evaluate other
people’s work, we will be asking each of you to grade the progress report for
other projects. The peer review
assignments will be assigned individually, and thus the 20 points for completion are individual-specific, not group
specific. Your grading assignments for the projects will show up on Canvas, and has to be completed by
June 2nd,
. Note that the peer grades will not be the final grade for the project, but they will definitely be taken into
Project Status UC Irvine 2/ 3
CS 175: Project in AI (in Minecraft) Spring 2017
4.1 Grading the Website
Given the project assigned to you, check out the website and the video summary. You should grade the website
on the requirements above: Is the team name unique to the project? Is the first page not the default one, and
is somewhat interesting? Does the team page look cleanly formatted? Overall, does it look like a website with
polished content that can shared publicly? Give the appropriate score, and write a paragraph to justify it. You are
not supposed to evaluate the technical content of the website when doing this.
4.2 Grading the Video
Now, find the video on the status page, watch it, and evaluate it for the requirements from the video. Primarily,
was the video easy to understand and follow? Was the problem setup clear from the video? Was the general
approach evident? What did you think of the implementation, does it seem like a reasonable prototype of the
goals of the project? As before, provide a grade, and justify it in a paragraph, as the peer evaluator.
4.3 Grading the Report
Finally, read the status report, available on the
page, and evaluate it for the criteria described above.
Was the summary adequate for you to understand the problem? Is the approach described in the appropriate
level of detail? Were the experiments evaluating all the aspects of the project you think are important? Are there
obvious baselines the group did not take into account? What did you think of the results? Do you think the plan
for the next few weeks look reasonable? When providing the grade, justify it by using as many paragraphs as
needed (I expect at least one each for the approach and for the evaluation).
for some in-
structions on how to perform peer reviews in Canvas. A “peer review” rubric
is available for you to use when grading the above three parts of each project.
Project Status UC Irvine 3/ 3