Minecraft as a Platform for Project-Based Learning in AI

I have used Minecraft as the platform for teaching AI via project-based learning. Minecraft is an open-world sandbox game with elements of exploration, resource gathering, crafting, construction, and combat, and is supported by the Malmo library that provides a programmatic interface to the player observations and actions at various levels of granularity. In Minecraft, students can design projects to use approaches like search-based AI, reinforcement learning, supervised learning, and constraint satisfaction, on data types like text, audio, images, and tabular data. I have offered an open-ended, undergraduate AI projects course using Minecraft that includes 90 different projects, covering themes that ranged from navigation, instruction following, object detection, combat, and music/image generation.

This work was also published as the following:

  • Sameer Singh.Minecraft as a Platform for Project-Based Learning in AI. AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI). 2020 Conference
    PDFWebsitePosterSpotlightAAAI Page, Abstract, BibTex ]
    Undergraduate courses that focus on open-ended, projectbased learning teach students how to define concrete goals, transfer conceptual understanding of algorithms to code, and evaluate/analyze/present their solution. However, AI, along with machine learning, is getting increasingly varied in terms of both the approaches and applications, making it challenging to design project courses that span a sufficiently wide spectrum of AI. For these reasons, existing AI project courses are restricted to a narrow set of approaches (e.g. only reinforcement learning) or applications (e.g. only computer vision).
    In this paper, we propose to use Minecraft as the platform for teaching AI via project-based learning. Minecraft is an open-world sandbox game with elements of exploration, resource gathering, crafting, construction, and combat, and is supported by the Malmo library that provides a programmatic interface to the player observations and actions at various levels of granularity. In Minecraft, students can design projects to use approaches like search-based AI, reinforcement learning, supervised learning, and constraint satisfaction, on data types like text, audio, images, and tabular data. We describe our experience with an open-ended, undergraduate AI projects course using Minecraft that includes 82 different projects, covering themes that ranged from navigation, instruction following, object detection, combat, and music/image generation.
    @inproceedings{malmo:eaai20,
      author = {Sameer Singh},
      title = { {Minecraft as a Platform for Project-Based Learning in AI} },
      booktitle = {AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI)},
      doi = {10.1609/aaai.v34i09.7070},
      pages = {13504-13505},
      year = {2020}
    }

Offerings

Examples

These projects are not neccesarily the best, but they capture the variety of projects that have been proposed and implemented by the students. The websites below link to the YouTube videos of their “reports”, and their code is available online as well.

Acknowledgements

I would like to thank Microsoft Research for providing and supporting Malmo, Moshe Lichman for his help in the first offering, and other staff that helped made this course possible, Zhe Wang, Stephen McAleer, and Yasaman Razeghi.