Advanced AI in Games

Advanced AI in Games

[Course Plan][Projects][Literature][F.E.A.R. Platform Demos]

Lectures: Tuesdays (and Wednesdays), Fall 2008 - 10:00-12:00 room 4A22

Lab (Exercise) Sessions: Wednesdays 13:00-15:00 @ "small" GameLab (2A54)

Teachers: Georgios Yannakakis (yannakakis[at]itu[dot[dk) (office: 4B09, phone: 7218 5078)
Teaching assistants: André T. Johansen (atjo[at]itu[dot]dk)

Course Description: The primary goal of the course is the understanding, design, implementation and use of nouvelle AI techniques for generating efficient intelligent behaviors in games. Additional focus will be given to state-of-the-art AI algorithms for improving gameplay experience.

During the course students will learn to

Identify tasks that can be tackled through advanced AI techniques and select the appropriate technique for the problem under investigation.

Design and implement efficient and robust advanced AI algorithms

Evaluate and test the algorithms in commercial-standard game productions.

Course Format: Teaching consisting of 6 weeks of intensive lectures and mandatory individual exercises and 8 weeks of independent group work on a game production. Lectures take place on Tuesdays from 10:00 to 12:00. There will be occasional additional lectures on Wednesdays from 10:00 to 12:00 during the first 6 weeks of the course. Lectures from industry people and other AI academic specialists are planned during the second 8-week phase of the course.

The hand-in: The final product for this course is a written project that includes:

Report (for the group project)
CD with the game
A video demo of obtained AI behaviors that includes game-play footage
N.B. A report of the individual project will be submitted in week 7 of the course. Students must pass their individual assignment in order to be eligible for the final examination of the course.

Literature: Articles/Chapters available from this webpage.

Examination: External examiner, 7-point marking scale, B4: Oral examination with written work but without time for preparation at the exam. The hand-in must be submitted by 17-Dec at the Exam Office no later than 15:00. Exams then take place 21-Jan 2009.

Students Eligible for the Exam:

Morten Silcowitz

Antonio Albuquerque Leitao

Patryk Makowski

Jens Peter Rosenkvist

Kevin Hejn

Course E-mail: mvai_e2008@itu.dk

Course Blog

Lecture Slides

Course plan (subject to change)

(last updated: 06/11-08)

Week	Date	Lectures	Readings	Individual Assignments (Production Tasks)	Group Project (Production Tasks)
1	26-Aug	Introduction - Lecturer: Georgios Yannakakis Class content, structure and projects Objectives of the course Explain what is possible on this course and expectations for final projects. Levels of control for AI	none	Download the LUDO game, the java class ANN + GA templates Download the Report template for Word and LaTeX users.	none
2	02-Sep	The Real Introduction - Lecturer: Georgios Yannakakis What is Game AI; why is it useful? Pathfinding Believable AI Agent Design Finite State Machines	How to report and present scientific work: [Link] AIGPW2: S. Rabin, Common and Promising Game AI Techniques	Begin to work/familiarize with the LUDO game Play LUDO and think about potential modeling of your LUDO agents: input (perception), output (action)	All: Getting in teams All: Team name, member names and email addresses sent to Georgios before week 3 (09-Sep).
2	03-Sep	Introduction to Artificial Neural Networks - Lecturer: Georgios Yannakakis	Chapter 3 of Course Notes		Own Game: Make a 1-page game concept document (See Tim Ryan's "Guidelines for a Game Concept"., Game Development class guidelines) Own Game: The document should clearly state the two advanced AI control levels you are considering. Own Game: Send concept doc as a pdf - with your team name clearly stated - to Georgios before next Tuesday (09-Sep). (Note if you have not finalized a team name you need to do so.) Ms. Pacman: Send team name to Georgios before next Tuesday (09-Sep). (Note if you have not finalized a team name you need to do so.) Ms. Pacman: Download the required screen-capture software and familiarize with it.
3	09-Sep	Multi-Layered Artificial Neural Networks - Lecturer: Georgios Yannakakis	Chapter 4 of Course Notes AIGPW: A. Champandard, The Dark Art of Neural Networks AIGPW2: P. Sweetser, How to Build Neural Networks for Games	Work on the Artificial neural network (trained through back-propagation/supervised learning) approach in LUDO	none
3	10-Sep	Genetic Algorithms - Lecturer: Georgios Yannakakis	Chapter 9 of Course Notes AIGPW: F. D. Laramee, Genetic Algorithms: Evolving the Perfect Troll AIGPW2: P. Sweetser, How to Build Evolutionary Algorithms for Games	Work on the Artificial neural network (trained through back-propagation/supervised learning) approach in LUDO	none
4	16-Sep	Fuzzy Logic - Lecturer: Georgios Yannakakis	PGAI: Chapter 10 AIGPW: T. Alexander, An Optimized Fuzzy Logic Architecture for Decision-Making	Report experiments, results and conclusions of the ANN approach	none
4	17-Sep	Reinforcement Learning - Lecturer: Georgios Yannakakis	Reinforcement Learning: A Survey by Kaelbling et al. AIGPW2: J. Manslow, Using RL to Solve AI Control Problems G. Teusaro, TD-Gammon	Work on the Genetic Algorithm approach in LUDO	none
5	23-Sep	Top 5 reasons not to use AI in Games and Next-gen Turing Tests - Lecturer: Peter Andreasen, IO-Interactive	none	Work on the Genetic Algorithm approach in LUDO	none
5	24-Sep	Hybrids - Lecturer: Georgios Yannakakis	S. W. Wilson, Classifier fitness based on accuracy O. Sigaud, and S. W. Wilson, Learning classifier systems: A survey AIGPW3: R. Cornelius et al, Constructing Adaptive AI Using Knowledge-Based NeuroEvolution Stanley et al., Real-time neuroevolution in the NERO video game	Report experiments, results and conclusions of the GA approach	none
6	30-Sep	Advanced Topics 1 - Lecturer: Georgios Yannakakis	AIGPW2: R. Houlette, Player Modeling for Adaptive Games AIGPW3: P. Spronck, Dynamic Scripting Yannakakis and Hallam, Towards Optimizing Entertainment in Computer Games Andrade et al., Automatic computer game balancing: a reinforcement learning approach Spronck et al., Adaptive game AI with dynamic scripting	Work on the Reinforcement Learning approach in LUDO	none
6	01-Oct	Good Experimental Methodology - Lecturer: Prof. John Hallam, University of Southern Denmark	Eiben and Jelasity, A Critical Note on Experimental Research Methodology in EC	Work on the Reinforcement Learning approach in LUDO Report experiments, results and conclusions of the RL approach Submit best LUDO player with instructions on how to install/run to Andre by Friday, 03-Oct, 15:00	none
7	07-Oct	LUDO Tournament/Competition - Organizers: André T. Johansen/Georgios Yannakakis	Hand-In Guidelines and Report template [Word, LaTeX]	Compare all three approaches Add an overall conclusion/discussion section in your report including competition outcome. Hand-in LUDO report (including tournament outcome discussion) and CD with java classes to Georgios by Friday, 10-Oct, 15:00.	Own Game: Think about which tools (approaches) will solve the problems you are considering; revisit your concept document and apply changes. Own Game: Apply all lessons learned during the LUDO AI development to your group project; any additions/changes in your concept document? Ms. Pacman: follow Own Game tasks (excluding revision of concept document).
8	14-Oct	Autumn Holiday Week
9	21-Oct	Advanced Topics 2 - Lecturer: Georgios Yannakakis Notification of LUDO report decision	Picard et al., Toward machine emotional intelligence: analysis of affectivephysiological state Yannakakis et al., Entertainment Capture through Heart Rate Activity in Physical Interactive Playgrounds Rani et al., Maintaining Optimal Challenge in Computer Games through Real-Time Physiological Feedback		Own Game: Make a level, try playing it. Revisit concept document: Does it need to be changed? Own Game: Is this level suitable for the problems/tasks under consideration? Does it need to be changed? Ms. Pac-Man: Work on your first AI approach
10	28-Oct	Project Work (TA available during lab session hours to assist)			Own Game: Polish level and characters All: Work on your first level of advanced AI control/Ms Pac-Man AI approach
11	4-Nov	Project Work (TA available during lab session hours to assist)			All: Work on your first level of advanced AI control/Ms Pac-Man AI approach
12	11-Nov	Project Work (TA available during lab session hours to assist)			All: Evaluate/Test your first level of advanced AI control/Ms Pac-Man AI approach. All: Work on your second level of advanced AI control/Ms Pac-Man AI approach.
13	18-Nov	Project Work (TA available during lab session hours to assist)			All: Work on your first level of advanced AI control/Ms Pac-Man AI approach.
14	25-Nov	Project Work (TA available during lab session hours to assist)			All: Evaluate/Test your second level of advanced AI control/Ms. Pac-Man AI approach
15	2-Dec	Project Work (TA available during lab session hours to assist)			All: Make a marketing trailer for your advanced game AI. Include game-play footage.
16	9-Dec	Project Work (TA available during lab session hours to assist)	Hand-In Guidelines and Report template [Word, LaTeX].		All: Polish final hand-in, including demo video.
17	16-Dec	Project Work (TA available during lab session hours to assist)			All: Polish final hand-in, including demo video.
	17-Dec	Hand-In due (before 15:00)
	t.b.a.	Play Day/Competition
	21-Jan	Exam

Projects

There are two classes of student projects for this course; individual mandatory assignments and group projects. Individual assignments involve the well-known LUDO (developed by David Christensen and Kasper Stoy) game. Each student has to generate three (3) different LUDO players to compete against the hand-crafted agents that come with the LUDO software (see details below).

There are two alternatives for group projects (see section Group Projects below)

Individual (Mandatory) Assignments: LUDO

LUDO is well-documented and implemented in java. Download the source code, the instructions of the game simulator as well as the NeuralNet and EA (GA) template classes.

You have to generate three different intelligent LUDO players using the following approaches:

Multi-Layer Perceptron (back-propagation/supervised learning)
Genetic Algorithms
Reinforcement Learning

Competition

There will be a competition day where all generated agents will compete against each other. Students will have to report on their methodology and results by evaluating each approach used. See hand-in guidelines.

Group Projects

Groups can choose among the following alternatives regarding their project

1. Own Game: Built two levels of advanced AI (Fuzzy, Neural Networks, GA, RL, hybrids) control in a game platform of your choice.

2. Ms. Pac-Man: Built two efficient Ms Pac-Man players.

Own Game

1. Choose a game platform/framework among the following

Unreal Engine 3 (highly recommended - technical support provided through the game development class webpage and the TA)
XNA
Source (not recommended)
.NET
Java
Other (you may propose any platform/framework/language you are most familiar with)

N.B. For any platform used you have to demonstrate demos of successful previous work.

2. Built 2 different levels of control using any advanced AI techniques covered in class. More specifically, the game:

Should be graphically (preferably) 3D. 2D is also acceptable.
Can be created in any game engine (see above); focus on AI; e.g. don't built a level from scratch, use available levels/games in Unreal/XNA etc.
Must embed two levels of advanced AI control

3. Your hand-in report should include results from testing and evaluation experiments as well as a video trailer of the obtained behavior (see hand-in guidelines)

Ms Pac-Man

The Microsoft Ms Pac-Man version (installed in all machines of Game Lab) and the screen-capture software developed by Jonas Flensbak will be used for your experiments. Download Jonas' .NET framework and documentation from here. Jonas can provide technical support if required.

Although computational intelligence methodologies (GAs, ANNs, Fuzzy, RL) are encouraged, groups can choose any two advanced AI approaches covered in the class (combinations are also possible, e.g. an RBS with an ANN). Other methodologies beyond the material covered in the class can also be proposed and discussed.

The best Ms. Pac-Man controller will enter the competition at the next Computational Intelligence and Games (CIG) Symposium. There will eventually be a first prize for the winner of the competition (that was 500 $ (!) at the Congress on Evolutionary Computation 2007). Fame is also included in the prize :)

Your hand-in report should include results from testing and evaluation experiments as well as a comparative study among the two Ms. Pac-Man behaviors obtained (see hand-in guidelines). A trailer video of the two obtained behaviors should accompany the report.

Literature

Course Notes: Peter Ross and John Hallam, Neural networks: an introduction, AI Applications Institute, University of Edinburgh, 1999

AIGIS: Michael Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems, Addison Wesley, 2004

AIGPW: Steve Rabin (editor), AI Game Programming Wisdom , Charles River Media, 2002

AIGPW2: Steve Rabin (editor), AI Game Programming Wisdom 2, Charles River Media, 2004

AIGPW3: Steve Rabin (editor), AI Game Programming Wisdom 3, Charles River Media, 2006

PGAI: Mat Buckland, Programming AI by Example, Wordware Publishing, 2005

Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, Prentice-Hall, 2002.

FEAR Platform Demos

Game AI behaviors can be observed using the FEAR platform for Quake 2 which accompanies the book "Alex J. Champandard, AI Game Development: Synthetic Creatures with Learning and Reactive Behaviors, New Riders Publishing, 2004.". Further on-line AI game development guidelines can be found in the book's website.

The online guide for setting up and using the FEAR platform.

Quick Guide (FEAR Platform Installation for watching animat demos)

Download the FEAR SDK, the Binary FEAR demo, the Quake 2 Demo and the latest Quake 2 binary patch.
Install Quake 2 demo: Click on the executable, and follow the on screen instructions to place the demo in C:\Games\Quake2
Install Quake 2 patch: Be sure to install the patch exactly in the same location as the Quake 2 game. By default this is C:\Games\Quake2 (just copy the extracted files in that directory).
Install the full FEAR SDK: Click on the executable, and follow the on screen instructions.
Extract the Binary FEAR demo in your desktop
Right click on My Computer, then select Properties. From there, there should be an Advanced tab --> Environment Variables. Set a new variable QUAKE2 (if it does not exist) to the value C:\Games\Quake2.
Game DLL: Copy the gamex86.dll from the extracted binary FEAR demo subdirectory \aigd-1.0.0-demo\Quake2\fear and paste it in C:\Games\Quake2\fear
Copy all .dll "brain" files from the extracted binary FEAR demo subdirectory \aigd-1.0.0-demo\Quake2\fear\modules and paste them in C:\Games\Quake2\fear\modules
Create a server: Create a shortcut of c:\Games\Quake2\quake2.exe in your desktop (or wherever you like), right-click it --> properties --> Shortcut tab --> Target : C:\Games\Quake2\quake2.exe +set game fear +set dedicated 1 +set deathmatch 1 +map q2dm1
Player mode: Follow 9 and set Target : C:\Games\Quake2\quake2.exe +set game fear +connect 127.0.0.1
Spectator mode: Follow 9 and set Target : C:\Games\Quake2\quake2.exe +set game fear +connect 127.0.0.1 + set spectator 1
Run the server.
You can add a Number of animats by writing sv add AnimatName Number on the server window. You can set the Number of animats through the bots Number command.
Run either player or spectator mode to see how your animats behave.