Escaping the harsh realities of the year 2020 has taken on various forms. For some, the year 2020 has been rife with reading, podcast listening, or late-night binging of the latest Netflix series. For me, nothing says departure from reality better than building an artificially intelligent agent to outperform a duo of ghosts who are hell-bent on eating it. Yes, you guessed correctly, I’m talking Pac-man and I’m talking AI. Let’s get started!

The Game Set-Up

The game will be played on a tiled game board containing both our characters and objects. Time in this context works in a discrete manner rather than continuous, and as a consequence, the following discussion will refer to time in step (e.g. the kth time step). Within this world, there exists:

• Pac-man — The Protagonist
• Inky and Clyde — The Antagonists
• Pellet — The food source of our hungry friend
• Power Pellet — the object that renders Pac-man’s adversaries edible.

Figure 1: Pac-man game board (Image by Author).

The interactions between these characters and the world around them are encapsulated within a complete board representation of the game — otherwise known as the state. Each state represents the world at a specific point in time and any action performed by either Pac-man or the ghosts will result in a change of state. The actions available to either Pac-man or the ghosts are movement-based actions that dictate whether the entity moves up, down, left, or right. Furthermore, like any good game, there exists a point system that is altered slightly from the traditional game.

• For every Pellet that is devoured, the score increases by 10.
• For every change of state, there incurs a living penalty of 1. This is specifically in place to prevent the AI agent from finding itself in a pattern that forever evades Inky and Clyde thereby making it impossible to complete the game.
• If the AI agent is able to complete the game, then there will be an increase of 500. The considerable increase incentivises the agent to finish the game.
• Similarly, a death penalty is served to de-incentivise the agent to exit the game by being eaten by the ghost.
• Eating either ghost after devouring a Power Pellet, results in a reward of 200.

#mathematics #pacman #reinforcement-learning #data-science #inside-ai #ai