A puzzle for AI

A puzzle for AI

I will tell you about one of them — how I taught a Reinforcement learning (RL) agent to play the famous puzzle game called “2048”. The article deliberately will not contain code, mathematics, state-of-the-art approaches and the latest discoveries in the field, so people who are well acquainted with RL will not discover anything new for themselves.

Hi! My name is Rinat Maksutov, I work at Intelligent Engineering Services group of Accenture Technology in Russia, and I lead custom development projects. Over the course of my long career at Accenture, I have experienced many different areas: mobile development, front-end, back-end, and even data science with machine learning. However, my story will not be about work, but about a hobby. I really enjoy learning and exploring new areas via my personal pet projects. Today I will tell you about one of them — how I taught a Reinforcement learning (RL) agent to play the famous puzzle game called “2048”. The article deliberately will not contain code, mathematics, state-of-the-art approaches and the latest discoveries in the field, so people who are well acquainted with RL will not discover anything new for themselves. This article is a story for the general public about how I set myself an unusual goal and achieved it.

Our company invests heavily in ongoing employee training. For example, last year a program was launched where employees can take one of the Nanodegree on Udacity for free (Nanodegree is a set of several courses with a final project). I have already done Deep Learning Nanodegree on this platform, so this time I decided to take the course on reinforcement learning.

The course reveals the basics of RL very well, but has one big drawback: the educational projects that are offered on the course are based on ready-made tasks — in which the environment the agent operates in has already been written for you by someone. That is, you just have to implement the learning algorithm and tweak the hyperparameters in order to reach the target score and pass.

Therefore after completing the course, you will not be able to fully apply RL and solve your own problems, because you have studied only part of this area. And the questions of how to properly build an environment for an agent, how to formalize a task for it, how to correctly assign rewards for various actions — remain outside the brackets, and you will have to figure it out on your own (what all these terms mean, I will explain below).

To close this gap, I tried to solve some problem that had not been solved by anyone before (at least with the help of RL), and use it to study various aspects of building environments for agents. As such a task, a mechanically simple puzzle game 2048 was chosen (you can play it in the browser here: https://play2048.co/ or search on Google Play or App Store for your smartphone). In this game, the player, by moving cells in one of four directions (up, down, right, left), needs to combine cells with the same value, and try to collect a cell with the maximum possible value. When you make a shift, a new two (with the probability of 0.9) or a four (with the probability of 0.1) appears on a random free cell. The game ends when there are no empty cells left on the field and the player cannot combine any cells.

Despite the name, 2048 is not the maximum possible cell value in the game. With proper training, you can get to values 4096, 8192 cells, and so on. The maximum cell value theoretically possible is 131,072, that is 2 ^ 17

machine-learning artificial-intelligence reinforcement-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

AI(Artificial Intelligence): The Business Benefits of Machine Learning

Enroll now at CETPA, the best Institute in India for Artificial Intelligence Online Training Course and Certification for students & working professionals & avail 50% instant discount.

Learning in Artificial Intelligence - Great Learning

What is Artificial Intelligence (AI)? AI is the ability of a machine to think like human, learn and perform tasks like a human. Know the future of AI, Examples of AI and who provides the course of Artificial Intelligence?

What is Machine learning and Why is it Important?

Machine learning is quite an exciting field to study and rightly so. It is all around us in this modern world. From Facebook’s feed to Google Maps for navigation, machine learning finds its application in almost every aspect of our lives. It is quite frightening and interesting to think of how our lives would have been without the use of machine learning. That is why it becomes quite important to understand what is machine learning, its applications and importance.

Artificial Intelligence, Machine Learning, Deep Learning 

Artificial Intelligence (AI) will and is currently taking over an important role in our lives — not necessarily through intelligent robots.

How To Get Started With Machine Learning With The Right Mindset

You got intrigued by the machine learning world and wanted to get started as soon as possible, read all the articles, watched all the videos, but still isn’t sure about where to start, welcome to the club.