======Game AI Problem Solving Challenge, Fall 2017 ======

**Design & Code: [[http://ru.is/~davidthue|David Thue]]** ([[http://cadia.ru.is|CADIA]] & SCS @ RU)
 
=====Introduction=====
Artificial Intelligence (AI) is an important part of many computer games, where it's often used to create dynamic challenges for players to overcome. In this problem solving task, your objective is to create an AI controller for a space ship in a simple 2D game. Survive by dodging incoming hazards, and earn a high score in Challenge Mode by shooting asteroids along the way. Welcome to Code Ship.

{{ :public:problem_solving_challenge_2015:codeship.png?nolink&600 |}}
=====Getting Oriented=====

The screen is split into two main parts: the Playing Field on the left, and the Code Window on the right. By putting code into the Code Window, you can control the orangey/grey space ship that starts near the bottom of the Playing Field. 

====The Playing Field====

The Playing Field shows you the action in the game. Click the Play button to start the action, the Pause button to pause time during a run, and and the Reset button to start a new run. Hovering with your mouse will show tool tips that describe the different buttons.

===Training Mode vs. Challenge Mode===

The game can be run in two modes. In **Training Mode**, you can watch your ship as it flies through a stream of asteroids that will always be the same after each reset, and you can even pause and tweak your code partway through a run. This makes Training Mode really useful for testing your code, but any high scores that you tally there won't get counted. The asteroid field in Training Mode is infinitely long, so you can practice there as much as you want.

In **Challenge Mode**, your ship will fly through a different stream of asteroids, but you won't be able to see what happens and you won't be able to edit your code during a run. This mimics what it might be like to actually send an AI probe into space; you can't be beside it to watch what happens first-hand, but you can get some reports back from its sensors. The asteroid field in Challenge Mode has a fixed length (24 Challenge Zones). Your progress will be shown on-screen, and your high scores will be saved when your ship meets its end or when you successfully complete a run.

To switch between modes, click the toggle button above the Playing Field and then reset the game.

====The Code Window====

The Code Window is your main interface to the game. While it supports the functions of a basic text editor, we **highly** recommend that you __write your code in an external editor__ (Notepad++ for Windows and Coda for Mac are both pretty good) and paste the code in for execution. To do so, use the Paste button at the bottom right of the Code Window. //Browsers can crash at any moment, so make sure your code is safe on the outside!//

**Your code will be executed once every second.** If you Pause the game, time will flow forward until the next second, your code will execute once, and then time will stop. Pressing Play at that point will cause time to resume. __You do **not** need to do anything special to make your code run again and again - it will happen automatically!__


=====Guts & Glory=====

To succeed in Code Ship, your AI controller will need to perform two kinds of task: **sensing the environment** and  **sending commands to your ship**.

====Sensing the Environment====

There are two things that your code can read about the environment:

  * **grid[][]**: a 2D array of strings with 7 rows and 5 columns. Each string is a single character and represents something on the Playing Field.
  * **weaponReady**: a bool about your ship's weapon.

For example, you can say things like:

    // print out grid cell at row 0, column 1
    print(grid[0][1]);
    
    // test if the gun has finished recharging
    if(weaponReady)
    {
        print("I can shoot now!");
    }

====Sending Commands====

There are two types of commands that you can send to your ship: movement and weapons
  * **moveX**: an integer between 0 and 4 stating which column the ship should move to.
  * **moveY**: an integer between 0 and 6 stating which row the ship should move to.
  * **shootFirst**: a bool stating whether the ship should try shooting before it moves.
  * **shootLast**: a bool stating whether the ship should try shooting after it moves.

For example, you can say things like:

    // move to row 5 (Y-axis), column 3 (X-axis)
    command.moveX = 3;
    command.moveY = 5;
    
    // try to shoot before moving
    command.shootFirst = true;     // like Han
    
    // try to shoot after moving
    command.shootLast = true;

**Your code will be __automatically__ executed once every second.**

=====Basic Syntax=====

Programming in Code Ship should feel familiar to anyone with general knowledge of C, C++, or JavaScript. Technically, it runs a simplified version of JavaScript (JS), so many functions that you could use in JS will work here as well.

====Variables====

Unlike in C or C++, you don't need to declare your variables, and their types will be automatically determined based on the data that you put inside them. Some examples:

    i = 0;            // makes a new integer called i because 0 is an integer;
    posX = 3.5;       // makes a new float called posX because 3.5 is a float
    output = "hello"; // makes a new string called output because "hello" is a string
    shootNow = false; // makes a new boolean called shootNow because false is a boolean

====Conditionals====

Conditionals are identical to C and C++. An example:

    // test grid location [0][2] and try shooting if there's an asteroid there and our weapon is ready
    if(grid[0][2] == "A" && weaponReady)
    {
        command.shootFirst = true;
    }
    
====Loops====

Loops are almost identical to C and C++; the only difference is that you don't need to declare your variables in 'for' loops. Some examples:

    // print out the grid (the upper left corner is 0, 0)
    // for each row i, step along the columns with j and add the grid cell [i][j] to our output string
    output = "";    
    for(i = 0; i < 7; i++)
    {
        for(j = 0; j < 5; j++)
        {
            output = output + grid[i][j];
        }
        output = output + "\n"; // add a line break in between rows
    }
    print(output);


    // search along row 0 for the first asteroid
    // stop when col points to its column or no asteroid is found
    col = 0;
    asteroidFound = false;
    while(!asteroidFound && col < 5)
    {
        if(grid[0][col] == "A")
        {
            asteroidFound = true;
        }
        else
        {
            col++;
        }
    }
    

====Infinite Loops (or, "How to Crash your Browser")====

An infinite loop is a loop whose condition never becomes false. Infinite loops will crash Code Ship, and probably your browser along with it. If, for some reason, you'd like to see that happen, try entering the code below. **Warning**: you will lose everything that you've entered into the game!

    while(true)
    {
    }

If, on the other hand, you would rather **avoid** this sort of thing, please try to avoid writing infinite loops in your code. If you happen to write one anyway, just refresh the page to start over. (//You'll still have a copy of your code in an external editor, right?//) ...and don't forget: your code will be run __automatically__ once every second - you don't need to write your own infinite loop!
=====Grading=====

This project is meant to be tackled in teams of two, and your team's grade will be based on the highest score that your ship earns in Challenge Mode. To have your score be counted, be sure to **enter the names and kennitalas of your team members at the top of the game's interface**, and **ensure that you submit at least one score in Challenge Mode**. Scores are submitted (only in Challenge Mode!) whenever either your ship explodes or you complete a run. Only scores earned in Challenge Mode will count toward your grade; scores earned in Training Mode will not count. 

=====Hand-ins=====

To receive a grade for your ship's performance, you must **register your team in Canvas** and **submit a .txt file containing your ship's code**.

=====Out in the Real World=====

The challenge of automatically controlling an agent in a real-world environment (like a space ship in an asteroid field) is generally **really** hard, and computer games can provide a nicely simplified training ground for trying out new algorithms. Researchers at Google's DeepMind have been applying an AI technique called "Deep Learning" to this sort of problem, and the results are pretty impressive: the algorithm can "learn" to play Atari 2600 games from only the pixels on the screen and the current value of the score. Here's a video that shows Deep Learning in action, trying to learn Atari Breakout:

[[https://www.youtube.com/watch?v=V1eYniJ0Rnk|Google DeepMind's Deep Q-learning playing Atari Breakout]]

...and here's an article about it in the journal Nature: 

[[http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html|Human-level control through deep reinforcement learning]]

**Good luck!**