Hunt Sparra

Date of Award

2021

Document Type

Thesis

Degree Name

Bachelors

Department

Natural Sciences

First Advisor

Roy, Tania

Area of Concentration

Computer Science

Abstract

Since early A.I. research, board games such as checkers, chess, and go have been proving grounds for novel techniques. While A.I. has had great success with two-player competitive, deterministic games in recent years, little progress has been made in solving cooperativecompetitive games. This thesis explores one such game: Diplomacy. Set in pre-WWI Europe, the game forces players to negotiate with their 6 rivals as alliances and individuals vie for control of previous “supply center” cities. Prior work has produced rules-based A.I. that perform well but fall short of human-level play. More recent research has used machine learning to create A.I. that handily beat the rules-based A.I. but require a large dataset of human games for training. Over three iterations, this thesis develops Diplomacy machine learning A.I. that do not require a dataset, using the Cross-Entropy Method (CEM) and Deep-Q Networks (DQN). In Iteration 0, several hyperparameters and game state representations are rapidly tested to discover which hold the most potential for CEM and DQN in Diplomacy. Iteration 1 tests the improvement of basic CEM and DQN agents, using the hypermeters and representations discovered in Iteration 0, over 100 thousand games. Iteration 2 improves upon both A.I. techniques by “masking” invalid actions, preventing them from being chosen. By the end of Iteration 2, both CEM and DQN dominate in a simplified Diplomacy environment against a random A.I.

Download

Share

COinS