Dragons and Data: Update #1

Teaching a computer to play a game is much more complicated than it sounds.

No, really, it only took me one summer in elementary school to become a videogame aficionado, so it seems intuitive that a computer with much more processing power than my developing adolescent brain could do it quicker! But computers have a much higher learning curve when it comes to abstract concepts like “learning.” If you think about your personal learning process, it’s very individualized: there are kinesthetic learners, vocal learners, visual learners, the list goes on and on. With computers, the whole idea is that all computers can perform the same set of tasks. They are all standardized, with each of their components communicating through bits of binary code. So there has to be an inherently different way for computers to learn, and when learning something as specific as a text-based adventure game, that learning process can be long and arduous.

I began my research by investigating the nebulous idea of “machine learning,” or the different ways that a computer can begin to classify and process data. There are two major types of machine learning: supervised and unsupervised. With supervised learning, a machine trains itself on hand-labeled data and categorizes new inputs based on the given data. In terms of Natural Language Processing (NLP), which is the computational understanding and tokenization of natural languages like English or French, the training data would be text corpora- essentially large bodies of raw text. For example, when training an algorithm on a list of the most common male/female names, you would want the algorithm to look at certain features, like the last letter of the name, and categorize the data based on that. So give the algorithm a thousand labeled male and female names from a “names” corpus, keep track of the last letter, and then give it a new name (not in the training data) and see if it can correctly classify the name as male or female based on the last letter. However, give the algorithm too many features to examine, like length of the name, whether it contains certain letters or not, how many vowels, etc, and the algorithm will overfit new data. This means that it looks at the data so specifically that it can no longer make generalizations about the data and is unable to categorize it correctly. There’s also another pretty glaring issue with this kind of learning: user bias. Depending on how the training data has been labeled, the computer will start to adopt the user’s bias as its own. This can become an especially dangerous issue when machine learning applications start to become more widespread, seeing as the computer science field isn’t exactly known for being the most diverse in terms of gender, sexuality, and race.

So, I turned my research towards the more abstract type of machine learning: unsupervised. This is where the computer picks out any observable patterns in the data and develops its own “labels” for categorization. My first step in developing an algorithm that ekes out patterns in text was to develop an algorithm that can actually read text and discern important features, which requires me to have a comprehensive understanding of NLP and, more generally, the english language.

Oh boy.

The past few weeks, I’ve been working my way through a textbook about Python’s NLP library, called the Natural Language Toolkit (NLTK). I’ve learned quite a tremendous amount of new coding skills that will be applicable to my future career in computer science, but more specifically, I’ve gained an understanding and appreciation for NLP, which is a super cool sub-area of coding. Besides writing some comprehensive notes about the NLTK, I’ve began coding a simple “Player” program. Since there isn’t a readily available corpus for text adventure games to train my algorithm, I have to develop a program that can take simple text games (ones that I will have to provide it), develop a dictionary of important words, and link those words to a win result or a loss result. That way, it will begin to notice patterns in the words that lead to a losing result. For the purpose of this research project, I’ll have to make sure that all of my text adventure games are simple, structured, and consistent. The point of this project is less about developing a revolutionary computer algorithm and more about developing my understanding of how a computer learns. It’ll be interesting to see what words the algorithm picks up on as being indicative of success/failure.

Currently, my “Player” can take in a simple script, parse each phrase (not sentence), detect keywords (those are the words in all caps that a normal human player would use to make choices, like “LEFT,” “RIGHT,” “CAVE,” etc.), tag words with their part of speech, and organize and upload every important word (nouns, adjectives, verbs, and adverbs- I decided on these because they are most likely to have valuable meaning) to a constantly updated dictionary. My next steps are to write more scripts so the program can build its dictionary, link the “Player” to the actual game program so that it can make choices, and figure out how to implement a way for the algorithm to categorize the data. These are the goals I would like to accomplish by the end of the summer, but there are more ways to continue with this project in the long term, such as by implementing the “WordNet” corpus, which can connect words to their synonyms.

I look forward to working more on this project, and I’m excited to be learning so much about the complex topics of machine learning and NLP. By building a foundation of understanding now, I’ll be able to grasp more advanced sub-fields and succeed not only in my future classes, but in my future career. I’m hoping that with this knowledge, I’ll one day be able to revolutionize the way we tell stories or help eliminate some of the human bias that we subliminally project onto our technology.

Link to abstract

Comments

  1. jbdillard says:

    This seems very interesting. Regarding your comparison of machine learning to human learning, its important to remember that the human thought process is generated by billions of neurons signaling each other. The raw processing power of the human brain actually exceeds even the largest computer clusters in the world. Computers just seem faster because they solve problems directly rather than simulating an entire consciousness and thought process like the brain does. Your comment about human bias leaking into supervised learning methods is an important acknowledgement that many people seem to overlook. I don’t know if you’ve looked at other examples of machine learning in video games, but I remember that last year a group called OpenAI were able to train an AI to beat the best DOTA 2 players in the world in 1v1 matches. I believe they used unsupervised learning, and trained the AI by having it play against itself millions of times a day until it learned on its own. Here’s an article I found on it for reference: https://www.theverge.com/2018/6/25/17492918/openai-dota-2-bot-ai-five-5v5-matches . Good luck with the rest of the project.