Only people who never talk to game creators could fail to notice that the games industry is in a state akin to civil war over the usage and implications of generative AI, wherein management waxes lyrical over opportunities to deliver more game with fewer people, while many developers live in fear and loathing of a technology they consider little more than organised theft of intellectual property in the service of replacing creative labor with unfeeling algorithms. Apparently, many prominent Silicon Valley voices opining on the future of games and AI never talk to actual game creators.

Perhaps more surprising, even to game creators and their confidantes, is that the games industry has always had a complicated relationship to AI. There has always been both an interest among game developers in the potential opportunities afforded by AI, and a widespread scepticism towards AI, and particularly the type of AI that comes out of research laboratories. The interest is easy to understand. For has AI not always been the future of games? Video game creators frequently attempt to do the impossible, or at least the currently technically impossible. For video games to be full of interesting characters, complex mechanics, living worlds, well-told stories and so on, it would seem that we need powerful AI.

The widespread scepticism towards AI, even before the current wave of generative models, might be harder to understand. Allow us to digress a bit to illustrate it.

Twenty years ago, one of the authors of this chapter, Julian Togelius, was in the middle of his PhD. His work initially focused on evolving neural networks to play simple games, specifically a car-racing game he had built himself as a research prototype. He had gotten some nice results and published a few papers, one of which even won an award. He was proud of his work and wanted to show it to game developers. So he did. The responses ranged from polite ”how cute” comments, to developers aggressively pointing out the worthlessness of his work and how preposterous it was for him to think he would have anything to offer the games industry. One blog post attacking his work had the title “the sorry state of academic crack-smoking”.

The most forgivable of Julian’s crimes was that the methods he had developed were computationally expensive, both to run and to train. Video games, he was told repeatedly, made full use of the computer, and there was simply no time left to run complex AI calculations: the processor (CPU) was running hot calculating everything that happened in the game, and the graphics card (GPU) was busy rendering this in beautiful, high-res (for the time) graphics. No idle cycles for neural networks.

Speaking of neural networks, even using these methods at all was a sin in the eyes of many game developers. Many of them had tried neural networks back in the 1990s, when they were last popular, and found them unreliable; you could never be quite sure what result you would get. If you let a neural network control your game character, who knew what it might do? Many game designers see their work as guiding the experience of the player, which means that they want as much certainty as possible about what the player will encounter. Such designers had no use for a neural network or some other pesky AI that introduced randomness or, worse, had a mind of its own.

Julian’s most severe crime, however, was focusing on the wrong problem. He had been training his neural networks to play the games as well as possible, but, as various game developers pointed out, this was not really interesting. If they wanted an opponent in a racing game that would drive faster laps than the player, they could just cheat by making the opponent’s car faster, or better at holding onto the road. They were right, in that having an AI play a game better does not, by itself, make the game more enjoyable for humans. This was useful advice that Julian took to heart. Since then, key theme in the research programs of both authors of this essay has been the use of AI to support game designers and to better understand games.

Lest we give the impression that video game developers don’t know or care about AI, let us assure you that this is not so. Plenty of game design inventions came from the judicious application of AI methods. Most games are modeled as finite state machines, an early AI invention, in which different states control the flow of the game. For example, you have a menu state, playing state, win state, etc. Non-player Characters (NPCs) are often also modelled as finite state machines. For example, the ghosts in Pac-Man switch between wandering, chasing, or running away. Pac-Man used simple pathfinding to guide its ghosts, but from the 1990s onward, the A algorithm became ubiquitous in video games; simply put, it’s how NPCs navigate, and several genres, from real-time strategy to first-person shooters, would not exist in their current forms without it. Meanwhile simulated sensors, pioneered by Thief (1998) with inspiration from robotics research, are crucial for stealth-style gameplay everywhere, as in the Assassin’s Creed series. More recently, the introduction of the goal order action planning method in F.E.A.R. (First Encounter Assault Recon*, 2006) increased the repertoire of things NPCs could do.

Other AI innovations have impacted, or even given rise to, specific genres. SimCity (1989), a genre-defining game if there ever was one, built heavily on cellular automata and systems dynamics theory; various games based on alternative or “natural” input modalities, such as those for the Kinect, rely extensively on modern computer vision; games that feature extensive crowds (such as Assassin’s Creed) often rely on swarm intelligence techniques such as the boids algorithm.

Then there are the one-offs: games that were built on some specific AI method, but which did not spawn their own genres. In Creatures, from 1996, the eponymous creatures were powered by a neural network and had reproduction with heritable characteristics, meaning that the game loop implements a form of genetic algorithm. Another interesting example is Black and White, a commercially and critically successful game from 2001 in which the core gameplay involves the player teaching a giant creature to do their bidding; this is accomplished using reinforcement learning, an AI technique that is infamously tricky to get right. Neither of these games gave rise to their own genre, but Black and White had a massive impact on AI research, albeit indirectly: one of the game’s programmers, Demis Hassabis, went on to co-found DeepMind, one of the leading AI labs.

Just as there are many different types of games, and just as connoisseurs of chess, Street Fighter, and Dungeons and Dragons don't necessarily have much in common, there are many different technologies called AI. While for many, AI is synonymous with large language models, AI as a field has existed for 70 years and created a fascinating array of methods across search, optimization, constraint solving, knowledge representation, supervised, unsupervised, and reinforcement learning. LLMs build on many of these. In addition to the academic research on AI, there is also a separate body of work on AI from inside the games industry aiming at solving video game-specific problems. This includes innovations such as behavior trees, which were popularized by Halo (2001) as a way of encoding NPC behavior, and numerous variations of pathfinding algorithms. This work is not typically published in the academic literature, and largely invisible to “mainstream” AI researchers.

As we noted earlier, plenty of things you might want to do in video games would require powerful AI, which may not exist yet, or which may be beyond the capabilities of current consumer hardware. In other words, video games have in some sense always been constrained by the lack of AI. From this perspective, quite a few common design patterns in games can be seen as workarounds for the lack of AI. For example, take dialogue trees: a way of representing dialogue as pre-written lines and responses that the player can choose. Nobody thought that dialogue trees were the best way to handle dialogue in a game; it was just what worked on the hardware of the early 1980s, when computer role-playing games were invented, and since then this design convention has become ubiquitous. This is how writers know how to write game dialogue, and how designers know how to incorporate it into the overall game design. It is also what players expect from dialogue in a game; if you introduced a completely different dialogue format, players might be confused.

This means that you cannot simply plug modern AI methods into games that are not designed for them. For example, if you take a game designed to work with dia logue trees and replace them with LLMs, you will most likely ruin the game, because while LLMs are amazing in many ways, they also have their own shortcomings. If you can ask the NPC about anything at all, how does the game make sure that you eventually get the piece of information you need to proceed? And what if you ask about something that is completely unspecified in the game, such as the price of a liter of milk? Does milk even exist in the game world? LLMs offer very different affordances compared to dialogue trees, which demand different types of game design.

In general, one of the biggest issues of integrating modern AI (such as LLMs) with games is the lack of control. As mentioned above, much of the designer's work involves crafting a space or experience for the players to enjoy or experiment with. Most generative AI methods, even with all the prompting and control methods, can’t guarantee a specific output. This issue can be solved if the generated content can be easily quantified, such as the number of enemies in the level, or the existence of a path between the player and the exit, because then we can always give feedback to the model and ask it to update the content. But this does not always work when the AI is involved in tasks with more qualitative criteria: if there is no easy way to judge the output, you cannot construct a loop that regenerates it when it’s not working well.

As in other forms of art, games carry the designer’s intention and message. In games, this is expressed through the creation and orchestration of mechanics, in the designed levels, the written narrative, etcetera, and it is delivered to the players when they play the game. Playing the game creates a conversation between the player and the designer where the player experiences the designer’s intentions, but that experience is also shaped by the player’s own actions. This creates a bigger problem when a game is fully designed by AI. Although modern AI can tell you why it created a particular system, the reasoning is frequently generic, and might not feel intentional in the end. (Many would also argue that the artificial system can never have an intent, only a simulacrum of one.) This results in resistance among games companies and designers, as the output from these systems doesn’t fit with their intentions regarding how their message should be represented in a game. It can also result in players feeling betrayed, as they wanted to have a conversation with the designer and their intentions, rather than an AI system; this is evident in the popularity of games by “auteur” designers with strong intent and personal styles.

Going forward, designers/developers might want to change their way of thinking about games. But players, too, may find it in their interest to be more forgiving and embrace absurdity. Both groups might look at new games as being similar to improvisational comedy, where the audience enjoys the emergence of surprising new interactions, and the performing comedians are highly adaptive to whatever situation arises.

As well as commercial game developers, we believe that casual creator communities will also benefit from new AI. Casual creators use games to express themselves and their ideas. This can be compared to how people all over the world express themselves via posting on social networks, although creating new playable experiences takes much more effort. Traditionally, creating entire games has been very hard, so casual creators used modding tools (programs that allow you to change specific features in existing games) to express themselves. However, while modding is easier than creating games from scratch, it still has a learning curve which pushes a lot of creators away. It also forces casual creators to work in the same genre as the base game they’re modding, so their self-expression has to align with the genre itself. Most personal expression games use narrative or level design to carry the creator’s intention, rather than game mechanics. Modern AI could enable casual creators to create new mechanics, thereby enabling new routes for self-expression, as well as lowering the technical bar: instead of learning these tools, a creator could just chat with an LLM and give it feedback until their game is realised.

The huge amount of work that is needed to create even something as simple as Tetris or Pac-Man compared to other media means that there are many under-explored opportunities for games. For example, you can create a meme in less than five minutes, while creating a game would traditionally at minimum take a couple of hours, which may explain why games as fast consumable products are not yet a big thing. Tools such as GIPHY Arcade are a step in this direction, but it restricts the available mechanics to a limited set while allowing the user to select the assets. Modern AI can now generate these types of games very quickly: you can simply open an LLM and prompt it to create a game based on an inside joke with a friend, and then send that game to them as a response in a chat. But while the tools exist, the platforms that would allow for easy sharing and running of these consumable games are not there yet.

It is likely that the relationship between AI and games will continue to be complicated. As the history of technological change suggests, it is natural to feel threatened when the ground is moving beneath your feet. We are not fans of trying to replace human creators; games are created by humans, for humans. But this should not stop people from exploring all these new possibilities that modern AI has opened up. So many things are now possible, that were firmly in the realm of science fiction a few years ago: games that are created on the fly to express a thought; games that adapt to the player and even grow with them; games that are literally infinite; games in which every character has a rich inner life; games for machines or animals; games that express the news of the day; games with unmatched fidelity to the real world; games that teach and test; games like nothing we’ve ever seen before. We can also make the types of games that already exist, but iterate on them faster, with many repetitive development tasks such as testing done automatically.

The technology exists, but the design largely doesn’t. We don’t yet know how to use these amazing new technologies to create new types of games. And this is one thing that AI will never be able to solve; we need human designers and casual creators—including you!—to think hard about how to design with AI as a material. It will be fun!

JULIAN TOGELIUS is a professor at New York University and co-founder of modl.ai, an AI-driven game testing company. Most of his research is about the intersection of AI and games, and he has written several books and hundreds of papers about this. His favorite games include the Civilization series, the Battle of Polytopia, and Hades, as well as big open-world games such as Zelda: Tears of the Kingdom, Elden Ring, and The Witcher III. He also plays quirky indie games on his phone.

AHMED KHALIFA is a lecturer at University of Malta and a game designer and developer in his free time. Most of his research is about procedural content generation, especially level generation, but that doesn’t stop him from exploring the rest of the fascinating research field of AI and games. He released more than 10 full games and more than 30 prototypes during game jams and competitions. His latest game, Queen Boat, was featured at Queerness and Games Conference and OpenScreen at AMAZE festival Berlin. His favorite games are mostly independent games such as Wandersong, Chicory, Balatro, Hades, and World of Goo.