Ask HN: Game devs, what is the hardest part about designing an AI for your game?

Mixing Concepts

The biggest surprise I have found on this subject is this strange entanglement of ideas between "AI designed to kill the player" ("Military" AI), and "AI designed to make the game fun ("Game" AI). This is like comparing a F-22 stealth fighter to a 747 commercial airliner -- sure, both are technically "jets", but they serve completely different functions.

The only AI we should be talking about here is AI designed to make a game more engaging and enjoyable for the player. That may include emulating a tiny bit of behavior that resembles military tactics, but only where games use that as a topical facade -- unless you play The Sims waaaay differently than I do.

Easy Mode

There are several posts here where people assert that "they don't want a game to be hard, that would be bad". As a counterexample, I would invite these individuals to view the awards, testimonials, and revenue from the ["Souls"](https://en.wikipedia.org/wiki/Souls_(series)) series (Demon's Souls / Dark Souls). I have spoken to psychology experts at game development conferences, as well as professors that explicitly study psychology in games -- and they are still struggling to define and quantify the exact elements that came together to make this series such a hit. But there is no denying that it is an absolute favorite, with a huge following, and tremendous profits to prove the efficacy of the model.

Obviously, I'm not advocating that "every game should have a Souls-style difficulty curve". That series caters to a specific audience -- and there are an ocean of subtle differences between fans of the series. The point is: broad statements like that go directly against a mountain of evidence. We need to think about this issue with its full (tremendous!) complexity, and how to leverage some immense computing power to address it. Speaking of which...

Based on your Purchase History...

Right now, companies like Amazon and Google are mashing up the entire history of everything you've purchased, every web page you've ever visited, along with demographic and social media data about you. They can then deliver a product recommendation / advertisement / etc. within milliseconds that targets your specific needs, even for things you didn't realize you wanted / needed yet.

If that can be done in milliseconds, then the same principles can be applied to tweaking the difficulty for players in real time. You don't even need to maintain a hot internet connection to constantly feed some huge cloud implementation at all times -- some local processing combined with daily or weekly micro-updates can do the trick. For a perfect example of local / cloud balance, check out SnapChat -- that thing uses the local computing power on your phone to identify your face, and then attach various 3D objects and effects to it in real time. That's done at 20-30 FPS on a smartphone without any special dots or markers to help "anchor" the models. That's amazing. And it's all possible because the [Convolutional Neural Network](https://en.wikipedia.org/wiki/Convolutional_neural_network) models that make that possible get trained on the cloud and downloaded every now and then.

So, yes -- you can code a game to dynamically customize itself for every single player. This is the [exact point of my video series](https://www.youtube.com/watch?v=hTDSGU2vAow) about this subject. Thus far I've made over a half-dozen Skyrim mods that demonstrate some of these concepts, and our team is expanding into other games that are finally opening up their engines to 3rd parties. If folks would like to talk about the nitty-gritty of all that, I'm happy to chat.

I think any game with strategy is going to have to use reinforcement learning to have any hopes of being understood by its players. Concerns around cheesing strategies can be mitigated for real time games using constraints and cheesing in turn based strategy games is a symptom that maybe the game you're working on isn't that strategic.

Ultimately Chess, Go, Dota, Starcraft are compelling games to watch and play because the skill cap is so high. No hand coded AI for such a game could ever prepare you to enjoy playing any of those games professionally or get to a point where you can understand professional play. High level strategy in those games would never even be discovered had we not had a large population of players playing those games for long hours.

Odds are most strategy games won't be as successful as any of the above games but the newcomers could still have compelling strategic elements that gamers could explore had they been able to find other players that are slightly better than they are so they learn but at the same time aren't frustrated by too big a skill gap. A small multiplayer community is why the strategy game market is pretty bleak except for a few large players.

Reinforcement learning can provide a better experience to your players because you the game developer don't need be the world expert at the strategies of your game to create a top level AI. You just make sure the rules of the game are there and self play will take care of the rest. You can constrain the AI such that it's enjoyable by both your newest and most devoted players. You can discover OP strategies and better guide your design process. Excel is great for balancing but if you want to make a game complex enough then balancing will be a tremendous challenge without a larger team.

If you're interested in leveraging reinforcement learning for your game, I run yuri.ai which helps game developers do exactly that. So reach out to me at mark@yuri.ai if I can help!

I programmed some new API functions for the AI scripting language in a popular RTS game once. I think the most tedious part is exposing the game data in an intuitive/helpful way for someone in a design role. It's a bit of an odd mix of optimization, computational geometry, and finding the right level of expressiveness.

For instance, a designer might say "It'd be great if I could put a tower between myself and the enemy town, but kind of at the border of mine, but only if I don't have enough money for a cool castle and I'm not overwhelmed." There's a lot of fuzzy logic in statements like those. The API between the game and the "AI" usually has to be general enough not to need spatial information, but also specific enough not to do things that feel unintuitive or silly to the player.

It's worth noting that machine learning helps avoid a lot of these issues, but I find AI tends to be designed for an feeling or experience, so I'm not sure if things like the Starcraft AI competition bots tend to make better products. Perhaps if ML could make the building blocks for designers to design AI?

I’ve been designing/coding games since I was 11yo and have a degree in gamedesign & dev. The hardest part isn’t the technical side of the game’s AI. It’s finding the balance of an AI that makes the gamer feel powerful enough to be fun- but not too powerful to become bored - and not find it too hard that he/she becomes demotivated. Obviously every player is different so finding a balance that can keep the gamer excited enough to continue is extremely hard. Getting the emotional-experience side of the player into the sweet zone is hard. One would say ‘why not make the AI such that it adapts to the user’. Which is 50% of the answer. But the challenge then becomes that you don’t want good players to be at the same level as bad players when the AI creates a similar experience for both. You want both players to enjoy the game, but not in the same way.

I've never specialized in AI but from my experience on game teams most of the challenges revolve around making the AI 'fun' to play against, which is not generally very well defined. It's also about taking feedback from designers and playtesters and figuring out how to adjust the AI to address that feedback. Machine learning techniques aren't really used much in game AI, partly due to these aspects. It's not necessarily obvious how to adjust some black box ML model in response to feedback from playtesters on balance issues or from designers on making the AI more or less 'agressive' or other qualitative feedback. Usually it's more a collection of rules and heuristics with various parameters that can be adjusted to tune the behavior and allow for different difficulty levels etc.