AI Learns to Drive: Stop Fiddling. Let It Cook.

GHub Platform · Гайд · 0 ответов · 17 просмотров

There are currently no guides for this game. Which is slightly surprising, because there seem to be quite a few players who have reached the same conclusion by the looks of the reviews: My AI was doing really well, then it suddenly became terrible. I changed some parameters. It got worse. I changed some more. It still got worse. This AI sucks. Refunded. So here's the guide I wish more people had before they started. My advice is simple: stop fiddling and let it cook. This is AI Learns to Drive , not Teach AI How to Drive . That's an important distinction. The temptation is to look at your cars and think: "That is a terrible line. I should reward this." "Why did it hit the wall? I should penalise that." "Why is it drifting? I don't want it drifting." "Why isn't it going faster? I'll increase the speed reward." "Why is it doing that? I'll change another parameter." And then you change something else. And something else. And something else. Suddenly your AI is worse than when you started. The problem isn't necessarily the AI. You keep changing the experiment. The game gives the evolutionary system a collection of mechanisms — mutations, crossover, selection, rewards, decay and so on — and lets it search for behaviours that produce better results. Your job is primarily to define what success looks like . The AI's job is to discover how to achieve it . Those are not the same thing. If you continually change the rewards, parameters and evolutionary settings because you don't like what the cars are doing, you're effectively changing the question while the AI is trying to answer it. Good. That's exactly when things can get interesting. An AI may discover a strategy that gets it most of the way around a track. It then starts exploring variations that might be better. And exploration can look awful . Absolutely spectacularly awful. A population that was previously completing most of a track can suddenly start crashing on the first corner. Distances can collapse. Lap times can get worse. Cars can start taking completely different lines. Some of them may do things that make you wonder whether the neural network has suffered a small electrical fire. This does not necessarily mean the AI has forgotten how to drive. It may mean it is searching. Suppose the AI has discovered a reasonably good way through a corner. It now starts experimenting with that corner. Some variations are worse. Some are much worse. A few are approximately as good. And occasionally one is better. The important part is that the better solution may require changing something that looked like it was already solved. For example: Corner A → Corner B → Corner C You might think Corner B is solved because the car gets through it quickly. But perhaps the line through B determines the speed and heading with which the car reaches C. So the AI changes B. Suddenly C becomes worse. That can look like a regression. But it may actually be discovering that the old solution to C only worked because B was slower . Now it needs to find a new C solution compatible with the faster B. This can propagate through the track. A faster exit from one corner can create a problem at the next corner. Solving that problem can create a problem after that. Eventually the system finds a combination that works better as a whole. This is why watching the middle of the pack can be fascinating. You can sometimes literally see the population exploring different solutions to the same section of track before eventually converging on one. This is probably the biggest trap. Humans have an enormous tendency to judge driving visually. We see: drifting wall contact strange racing lines cars turning around cars taking apparently unnecessary detours and think: That's bad driving. Maybe it is. But the AI doesn't care whether the line looks elegant. If an apparently ridiculous manoeuvre produces a better result under the game's fitness function, that manoeuvre gets a chance to survive. And sometimes the ridiculous behaviour is only an intermediate step toward something much more sensible. A car might initially use a wall to recover from a bad trajectory. Later it might discover it doesn't need the wall. Later still it might discover a completely different racing line that makes the original behaviour obsolete. If you had aggressively penalised the original behaviour at the first stage, you might have prevented the system from exploring the route that eventually led somewhere better. The best individual is useful, but it can also be misleading. One car might produce an amazing result once. That doesn't necessarily mean the population has learned anything. What you really want to see is the good behaviour propagating . A useful pattern is: One exceptional run Several runs approach it The median improves The population starts behaving similarly The old "best" becomes the new normal Exploration starts again That last bit is important. A new best result is not necessarily the end of learning. It may be the beginning of another round of optimisation. Imagine your AI normally reaches 8,000 metres. Then you leave it running. Suddenly the median falls to 6,000. It looks broken. But if, during that period, the population is trying radically different trajectories, that may simply mean it is spending its evolutionary budget on exploration. Eventually you might see: 6,000 → 6,500 → 7,000 → 7,500 → 8,000 → 8,100 That's the moment you know the exploration found something useful. The temporary collapse was not necessarily wasted. The population was testing hypotheses. This is where I think a lot of players get themselves into trouble. If you change the reward structure every time you dislike the current behaviour, the AI never gets a stable target. Imagine teaching a real learner driver like this: "Accelerate here." They do. "No, brake." They brake. "Actually accelerate." They accelerate. "Turn earlier." They turn. "Why are you turning earlier? Stop doing that." They stop. Then, just as they finally find a good racing line, you grab the steering wheel and yank them somewhere else because you think your new line is better. Eventually the learner isn't learning to drive. They're learning that the instructor is unpredictable. The same basic principle applies here. If you have established a sensible fitness function, give it time to work . For me, the process is surprisingly simple. 1. Pick a sensible objective Decide what you actually care about. If you want the car to go a long way, reward distance. If speed matters too, give it some weight. You don't necessarily need to encode every bit of driving behaviour you personally think is desirable. 2. Start Training Then watch what happens . Not just the best score. Watch the median. Watch the spread. Watch the actual cars. 3. Expect Chaos There will probably be periods where things look worse. Don't immediately assume something is broken. Ask: What are they experimenting with? 4. Let Promising Behaviours Propagate When a new strategy appears, give the population time to refine it. The first version may be ugly. The second version may be slightly less ugly. The twentieth version may suddenly be brilliant. 5. Save Good Generations This is the one safety net I would recommend. If you have a genuinely excellent generation, save it. Not because you should constantly restore it whenever the AI gets worse. Save it because evolutionary experimentation occasionally wanders very far from useful territory. Think of it as a checkpoint, not a steering wheel. 6. Only Intervene When You Have Evidence If something consistently fails for a very long time and there is no sign of useful exploration, then investigate. But don't panic because the AI had twenty generations where it appeared to forget how corners work. The AI doesn't know your idea of what good driving looks like. And that's the point. You may think th