Editor’s Note (2/22/23): This work was first published on February 11, 2022. Gran Turismo players can now play against AI Gran Turismo his Sophie in the latest version of the game, so we’re republishing it.
Race car drivers must brake, steer and accelerate at precise times to sprint around corners along the fastest ‘racing line’ without losing control. This process depends on the limits of friction and is controlled by known laws of physics. That means self-driving cars can learn to complete laps at the fastest possible speed (as some cars already do). But when the automated driver has to share space with other cars, this becomes even more of a problem. Scientists have now solved this problem virtually by training artificial intelligence programs to outrun human competitors in the surreal racing game Gran Turismo Sport. The findings may lead self-driving car researchers to new ways to make this technology work in the real world.
Artificial intelligence has already conquered human players in certain video games such as StarCraft II and Dota 2.Published in Nature“In most games, the environment defines the rules and protects users from each other,” he explains. “But in racing, the cars are very close to each other and there is a very sophisticated sense of etiquette that needs to be mastered and deployed. [AI] Agent. To win, we have to respect our opponents, but we also have to maintain our driving line and not give way. “
To teach the program tricks, Sony AI researchers used a technique called deep reinforcement learning. They rewarded the AI for specific behaviors, such as staying on track, maintaining control of the vehicle, and respecting race etiquette. Then I experimented with different race methods that allowed me to set up my program freely to reach those goals. Sony’s AI team trained several different versions of the AI, called Gran Turismo Sophy (GT Sophy), each specialized to drive a specific type of car on a specific track. Next, the researchers pitted the program against a human Gran Turismo his champion. In the first test conducted last July, a human achieved his score for the best overall team. In his second run in October 2021, AI broke through. Both individually and as a team, they beat human opponents and achieved the fastest lap times.
Human players seemed to take the loss lightly, and some enjoyed playing wits against the AI. Erica Marcus Kato, Director of Strategy and Partnerships at Sony AI, said: “The lines used by the AI were so tricky that you could probably do it once. Emily Jones, FIA Certified Gran Turismo Championship 2020 World Finalist and later against GT Sophy, said: Although Jones says he felt a little helpless in competing with AI, he said the experience was impressive.
“Racing, like many sports, is all about getting as close to the perfect lap as possible, but never really getting there,” says Jones. “For Sophie, Perfect wrap. It couldn’t get any faster. “
Sony’s team is currently developing the AI further. “For each car track combination, we trained an agent that was a version of GT Sophy,” he says. “And one of the things we’re looking at is can we train a single policy that can run on any car on any track in the game?” We may also be working with the developers of Gran Turismo, a subsidiary of Sony Interactive Entertainment’s Polyphony Digital, to incorporate a version of GT Sophy into a future update of the game. To do this, researchers have tweaked his AI performance so that he can be a challenging opponent, even if the player is less skilled than the champions he’s tested AI with so far. but not invincible.
Gran Turismo offers realistic approximations of specific cars and specific trucks, and the unique physics parameters that control each, so this research may have applications beyond video games. A software engineer at the artificial intelligence research company OpenAI, the OpenAI Five project that beat humans in Dota 2. (Chan is not involved in his GT Sophy investigation.)
“Gran Turismo is a very good simulator. It’s gamified in some ways, but it’s really faithful to the many differences you can get with different cars and different courses,” says Stanford University’s Professor J. Christian Gerdes said: Mechanical engineering, which was not involved in new research. “In my opinion, this is the closest thing he has to someone who published a paper that said AI could compete with humans in a racing environment.”
However, not everyone fully agrees. “In the real world, you have to deal with things you have to avoid: bicycles, pedestrians, animals, things that fall off your truck and onto the road, bad weather, vehicle breakdowns,” he says. Steven Shradover is a research engineer in the California Partners for Advanced Transportation Technologies (California PATH) program at the University of California, Berkeley Transportation Research Institute. Nature paper. “Nothing like that appears in the game world.”
But Gerdes said the success of GT Sophy has challenged certain assumptions about how self-driving cars are programmed, and that it could still be useful in the future. Self-driving cars can make decisions based on the laws of physics and AI training. “If you look at what’s in the literature, and to some extent what people are putting on the road, motion planners tend to do physics-based optimizations, and the recognition and prediction part goes to AI.” Gerdes says. But in GT Sophy, the AI’s motion planning (such as deciding how to approach corners on the high end of performance without causing crashes) was based on his AI side of the formula. “The lesson for self-driving car developers is that data points suggest that we may need to revisit some of our preconceived notions that certain parts of this problem are best done in physics. There is,” he says. “AI might be able to play there, too.”
Gerdes also suggests that GT Sophy’s work could provide lessons for other areas of human-automated system interaction. In Gran Turismo, he points out, AI will have to balance the difficult problem of achieving the fastest route on a track with the difficult problem of smoothly interacting with humans, who are often unpredictable. “If you have an AI system that can make sophisticated decisions in its environment, it could be applied not only to autonomous driving, but also to interaction with robot-assisted surgery, machines that are useful in the home,” he says. says Mr. If there is a task for humans and robots to work together to move something, it is in some ways much trickier than robots trying to do it on their own. “
A version of this article titled “AI Champions” was adapted for publication in the May 2022 issue Scientific American.
