To advance robot swarms, UB engineers turn to video games

Wide view of engineering research lab, showing students setting up an experiment involving mini-drones, high-speed cameras and computers.

Inside UB's SMART Motion Capture Lab, students create a simulated environment to demonstrate how autonomous air and ground robots can work together. Credit: Douglas Levere, University at Buffalo.

Gamers’ brain waves, eye movements could help engineers improve artificial intelligence for autonomous robots

Release Date: January 22, 2020

Souma Chowdhury head shot.

Click here for a high-resolution photo of Souma Chowdhury

“We don’t want the AI system just to mimic human behavior; we want it to form a deeper understanding of what motivates human actions. That’s what will lead to more advanced AI.”
Souma Chowdhury, assistant professor of mechanical and aerospace engineering
University at Buffalo

BUFFALO, N.Y. — The key to improving robot swarm technology may lie within video games.

That’s according to a research team from the University at Buffalo Artificial Intelligence Institute, which received a $316,000 federal grant to study the decisions people make — as well as biometric information such as their brain waves and eye movements — while gaming.

Researchers will use this data to build artificial intelligence they believe can improve coordination among teams of autonomous air and ground robots.

“The idea is to eventually scale up to 250 aerial and ground robots, working in highly complex situations. For example, there may be a sudden loss of visibility due to smoke during an emergency. The robots need to be able to effectively communicate and adapt to challenges like that,” says the grant’s principal investigator, Souma Chowdhury, PhD, assistant professor of mechanical and aerospace engineering in the School of Engineering and Applied Sciences.

Co-investigators include David Doermann, PhD, director of the UB’s Artificial Intelligence Institute and SUNY Empire Innovation Professor of computer science and engineering; Eshan Esfahani, PhD, associate professor of mechanical and aerospace engineering; and Karthik Dantu, PhD, assistant professor of computer science and engineering.

UB student works with mini-autonomous robot in front of a projected simulation of a city.

The idea is to eventually scale up to 250 aerial and ground robots, working in highly complex situations.

Swarm robotics research is inspired by many sources; ant colonies and schooling fish are examples. But the potential to improve AI systems by learning from humans is enormous, says Chowdhury, a member of UB's Sustainable Manufacturing and Advanced Robotic Technologies (SMART) Community of Excellence.

The study, which is funded by the Defense Advanced Research Projects Agency, will center on real-time strategy games. These are time-based (as opposed to turn-based) and involve using resources to build units and defeat opponents. Examples include StarCraft, Stellaris and Company of Heroes.

Students will play a basic real-time strategy game developed by the research team. In addition to recording the decisions the gamers make, researchers will track their eye movements with high-speed cameras and their brain activity through electroencephalograms.

Students wears a device that measure the electrical activity in his brain while he plays video games during experiments for the project.

Inside UB's SMART Motion Capture Lab, researchers measure brain waves from students who play video games that simulate environments for autonomous robots. Credit: Douglas Levere, University at Buffalo.

The team will use the data to create artificial intelligence algorithms that guide the behavior of autonomous air and ground robots.

“We don’t want the AI system just to mimic human behavior; we want it to form a deeper understanding of what motivates human actions. That’s what will lead to more advanced AI,” Chowdhury says.

Eventually, the team will integrate and evaluate artificial intelligence it develops into more sophisticated virtual environments developed by DARPA’s partner organizations.

“This project is one example of how machine intelligence systems can address complex, large-scale heterogeneous planning tasks, and how the University at Buffalo Artificial Intelligence Institute is tackling fundamental issues at the forefront of AI,” says Doermann.

Media Contact Information

Cory Nealon
Director of News Content
Engineering, Computer Science
Tel: 716-645-4614
cmnealon@buffalo.edu
Twitter: @UBengineering