FrodoBots and YGG team up on gamified AI and robotics with Earth Rover Challenge

The competition, co-organized with researchers from Google Deepmind and other research institutes, will take place during the prestigious IROS 2024, the IEEE/RSJ International Conference on Intelligent Robots and Systems, from October 16-17 in Abu Dhabi.

An “AI vs Humans” Competition

The Earth Rover Challenge is a first-of-its-kind competition where 5 AI teams and 5 human gamers will remotely control small-sized sidewalk robots while completing real-world navigation missions across multiple cities including Abu Dhabi, Singapore, Taipei, Stockholm, and more.

Participating AI teams include research labs from UC Berkeley, UT Austin, Carnegie Mellon University, National University of Singapore, Seoul National University, etc. Human gamers will be selected from the top leaderboard of thousands of YGG gamers previously playing FrodoBots’ first gaming title: Earth Rover.

“At FrodoBots, we believe the intersection of robotics and gaming could create unique gaming experiences for gamers while making meaningful contributions to Embodied AI research by
collecting real-world datasets from these IRL gaming sessions,” said Cho.

The company has about a dozen people, and it raised a small amount of funding from cryptocurrency enthusiasts. Cho said the company will likely raise additional money.

Gaming and the new gig economy

FrodoBots’ collaboration with YGG advances the gaming-enabled gig economy initiated by the play-and-earn movement. Through this competition, the partners hope to showcase the potential of robotic gaming as a tool for large-scale teleoperation data collection.

YGG has a network of Web3 gamers who can do a lot of the work and earn something while doing it.

“Participating in this competition opens up exciting new opportunities for the gamers in our community,” said Beryl Li, cofounder of Yield Guild Games, in a statement. “This collaboration exemplifies the Future of Work and how web3 gaming can have tangible impact beyond the digital realm and help contribute to advancements in AI and robotics, while having fun.”

YGG will bring gamers into the project, as they will be able to compete in a fun experience that ultimately will determine who gets to go to the Abu Dhabi tournament. There may also be a way to incentivize players with tokens. If the players undertake difficult drives or useful, drives they will get points. If they drive in an empty park, they will get very few points. If they do a mission and bring back the robot car to the right point safely, they can get points. So far, there are thousands of entries from gamers.

In the tournament, the AI will try to do the same thing the human drivers do and try to beat them in the challenges. The more the AI can learn from the human data sets, the better the AI will get at driving.

Advancing AI robotics research

As part of the competition, FrodoBots has open-sourced a 2,000-hour driving dataset, the largest of its kind in the public domain. In addition, participating AI teams will have access to FrodoBots’ global fleet of robots deployed in over a dozen cities across all six continents, allowing for AI model evaluation on an unprecedented scale under “in the wild” settings.

On the research side, Michael Cho started engaging with with the researchers at DeepMind, the Google-owned research lab for AI. That’s how the idea for the AI vs. human gamer competition arose. It’s now dubbed the Earth Robot Challenge, and it will happen in Abu Dhabi next month.

“As a researcher, I was really intrigued by FrodoBots’ unique approach to solving the data bottleneck problem in AI Robotics research by introducing gaming to the mix,” said Jie Tan, senior staff research scientist at Google Deepmind (which is not part of the project), in a statement. “More importantly, the large fleet deployed in diverse locations provided by FrodoBots created an ideal benchmark to evaluate robot policies in real-world settings. I’m glad to co-organize this competition alongside my colleagues and the FrodoBots team where we’d test the progress of autonomous AI models against human performance.”

“Robotics and AI in the last few years have seen tremendous exciting advances,” said Ted Xiao, senior AI researcher at Deepmind, in an interview. “With systems like AlphaGo, the LLMs are great at being chatbots or at solving math problems. They can also play games like DOTA or chess. But can AI actually control real world robots and actually act in the real world. That’s a constant North Star for a lot of scientists and researchers, including myself.”

Xiao added, “And I think with FrodoBots, the very exciting opportunity is that this is a globally distributed evaluation setup where we really have these really awesome platforms employed in multiple cities around the world, where we can really test the leading edge capabilities of the latest and greatest AI models and how good they are at operating and navigating in these urban environments.”

In robotics research today, training and evaluations are done in laboratory settings or in limited indoor environments, which are super controlled and very sterile, he said.

“They’re almost like Hollywood scenes, almost we’re trying to set up these warehouses which look like the real world. We’re trying to measure real capabilities,” Xiao said.

Xiao added, “I think FrodoBots gets the opportunity to test in a very difficult and challenging setting. It’s not completely out of imagination that the latest AI models can operate in these settings where it’s more like a video game.”

There are first-person navigation and driving simulators on the market, and you can imagine that AI is going to start doing as well as humans. This competition is a motivation for the researchers in this space to make the technology better and for the humans to bring their best skills to the competition.

Some of this reminds me of other “citizen science” projects that gamers have gotten involved with, like SETI@Home, which was created to uses unused computing power to help with the Search for Extra-Terrestrial Intelligence.

David Hsu, Provost’s chair professor at the National University of Singapore, said in a statement, “Beyond the open-source dataset, this competition provides a common set of diverse natural environments to deploy and evaluate competing research approaches ‘in the wild’, something that’s sorely needed in robotics research. It also pits robot autonomy against human intelligence. It’s a true test of generalization, the key challenge of robot learning.”

FrodoBots plans to continue co-hosting such competitions with researchers while expanding the scope to include other robotic tasks with new robotic gaming titles in its push to make meaningful contributions to Embodied AI research.

FrodoBots wants to transform the landscape of gaming and Embodied AI research by crowdsourcing real-world datasets through its unique robotics gaming titles. With games like Earth Rovers where players earn points by exploring the world via teleoperated sidewalk robots, and Octo Arms, which challenges gamers to solve 3D puzzles with robotic arms, FrodoBots is merging entertainment with robotics.

Origins

Around the spring of 2022, Cho started the project with his brothers during the pandemic to keep them occupied, and they decided to focus on a self-driving robot. Rather than build an expensive one, they focused on a small robot that could be built for $500 or less. Those cars are also far safer to drive on roads (or sidewalks) in the real world.

“It struck me that a lot of the cyber robot projects were pretty expensive, as it look like they were costing $10,000 to $20,000 each,” Michael Cho said. “I felt like the barrier to getting to [level 5 self-driving vehicles] was not the lack of hardware but maybe the lack of a data set.”

He added, “If you end up having really expensive robots, that actually goes against your goal to collect more data. What if we just designed the thing as a toy, right? Just think of it as an RC car. Then maybe we can get the cost down to $500. We were naive enough to take this on, but we definitely believe that the thesis has come true. In fact, we are starting to sell this robot car for $300 each.”

His brother Sam Cho decided to join as a cofounder of FrodoBots, and Sam runs the hardware team in China.

Game thinking

At first, the founders weren’t thinking about a game. The remote company started deploying its robots in different places in the world to start collecting driving data.

“It turned out it was pretty fun to drive it, and then we thought maybe there is something here like Euro Truck Simulator,” which is a game where you just drive trucks on realistic maps in the real world. It turns out that these games may seem boring but they appeal to a certain kind of gamer. The joy of these games has to do with driving through some unknown part of the world and discovering what it’s like.

Michael Cho showed me videos of driving the small RC cars around remotely, and it was funny to see puzzled pedestrians walking past the cars. There’s a speaker microphone in case the human remote driver needs to talk to someone nearby.

Since it’s not inherently fun to do, the company offers incentives in the form of points for the players, who can get on a leaderboard. It will also lean into rewards by incentivizing people through Web3 tokens (again, here’s where YGG comes in). A player can buy a car for $300, but a bored player could lease it to other players and then start making money from it.

The game and robotic car has been tested in more than 20 cities with both AI and humans. Now the company is adding a feature that is “like Pokemon Go, but in real life,” Michael Cho said.

“As you drive around, you can hunt for rare NFTs. You can imagine that “most of the time you find something silly, but you can image you occasionally get luck,” Michael Cho added. “I think there is potential for doing something really interesting with augmented reality. Right now, it’s very simple. I think there is a lot to the game design that we want to explore.”

He added, “We know we’re not game designers in any way. We’re just proving it almost like this can be done. But, over time, we want to open source a lot of these tools so the other game developers can build their own robotic games.”

Michael Cho said the company has been lucky because it’s videos have gone viral a company of times.

Taking care of the small RC cars

Some of the missions for humans are difficult. One could last 43 minutes and take the car through very crowded streets. Cars need to be aware of pedestrians and slow down or move out of the way.

There have only been a couple of incidents in the past two years when humans have picked up the cars and disabled them, like throwing them in the river. But Michael Cho said the robots are pretty robust and can take some damage. They travel at a slow speed of three kilometers per hour.

The robots are small enough so that the risk of physical harm to humans in the world is small. In each place with a robot car, there will be someone to pick up the car and bring it back to charge it.

The humans vs. AI competition

In the competition, there will be five AI teams going up against five human gamers. The AI will try to do what the humans are already doing — completing navigation missions in real world. YGG gamers are engaged in the driving, and the top five players on the leaderboard will get to go to Abu Dhabi.

“It would be quite a spectacle,” Michael Cho said. “The five robotic teams will help with navigation research, and they’re coming from some of the top labs in the world.”

Michael Cho showed off some AI-controlled remote driving RC cars.

“They built these using our driving data set,” he said. “I would really love to see robotic gaming take off. You can imagine someone buying something almost like a game controller and then play a kind of 3D puzzle.”