A trio of students from the University of British Columbia has put their expertise to the test by building a single-player, artificial intelligence (AI)-controlled air hockey table. This is interesting enough, but what really takes the biscuit is that the AI managed to learn to play purely in simulation.

Typically, AI-controlled robots like this are trained through trial and error in the “real world” by failing and learning from thousands of practice runs. While effective, this is time-consuming and can damage hardware.

The UBC team, however, decided to see if they could train the AI in the digital space, make its mistakes there first, and then be unleashed on a real air hockey table. To this end, they built a highly accurate digital twin of the table to train the AI.

Once the training was complete, they then copied the trained AI into the real robot to see how it performed against a human opponent. And the results were surprising.

Despite any real-world experience, the AI-controlled robot air hockey player was able to put up a real challenge. All this straight “out of the box,” as it were.

Tempered digitally for the real world

What’s also interesting here is that AI typically struggles to learn tasks like air hockey. This is because the puck, in particular, moves very fast, can move unpredictably, and is heavily influenced by subtle changes from interactions with paddles and walls.

Any robot player also has to factor in time lags from motors, cameras, voltage fluctuations, mechanical vibrations, and imperfect puck tracking. So, even tiny errors can result in game-losing miscalculations.

To this end, the team purposefully designed the AI’s training environment to be imperfect. This is a diversion from most training scenarios, which can be “too perfect,” but the real world clearly isn’t.

So, the team factored in things like uneven rails, warped tables, inconsistent rebounds, power supply dips, and camera latency. Called “domain randomization,” this enabled the AI to learn to expect the unexpected and react as best it can, just like a human player has to.

This, the team explains, teaches the AI to predict a range of likely outcomes from a puck bounce rather than an exact outcome based on a set algorithm. So, the AI learned to expect a puck to be in an approximate place in the future and react accordingly.

To speed up training, the team ditched the idea of using normal physics engines like Unity and Unreal and instead turned to something called “soft actor critic” training. In short, this is a sort of carrot-and-stick-based learning environment.

Air hockey is a punishing sport

Within it, the AI takes action and gets rewards or punishments based on its performance. Over millions of simulated games, the AI gets better and better at playing the game and adapting to all the built-in chaos of the game.

When applied to the real world, the AI player was also giving a helping hand using a special overhead camera and a retroreflective tape-coated puck. This helps the AI “see” the puck more clearly at 120 frames per second.

Beyond air hockey, the team’s innovation also has some potential applications for training other autonomous systems like drones, vehicles, robots, etc. If the training of these platforms could be sped up and made more realistic, similar sim-to-real transfers could be a huge boon.

If you want to learn more, details are available on the project’s official GitHub page.

💾

Electric vehicle startup Harbinger has reportedly teamed up with American Rheinmetall to develop various trucks for military use. Based on the former’s medium-duty hybrid and electric truck chassis, the plan is to develop trucks ranging from robotic cargo haulers to battlefield logistics and resupply vehicles.

Since Rheinmetall has extensive experience as a defense contractor, it will be able to provide expertise on survivability and integration into military supply chain operations.

The new vehicles will use something called “drive-by-wire,” which essentially means steering, braking, and acceleration will be electronically, not mechanically, operated. Such a setup will be essential for remote operations, autonomous operations, and potentially artificial intelligence-assisted driving.

The new vehicles will also form part of the Pentagon’s drive for what it calls “attritable mass.” In essence, this means they are looking for ways to field hardware that is cheap enough to be essentially disposable if needed.

Harbinger of the future of army logistics?

This is especially the case when trained personnel are thrown into the mix. Losing a $50 million vehicle and its crew is not only expensive but should be avoided if at all possible.

And this is where Harbinger and Rheinmetall’s venture potentially comes into its own. If dangerous logistics and supply runs can be achieved using cheap autonomous electric vehicles, the costs of losses become more acceptable.

Electric vehicles (EVs), in particular, offer some interesting benefits in this role too. First and foremost, they are relatively quiet in operation, making them better suited for stealth logistics, reconnaissance, and nighttime resupply.

They also produce less of a heat signature compared to internal combustion engine-driven vehicles. EVs also benefit from massive onboard power, which can be used as a kind of mobile giant battery to power/recharge other electronic assets like drones, electronic warfare (EW) systems, and potential directed-energy weapons.

This new venture is looking, specifically, at providing an Extended Range Electric Vehicle (EREV) like its announced “Praesidia.” In short, this is basically a militarized, ruggedized hybrid electric-diesel vehicle that provides the best of both worlds.

Such a platform will have an excellent range and reliability, making it very attractive for military operations in a range of theatres. Scant details have been officially released, but according to Harbinger, the Praesidia should have a range of around 500 miles (805 km).

Any army marches on its stomach

This vehicle, according to Harbinger, will also provide “100kW Exportable power for counter-drone systems, directed energy, radar, and off-grid needs like forward operating bases.”

“Harbinger has always built for the toughest commercial missions, and the Warfighter’s mission is the toughest of all,” explains John Harris, Co-Founder and Chief Executive Officer of Harbinger.

“Partnering with American Rheinmetall, one of the most capable ground systems integrators serving the DoW, allows us to bring our autonomy-ready platform at a price point that makes true attritable mass possible,” he added.

Together, we can give Soldiers robotic systems that are affordable enough to be everywhere they’re needed, and tough enough to do the job when they get there. Most importantly, by taking the driver out of the vehicle with advanced autonomy and teleoperation, we can keep service men and women out of harm’s way,” he said.

💾