❌

Reading view

Drones learn to squeeze through narrow gaps using onboard AI control

Unmanned aerial vehicles (UAVs), commonly known as drones, are now widely used for various purposes, ranging from filmmaking and aerial photography to industrial inspection, precision farming and reaching obstructed areas during emergency response missions. While many existing drones can move swiftly in their surroundings and circumvent large obstacles, most still struggle in cluttered environments. In addition, they are often unable to execute maneuvers that would allow them to safely pass through small gaps or reach secluded areas.

  •  

The Surprising Fix for Robot Traffic Jams

Wheeled Robots Crowd Density ExperimentHarvard researchers found that adding a little randomness to robot movements can prevent gridlock and dramatically boost efficiency in crowded swarms. Imagine a swarm of robots sent into a tight space to handle an urgent job, such as cleaning up an oil spill or assembling complex equipment. At first, adding more robots speeds things up. [...]
  •  

Aircraft Carrying Swift Boost Satellite Takes off From NASA Wallops

An airplane taking off with a rocket attached to the fuselage
Northrop Grumman’s Stargazer took off Thursday, June 18, 2026, from NASA’s Wallops Flight Facility in Virginia. The modified L-1011 aircraft carried the company’s Pegasus XL rocket hosting LINK, a robotic servicing spacecraft from Katalyst Space. LINK will launch from the air-deployed Pegasus XL later this month on a mission to boost the orbit of NASA’s Neil Gehrels Swift Observatory.
NASA/Jeanette Kazmierczak

An airplane carrying a rocket loaded with a robotic spacecraft designed to raise NASA’s Neil Gehrels Swift Observatory departed the agency’s Wallops Flight Facility in Virginia on Thursday, June 18.

Stargazer, a modified L-1011 operated by Northrop Grumman, took off for Kwajalein Atoll, part of the Republic of the Marshall Islands in the South Pacific Ocean. Attached to the belly of the aircraft was one of the company’s Pegasus XL rockets with LINK inside.

NASA contracted Katalyst Space to design and build LINK to rendezvous and dock with Swift and lift it back to a higher altitude.

Spacecraft in low Earth orbit experience drag caused by our planet’s atmosphere, which gradually reduces their altitude if they do not have propulsion systems to counteract the effect.

Recent solar activity magnified this effect on Swift, which began to sink faster than anticipated.

Katalyst selected Northrop Grumman’s Pegasus XL launch vehicle as the best means of reaching Swift on a short timeline based on orbital and programmatic needs.

A plane taxiing with a rocket attached to the fuselage
Stargazer will carry Pegasus and LINK to Kwajalein Atoll, part of the Republic of the Marshall Islands in the South Pacific Ocean with stopovers in California and Hawai’i.
NASA/Jamie Adkins

Later this month, Stargazer will carry the rocket to an altitude of around 40,000 feet before releasing it.

After several seconds in free-fall, Pegasus XL will fire the first of its three-stage rocket motors, delivering LINK into orbit in about 10 minutes.

Learn more about the Swift mission at:

https://science.nasa.gov/mission/swift/swift-boost-mission/

By Jeanette Kazmierczak
Goddard Space Flight Center, Greenbelt, Md.

The post Aircraft Carrying Swift Boost Satellite Takes off From NASA Wallops appeared first on NASA Science.

  •  

Near Absolute Zero, This Transistor Starts Acting Like a Brain Cell

Brain AI Chip Artificial Intelligence Technology Data StreamA single transistor that behaves like a brain cell in the deep freeze could help unlock the next generation of quantum computers and space exploration systems. Researchers at the University of Hong Kong (HKU) have developed a new type of brain-inspired electronic hardware that can operate at temperatures close to absolute zero. The breakthrough could [...]
  •  

NASA’s new prototype rover completes 16-mile autonomous desert trek

NASA’s Jet Propulsion Laboratory (JPL) has developed a 4-foot-long, four-wheeled prototype designed for high-speed, long-distance autonomous exploration on the Moon and Mars. 

Dubbed Exploration Rover for Navigating Extreme Sloped Terrain (ERNEST), it underwent field tests in the Colorado Desert. The robotic rover successfully traveled 16 miles over 37 hours with minimal human intervention.

“This testing is helping us refine the mobility hardware and autonomy software to navigate extreme distances across a wide range of terrain and lighting conditions anticipated on the Moon,” said Issa Nesnas, a principal technologist at JPL. 

Active suspension system

For the last thirty years, NASA has used a mechanical setup called the “rocker-bogie” suspension system. It is the architecture behind every Mars rover from Sojourner to Perseverance that keeps their wheels grounded.

But it has some problems. Send a passive rover up a loose, sandy slope steeper than 15 degrees, and it will likely get stuck, spin its wheels, or risk a catastrophic tip-over. 

The ERNEST prototype introduces an innovative active suspension system that manages its own weight distribution. 

Equipped with four steerable wheels and a front-powered gimbal, the rover can navigate extreme terrain using unique gaits like squirming, wheel-walking, obstacle-climbing, and even driving sideways. 

Plus, a built-in clutch mechanism allows ERNEST to switch back to a passive mode when efficient, less-demanding driving is needed, blending high-capability mobility with energy conservation.

The 4-foot-long prototype is a testbed designed to advance both robotic autonomy and extreme terrain mobility for future Moon and Mars missions.

ERNEST achieves higher speeds and longer driving distances through its enhanced independent decision-making and a unique ability to lift its individual mesh wheels over obstacles that would trap current Mars rovers.

Ultimately, these technological improvements will inform the design of next-generation rovers capable of exploring previously inaccessible, rough extraterrestrial landscapes.

During recent testing, the vehicle clocked speeds up to 0.6 mph (1 kph) over 37 hours of driving across seven days — an order of magnitude faster than the autonomous navigation speeds of Curiosity and Perseverance. 

“You could do a science road trip across the Moon — or Mars — with this vehicle,” said James Keane, a JPL planetary scientist working on lunar missions.

Preparing for the rough, unknown terrain

Using reinforcement learning, computers ran thousands of hours of virtual simulations over single weekends, punishing the virtual rover with impossible physics scenarios until it learned the optimal way to climb, crawl, and survive.

The software was then loaded directly into the physical rover. Engineers took it out into the desert heat and turned it loose. ERNEST handled the rest. 

The team even ran the rover through the deep, confusing shadows of dusk and dawn. This simulated the extreme lighting conditions future astronauts and robots will face at the Moon’s south pole.

Building on this success, a new project will merge ERNEST’s active suspension control with long-range intelligent navigation. This will allow the rover to plan paths, deciding when to climb surmountable obstacles and when to steer clear of major hazards. 

It will help to prepare future missions for the most rugged landscapes on the Moon and Mars.

💾

NASA's JPL has developed a 4-foot-long, four-wheeled prototype designed for high-speed, long-distance autonomous exploration on the Moon and Mars. 
  •  

Three-armed Sashimi-Bot learns to slice and serve fish like a pro

If you ever need help in the kitchen cutting fish into appetizing bites, a new three-armed robot may be able to help. And that's no easy feat for robots. While they are generally good at picking up rigid objects, something slippery that can change shape when touched, such as a fish, poses a range of challenges. Then, if you add slicing into the mix, it becomes even more difficult because the robot has to cope with the fish changing shape as it is handled and cut.

  •  

Video: New robotic system achieves 99.5% success in fast auto factory wire assembly

Canadian robotics company Sanctuary AI has reported a significant achievement in industrial automation.

The firm said its system achieved a task success rate of more than 99.5 percent during a complex wire-plugging task for a global Tier 1 automotive supplier. 

The operation completed in 2.54 seconds, meeting the customer’s live-production benchmarks and demonstrating high-speed, reliable performance in a manufacturing setting. 

Sanctuary AI said the milestone highlights its strategy of deploying Physical AI on commercial robots to speed industrial adoption and support future industrial humanoids.

Factory automation leap

The achievement reflects Sanctuary AI’s strategy of deploying its Physical AI platform on existing and next-generation industrial robotic systems, rather than waiting for widespread commercialization of humanoid robots. This hardware-agnostic approach enables the company’s AI models to operate across different robotic platforms while accelerating adoption in manufacturing environments.

Sanctuary AI demonstrated high-performance Physical AI capabilities in an industrial manufacturing environment, achieving a task success rate exceeding 99.5 percent in a complex wire-plugging operation for a global Tier 1 automotive supplier. The system completed the task with a cycle time of 2.54 seconds, matching the throughput requirements of the customer’s live production line and meeting established industrial performance benchmarks.

The deployment focused on a plug-insertion process involving flexible wire assemblies moving along a conveyor. Such tasks are particularly challenging for automation because the cables can bend, twist, and shift position dynamically during transport, requiring continuous perception, adaptation, and precise manipulation.

According to the firm, the AI-powered robotic system successfully detected, tracked, aligned, and inserted the connectors while maintaining production-line speed and reliability.

“Manipulating a flexible wire into a moving target on a live conveyor is exactly the kind of contact-rich dexterity problem that has kept tasks like this out of reach for traditional automation. Solving it required models built around performance from day one with reliability, cycle time, and safety measured against real production benchmarks,” said Olivia Norton, co-founder and CTO of Sanctuary AI, in a statement.

AI power industries

Sanctuary AI develops Physical AI systems designed to enable robots to perform complex tasks in real-world industrial environments. Unlike traditional industrial automation, which relies on fixed programming for repetitive operations, Sanctuary AI’s approach combines advanced perception, reasoning, and manipulation capabilities, allowing robots to adapt to changing conditions and handle tasks that require dexterity and decision-making.

The company’s Physical AI platform processes information from cameras, sensors, and robotic actuators to understand its surroundings and execute actions in real time. This enables robots to identify objects, track movement, plan motions, and manipulate components with a high degree of precision. The technology is particularly suited to manufacturing environments where variations in object position, orientation, or movement make conventional automation difficult.

A key feature of Sanctuary AI’s strategy is its hardware-agnostic design. The AI can be deployed on existing industrial robotic systems as well as future robotic platforms, reducing the need for specialized hardware. This allows manufacturers to integrate advanced AI capabilities into current production lines while preserving existing infrastructure investments.

The platform is designed to automate labor-intensive tasks, including assembly, material handling, inspection, and component insertion. By combining real-time perception, motion planning, and adaptive control, Sanctuary AI aims to help factories improve productivity, maintain consistent quality, address labor shortages, and prepare for the next generation of intelligent industrial robots.

“By focusing on a performance-first approach to Physical AI models, we’re delivering value to customers today, on an AI platform that will also scale to the next generation of general-purpose systems,” said Norton, in a statement.

💾

Sanctuary AI achieves 99.5 percent success on a wire-plugging task, matching factory production speed with Physical AI.
  •  
  •  

Humanoid robots: How human-like machines could change our daily lives

Humanoid robots are no longer science fiction: They can take on tasks that challenge humans and have the potential to play an important role in industry, services or private households in the future. At Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), researchers are working on hardware and software for human-like robots. Dr. Sebastian Reitelshöfer from the FAU Chair of Manufacturing Automation and Production Systems explains in an interview how humanoid robots could support us in the future and why this future technology on two legs could be an opportunity for northern Bavarian industry.

  •  

Sanders bills AI companies for $7T public fund, aims $1,000 payments for US citizens

Sen. Bernie Sanders has unveiled a proposal that would give Americans a direct financial stake in the country’s booming artificial intelligence industry. The Vermont independent introduced legislation Thursday that would create a national sovereign wealth fund tied to major AI companies. Under the proposal, the federal government would hold a 50% ownership stake in qualifying AI firms and distribute annual payments to U.S. citizens.

Sanders argues that today’s AI systems rely heavily on content, knowledge, and creative works produced by generations of people. Because AI companies benefit from that collective human output, he says the public should share in the industry’s financial gains. The legislation, called the American AI Sovereign Wealth Fund Act, comes as lawmakers debate how to regulate AI and address concerns about wealth concentration in the technology sector.

Public stake in AI

The proposal would apply to AI companies generating more than $200 million in annual revenue. Those firms would become subject to a system that grants the public a significant ownership interest through a federally managed investment fund. Sanders estimates the fund could reach a value of roughly $7 trillion based on current valuations of leading U.S. AI developers.

The future of AI must not be decided behind closed doors by billionaires seeking to maximize their power and profit.

It should be decided by the American people.

That's why I'm introducing the American AI Sovereign Wealth Fund Act. pic.twitter.com/sbC0YMT90f

— Sen. Bernie Sanders (@SenSanders) June 18, 2026

The senator says the structure would allow Americans to benefit directly from one of the fastest-growing industries in the economy. He projects that citizens could initially receive annual payments of around $1,000. Those payouts could increase as the AI sector expands.

Speaking to reporters, Sanders said the payments would likely grow if AI companies continue generating larger profits. He also suggested future revenue could help support public priorities such as healthcare, education, and housing.

Collective value argument

At the center of the proposal is Sanders’ belief that AI companies derive value from information created by society as a whole. The legislation argues that AI models train on vast amounts of human-created material, including books, journalism, software code, scientific research, artwork, videos, and other forms of knowledge.

Sanders contends that a small group of technology executives now captures most of the economic benefits generated from those resources. Supporters of the measure compare AI to publicly owned natural resources. In that view, companies extracting value from a shared asset should return a portion of that wealth to the public.

Model draws from funds

The bill takes inspiration from sovereign wealth funds already operating across the United States and around the world. States such as Texas and New Mexico have long used investment funds supported by natural resource revenues to finance public programs, particularly education. Dozens of countries, including Norway, China, and the United Arab Emirates, also operate sovereign wealth funds that manage public assets and investments.

If Congress approves the legislation, a seven-member independent commission would oversee the fund. The president would nominate commissioners, while the Senate would confirm them.

The commission would represent public interests in qualifying AI companies and manage the government’s ownership stake. While the proposal faces significant political hurdles, it adds a new dimension to the growing debate over who should benefit from the economic gains created by artificial intelligence.

  •  

Video: Child-friendly Codey robot showcases embodied AGI future in education, healthcare

A Seattle-based robotics company has developed a three-foot-tall humanoid robot designed for use in public attractions and venues.

The AI-powered robot named Codey can navigate autonomously, engage in conversations without remote human control, and make independent decisions.

While not yet capable of performing complex physical tasks such as fetching objects, Codey is being developed for future roles in healthcare, education, and hospitality.

The company says the robot is designed to interact naturally with people, particularly children, in public-facing environments.

Codey targets companionship

Codey is designed to serve as an interactive companion in public-facing environments. Standing about three feet tall, the robot features glowing hazel eyes, expressive facial animations, and a friendly appearance intended to engage users, particularly children.

Powered by AI, Codey can navigate autonomously, hold conversations, and make basic decisions without requiring a remote human operator. Unlike many robots that rely heavily on teleoperation, Codey is designed to function independently, enabling more natural and responsive interactions with people. The robot uses a combination of sensors, onboard computing, and AI software to understand its surroundings and communicate with users in real time.

Mind Children plans to conduct pilot studies this year in museums, hotels, and art galleries to gather data and refine Codey ahead of mass production. One of the company’s near-term objectives is to enable the robot to perform simple physical tasks, such as pressing buttons. A second-generation version of Codey is scheduled for release in 2027. To support development, Mind Children has launched a crowdfunding campaign and has raised more than $600,000 toward its $1 million funding target.

In the longer term, Codey is expected to be introduced into healthcare, education, and hospitality settings. Potential applications include assisting visitors with information, providing companionship, supporting learning activities, and helping organizations deliver more engaging customer experiences.

Embodied AGI advances

At the core of Mind Children’s robotics platform is Hyperon, the decentralized artificial general intelligence (AGI) framework developed by SingularityNET. The system is designed to give robots advanced cognitive capabilities that go beyond traditional rule-based automation, enabling them to reason, learn, adapt, and make decisions in dynamic real-world environments.

By integrating Hyperon into its humanoid robots, Mind Children aims to create machines capable of understanding context, processing complex information, and responding more naturally to human interactions. The framework incorporates knowledge representation, reasoning engines, and human-inspired motivation systems that help robots prioritize tasks, pursue goals, and adapt their behavior based on changing circumstances. This approach is intended to move robots closer to exhibiting flexible, human-like intelligence rather than performing only pre-programmed actions.

Unlike conventional AI systems that rely on centralized computing and data control, Hyperon operates within SingularityNET’s decentralized ecosystem. This architecture allows AI services, knowledge resources, and computational capabilities to be distributed across a network, promoting collaboration, scalability, and interoperability among different AI agents and robotic platforms.

The partnership between Mind Children and SingularityNET positions the company among the first developers pursuing embodied decentralized AGI—an emerging field that combines advanced artificial intelligence with physical robotic systems. The goal is to create autonomous robots capable of perceiving, reasoning, learning, and interacting with the physical world while continuously improving their capabilities through decentralized intelligence networks.

💾

Codey, a child-friendly AI humanoid robot, is set for public deployment as Mind Children expands its capabilities.
  •  

US firm unveils humanoid and robots with new ecosystem for embodied ‘intelligence’

US robotics firm Faraday Future (FF) has unveiled its first-half embodied AI (EAI) robotics lineup, introducing a full-form Robot World that spans six major product series.

The company also launched what it describes as the world’s first Three-in-One EAI Robotics Education Ecosystem Strategy, aimed at connecting robotics education, industry, and consumer applications.

During an event at its new headquarters in El Segundo, California, Faraday Future debuted the Futurist humanoid robot and the FX Navi quadruped robot, with the latter priced at $1,990 and available for immediate sale and delivery.

The California-based firm also announced the FF Robotics Partner Program, designed to expand global collaboration across its robotics ecosystem.

Robotics ecosystem unveiled

FF has unveiled a broad EAI robotics ecosystem, introducing four new robots, an open developer platform, and what it describes as the world’s first Three-in-One EAI Robotics Education Ecosystem Strategy aimed at accelerating consumer and educational adoption of physical AI technologies.

Leading the launch is the All-New Futurist humanoid robot, positioned as FF’s flagship professional-grade platform. Standing about 5 feet 8 inches tall and weighing approximately 121 pounds (54 kilograms), the robot is 14 percent lighter than its predecessor and features a redesigned structure for greater stability and more natural movement. Equipped with 31 degrees of freedom and a knee-joint torque of up to 320 Nm, the humanoid can run at speeds approaching 11 mph (17.7 km/h).

According to the firm’s statement, it is powered by a dual-battery system that offers up to 6 hours of runtime. It is built on a Vision-Language-Action (VLA) architecture combined with a World Model for advanced perception, reasoning, and autonomous decision-making. The robot is designed for commercial reception, guided services, industrial handling, warehouse operations, and hazardous-environment tasks. Additional specifications and pricing will be announced at Automate 2026 in Chicago on June 22.

FF also launched the FX Navi, a quadruped robot designed for home and classroom use. Marketed as the first foundational EAI learning quadruped robot in the United States, Navi combines robotics hardware with a dedicated STEM education ecosystem. The robot features 12 motorized joints, weighs 8 kilograms, and can navigate everyday indoor environments. A smartphone inserted into the head module serves as its computing platform, supporting both iOS and Android devices.

Users can customize the robot through downloadable and printable 3D head designs, while over-the-air updates will continuously add new capabilities such as autonomous following, multimodal perception, and natural interaction. Navi starts at $1,990, with optional development and curriculum packages available for students, educators, and developers.

One brain strategy

The company also previewed two upcoming humanoid robots. Master Mini, approximately one meter tall, is designed for education and robotic sports competitions, while Nova, standing about 50 centimeters tall, is intended as an entry-level educational and companion robot for children.

FF said its robotics strategy is built around a “one brain, multiple forms” approach designed to power a wide range of embodied AI robots from a shared intelligence platform. At the core of the strategy is FF’s evolving EAI Brain, based on a Vision-Language-Action (VLA) architecture and World Model, supported by an open-source developer ecosystem. The company says the platform is designed to operate across different environments, perform tasks autonomously, and adapt to new applications through its Skills and Agent system.

Rather than focusing on a single humanoid robot, FF is developing multiple robot form factors under its EAI Robot World concept, allowing a common AI brain to be deployed across different devices and use cases. The ecosystem is further supported by an EAI Data Factory that gathers real-world operational data and a global industry network aimed at improving manufacturing efficiency and reducing costs. Together, these elements form an expanding ecosystem designed to support large-scale deployment of embodied AI technologies.

FF additionally launched its open developer platform, featuring tools such as Brain Blocks visual programming, natural-language coding, robot personality customization, motion-generation software, and SDKs for professional developers. Through its new Agent Skill Store and Developer Ecosystem Program, FF aims to encourage contributions from students, educators, researchers, and software developers while expanding robotics adoption across homes, schools, and industry.

💾

Faraday Future launches humanoid and quadruped robots, unveiling a new embodied AI ecosystem and education strategy.
  •  

Europe builds robotic arm that can see, feel and handle samples for Moon missions

European Space Agency (ESA) engineers are assembling one of Europe’s most advanced robotic arms, a system designed to handle samples and perform complex tasks during future missions to the Moon and Mars.

Known as the Sample Transfer Arm (STA), the robot can extend up to 2.4 meters and move with seven degrees of freedom, giving it a range of motion similar to a human arm. The system is equipped with cameras, sensors, and onboard electronics that allow it to perceive its surroundings and perform tasks with a high level of autonomy.

The robotic arm was originally developed as part of the joint NASA-ESA Mars Sample Return campaign. Its role was to transfer Martian samples collected by NASA’s Perseverance rover for eventual return to Earth.

Although the future of Mars Sample Return remains uncertain, the technology is now being positioned for a wider range of applications, including future lunar exploration missions.

Human-like space helper

The arm features a shoulder, elbow, and wrist configuration similar to a human arm. At its tip is a gripper capable of handling objects with millimeter-level precision.

The system also includes a sophisticated force and torque sensor that acts as a sense of touch. The sensor measures how an object is being pushed, pulled, or twisted in three-dimensional space, allowing the arm to manipulate items with greater accuracy.

Position sensors embedded throughout the joints continuously calculate the location of the arm’s tip, helping maintain precise control during operations.

Engineers say this level of precision could prove useful for a variety of activities in space, including collecting geological samples, transferring equipment, and supporting astronauts during surface missions.

The robotic arm also contains its own electronics unit, which acts as the system’s control center. Cameras mounted on the arm provide visual awareness, enabling the robot to detect and interact with objects in its environment.

Ready for testing

Teams at Italian aerospace company Leonardo are currently completing the integration of the system at the company’s facility in Nerviano, near Milan.

In the coming weeks, engineers will begin testing the arm in simulated space environments to evaluate its performance before any future deployment.

The project brings together companies and suppliers from across Europe. The industrial consortium is led by Leonardo and includes GMV and AVS from Spain, Maxon from Switzerland, 3DPlus from France, and COMOTI from Romania, along with suppliers from Denmark, Greece, and Germany.

The arm incorporates several specialized systems designed for spaceflight. These include a flexible wrist mechanism that allows softer contact with objects, a flat electrical harness that carries hundreds of signals throughout the arm, and structural mechanisms that help withstand the forces experienced during launch and landing.

European space agencies view advanced robotics as a key capability for future exploration missions, particularly as crews prepare for longer stays on the Moon and eventual missions to Mars.

With its ability to see, feel, and autonomously manipulate objects, the Sample Transfer Arm could become an important tool for handling samples and supporting scientific operations beyond Earth.

  •  

Can AI Help You Find Your Lost Keys?

In a groundbreaking stride toward enhancing robotic cognition, scientists at MIT have unveiled an innovative memory framework that equips robots with the capacity to create, retain, and retrieve extensive, richly annotated spatial memories over extended periods. This advancement marks a significant leap beyond traditional robotic mapping techniques, enabling robots to comprehend and interact with complex environments in ways that closely mirror human spatiotemporal reasoning. The development promises transformative impacts on robotic collaboration with humans, particularly in dynamic and large-scale settings such as industrial facilities and urban landscapes.

Traditional robots, while adept at constructing geometric maps and executing predefined tasks, lack the nuanced memory capabilities that humans effortlessly employ. Consider a factory worker recalling the exact location of a partially assembled component from the previous day—a routine yet intricate cognitive task. Robots confronted with this scenario traditionally falter because their mapping systems fail to integrate detailed object descriptions and temporal context seamlessly. MIT’s new framework addresses this critical deficiency by embedding rich semantic information directly into the spatial maps robots generate as they navigate, thus producing a coherent, language-accessible mental model of their environment.

At the core of this system is a method termed Describe Anything, Anywhere, Anytime, at Any Moment (DAAAM), which synergizes advanced computer vision with robust spatial mapping. DAAAM endows robots with the ability to tag objects with descriptive annotations as they explore. For instance, a robot might label a building as the “Stata Center,” noting its architectural style, or observe a collection of bicycles and recall specifics such as a red bike sporting a flat tire. Crucially, these annotations are spatially organized within a three-dimensional map, allowing the robot to group objects logically by their locations and create a persistent, queryable memory that supports efficient retrieval.

One of the distinguishing challenges in realizing such a system lies in balancing the richness of data with the constraints of real-time operation. Existing approaches to detailed environmental annotation are computationally intensive, often taking precious seconds to process a handful of objects, thereby rendering them impractical for dynamic robotic applications. To solve this bottleneck, the MIT team engineered an optimization technique for keyframe selection, enabling the robot to identify and annotate images that offer the clearest and most comprehensive view of multiple objects simultaneously. This selective strategy accelerates the annotation process by an order of magnitude, permitting the robot to construct and update its semantic map as it moves without latency.

Beyond data acquisition, the ability to efficiently query and extract relevant information from the amassed database of spatial and semantic knowledge is vital. The researchers integrated a sophisticated large language model (LLM) enhanced with tailored toolsets designed to mitigate common issues such as hallucinations and to refine the relevance of retrieved data. This framework allows for rapid, accurate responses to complex spatial-language queries like, “Where did I leave my wallet?” or inquiries about specific landmarks within an indoor or outdoor environment. By leveraging semantic search capabilities that consider both linguistic cues and geographical context, the robot can pinpoint targets with remarkable precision and speed.

The practical implications of this technology are profound. In manufacturing settings, a robotic assistant could be dispatched to retrieve components based on natural language queries referencing past events and locations, thus augmenting human productivity and safety. Similarly, augmented reality systems could harness this structured long-term memory to guide maintenance personnel through complex infrastructure, flag anomalies based on historical data, or assist commuters in navigating public transportation hubs with personalized, context-aware directions.

MIT’s approach signifies a departure from conventional 3D mapping systems that either sacrifice descriptive depth for computational efficiency or rely on rich annotations that are prohibitively slow to generate at scale. By fusing high-level semantic perception with spatial cognition underpinned by real-time processing capabilities, DAAAM lays the foundation for a new class of robots that can engage in sophisticated spatial-temporal reasoning analogous to human common sense.

Ongoing research aims to extend this framework’s scope to encompass temporally dynamic events, enabling robots not only to remember object locations but also to encode significant occurrences within their environment. Integrating confidence metrics into the system’s responses is also a priority, enhancing the reliability and interpretability of information supplied to human users in collaborative scenarios. The vision is to cultivate a versatile, generalist robotic agent capable of executing diverse tasks on demand through naturalistic human-robot interaction grounded in shared language and understanding.

The robustness of the DAAAM system was empirically validated through comparative experiments, demonstrating superior accuracy over leading existing methodologies by margins ranging from 21 to 53 percent, contingent on the nature of the queries. The project’s intersection of computer vision, robotics, and natural language processing underscores a multidisciplinary approach vital for advancing intelligent autonomous systems that are primed for real-world deployment.

With the proliferation of autonomous machines across myriad domains, the advent of this long-term spatiotemporal memory framework addresses a pivotal gap in robotic intelligence. Robots equipped with such memory capabilities can transcend static, pre-programmed functions, adapting fluidly to evolving tasks and environments while fostering seamless collaboration with humans through shared understanding. As such, MIT’s contribution is poised to accelerate the integration of robots as capable and context-aware partners in everyday human endeavors.

This research, funded partly by the U.S. Army Research Laboratory and the Office of Naval Research, was publicly disclosed in a paper authored by Nicolas Gorlo, Lukas Schmid, and Luca Carlone and presented at the Conference on Computer Vision and Pattern Recognition (CVPR). Luca Carlone, the principal investigator and a professor at MIT’s Department of Aeronautics and Astronautics, emphasizes that the technology was developed with the goal of endowing robots with human-like language-based spatial reasoning, a foundational step toward more intelligent and helpful machines.


Subject of Research: Robotics, Artificial Intelligence, Long-Term Spatial Memory for Robots

Article Title: MIT Develops Real-Time Long-Term Memory Framework for Robots Combining Semantic Understanding with 3D Mapping

News Publication Date: Not explicitly stated; presented at CVPR 2024

Web References:

Image Credits: MIT

Keywords

Artificial intelligence, Robotics, Spatiotemporal memory, Computer vision, Long-term memory, Language models, Human-robot interaction, Autonomous systems, Machine learning, Semantic mapping, Real-time processing, Augmented reality

  •  

AI coding agents taught robots how to install GPUs and cut zip ties

What happens when you give AI coding agents a lab full of robotic arms, some compute resources, and a “generous token budget” for teaching the robots various tasks? The agents can apparently figure out a training regimen that teaches the robots to successfully cut zip ties and even insert GPUs into thin sockets on motherboards.

That glimpse into how AI can act in a fully autonomous way to automate robot training was made possible by a new agent harness framework—software that wraps around AI models to enable their use of various tools while also providing capabilities such as memory, context, constraint, and feedback loops. That agentic harness, called ENPIRE, was developed by robotics researchers at the Nvidia GEAR (Generalist Embodied Agent Research) lab alongside collaborators from Carnegie Mellon University in Pittsburgh and the University of California, Berkeley.

“A part of our NVIDIA GEAR lab now self-improves tirelessly overnight,” wrote Jim Fan, director of AI at NVIDIA, in a LinkedIn post. “We just read the reports in the morning.”

Read full article

Comments

© NVIDIA

  •  
❌