Whether consciousness arises entirely within the brain or involves processes beyond it has long been debated in scientific and philosophical circles. Historically, most academics have associated consciousness with neurons firing and communicating information; however, precisely how neural activity gives rise to subjective experience remains an open question.
Neuroscientist and consciousness researcher Christof Koch, a former faculty member at institutions including MIT and Caltech, challenges academics to take a closer look at what philosophers call the “hard problem” of consciousness: how the physical processes of the brain could produce subjective experiences in humans.
Earlier this year, Koch presented his arguments at the “Behind and Beyond the Brain” Symposium organized by the Bial Foundation, where he argued that the nature of consciousness, as well as phenomena like near-death experiences, may require a broader scientific framework.
Koch focused on three areas. First is the physical brain itself; second is the question of how we can truly agree on what is “real.” What does blue actually look like to you? What does pain feel like? These questions are difficult to answer scientifically as universal truths.
Third are unexplained experiences that may not fit neatly within a materialistic scientific framework, such as near-death experiences.
Revisiting an old approach
Looking at older philosophical models might lead researchers toward new understandings or modern ways of approaching consciousness. Koch points to concepts such as idealism and panpsychism.
Idealism holds that consciousness or mind is fundamental to reality—mind comes first, matter second—while panpsychism holds that consciousness is a fundamental feature of the universe.
If mainstream science adopted such a shift, it could profoundly change how we understand ourselves. A paradigm shift of sorts may be coming for humanity, potentially through the integration of these ideas and a move away from strictly materialistic approaches to consciousness.
Koch points to these concepts as potential frameworks in which consciousness remains a fundamental aspect of reality. He also supports the controversial Integrated Information Theory (IIT), which proposes that systems possessing sufficiently integrated information may have some form of subjective experience.
Consciousness in Perspective
For additional perspectives, The Debrief reached out to Julia Mossbridge, a neuroscientist and Sr Distinguished Fellow in AI, Mind and Society at Florida Atlantic University, and co-author of Transcendent Mind: Rethinking the Science of Consciousness (2017), the first American Psychological Association textbook on consciousness to not assume consciousness is reducible to brain activity.
“To many neuroscientists, the brain is starting to seem more like a radio receiver and potentially also a transmitter of consciousness,” Mossbridge said in an email to The Debrief. “The idea that’s losing more ground with every new credible report of so-called ‘anomalous cognition’ is the old one—that the brain creates consciousness like the gallbladder creates bile. Maybe it’s more apt to compare the brain to the heart—it doesn’t create blood, it circulates it. But because you can use a machine to circulate the blood, it’s very clear the heart is important, but in modern times, not essential for circulation.”
Mossbridge points to reports of what she calls “veridical conscious observations” that occur when a patient’s brain shows little or no measurable activity to researchers, or what is described as “brain death,” as proof that consciousness may not be produced entirely by the brain.
These experiences led Mossbridge to consider the brain as a “transformer of consciousness rather than a creator of it.” She compares the relationship to a broken radio: “If you have a disruption in your brain, like a broken radio, you might have a disruption of consciousness.”
To test this hypothesis, Mossbridge says researchers could investigate whether people can perceive and later accurately recall “veridical information” while in deep sleep or under general anesthesia. Although she acknowledges that even positive findings might have another explanation, noting that they “could be explained by basic psychic functioning using some unknown physical mechanism.”
“So if that kind of experiment produces a positive result, and is replicated, you either have to argue that psychic ability is real but is something the brain does nonlocally (like maybe through entanglement) or that consciousness can wander from the body and observe things, or both,” Mossbridge concluded.
“The physical but nonlocal explanation is a bit like a heart pumping the blood, the brain guiding consciousness. The consciousness-wandering explanation is a bit like a heart being fully separated from the blood–and another method is used to circulate consciousness.”
“Either way, the standard model fails,” she added.
Taken together, Koch and Mossbridge’s perspectives point to a provocative possibility: that understanding consciousness may ultimately require science to reconsider some of its most fundamental assumptions about the relationship between mind, brain, and reality.
Lumus is expanding its partnership with Quanta this week, which brings a licensing agreement between the two companies for next-gen waveguides.
Lumus states Quanta will manufacture its next-gen waveguides: the Z-30 2.0, A-Lens 30 (30-degree FOV), and A-Lens 50 (50-degree FOV).
The company was at CES 2026 where it stated it's future for waveguides for AR glasses will be "thinner, lighter, and easier to produce."
This morning (Sept 1), Lumus, a developer of waveguides for several big-name brands, announced a major partnership expansion.
In a press release, Lumus announced that it has expanded its partnership with Quanta Computing. This expansion sees the two companies reaching a licensing agreement "to manufacture three new geometric waveguides for AR glasses." The post states Lumus is developing three new waveguides: the Z-30 2.0, A-Lens 30 (30-degree FOV), and A-Lens 50 (50-degree FOV).
Thanks to the new licensing agreement, Quanta will handle manufacturing. What we can expect from the Z-30 2.0 is a 30-degree FOV "in a thinner, lighter optical engine." Lumus adds that its latest technology increases the display's brightness, alongside direct prescription lens bonding. The two new A-Lens waveguides (30 and 50) are Lumus' "lower-cost" production solutions.
The next-gen A-Lens waveguides are reportedly cutting down the weight and thickness by 50%. However, Lumus states it maintained its image quality and efficiency with this upgrade. This is an ongoing trend that Lumus is trying to introduce into this advancing world of AR glasses. Trimming down waveguide weight and thickness is key, but so is image quality, brightness, and cost.
Earlier today, Lumus' CEO, Ari Grobman, said, "Our relationship with Quanta has always been built around solving the practical challenges of bringing advanced AR optics into mass production. This agreement allows us to bring our next generation of lower-cost waveguides into manufacturing with a partner that understands both the technology and the production requirements behind it."
Here comes the future, and Lumus is leading it
(Image credit: Lumus)
As mentioned before, Lumus creates waveguides for several brands, even for Meta and its Ray-Ban Display Glasses. Earlier this year, Meta announced that it was doubling the production of its Ray-Ban glasses to meet demand. This was something Lumus echoed in its press release this morning. The demand for smart glasses continues to rise, but there's room for improvement. Its work to reduce the weight and thickness of its waveguides for AR glasses is one such upgrade.
Additionally, the lower-production-cost A-Lenses are designed to help bring more products to consumers. We'll have to see how things pan out going forward.
On a related note, Lumus was at CES 2026. This major tech event is where the company first stated what its plans were for the future of AR glasses. Lumus said, "That's the future for us—making these waveguides thinner, lighter, and easier to produce." Its announcement today further echoes that sentiment.
Meta announces that its neural handwriting is now rolling out in its EAP (Early Access Program) for the Ray-Ban Display.
Users are recommended to write with their index finger on a stable surface, and allow the AI to correct any typos or mistakes.
Meta confirmed this was on the way for the Ray-Ban Display in December.
Meta's neural band is an important part of its smart glasses experience. It brings convenience, and Meta says it's offering a new feature in Early Access for silent AI interactions.
If you've got a pair of Meta smart glasses, the company has announced a silent handwriting feature in Early Access using the Neural Band. In a press release, Meta says users can write "AI queries with their index finger when their display is on." It states users can get answers to questions, play a song, pull up directions, and more without much effort—just a finger. Meta went into more detail about this EAP (Early Access Program) feature.
If you're interested, Meta says users must ensure their Neural Band is paired with their glasses and that everything is updated, as well as enroll in the EAP. To enable neural handwriting, head to Meta AI app > Settings > select your glasses > Early Access > toggle Meta AI with neural handwriting. For the "best results," Meta recommends users write with their index finger. It states you should "anchor your wrist on a hard surface and use your index finger to write." What's more, you should write in print, not cursive.
To ensure the Neural Band and neural handwriting feature captures what you're writing properly, you should write smoothly and at a consistent pace. Meta says users shouldn't worry about typos, nor should they wait for their HUD to catch up to what they've written. Continue regardless, and Meta AI will automatically fix any errors.
Neural handwriting is headed to the Meta Ray-Ban Display in the EAP.
Writing in the air
(Image credit: Android Central)
It was teased in December 2025, and now it's finally here. Meta confirmed late last year that it planned to empower its Ray-Ban Display further with "virtual" handwriting. We've now seen this arrive as neural handwriting. This EMG handwriting feature was confirmed to leverage the Neural Band with the user's drawing and gesture inputs. The company also teased Instagram Reel support for its glasses, too.
The neural band was an object of interest for Android Central's Nicholas Sutrich last year, too. To him, it felt like a true device straight out of a Sci-fi movie. To Meta, the sEMG band (neural band) is a requirement for "true AR smart glasses products." Users wouldn't need to rely on a screen to type their commands. The band captures your muscle movements to act out what you're writing or capture your gestures. It's silent and completely hands-free (technically).
Android Central's Take
The neural band is pretty cool, I have to admit. But bringing this "virtual" or neural handwriting just seems like the next logical step for Meta. It can already capture gesture for scrolling through song lists, why not do more? Why not make it so writing a quick message or asking its AI is effortless with single movements?
Scientists from the Beijing Institute of Technology have revealed a colloidal quantum dot (CQD)-based infrared-to-visible upconverter that transforms the infrared spectrum into full color, allowing people to ‘see’ the infrared in the visible light spectrum.
The research team behind the novel invention said their upconverter can be used in eyeglasses to improve human vision and for augmented reality (AR) applications. They also reported a successful implantation of an upconverter in mice, opening the possibility of an implantable photoreceptor to repair or improve vision in human eyes.
Attempts to Convert Infrared to Full Color Face Limitations
In a published study detailing the novel, full-color IR glasses, the human eye is “inherently blind” to infrared radiation. Although IR radiation interacts with the retina, infrared wavelengths lack the energy to trigger the photoisomerization of the retinal chromophore that lets people distinguish colors. As a result, humans don’t have visual access to over half of the solar spectrum.
Still, the researchers write, “harnessing this imperceptible radiation is critical, as infrared perception unlocks a vast reservoir of inaccessible information.” For example, this added visual information enables critical functions in other animals, such as navigation in darkness or detecting predators and prey from their heat. Still, they observe that our natural visual-light spectrum vision is “utterly inadequate” to access this information that so many animals rely upon.
In their study, the researchers highlight recent advances in accessing the IR spectrum, including “ocular injectable photoreceptor-binding upconversion nanoparticles” that convert infrared light into visible emissions. Other studies have also evaluated wearable lenses with nonlinear upconversion projecting infrared-transformed visible light onto the cornea and photovoltaic nanowire networks.
Although these approaches achieved some IR sensitivity, the study authors said they remain “fundamentally constrained” to narrow near-infrared (NIR) spectral response. These prior efforts also relied on high-intensity laser excitation that the team notes is “incompatible with natural infrared radiation.” The result is a “mostly monochromatic output” lacking infrared spectral discrimination in full color.
Subtle IR Wavelength Variations Facilitate Upconversion
For their new approach, the research team investigated a mercury telluride (HgTe) CQD-based infrared-to-visible upconverter. They note that this material combination leverages the material’s “quantized energy level states” in synergy with a separate, dual-emissive-layer organic light-emitting diode (OLED).
Specifically, the engineered layers captured IR photons in strategically engineered hole-trapping barriers. The team explains that this “allows for subtle differentiation of infrared input by tuning the emitted visible color and luminance” based on infrared wavelength and intensity.
Above: The infrared-to-full-color upconverter is applied as (A) the wearable eyeglass and (B) the new-generation retinal photoreceptor (Image credit: Fu, C., Zou, J., Yang, X., Hao, Q., Tang, X., & Mu, G. (2026). Multispectral infrared-to-full-color upconversion expanding human vision).
“This upconversion yields a discrimination sensitivity to subtle infrared variations exceeding two orders of magnitude higher than conventional single-color modes, capitalizing on the human eye’s intrinsic superiority in chromatic differentiation over simple luminance contrast,” they explain.
To test their material’s IR upconversion ability, the researchers designed a pair of lightweight (23 grams) semi-transparent wearable eyeglasses. The team said these tests were a success, achieving “high-resolution infrared imaging” derived from the infrared spectral information. However, instead of monochrome, the research team said that data is displayed as full-color visible patterns directly onto the retina, “enabling intuitive infrared vision without obstructing natural sight.”
Surpassing the Evolutionary Boundaries of Biological Photoreception
Although the current version of the full-color, infrared-capable glasses is still in the development stages, the team said the technology could lead to advancements in visual prosthetics, augmented reality (AR), and “navigation under environmental conditions where normal sight is degraded.”
Following the successful validation in mice, the team said their technology could also serve as the technological basis for an implantable new-generation retinal photoreceptor. In that design, the team said an implanted device could transform infrared light into visible light emissions “that stimulate light-sensitive proteins on retinal neurons to bypass damaged photoreceptor cells and potentially restore visual function across both the visible and infrared spectra.”
“By surpassing the evolutionary boundaries of biological photoreception, this technology paves the way for next-generation visual prosthetics, high-fidelity augmented reality, molecularly sensitive substance identification, and robust navigation in degraded visual environments,” they conclude.
Christopher Plain is a Science Fiction and Fantasy novelist and has spent the last six years as Associate News Editor and Head Science Writer at The Debrief. Follow and connect with him on X, learn about his books at plainfiction.com, or email him at christopher@thedebrief.org
Meta may dominate the smart glasses zeitgeist, but it's far from the only player — my favorite brand in the space is actually Xreal, with its range of impressive but pricey wearable-display specs. And with the launch of its new xbx brand, Xreal's lineup now includes more budget-friendly options too.
If money were no object, Xreal’s One Pro glasses are easily the crème de la crème of smart specs. Their Bose-tuned audio is impressive for such a small package, their electrochromic dimming tech is perfect for swapping between clear vision and shaded for optimal viewing conditions, and their unique optic engine displays their image more vividly than the competition, with fewer visible distortions.
The only issue is their price — $649 / £579 (they aren't available directly in Australia), absent any discounts, although they can be found on sale occasionally.
If you’re on a budget, my preferred pick has instead been RayNeo. Its Air 4 Pro specs debuted earlier this year, and not only are they affordable at just $299 / £289, they still come with solid specs, including HDR10 support.
With its new xbx a01+, however, Xreal might have just claimed the budget crown for itself.
The RayNeo Air 4 Pro glasses (Image credit: Alastair Jennings)
Budget brilliance
Now the elephant in the room is that Xreal's specs cost a bit more. At their current $299 / £269 discounted price at Xreal.com they're on par with or cheaper than RayNeo's glasses; however their official price is $359 / £349, and while they aren’t discounted at the moment, we’ve regularly seen the RayNeo Air 4 Pros selling for $249 / £249 (RayNeo also doesn't seem to sell directly to Australia) or cheaper.
At the same time, the Xreal xbx a01+ glasses boast a handful of improvements which warrant a slightly higher price.
For a start, their brightness tops out at 1,600 nits, while the RayNeo Air 4 Pro can only hit 1,200 nits. This extra brightness should allow you to get more out of the HDR10 support, offering more room for the light vs dark contrast and vivid colors to breathe across the video format’s visual range. Highs can look more high, helping lows look lower by comparison.
(Image credit: Future / Hamish Hector)
Aiding the image quality are the swappable lens covers.
While advertised as giving you the ability to choose which color of sunglasses cover you most want to complement your style, the xbx's box also includes a completely opaque black cover you can use with your specs. Because the specs are essentially virtual projectors, a darker environment ensures the highest-quality viewing conditions, though I found the regular sunglasses were fine even in bright sunlight.
The xbx glasses are lighter too, weighing just 62g compared to the RayNeo Air 4 Pro's 76g. Admittedly I haven't found them majorly more comfortable, but as I kicked back with them in a sun lounger while on vacation for hours on end I was relaxed enough to doze off on several occasions.
The only tradeoff is, I'd say, the audio, which doesn't seem to involve Bose tuning this time, and isn't quite as impactful as the Air 4 Pro's. That said, I'll almost always use earbuds or headphones with my display glasses, because they’re better in terms of quality and minimize sound leakage, so this downgrade didn't bother me.
(Image credit: Future / Hamish Hector)
Who wins? You decide
You'll also have to contend with Xreal's standard optic engine too. This is the prism that reflects the micro-OLED display into your eyes, and the standard design is susceptible to external reflections — most commonly light bouncing off the top of the specs and into your view. Pictures will also be a little smaller thanks to a reduced field of view, and can be a little blurrier at the frame edges.
Only Xreal's One Pros boast the improved design, but having experienced it, it's tough to go back to the standard optic engine.
To be fair, though, that’s mostly a me problem, and if you’re new to the display glasses world the Xreal xbx a01+ and RayNeo Air 4 Pros should be perfectly adequate performance-wise.
(Image credit: Future)
As for which of the budget specs you should go for — whichever is cheaper would be my suggestion.
The technical improvements of the xbx specs are solid, but if you can save $40/£40 or more on the RayNeos, their few deficiencies can be overlooked.
That said, if stylish, colorful designs are a must, alongside a marginally better image quality, then you'll be very pleased with the new Xreal xbx a01+ glasses.
I’ve tested dozens of smart glasses for TechRadar, but every time I get a new pair of Meta Ray-Bans to try, I’m reminded why they’re so popular — because yet again my past three weeks with the Ray-Ban Meta Optics have been a joy.
Mostly.
Much like Meta’s previous smart specs, you’ll find a host of technological features crammed into the eyewear.
There are speakers for open-ear audio, which is perfect for listening to casual tunes, taking a call, or hearing the AI’s responses to your queries — with the glasses’ microphones also doing a great job at picking up what you’re saying with minimal noise disruption.
Dedicated earbuds will sound better for sure — the bass isn’t as emphatic as I like with Meta’s Optics glasses (or its other models) — though I did enjoy listening to music through my glasses for hours on end with no major complaints.
(Image credit: Future / Hamish Hector)
As mentioned, there’s also Meta’s AI, which is accessed through the connected phone app and voice commands on your specs. It can answer questions you ask it and, with your permission, see what you see by accessing the Meta Optics’ camera to provide additional context to your questions, and interact with connected apps to play music requests or tell your health and fitness stats.
I’m not a big AI user except when I’m on holiday or just generally acting like a tourist. On our honeymoon, my wife used Look and Ask to check the ingredients on hair care products to make sure she wasn’t allergic, and I got the AI to supply me with phrases to use when I realized my limited Spanish wasn’t going to cut it.
The AI’s also perfect for identifying landmarks and telling you more about historic sites. I relied on it so much on a trip I took to Italy with my other Meta glasses a while ago — it helped me understand the ruins of Pompeii and better learn the history of the Colosseum.
A photo taken with my Meta Optics (Image credit: Future / Hamish Hector)
Beyond identifying foreign signs and buildings, the cameras are ideal for first-person video. I used them to snap photos and videos of my wife and me, and they came out really well — they're the ideal way to record memories digitally, as you can still live in the moment rather than being cut off behind a phone screen.
I’ll talk more about cameras in a moment, but before that I want to focus on the key way the Optics is different to Meta’s other Ray-Ban smart specs: it's comfort and fit.
Built to last all-day
Meta’s smart glasses are chunkier than non-intelligent eyewear, which can make them feel more rigid and less comfortable than regular specs. To make the Optics better suited to all-day wear (something I can confirm they handle effectively after wearing these glasses for hours on end over the past few weeks), you’ll find interchangeable nose pads and adjustable temple tips to help you sculpt them to your face shape.
The correct nose pads have helped the specs sit better on my head without slipping down, while the temple tips can be carefully manipulated (either at home or at an optician) to help the Optics grip your face without applying too much pressure to the sides of your head.
(Image credit: Future / Hamish Hector)
If you’re considering getting some yourself, my final design note is this: pay extra for transition lenses.
The Ray-Bans I’ve been testing are outfitted with the Transitions XTRActive option, meaning they change between clear and shaded when exposed to sunlight — even when I’m in the car — and the lenses changed so quickly as I moved between different environments (something I noticed a lot when going between indoor and outdoor spaces on my honeymoon to Tenerife).
My non-smart eyeglasses boast Transitions Gen S lenses, and I’m also happy with them, or you can opt for polarized Transitions if you want even more light protection, if you’re also willing to pay a bit extra.
Regardless of which type you opt for, Transitions lenses are essential because they maximize the usefulness of your smart specs. Without them, you’re limited to either clear glasses you can only use indoors or on darker days, or shaded glasses that can only be worn outdoors when the sun is brightly shining.
Why did there have to be a but?
Now, the elephant in the room is that Meta’s smart specs are facing increasing amounts of scrutiny from the public because of their cameras.
The upshot seems to be a wave of negativity: I conducted a poll through one of my recent stories, and over half of all respondents said they think smart glasses should be banned.
For its part, Meta is doing plenty to combat these issues: it cracked down on Facebook Marketplace listings for glasses modding services and issued an update to prevent the camera from working if the recording light on your smart eyewear is disabled, and it’s also tackling Instagram profiles belonging to creeps like pickup artists who promote using the glasses to harass people.
I still think more needs to be done on the privacy side of things — at the very least being much more explicit about what features share captured content with Meta, though ideally all of your content would remain private through an Apple-like Private Cloud Compute server setup.
I want to just love my smart glasses (Image credit: Future / Hamish Hector)
As of now, I’m admittedly very nervous to wear my Meta Optics. I haven’t been worried about going around wearing my Meta Ray-Ban and Oakley specs in the past, but with growing public anger over the way smart specs are being misused, I have a genuine concern someone will see me wearing them and act violently towards me.
Or I could simply be banned from entering a space while wearing them. Some Comic-Cons, offices, and courts don’t let you in wearing Meta-style smart glasses. Especially for something as essential as prescription glasses, and pricey ones at that, being constantly nervous while wearing them or barred from entering certain spaces isn’t ideal.
There is genuinely so much to love about the smart glasses, and if you haven’t yet got a pair and wear prescription eyeglasses, the Meta Ray-Ban Scriber Optics styles are the ones I’d most highly recommend.
ARK augmented reality describes a form of spatial interaction in which artificial intelligence does more than place digital objects in the physical world. It uses knowledge from foundation models to understand unfamiliar scenes, generate useful information and adapt its response to the person, task and environment.
The concept comes from a 2023 research paper titled “ArK: Augmented Reality with Knowledge Interactive Emergent Ability.” Its authors proposed a system that transfers knowledge from large foundation models into new physical and virtual environments. Instead of training an AI agent separately for every room, object or activity, ArK uses existing knowledge memory to interpret what it sees and generate appropriate 2D or 3D content.
Traditional augmented reality places information over reality. ARK augmented reality tries to understand reality before deciding what should appear.
That distinction creates a compelling direction for Apple. Vision Pro already blends digital content with physical space. Apple Intelligence adds language, image understanding and personal context. Visual Intelligence lets users look at objects, text and scenes before asking for information. Siri AI adds a conversational layer. ARKit, RealityKit and visionOS provide the spatial foundation.
Apple now has many of the components required to build a knowledge layer over the physical world.
ARK Augmented Reality Goes Beyond Digital Overlays
Most familiar augmented reality experiences are relatively fixed. A furniture app places a digital sofa in a room. A navigation app displays arrows over a street. A museum app identifies an artwork. A repair app shows where a component belongs.
These experiences can be useful, but they usually depend on predefined content and rules. The system recognizes a known object or location, then presents prepared information.
ARK augmented reality proposes something more adaptive. A knowledge-enabled system could enter an unfamiliar environment, interpret what is present and generate assistance based on a broader understanding of objects, relationships and possible actions.
A user looking at a complicated electrical panel would not only see labels. The system could identify the panel type, explain the purpose of each section, compare the arrangement with standard configurations and warn the user not to touch dangerous components.
A student examining a machine could ask how it works and receive a spatial explanation attached to its real parts. A surgeon could review anatomical information positioned around a medical model. A technician could receive repair guidance even when the exact equipment had not been included in a previous training dataset.
The display becomes less like a floating screen and more like an informed interpretation of the environment.
Image Credit: Apple Inc.
Why Apple Is Positioned to Build It
Apple’s advantage would not come from inventing every part of ARK augmented reality. It would come from combining hardware, software, artificial intelligence and personal context into one controlled ecosystem.
Vision Pro already understands rooms, surfaces, hands, eyes and objects well enough to place digital content convincingly in physical space. Its cameras and sensors continuously map the environment while visionOS manages windows, volumes and immersive experiences.
Apple Intelligence provides another layer. Foundation models can interpret language, summarize information, generate content and connect requests with app actions. Visual Intelligence can identify text, images and real-world objects. Siri AI can act as the conversational interface.
The Apple ecosystem also provides personal context that a generic AR platform may not have. Calendar knows the user’s schedule. Maps understands location. Photos contains visual history. Notes and Mail hold project information. Health contains personal trends. Home understands rooms, accessories and automations.
An Apple-built knowledge layer could combine the environment in front of the user with the information already available across trusted devices and apps.
A person preparing for a meeting could look at a conference room and see the agenda, participant names and relevant documents arranged around the table. A traveler could look across a train station and receive directions connected to the correct platform, departure time and digital ticket. A homeowner could examine a room and see live information from temperature, air-quality and energy sensors.
The physical environment becomes the interface.
Vision Pro Is the Laboratory
Vision Pro is the obvious development platform for this idea because it already offers the cameras, displays, sensors and processing required for advanced spatial understanding. Its size and price limit broad adoption, but those same characteristics allow Apple and developers to test experiences that would be harder to build first on lightweight glasses.
ARK augmented reality on Vision Pro could become especially useful in enterprise environments. Apple already promotes the headset for employee training, product design, field service, collaboration and specialized visualization.
A worker could enter an unfamiliar facility and receive contextual instructions attached to real equipment. An engineer could inspect a physical prototype while design changes, performance data and maintenance history appear around it. A new employee could learn a workflow through spatial guidance generated from company manuals and operational knowledge.
The system would not need to display every available fact. It would need to understand which information fits the current task.
That is where AI knowledge becomes more valuable than a traditional AR overlay. Too much information can be as disruptive as too little. A strong system should filter, prioritize and explain rather than cover the room with floating labels.
The iPhone Could Bring ARK to More People
Vision Pro may be the development environment, but iPhone could provide the scale. The iPhone already combines cameras, LiDAR on Pro models, location, motion sensing, Visual Intelligence and Apple Intelligence.
An iPhone-based ARK experience would be less continuous because users would need to hold up the device. It could still deliver useful knowledge interaction without requiring a headset.
A user could point the camera at a plant and receive care guidance based on its species, visible condition, local weather and placement in the room. Someone assembling furniture could receive generated guidance positioned over the actual parts. A shopper could compare products by looking at them rather than manually searching for each model.
The difference from ordinary visual search would be interaction. The user could ask follow-up questions, change the goal and let the system update the visual guidance.
Future glasses could eventually make the experience persistent. The iPhone would help Apple develop the intelligence, privacy controls and app ecosystem before lightweight eyewear becomes ready.
A World That Explains Itself
The most ambitious version of ARK augmented reality is a world in which information is attached to objects, places and situations only when requested or needed.
A city could become understandable without searching through maps, review sites and separate transit apps. Buildings could reveal history. Streets could explain temporary closures. Restaurants could surface reservations, dietary options and accessibility details. Public transport could show the correct entrance and platform.
Inside a home, Apple’s knowledge layer could explain energy use, identify maintenance problems and guide repairs. In a classroom, physical models could become interactive lessons. In healthcare, patients could receive visual instructions for approved home procedures or medication devices.
The experience would depend on context rather than menus. The user looks, asks and receives information positioned where it belongs.
This is the transition from augmented reality as decoration to augmented reality as understanding.
Privacy Will Decide Whether It Works
A knowledge layer capable of understanding the physical world also creates serious privacy questions. The system may see homes, workplaces, documents, faces, screens and private conversations. The more context it receives, the more useful it becomes. The same access can also make it invasive.
Apple’s privacy architecture could become one of its strongest advantages. On-device processing would allow some scene understanding without sending continuous camera feeds to external servers. Private Cloud Compute could handle more complex requests while preserving stricter controls than an ordinary cloud AI service.
Apple would also need visible controls over what is analyzed, remembered and shared. A user should know when the system is interpreting the environment and whether information is temporary or retained. Workplaces and public spaces would need rules for recording, identification and sensitive data.
ARK augmented reality cannot become socially acceptable if people feel that nearby devices are constantly cataloging them.
The successful version would behave less like surveillance and more like intentional assistance. The user asks for understanding. The system processes only what is needed. The knowledge disappears when the task ends unless the user chooses to save it.
Image Credit: Apple Inc.
Developers Could Build Knowledge Worlds
Apple would not need to create every knowledge experience itself. visionOS, ARKit, RealityKit and App Intents could let developers connect specialized knowledge with physical environments.
A medical company could build guidance for approved equipment. A manufacturer could create spatial maintenance systems. A university could attach interactive explanations to laboratories and museums. A retailer could create product comparisons based on inventory and customer preferences.
Apple could provide the shared intelligence layer: scene understanding, object recognition, language interaction, privacy permissions and spatial presentation. Developers would provide the trusted domain knowledge.
This division would be essential. General AI can explain many things, but professional environments require verified information. A generated answer about a historical building has different consequences from an answer about industrial machinery or medical treatment.
Apple’s platform could distinguish between general knowledge and certified domain sources, allowing users to see where information comes from before acting.
From Spatial Computing to Contextual Computing
Apple describes Vision Pro as a spatial computer because digital content can occupy physical space. ARK augmented reality suggests the next progression: contextual computing.
A spatial computer knows where digital content is placed. A contextual computer understands why the content is useful in that location, what the user is trying to accomplish and which knowledge should appear.
That would connect several Apple strategies that currently look separate. Vision Pro supplies spatial awareness. Apple Intelligence supplies reasoning and generation. Siri AI supplies conversation. Visual Intelligence supplies real-world understanding. The ecosystem supplies personal context.
Together, they could create a system that does not wait for users to open the correct app and search through the correct menu. The environment itself becomes the starting point.
Apple’s opportunity is to make that interaction restrained rather than overwhelming. The company does not need to cover reality with digital noise. It can use design, privacy and contextual intelligence to make information appear only when it improves the moment.
ARK augmented reality offers a useful description of that future. Reality remains visible. Knowledge becomes interactive. The computer begins to understand the world in front of the user instead of asking the user to translate everything into a search box.
It’s been several years since generative AI and large language models (LLMs) took the world by storm. LLMs surpassed earlier natural-language systems at generating text, while diffusion models enabled generating images, music, and videos.
These generative AI models work well in the digital world, but on their own, they have limited capabilities to comprehend the three-dimensional physical world and other spaces. This includes the objects occupying an area, how they relate to each other, tracking movement, and answering complex questions requiring an understanding of dimensions, distances, motion, and collisions.
Spatial intelligence is an AI capability that allows models to reason about three-dimensional space. These models can generate 3D scenes of the world and other spaces. This content can then be displayed through traditional renderers, game engines, or AR/VR systems that use spatial computing techniques. But it’s the spatial intelligence model’s ability to connect natural language with 3D models that has the most applications in robotics, manufacturing, construction, and other physical environments.
Dr. Fei-Fei Li, often called the godmother of AI, published a manifesto on how spatial intelligence is AI’s next frontier, contrasting it with LLMs. “While current state-of-the-art AI can excel at reading, writing, research, and pattern recognition in data, these same models bear fundamental limitations when representing or interacting with the physical world,” wrote Dr. Li. “Our view of the world is holistic—not just what we’re looking at, but how everything relates spatially, what it means, and why it matters. Understanding this through imagination, reasoning, creation, and interaction—not just descriptions—is the power of spatial intelligence.”
The concept of spatial intelligence isn’t new and was described in Howard Gardner’s book, Frames of Mind, in 1983. Recent breakthroughs, including the launch of World Labs’ Marble and its $1 billion funding round, and competing approaches from Google’s Genie 3 and Nvidia Cosmos, should put spatial intelligence and world models on more R&D road maps.
What are spatial intelligence models?
It’s important to develop AI literacy and understand the terminology and concepts related to the physical world and 3D AI technologies:
Spatial intelligence encompasses specialized approaches such as geospatial models for mapping the physical world and building information modeling (BIM) for modeling physical structures. It also extends to generative 3D, robotics, and physical reasoning applications.
World models are a class of neural network architectures and are currently a prominent approach to building spatial intelligence.
Digital twins are live, virtual replicas of physical assets that combine 3D models with real-time sensor data. Spatial intelligence, an emerging capability of digital twins, adds natural-language prompting, generative scenario exploration, and physics-aware reasoning.
Spatial computing refers to digital content anchored in and interacting with physical space, sensed and rendered in three dimensions and delivered through AR/VR and mixed-reality systems.
“Spatial intelligence models go beyond pixels to understand the 3D structure of the world—how objects are positioned, how they move, and how they interact,” says David Fattal, founder and CTO at Leia. “This enables applications like more realistic video generation, spatial computing interfaces, and AI systems that can reason about physical environments. As real-world 3D data becomes more available, these models will become foundational to the next generation of visual AI.”
Monitoring the built environment
To better understand spatial intelligence, let’s consider physical infrastructure such as bridges and buildings. The American Society of Civil Engineers estimates a $9.1 trillion investment is needed from 2024 through 2033 to achieve a state of good repair. When maintenance and monitoring lag, it can lead to major failures such as the 2022 collapse of the Fern Hollow Bridge in Pittsburgh.
Spatial intelligence and the development of digital twins may help identify issues earlier and prioritize where investments are needed. “Spatial intelligence models serve as the 4D digital blueprints for our built environment, allowing us to visualize and predict the complex interactions between aging assets and the shifting ground beneath them,” says Patrick Cozzi, chief platform officer at Bentley Systems. “By synthesizing disparate geospatial data into a living digital twin, these models provide the foresight necessary to mitigate the hidden risks of structural fatigue and subsurface instability.”
There’s a significant challenge in bridge health monitoring and transitioning from manual, infrequent structural inspections to leveraging sensors, digital twins, and spatial intelligence. Cozzi adds, “This integration of continuous field data moves beyond static documentation, empowering agencies to evolve from reactive repairs to proactive, resilient asset management that safeguards the long-term integrity of our most critical public systems.”
Avoiding collisions
Bridges are largely static, but the real world is increasingly being occupied by autonomous systems such as self-driving cars, robots, and drones. And where there are moving systems, there is a risk of collisions.
“Spatial intelligence models are AI systems that reason about the physical world by combining vision, sensor data, and contextual cues to understand space, motion, and object relationships,” says Sudeep George, CTO at iMerit. “The value of spatial intelligence models lies not just in perceiving an environment, but in enabling machines to act within it safely and in real time. That is especially important in robotics and autonomous systems, where decisions must be made in complex, multimodal, fast-changing settings.”
To see one example, this tutorial for simulating robotic environments combines Nvidia Isaac Sim, an open source robotics reference framework, with spatial intelligence in Marble from World Labs.
Today’s collision detection systems, such as those used in autonomous vehicles, typically rely on modules for sensing, perception, planning, and control. Spatial intelligence models may offer improvements by assessing the collision risks of unidentified objects or by tracking objects that move out of sensor view. For example, Waymo’s World Model, built on Genie 3, is a simulator that generates complex weather conditions and other critical safety events.
For an out-of-this-world example, James Urquhart, field CTO and technology evangelist at Kamiwaza, has delivered several examples of spatial intelligence applications, including one for satellite collision detection and conflict analysis. Urquhart says, “Models that specialize in these types of data sets, as well as the physics and geography of the real world, enable faster and more accurate decision-making for tasks that depend on them.”
Applying spatial intelligence
Recent spatial intelligence announcements include creating 3D worlds from image or text prompts with Marble and simulating water physics, lighting, weather, and animal behavior with Genie 3. But difficulties remain in bringing spatial intelligence to physical-world use cases.
“Spatial intelligence and world models are laying the groundwork for future AI agents that will be able to interact in and with our physical world,” says Jason Corso, cofounder and chief scientist at Voxel51. “These models are significantly more challenging to develop and test, largely because the data underlying their development is complex, and it’s hard to handle all of the combinatorics involved in the physical world.”
In addition to learning the models and prototyping with them, development and data leaders need to review the data assets that will feed spatial intelligence models. “Spatial intelligence models translate location signals into a structured understanding of the real world, but they’re only as reliable as the data beneath them,” says Dan Adams, executive vice president and general manager of Enrich at Precisely. “The real unlock isn’t the model—it’s the reference layer with persistent identifiers, confidence metadata, and source lineage that lets AI reason about places, not just match strings.”
Even once applications are developed, there will be infrastructure challenges in deploying them at the edge. Ali Kayyam, principal research scientist at BrainChip, says, “The key to unlocking spatial intelligence at scale is having the low-power, event-driven hardware that can run it at the sensor in real time where it matters most.”
Where to get started
My suggestions for developers looking to get hands-on with spatial intelligence and world models:
A recent study documented a participant using an AI model to simulate a conversation with her late grandmother. According to her report, the experience offered a sense of closure that she had not experienced before.
Researchers at the University of Colorado Boulder call these AI agents ‘generative ghosts.’ These systems use data such as writing, voice recordings, or photographs from the deceased to create simulated conversations. Some platforms, such as Project December and Séance AI, generate text-based versions of journal entries and messages. Others, such as HereAfterAI, include voice and images. A few companies have also developed virtual reality experiences that let users interact with holographic representations of loved ones.
Jack Manning, the PhD candidate in Information Science who led the study, expected the sessions to feel unsettling. “We originally thought it might feel very Black Mirror creepy to people and make them uncomfortable,” he said. “I ended up being completely wrong. People thought it was amazing.”
How To Build a Ghost
The study included 16 participants, aged 22 to 50, each of whom joined a Zoom interview about a deceased relative or friend. During the session, a facilitator led the conversation while another researcher entered details into a large language model, generating an AI representation of the deceased in real time.
Each participant interacted with two versions of the AI model for about twenty minutes each. One version spoke as the deceased, using the first person, while the other described the person in the third person. A facilitator stayed present to offer support if the conversation became emotionally difficult.
The results indicated a clear preference for the version that spoke as the deceased individual, rather than about them. Researchers described this as a preference for ‘reincarnation’ over ‘representation.’
What People Will and Won’t Forgive
Participants tended to overlook factual mistakes or made-up details from the AI. What mattered more was whether the AI captured the right tone and language. If the model used words or phrases that felt out of character, people were more likely to disconnect from the experience. The study found that getting the voice and manner right was more important than perfect factual accuracy.
How the AI communicated also shaped the experience. Participants preferred short, to-the-point sentences and even the use of emojis, rather than the longer, more formal responses that AI models often produce. This suggests that future versions of these systems should pay close attention to conversational style.
A Technology Both Wanted, and Feared
All sixteen participants indicated that they would be willing to use the technology again. However, most also expressed concerns about the potential risks of individuals in grief accessing these AI systems without supervision.
Lead researcher Jack Manning said his own experiences with loss have shaped his view of the risks and benefits of AI grief tools. He acknowledged that negative outcomes are possible but also noted that the technology could help some users find a sense of closure and peace.
Jed Brubaker, the study’s senior author and an associate professor of Information Science, said the lab took on the work precisely because the field had outpaced the research. “To our knowledge, we are conducting the first user experience studies of simulated AI ghosts,” he said.
The research team intends to involve mental health professionals in future studies. The next phase will evaluate the psychological risks and benefits of prolonged use of AI grief tools to ensure that empirical research guides product development.
Generative ghost technology is already available through commercial platforms, and people are actively using these systems. The study highlights that voice, tone, and pacing are often more important to users than factual accuracy. As the technology develops, researchers are working to address the challenges of providing appropriate safeguards and responsible design.
Austin Burgess is a writer and researcher with a background in sales, marketing, and data analytics. He holds an MBA, a Bachelor of Science in Business Administration, and a data analytics certification. His work focuses on breaking scientific developments, with an emphasis on emerging biology, cognitive neuroscience, and archaeological discoveries.
The Samsung Galaxy XR headset is now available for preorder ahead of its July 8 launch in the UK, and while you can’t pick it up at Amazon, that doesn’t mean you can't snag a Prime Day-like deal or two.
Best of all, you won’t need to be a Prime or other subscription service member; Samsung’s discounts are available to every shopper through its online store — you’ll just need to bundle a few things with your purchase, or sign up to PayPal (if you haven’t already) and check out using that as your payment method.
For that PayPal offer, you’ll get £100 off using the code PAYPALXR at Samsung's online store. That’s a serious saving off the £1,699 asking price, but you only have until July 7 to take advantage of this offer.
Plus, if you’re looking to upgrade your Samsung tech, or dive deeper into its ecosystem, it has deals that will save you 10% on Galaxy smartphones, a Galaxy Watch, and Galaxy Buds.
Today's best Samsung Galaxy XR deals
Code: PAYPALXR If you buy the Samsung Galaxy XR headset using PayPal then you can save £100 with the code PAYPALXR. You only have until July 7 to use this deal, but if you’re strongly considering getting the Samsung Galaxy XR headset then it’s a hefty saving on the device for little to no effort (especially for folks with a PayPal account already).View Deal
Save 10% on Galaxy smartphones — August 4, 2026 Save 10% on Galaxy watches and buds — September 30, 2026 If you don’t want to simply buy a Galaxy XR headset, but also upgrade your whole Samsung setup, then this combo deal is for you, as you can save 10% on Galaxy gadgets. I currently use a Galaxy Watch 8 Classic, a pair of Galaxy Buds Pro 4, and a Galaxy S26 Ultra, and they are all superb, and this kind of discount only makes the tech even better.View Deal
Save 30% on a travel case & controllers — September 30, 2026 Lastly, much like the above tech offer you can also save big on an official travel case and Samsung controllers when they’re bought with the Galaxy XR headset. The controllers will be required for some titles, though most games rely on hand tracking so I’m not convinced you need a pair straight away. A case, however, is a must-buy. Whether it’s this one or an unofficial box, any VR headset is much easier to carry around with a case, and it helps keep the headset’s lenses from getting damaged by direct sunlight. Besides a silicone facial interface (which this headset lacks), a case is the only accessory every VR headset user should own.View Deal
After spending a week with the Samsung Galaxy XR headset, Lance Ulanoff called it “an impressive multimodal AI spatial computer,” though Android XR and the overall experience didn’t hold a candle to visionOS and the Apple Vision Pro, especially the M5 version.
He also noted that, at launch, Gemini felt less deeply integrated as he had originally imagined, and the lack of precision with controls dulled the ‘wow’ factor.
At the same time, the Galaxy XR headset is half the price of the Vision Pro, so technical downgrades are to be expected, and as Google and Samsung iterate on the software, you could find the Samsung Galaxy XR headset seriously ups its game — especially as more apps get XR support.
It delivers some impressive OLED visuals despite costing less, and can integrate with the full G Suite of apps for some spatial productivity. So, if you want something that balances work and play, with great performance, at a more affordable price point, and don’t mind missing out on the Quest gaming library, you’ll struggle to find something better than Samsung’s XR machine.
A new 3D image projection system from researchers at the UCLA Samueli School of Engineering and the California NanoSystems Institute (CNSI) marks a major step toward overcoming a longstanding problem for holographic technology.
Professor Aydogan Ozcan led the work reported in a recent paper published in Light Science and Applications, which describes the team’s new snapshot 3D image projection system, solving one of the technology’s most daunting issues.
“These results establish the diffractive 3D display system as a compact and scalable framework for depth-resolved snapshot 3D image projection, with potential applications in holographic displays, AR/VR interfaces, and volumetric optical computing,” the authors write.
3D Image Challenges
While 3D films viewed wearing special glasses were popular a decade ago, this type of 3D image projection is something more akin to the holodeck on Star Trek, projecting sophisticated, real-time images. While advances have been made in bringing such technology to life, many obstacles remain to creating convincing three-dimensional projections.
The snapshot 3D image projection system described in the paper combines a digital encoder with a passive optical decoder and uses a deep learning algorithm that operates end-to-end to overcome one of holography’s greatest challenges.
Interplane cross-talk is one of the main issues associated with 3D image projection. As 3D objects are sliced up and images projected at various depths, elements from one image can leak through to the image in front of it, destroying the illusion of a solid object existing in three-dimensional space.
Improving Holographic Technology
In the new system, an image first enters the encoder, which uses a Fourier-based neural network to separate spatial and frequency features into a stack of images. It then analyzes the positions where the images will be projected to simultaneously generate a 3D visual. Then the images are decoded and projected onto structurally optimized surfaces, where the depth-dependent field programming occurs as the light propagates.
The deep learning algorithm, building on recent advances in combining machine learning and 3D projection, optically routes the image slice to its correct depth while adjusting for in-plane leakage.
Together, this system creates a compact, high-fidelity display in a single shot, combining closely spaced axial planes to create the illusion of a three-dimensional object from projected light. These improvements provide for a more natural depth perception and increase visual comfort for the viewer.
The Future of 3D Image Technology
Advancing 3D imaging technology is essential for a variety of emerging applications, including augmented reality, virtual reality, holography, and immersive visualization. The rapid degradation of image fidelity caused by light bleed has been one of the primary issues impeding progress in these fields.
So far, the researchers have demonstrated their multi-plane snapshot image projection with a model of volumetric scenes comprising 28 axial slices. They also constructed a simpler real-world prototype to validate the model, made of only two planes. Experiments with the prototype corroborated the model’s results, producing outputs that matched the simulated numbers and achieved a similar image quality. Additionally, when the team ran their prototype against a projection without a diffractive decoder, their device exhibited considerably superior performance.
Between the simulations and laboratory tests on a physical prototype, the team has demonstrated that their work is scalable yet compact, enabling a 3D display with high axial resolution. In their current form, the researchers see near-eye displays, microscopy, and volumetric optical computing as the primary use cases, but continued development could open possibilities for multi-perspective holography and energy-efficient 3D systems.
Ryan Whalen covers science and technology for The Debrief. He holds an MA in History and a Master of Library and Information Science with a certificate in Data Science. He can be contacted at ryan@thedebrief.org, and follow him on Twitter @mdntwvlf.
Qualcomm CEO Cristiano Amon said Tuesday that the company is working on over 40 different AI wearable devices — including jewelry, earbuds with cameras, pins, and watches — a sign of how aggressively the chipmaker is betting that the next major computing platform won't be a phone.
There’s still no word on the final launch price, however. | Image by Xreal
The Project Aura glasses collaboration between Xreal and Google is now one step closer to being something you can buy. Reservations for the second Android XR device, now dubbed the Xreal Aura, are available for $99 starting today, with a full launch in the US, UK, Japan, Canada, and South Korea expected sometime this Fall.
The Xreal Aura follows Google's first Android-powered XR device, the Samsung Galaxy XR headset, which launched at $1,799 in October 2025. We still don't have a full price for the Aura - which Google has previously described as "a headset masquerading as glasses" - but Best Buy will be the first in-store retail partner whe …
This record-breaking crystal can transform from mirror-like to glass-like, opening the door to tiny optical chips, AR displays, and smart contact lenses. Creating truly “invisible” wearable technology such as smart contact lenses and ultrathin augmented reality glasses will require a radical rethink of how optical devices are built. Instead of relying on conventional bulky components, [...]
Login
