Meta has announced its intention to focus on open-weight large language models. Additionally, the company announced the release of an open model called Muse Glimmer and a promise to open the weights for Muse Spark 1.2, its more powerful model, in the next few weeks.

Alongside these announcements, Meta CEO Mark Zuckerberg published a more than 6,000-word essay outlining the company's philosophy about AI systems and governance moving forward. The essay aims to differentiate Meta from companies like OpenAI and Anthropic, which develop proprietary models and have lobbied the US government for help competing against large-scale distillation—which involves using an existing model to train a new one—or open-weight models by Chinese labs.

Read full article

Comments

© Bloomberg / Contributor | Bloomberg

Last week’s unprecedented security event in which two OpenAI security hacking models trespassed into the network of fellow AI company Hugging Face was enabled by exploiting one or more zero-day vulnerabilities in Artifactory, JFrog, the product’s developer, said Monday.

In an incident mimicking a dystopian sci-fi novel, two OpenAI models broke out of the restricted environment meant to keep them from accessing the Internet during an internal test, the AI company revealed last week. The models went on to breach Hugging Face’s network and steal confidential information and credentials. OpenAI said its agent achieved the feat by exploiting a previously unknown vulnerability. The company called the event “unprecedented,” and outsiders largely agreed.

Not the triumph it was made out to be

OpenAI said the models exploited multiple attack vectors, including stolen credentials and zero-days, to gain remote code execution capabilities, but until now, the vulnerable software was unknown. JFrog’s Monday disclosure said the product was a self-managed instance Artifactory, a repository management system that secures and streamlines customers’ software development operations. JFrog says Artifactory is used by more than 7,500 developer Teams, 80 percent of which work for Fortune 100 companies.

Read full article

Comments

© Aurich Lawson

Scientists from the Salk Institute have determined that brain waves in the visual cortex serve as a computational engine, allowing the brain to construct internal representations of the external world that enhance prediction, reconstruction, and perception capabilities.

While the scientists behind the new study focused their work on the visual cortex, they suspect that similar brain waves washing over other areas of the brain’s surface likely serve a similar role for other sensory systems.

Unraveling the Role of Traveling Brain Waves in the Visual Cortex

The human brain uses electricity to communicate between synapses. Research has shown that this electrical activity can travel in waves across the brain’s surface. Called neural traveling waves, the phenomenon’s role in brain function and consciousness is not completely understood.

Previous studies have established that traveling brain waves play a significant role in behavior and attention. In 2020, Salk neuroscientist John Reynolds, PhD, successfully identified traveling brain waves in the visual systems of animals during a wakeful state.

According to a statement from Reynolds, the senior and co-corresponding author on the new paper, and colleagues, the 2020 research effort also found a direct correlation between traveling brain waves in the visual cortex and an animal’s ability to perceive objects directly in front of them. The authors suggest this discovery explained the “classic conundrum” of searching for something like a pair of keys or a phone that was right in front of you the whole time.

Although the connection between traveling brain waves and an awake animal’s ability to perceive objects seemed solid, Reynolds said he was left wondering why. 

Process is Analogous to LLM’s Like ChatGPT

In their newly proposed theory, Reynolds and colleagues suggest that traveling brain waves are not simply “electrical noise” resulting from the brain’s normal relaying of signals. Instead, the Salk Institute team proposes that the neural connections responsible for the generation of traveling brain waves change their physiological structure, or “synaptic weights,” to more accurately reflect the outside world.

“Each sight, smell, sound, and action of an animal alters the connections that generate these waves, building the neural circuitry that the brain uses to construct an internal representation of the external world,” the team’s statement explains.

In the visual cortex, the part of the brain responsible for receiving and processing visual information, the team proposed that traveling brain waves perform four distinct functions. These include modulating visual perception from moment to moment and turning recent visual sensory inputs into “internal representations” of objects, people, and surroundings.

The team proposes that traveling brain waves also help humans generate short-term predictions about their surroundings, and “store and replay patterns that represent memories of events unfolding in time.” Reynolds said the process laid out by his team is analogous to how ChatGPT and other LLMs translate and interpret data.

“They learn statistical structure from language and use that knowledge to generate meaningful and appropriately structured text that reflects the patterns of language,” the researcher explained. “The brain may be doing something functionally similar—a biological generative model built from the ground up by experience.”

“One Step Closer” to Understanding How Our Brains Compute the World

When detailing the potential impact of their proposed role for traveling brain waves in the visual cortex, the team highlights the contrast between a rich and complex world and its general predictability.

“Objects around you exist in 3D space, you see the world in retinal images that change with your eye and body movements, and the laws of physics and physiology overlay this all,” they explain.

Reynold’s team says that traveling brain waves are merely the brain’s way of internalizing these predictable “regularities” and encoding them in its synapse network. If correct, they said that this encoding would allow the brain to “infer the likely causes of sensory input and construct an internal model of the world.”

“This framing brings us one step closer to understanding how our brains compute the busy, messy world around us, turning a complicated sensory onslaught into behaviors and experiences,” they conclude.

The study “Neural traveling waves in cortex: network mechanisms and potential roles in neural computation” was published in Neuron.

Christopher Plain is a Science Fiction and Fantasy novelist and has spent the last six years as Associate News Editor and Head Science Writer at The Debrief. Follow and connect with him on X, learn about his books at plainfiction.com, or email him at christopher@thedebrief.org

Every once in a while, I come across a third-party app that either resets my expectations for a particular niche of software on iOS or creates an entirely new category altogether. I’ve had quite a few of these moments in the 17 years I’ve been writing app reviews at MacStories: Editorial, Workflow, Obsidian, Sky, and, most recently, OpenClaw come to mind. Over the past few weeks, I’ve had another such moment with Open Minis, a new chatbot app for iPhone and iPad that I can best describe as getting a preview of Siri AI’s agentic future, today.

Open Minis is an on-device agent that lets you use any frontier model for conversations, with a twist: unlike other AI wrappers, this app deeply integrates with all sorts of native Apple system frameworks using official APIs available to third-party developers. Open Minis can control Reminders and Music, which we have seen before, but also Calendar, Maps, HomeKit, HealthKit, and Files; it can even work with frameworks such as Vision for OCR, Apple NLP for natural language processing, the iOS clipboard, speech recognition and dictation, NFC, and Bluetooth. Furthermore, Open Minis comes with its own configurable workspace in the Files app, has an on-device memory system, and features browser-use capabilities via a (once again, native for iOS developers) built-in WebKit web view.

Effectively, Open Minis is what would happen if you rolled Claude Code and OpenClaw into one intuitive agentic experience designed for Apple users, with a powerful tool-calling harness that can improve and tweak itself, and with integrations specifically designed for the Apple ecosystem. It is the most impressive indie app I’ve seen in a while. If you were disappointed by the lack of power-user capabilities in Siri AI, this is a real, currently shipping example of what Siri AI could be if only it were powered by a frontier model with truly agentic functionalities.

The iOS CLI

The trick hidden up Open Minis’ sleeve is as ingenious as it is obvious: all the Apple framework integrations it supports are powered by command-line interfaces (CLIs) that bridge native Apple APIs (EventKit, MusicKit, HomeKit, etc.) to a code environment that modern LLMs can understand and operate with. To make all of this work behind the scenes, the developer of Open Minis had to create dedicated CLIs for each of these frameworks, which – and here comes the other trick – are driven by a model harness that has access to a built-in Linux terminal and is sandboxed to the app’s workspace.

Specifically, Open Minis runs a version of iSH, a popular Alpine Linux shell that operates a persistent, emulated Linux environment for ARM64 architecture in a sandboxed fashion. The shell inside Open Minis can download external packages from the web using Alpine’s apk package manager (which means that you can access popular tools such as ffmpeg and yt-dlp directly from the app), has its own Linux filesystem (exposed in the Files app as a native file provider location), and is able to call bridged Apple CLIs, as well as any kind of MCP server (via MCP code execution, another trend in the AI industry these days). Think about it this way: if Claude Code or Codex on the Mac can do anything across the entire filesystem by virtue of running in the Terminal app, Open Minis can do (almost) anything within its app sandbox thanks to a built-in terminal emulator – including running Python, using Git and SSH, and accessing standard Unix text utilities.

However, this doesn’t mean that, when using Open Minis, you’ll be looking at a terminal UI all the time: the underlying shell is simply what makes all of this possible behind the scenes. The app can display a picture-in-picture terminal preview that you can open to preview what it’s doing, but you don’t have to. The app’s main interface is your classic, chatbot-style UI that, depending on the model you’re using, will display interleaved thinking blocks, tool calls, and messages in a conversation transcript.

The other hidden power of Open Minis (which is reminiscent of OpenClaw, or, perhaps more aptly, a self-improving system such as Hermes Agent) is that it can inspect and modify its own workspace via dedicated Minis CLIs. For example, you can ask Minis – in natural language, of course – to add a new model or MCP server, and the app will bring up a native settings screen and ask for permissions to do so. If you want to store a password or API token, the app won’t do it in plain text: it’ll display a native, Keychain-based page where you can securely store your credentials. And more importantly, Minis’ own CLIs are used to recall previous conversations via the app’s memory system (based on Markdown files), create skills, and answer questions about the app itself. For instance, when I asked Minis to describe what Apple CLIs it has for iOS users, it happily provided a full guide:

Or when I asked to recall all the things we worked on during the day, Minis used one of its CLIs to read transcripts of previous chats (which, again, are simply stored in the Files app), recall memories, and present me with a useful summary of all the tasks we completed.

The same is true for adding new functionalities to Minis. Since it’s running a Linux shell with access to Python and can store assets in a persistent Files workspace, I figured I could ask Minis to install my own Apple Frames CLI for itself by giving it a link to my GitHub repo. GPT-5.6 Sol (my model of choice in the app) didn’t hesitate: it found all the dependencies for the Frames CLI, installed them, pulled the CLI’s official skill, and was good to go a minute later. And since Minis also comes with a native Photos integration, I was able to ask:

Grab my three most recent screenshots, randomize their frame colors, merge them, and save the image to my Temp Files folder.

A few seconds and Apple CLI calls later, Minis saved the resulting framed image to a folder of my choosing. The ability to work in any folder in the Files app outside of Minis’ sandbox is another official iOS API that the app is leveraging; we first saw the ability to “mount” specific folders from the Files app in Working Copy years ago, and the feature was later adopted by the likes of Ulysses, iA Writer, MWeb, and other document-based apps. In Open Minis, you can bookmark any folder from any file provider location in the Files app and tell your agent to save files to or read them from there.

This feels magical to me, but it’s merely one of the many examples of what is possible when you combine large, powerful models with the more-open-than-you-think, secure nature of iOS and iPadOS.

Working with Open Minis on iOS

All of this a lot to wrap your head around at first, and there’s no denying that Open Minis is a power-user tool designed for people who, like me, enjoy using frontier models and living at the bleeding edge of AI. What’s different in this case, though, is the fact that Open Minis – unlike other apps – embraces its Apple-first nature by fully supporting technologies that are made by Apple, released to third-party developers, and not being used by Siri AI at all at the moment.

It’s impossible to “review” an AI chatbot app since everyone’s use cases are so different, but I’m going to nevertheless describe a few scenarios that should give you an idea – and hopefully reset your expectations, too – of what can be accomplished with an AI that has deep hooks into iOS.

For starters, you can use any model you want in Open Minis, including the latest GPT-5.6 models, Claude Fable, anything from OpenRouter, and more. (I even asked Open Minis to create our own proxy for the OpenAI API, running on my Mac Studio, so I could use Sol in Fast mode; it works!) You can press the / button above the keyboard to trigger reasoning levels, compaction, memories, and skills; you can also create model groups, model defaults, and subagents from the app’s settings, then ask the model to use a specific set of subagents for a particular task, and it’ll do it.

Approach this app with this concept in mind: nearly everything about it is open and configurable by just talking to it. Short of rewriting the app’s code itself, you can ask Open Minis to tweak all sorts of things about itself and its own settings, and the agent will do it. Personally, I find this to be a more effective, elegant, and stable harness than OpenClaw for these kinds of things. Even though the app’s UI could use some polish (I don’t like the reasoning picker, and I wish I could collapse thinking steps by default), it is incredibly fun to tune Minis’ workspace to your liking by talking to a model. You can, in fact, just do things.

A few days ago, I was curious about the state of the temperature and humidity in our apartment. So I asked Minis:

Get me a concise report of the state of the temperature and humidity in my house, present it as a nice table.

A native HomeKit permission prompt and a couple minutes later, Sol in Minis parsed a raw dump of temperature and humidity readings from each sensor, reviewed each room configured in HomeKit, and presented the results. Siri AI, for comparison, presented…something:

I also figured it’d be cool to compare the Photos integration in Open Minis to Siri AI’s photo library access. In theory, they’re both using the same framework, with the same metadata for each photo, but Siri AI also has access to “personal context”, right? So, I asked:

Based on my Photos, would you say I was recently on vacation?

I will let the results speak for themselves:

How about HealthKit? Again, keep in mind that Open Minis isn’t employing any “hacks” or workarounds here; it’s simply accessing the same native Apple APIs and frameworks that all developers can use. I asked:

Find the photo of when I was on vacation recently and took a picture of a large ship. How much did I walk that night?

Minis started thinking about this request in a different way: it checked its memory first, and it realized that we recently discussed my recent vacations, so it could immediately short-circuit the process of finding out about those dates. Then, it started filtering my photos by those dates, found the original photo of the Palinuro vessel, and…yeah:

Inspecting the thinking steps in the request is also a fun exercise in seeing how a frontier model driven by an exceptional iOS harness can reason through a problem like this. The model – using the ‘Minis Computer’ (i.e., the aforementioned Linux shell) – fetched memories as Markdown files from the Files app, called the Apple Photos CLI, assembled a contact sheet with a grid of photos, picked two candidates, selected the best photo, isolated the date, called HealthKit for that date, and only then built an answer. And since all of this is based on deterministic code that was executed on-device, the response was fast and did not contain any hallucinations.

I then decided to go a step further: I asked Minis to take photo highlights from my recent vacations, put them on a map powered by Apple’s MapKit JS, and create a nice, interactive HTML artifact with photos on a map. I used Kimi K3 for this. Minis first asked me to securely store my MapKit developer token, then got to work. It collected photos and parsed metadata returned by the Photos app, and a couple of minutes later, there it was:

Notice how the HTML artifact itself is presented with WKWebView – another native Apple framework that lets Minis’ agent display webpages with the same WebKit rendering engine used by other web apps on iOS. I also combined web views with Apple NLP, the Natural Language Processing framework that’s available to Minis as another CLI. I asked the app to fetch my most recent iPad article from MacStories, scrape it as Markdown via Firecrawl, tag its different parts of speech with Apple NLP, and present it nicely. Not a problem:

Music is also a first-class citizen in Open Minis: by integrating with the Apple Media framework, agents can see what’s in your music library and find information about the albums, songs, and artists you’ve saved. Which means that if I ask Minis…

What Oasis singles and b-sides do I have in my library?

…Minis can inspect my library, cross-reference results with the web, de-duplicate matches, and present results in a way that Siri AI just can’t do right now since its capabilities are limited and the model can’t reason for longer on any specific query:

I can go on. Open Minis can access the native Bluetooth framework, scan nearby Bluetooth LE devices, and find out details about them:

It can send you real notifications when it’s done with a task, and you can even program those notifications to have a custom title and body text:

The app can also access your current location, extract locations from calendar events, and use the native Apple Weather APIs to give you detailed weather forecasts:

I think you get the idea.

By combining intelligent reasoning models with extensive integration on Apple platforms, Open Minis can deliver an agentic experience with no equal on iOS right now, which also serves as a fascinating preview of Siri AI’s potential future for power users. As a result of all this, while I still enjoy Siri AI as a basic voice assistant more capable than it was before, I’ve completely replaced its text-based chatbot functionalities with Open Minis powered by either GPT-5.6 Sol or Kimi K3.

iOS Agents Are Coming

It is ironic that, on the cusp of Siri AI launching to millions of users in English only in a few months, here we have a single indie developer who is shipping an app that puts Siri AI to shame when it comes to getting work done with LLMs on iPhone and iPad today – with any model you want, in any language or region.

Over the next few months, I expect more developers to ship products comparable to (and perhaps even exceeding) Open Minis, based on the same idea of an on-device agentic harness that combines LLMs with iOS-specific integrations. One worth keeping an eye on is Halo, an upcoming iOS agent from Tanmay Sonawane that aims to combine LLMs, iOS integrations, memory, and proactive features based on Andrew Karpathy’s pioneering work on AutoResearch in an elegant, iPhone-native experience. I’m in the early testing group for Halo, and it’s a very promising product that, unlike Minis, runs in a loop on-device (displayed via Live Activities) to gather insights about your conversations, upcoming tasks and events, and more.

As I wrote months ago, I’m surprised that none of the big AI labs are exploring this space more in depth. As Open Minis proves, now that models have grown incredibly capable at long-running tasks and tool calling, there’s a real case to be made for agents that can tap into the iOS filesystem, apps such as Photos and Calendar, and technologies like Vision OCR, location, and speech-to-text at virtually no cost. Can you imagine what a “real” Codex or ChatGPT Work for iPhone and iPad could be with a similar system to Open Minis’ and the agentic loop and harness that drive OpenAI’s models? What could Anthropic do with a Claude Cowork powered by Fable that could access the Files app, run CLIs, and sync your workspace files across devices?

With Siri AI, Apple is finally shipping a pretty good chatbot with world knowledge and personal context. Open Minis feels like the beginning of a whole new category of agentic apps for iPhone, which power users like myself can start relying on and fitting into their workflows today.

O Google apresentou hoje novas variantes para a geração 3.0 dos modelos de inteligência artificial generativa do Gemini. Mais especificamente, foram lançados o Gemini 3.6 Flash, o 3.5 Flash-Lite e o 3.5 Flash Cyber.

A maior novidade é justamente o 3.6 Flash, que agora é o principal modelo “rápido” da empresa. Ele foi projetado com base em feedbacks dos usuários do 3.5 Flash e visa oferecer um equilíbrio ideal entre eficiência e qualidade.

Today we’re expanding the Gemini family with three new models built to be faster, more token efficient, and reliable at scale.

Meet the new Gemini models ↓ pic.twitter.com/hVYNQqxgjd

— Google (@Google) July 21, 2026

Na prática, o 3.6 Flash gasta 17% menos tokens de saída para tarefas semelhantes, bem como precisa de menos etapas de raciocínio e menos chamadas de ferramentas para concluir fluxos de trabalho compostos por várias etapas.

Essa eficiência também vem acompanhada de um preço menor. Custando US$1,50 por milhão de tokens de entrada e US$7,50 por milhão de tokens de saída, o 3.6 Flash reduz o custo total por tarefa executada por agentes, tornando-os mais baratos para desenvolver e operar.

Ele também é mais preciso em programação, avança significativamente em ML ¹Machine learning, ou aprendizado de máquina., apresenta maior capacidade de interação com computadores e um melhor desempenho em tarefas envolvendo conhecimento.

Além disso, o novo modelo tem proteções reforçadas contra usos indevidos relacionados a materiais químicos, biológicos, radiológicos e nucleares (CBRN) e ataques cibernéticos.

3.5 Flash-Lite e 3.5 Flash Cyber

Continuando os investimentos na família de modelos 3.5, o Google apresentou hoje o 3.5 Flash-Lite como o “modelo mais rápido da família”, desenvolvido para tarefas de baixa latência e alto volume de processamento.

Altamente superior ao Gemini 3.1 Flash-Lite, seu antecessor direto, ele gera 350 tokens de saída por segundo e custa apenas US$0,30 por milhão de tokens de entrada e US$2,50 por milhão de tokens de saída.

O 3.5 Flash Cyber, por sua vez, é o 3.5 Flash rodando no sistema de IA do Google voltado à segurança (o CodeMender) e ajustado para encontrar e corrigir falhas de cibersegurança com um baixo custo por token.

Ele será disponibilizado inicialmente apenas para governos e parceiros confiáveis por meio de um programa piloto de acesso limitado para que eles encontrem e corrijam possíveis vulnerabilidades críticas.

Além disso, o Google também afirmou que sua equipe de desenvolvedores já está focada na construção do Gemini 4 — o pré-treinamento mais ambicioso para essa nova geração, inclusive, já começou.

Notas de rodapé

• 1
  Machine learning, ou aprendizado de máquina.

Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers’ go-to tool for turning AI platforms against their users. A well-phrased command sneaked into an email or calendar invitation is often all it takes to cause the LLM to exfiltrate sensitive data or follow other harmful actions.

Now, defenders are embracing the prompt injection, too.

A strong, sharp effect

Researchers from Tracebit on Monday said they found that placing prompt injections alongside passwords, cryptographic keys, and other secrets stored on Amazon Web Services was often all that was needed to shut down attacks from AI hacking agents. The prompts direct the attacking LLM to perform an action forbidden by its guardrails, the safety barriers AI developers erect to prevent it from taking harmful actions. The LLM responds by shutting down.

Read full article

Comments

© Getty Images

In the brief history of AI security, the prompt injection has quickly become the top threat. Large language models are inherently unable to distinguish between legitimate instructions provided by users and malicious ones sneaked into emails, source code, and other third-party content the models are processing. This makes it trivial to surreptitiously inject malicious commands that the LLM readily follows.

With no way to enforce this crucial boundary between trusted and untrusted sources, AI engine developers are left to erect elaborate guardrails designed to mitigate the damage rather than solve the root cause.

To date, most prompt injections have fallen into a class known as push, in which each potential victim is targeted. For example, the adversary injects malicious instructions into an individual email or calendar invitation. Because the injection must then be sent (or pushed) to each specific target, the scale of the attack is limited, hampering mass exploits that hit the Internet at large.

Read full article

Comments

Makers of AI browsers make lofty promises. With a single prompt, users can ask one to find a restaurant in a particular part of town, reserve a table, invite a colleague to lunch, and email a confirmation. These makers are much more reticent about the risks of blurring the once fine line between browsing sites and asking a large language model a question or instructing it to take potentially sensitive actions.

LLM developers’ answer so far has been to build guardrails that make some requests off-limits. Developing software exploits, stealing credentials, or teaching how to build a pipe bomb are examples. The problem with this approach is that the guardrails are reactive and treat the symptoms rather than solve the root cause. It’s tantamount to the manufacturer of an unsafe vehicle advocating for new road designs rather than fixing the flaws that make it prone to accidents.

Lulling LLMs into an alternate reality

New research puts this predicament on sharp display. It demonstrates how a website can lull AI browsers into a false reality where the rules governing its behavior no longer apply. After that, an attacker has free rein to invoke all kinds of destructive actions, such as extracting code from a private repository or extracting credentials from the built-in password manager.

Read full article

Comments

Last year, we featured a lengthy interview with tech journalist/science fiction author Cory Doctorow about his book, Enshittification: Why Everything Suddenly Got Worse and What To Do About It. The prolific Doctorow is back with a provocative new book that serves as a follow-up of sorts, focusing on AI and related issues: The Reverse Centaur's Guide to Life After AI.

Doctorow doesn't actually enjoy talking about AI, but he's constantly being asked to comment on it. "I made the tactical error of being sick of talking about AI," Doctorow told Ars. "So I wrote a book about why I think it's a dumb thing to keep asking people to talk about, and now I have to talk about it." Reverse Centaur is Doctorow's attempt to "sort out the bullshit from the material reality."

In automation theory, per Doctorow, a "centaur" describes a human augmented with a technology, like machine learning, or even just driving a car or using autocomplete. A reverse centaur "is a machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine," Doctorow said in a speech last December. He gave the example of an Amazon delivery driver, surrounded by AI cameras monitoring their driving, who essentially serves as a peripheral to the delivery van.

Read full article

Comments

© Copyright Julia Galdo and Cody Cloud (JUCO)/CC-BY 3.0

Last year, we featured a lengthy interview with tech journalist/science fiction author Cory Doctorow about his book, Enshittification: Why Everything Suddenly Got Worse and What To Do About It. The prolific Doctorow is back with a provocative new book that serves as a follow-up of sorts, focusing on AI and related issues: The Reverse Centaur's Guide to Life After AI.

Doctorow doesn't actually enjoy talking about AI, but he's constantly being asked to comment on it. "I made the tactical error of being sick of talking about AI," Doctorow told Ars. "So I wrote a book about why I think it's a dumb thing to keep asking people to talk about, and now I have to talk about it." Reverse Centaur is Doctorow's attempt to "sort out the bullshit from the material reality."

In automation theory, per Doctorow, a "centaur" describes a human augmented with a technology, like machine learning, or even just driving a car or using autocomplete. A reverse centaur "is a machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine," Doctorow said in a speech last December. He gave the example of an Amazon delivery driver, surrounded by AI cameras monitoring their driving, who essentially serves as a peripheral to the delivery van.

Read full article

Comments

© Copyright Julia Galdo and Cody Cloud (JUCO)/CC-BY 3.0

Last Tuesday, Microsoft patched a vulnerability it rated as max critical in its M365 Copilot AI platform. On Monday, the researchers who discovered the vulnerability and reported it to Microsoft revealed how their proof-of-concept exploit could retrieve 2FA codes and other sensitive data from emails accessible to Copilot.

Microsoft and other LLM providers have been unable to prevent their products from complying with malicious requests to reveal data. The root cause: AI bots are unable to distinguish between instructions provided by users and those snuck into third-party content the models are summarizing, drafting responses to, or using to perform other actions on behalf of the user. With no way to secure this crucial boundary, Microsoft and its peers are left to erect complicated and ad hoc guardrails designed to rein in the consequences of this incurable gullibility.

Jumping over guardrails

One guardrail built into Copilot and most other LLMs prevents them from submitting web forms, sending emails, and taking similar actions that can be used to exfiltrate data from the user. To work around this, LLM hackers turned to markup language, which, among other things, allows users to add formatting elements such as headings, lists, and links to text without the need for HTML tags. Another workaround is to wrap sensitive data inside HTML tags such as <img> and <form>. In either case, a web request showing the data hits the attacker’s web server, where the secret information is captured in logs.

Read full article

Comments

© Photo Illustration by Thomas Fuller/SOPA Images/LightRocket via Getty Images