Last year, we featured a lengthy interview with tech journalist/science fiction author Cory Doctorow about his book, Enshittification: Why Everything Suddenly Got Worse and What To Do About It. The prolific Doctorow is back with a provocative new book that serves as a follow-up of sorts, focusing on AI and related issues: The Reverse Centaur's Guide to Life After AI.

Doctorow doesn't actually enjoy talking about AI, but he's constantly being asked to comment on it. "I made the tactical error of being sick of talking about AI," Doctorow told Ars. "So I wrote a book about why I think it's a dumb thing to keep asking people to talk about, and now I have to talk about it." Reverse Centaur is Doctorow's attempt to "sort out the bullshit from the material reality."

In automation theory, per Doctorow, a "centaur" describes a human augmented with a technology, like machine learning, or even just driving a car or using autocomplete. A reverse centaur "is a machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine," Doctorow said in a speech last December. He gave the example of an Amazon delivery driver, surrounded by AI cameras monitoring their driving, who essentially serves as a peripheral to the delivery van.

Read full article

Comments

Last year, we featured a lengthy interview with tech journalist/science fiction author Cory Doctorow about his book, Enshittification: Why Everything Suddenly Got Worse and What To Do About It. The prolific Doctorow is back with a provocative new book that serves as a follow-up of sorts, focusing on AI and related issues: The Reverse Centaur's Guide to Life After AI.

Doctorow doesn't actually enjoy talking about AI, but he's constantly being asked to comment on it. "I made the tactical error of being sick of talking about AI," Doctorow told Ars. "So I wrote a book about why I think it's a dumb thing to keep asking people to talk about, and now I have to talk about it." Reverse Centaur is Doctorow's attempt to "sort out the bullshit from the material reality."

In automation theory, per Doctorow, a "centaur" describes a human augmented with a technology, like machine learning, or even just driving a car or using autocomplete. A reverse centaur "is a machine head on a human body, a person who is serving as a squishy meat appendage for an uncaring machine," Doctorow said in a speech last December. He gave the example of an Amazon delivery driver, surrounded by AI cameras monitoring their driving, who essentially serves as a peripheral to the delivery van.

Read full article

Comments

Last Tuesday, Microsoft patched a vulnerability it rated as max critical in its M365 Copilot AI platform. On Monday, the researchers who discovered the vulnerability and reported it to Microsoft revealed how their proof-of-concept exploit could retrieve 2FA codes and other sensitive data from emails accessible to Copilot.

Microsoft and other LLM providers have been unable to prevent their products from complying with malicious requests to reveal data. The root cause: AI bots are unable to distinguish between instructions provided by users and those snuck into third-party content the models are summarizing, drafting responses to, or using to perform other actions on behalf of the user. With no way to secure this crucial boundary, Microsoft and its peers are left to erect complicated and ad hoc guardrails designed to rein in the consequences of this incurable gullibility.

Jumping over guardrails

One guardrail built into Copilot and most other LLMs prevents them from submitting web forms, sending emails, and taking similar actions that can be used to exfiltrate data from the user. To work around this, LLM hackers turned to markup language, which, among other things, allows users to add formatting elements such as headings, lists, and links to text without the need for HTML tags. Another workaround is to wrap sensitive data inside HTML tags such as <img> and <form>. In either case, a web request showing the data hits the attacker’s web server, where the secret information is captured in logs.

Read full article

Comments