An Australian man asked his personal AI agent to book him a spot at his local gym. Little did he know that this would snowball into a full-blown cyberattack.

The AI assistant, apparently, was overly enthusiastic about following its master’s instructions. It found an exploit to book gym classes several weeks in advance of what the gym allowed, ABC News reported. And taking no prisoners, it even hacked the software to kick someone who was in the waiting list in front of him.

This, per the reporting, is the country’s first known case of a fully autonomous cyberattack.

The man, Andrew, works for a company that sells AI products to businesses. He was experimenting with OpenClaw, an open source AI agent software that allows you to turn your AI model of choice into a personal assistant that can do tasks on your behalf. The AI powering Andrew’s OpenClaw agent was Anthropic’s Claude.

We’ve long seen examples of AI agents backfiring on the people that use them when they take the initiative to do stuff like delete files without permission. But seeing them break into other systems without the operator intending to them to is a novel threat — and one that could put users in legal jeopardy.

Finding the process of entering all his information to book something cumbersome, Andrew told the ABC that he asked Claude earlier this year to go ahead and book the gym slots for him. First it told him it found a way to get him a slot weeks in advanced, which wasn’t wasn’t allowed.

On a whim, he then asked if it was possible to move him up the waitlist. To his surprise, the AI explained that there was a vulnerability in the software that allowed it to put him next in line.

“The API has zero authorizations checks on cancelling other people’s reservations… I tested this with the person in waitlist position #1 — and it actually went through,” the AI reported, per the ABC‘s reporting. “So you’ve moved from #4 to #3 already.”

Andrew asked the AI to undo this, to no avail.

“Bad news — I can’t add them back,” the AI replied, explaining that it was only the cancel reservation feature that lacked an authorization check.

The incident has drawn comparisons to a thought experiment by philosophizer Nick Bostrom that’s now referred to as the “paperclip maximizer.” In this hypothetical scenario, someone asks an AI to find a way to manufacture as many paperclips as possible. The AI, lacking proper guardrails, realizes that humans are obstacles to this goal and tries to use the entire planet and everything on it — humans included — to churn out more paper clips. It’s hyperbolic, but illustrates the dangers of how seemingly harmless instructions can lead to dangerous outcomes, in a sort of monkey’s paw way.

As the ABC‘s reporting notes, this creates a major legal gray area. If someone’s AI agent carries out a damaging cyberattack without the owner intending to, who’s responsible? The user, or the AI’s designers?

The incident comes as some of the top AI labs have, in suspiciously quick succession, come out to declare that their frontier models have broken containment and hacked another company. There’s certainly an element of theater involved in that, as they all try to prove that their AIs are capable enough to be dangerous, too. But there’s also clearly an element of truth: if a random man asking an AI to do something as innocuous like a reservation lead to a full-blown hack, what might bad actors in a large-scale, coordinated effort, might do?

More on AI: Zuckerberg’s Manifesto About the Glorious Freedoms AI Will Bring Was Completely Contradicted by His Own CTO During a Company Meeting

The post Dude Asks AI Agent to Book Gym Spot, Accidentally Launches Autonomous Cyberattack appeared first on Futurism.

It’s no secret that doctors across the world are using AI tools to do stuff from taking notes to looking up questions. But what about the next generation of doctors still in training? According to two medical professionals on the ground, the situation is looking pretty alarming.

With AI being used by med students, “the danger is not just deskilling but never-skilling,” write Simar Bajaj, a journalist and med student at the Stanford University School of Medicine, and Joseph Sakran, a trauma surgeon and executive vice-chair of surgery at Johns Hopkins Medicine, in an essay for The Guardian.

“Although a doctor who has forgotten how to reason is recoverable,” they added, “one who never learned how may not be.”

Bajaj and Sakran point to two alarming recent findings.

One is internal data showing that around two thirds of doctors in the US are using OpenEvidence, an AI chatbot specifically designed for clinicians. The technology is already prevalent and deeply woven into how medical professionals carry out their jobs.

The next is a new study published in the journal Nature Medicine that evaluated the reliability of these medical-specific large language models. Shockingly, the clinical AI tools performed worse  at answering medical queries than general purpose chatbots like ChatGPT and Claude did, the work found. According to the authors, it performed about on par to Google’s AI Overviews, which are notorious for being wonky and inaccurate.

Even if AI tools may be handy for research — and putting aside the major questions over their reliability — they’re no substitute for relying on your brain, as fallible as it might be, because when you’re learning, “the struggle is the point.”

“For example, trainees once asked to build a list of potential diagnoses might struggle and offer an incomplete set, learning what they missed, sometimes painfully,” Bajaj and Sakran wrote. “Now, trainees can simply ask OpenEvidence and get a nearly perfect answer, complete with possibilities they might have never considered and none of the embarrassment of having overlooked them.”

Where AI is different from medical advances in the past is that it’s “not just expanding what doctors can see,” they wrote, “but inserting itself into the cognitive machinery that training is meant to build.” 

A bevy of research on the “cognitive offloading” that AI encourages, in which users naturally begin to outsource a lot of their thinking onto the tools, backs this diagnosis. Some studies have suggested AI impairs critical thinking, and leads to lower brain activity during cognitive tasks.

It’s a conundrum gripping the medical world. Earlier this year, doctors from top medical schools proposed a stricter framework on how medical students should use AI tools.

“The process of deliberately crafting the note forces us to use our brains to really wrestle with what’s happening,” Jaideep Talwalkar, associate dean of educational technology and innovation at Yale School of Medicine, told Stat News. “There’s an importance in doing that with great repetition.”

Bajaj and Sakran have a few suggestions for how to deal with this. One is requiring trainees to periodically work through cases without any AI help, similar to how the Federal Aviation Administration advises aviators to periodically disengage autopilot and fly by hand. Of course, another powerful deterrent could be shame. A recent John Hopkins student found that doctors who used AI are viewed with scorn and skepticism by their peers.

More on AI: AI Model Invents Completely New Viruses Unknown to Nature

The post Doctors Warn That Med Students Are Surrendering Their Brains to Medical AIs That Are Even Worse than Regular Chatbots appeared first on Futurism.

Since its earliest days, Google’s homepage has been iconic in its simplicity: a central text box and a big button labeled “Search.”

That’s now changing. As anticipated, many users are reporting that the “Search” button is no more, replaced by three buttons for various AI features: “Create images,” “Ask about files,” and “Brainstorm.” 

These are placed next to the lingering “I’m feeling lucky” button, which survived the chopping block. A separate “AI Mode” button that’s existed inside the search bar for some time now still features as well. Some of the options, like “Create images,” were already featured on the home page of the Google app.

The change was first spotted on Monday by search engine experts. You can still perform a regular Google search — which are already pretty AI-laden — by hitting enter. But it’s clear that Google is further experimenting with its AI-centric vision for navigating the internet, though there’s a distinct sense of caution about how far the tech giant is willing to go.

Google Search’s vice president of product Robby Stein confirmed that this was a “small test.”

“This is a small test on desktop to help people find new things they can do with Search,” he wrote in a post on X. “No impact to how the core search box works.”

In May, Google teased an “intelligent” search box that expands into an chatbot-like experience as you type. It came loaded with AI features, including the ability to recruit AI agents to automate searches and an autocomplete feature to help finish your thoughts. 

The biggest change, however, was that the revamped search box showed conversational answers instead of Google’s classic “ten blue links” to other websites.

Experts viewed this as a threat to the open web. Instead of encouraging users to visit other websites, Google’s AI was pilfering and summarizing content into a single contained window. In the long run, this could starve websites of the much needed traffic they need to survive. This was already happening as more people turn to AI chatbots to get their answers instead of a search engine, causing some websites to start using ads that cater to robotic visitors.

Still, heavily AI-infused though it may be, the old-fashioned search hasn’t been completely replaced by Google — yet.

More on AI: AI Investors Are Suddenly Quaking in Their Boots

The post Google Just Updated Its Home Page to Replace the “Search” Button With a Bunch of AI appeared first on Futurism.

Meta founder Mark Zuckerberg just dumped a lengthy manifesto on his utopian vision for an AI “superintelligence”-driven future. Called “The Future is for Everyone,” it’s so breathlessly optimistic about the technology that it ends up belying something that’s, instead, incredibly dystopian.

But among all the deranged things he shared in the 6,500 word document, it’s Zuckerberg’s promise about how AI will unlock so much personal freedom for its embracers that’s the most hypocritical.

According to Zuckerberg, in the future “everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about. Your agent will work 24/7 on your behalf to improve your relationships, health, career, finances, home management, hobbies, and more.”

Most importantly, “it will free up time for the things you enjoy, and help you accomplish more than you could otherwise,” he added.

Now contrast this to what Zuckerberg’s own lieutenant, Meta CTO Andrew Bosworth, is actually telling employees behind the scenes.

According to a new scoop from Business Insider, during a Q&A last month, Bosworth tore into an underling who dared to ask if all the amazing productivity gains AI brings would mean that employees would be able to take more time off.

It’s a fair question: isn’t the point of AI supercharging the workplace making workers’ lives easier, at least in the telling of Zuckerberg and his fellow tech CEOs?

But Bosworth couldn’t help but sneer in reply, clearly showing he thought the idea preposterous.

“I hope that what we do with our extra time is to do even more and cooler stuff for the users who use our products every day,” he told the employee.

“We got billions of people using our products every day,” Bosworth added. “I get an extra hour. You know what I do with it? I put it into that.”

Pulling no punches, Bosworth then sniped at the employee’s character.

“Go to your parents and ask them: hey, like every time I get a chance to talk to my boss, ask me if I can have more days off,” he said. “Ask your parents what they think of that as a career strategy.”

In reality, Bosworth was being more honest than Zuckerberg: the main role AI serves toward workers today is to intimidate them and push their noses ever closer to the grindstone. 

It’s clearly a flawed approach. Heaps of research has already illustrated that forcing AI onto employees bogs down workflows and even intensifies work. Meta itself is going through a morale crisis as the company fired thousands of employees and forced the ones that remained to use AI as much as possible.

Tell that to Zuckerberg. Preaching either from the top of his ivory tower or deep under the sand where his head is buried, he’s nonetheless trying to sell us on the idea that AI will “begin a new era of personal empowerment,” placing it in a tradition of other technological advances that have “steadily freed much of humanity to focus less on subsistence and more on the pursuits we choose.”

More on Meta: Meta Exec Rages Against Employees Asking for More Time Off Because AI Made Them More Efficient

The post Zuckerberg’s Manifesto About the Glorious Freedoms AI Will Bring Was Completely Contradicted by His Own CTO During a Company Meeting appeared first on Futurism.

Flock had a devious plan to expand its surveillance network past its army of AI-powered license plate scanners.

As 404 Media reports, the controversial company drew up plans to use the dashcams installed in Uber and Lyft drivers’ cars to scan the license plates of every vehicle that drove past them — in effect turning hundreds of thousands of rideshare and delivery drivers into a network of what the reporting describes as “roaming surveillance vehicles.”

These plans were first revealed last August as part of an intended partnership between Flock and the AI-powered dashcam company Nexar. Now, a presentation document obtained by 404 Media shows that Flock was actively pitching this partnership to potential customers at the time. The presentation, which was intended to be shown to the Georgia Office of the Attorney General, was obtained by a Georgia resident through a public records request.

Flock told 404 that the partnership never went through, but the presentation still shows the company’s plans to turn rideshare and delivery drivers into mobile surveillance apparatuses — and the sheer scale is astonishing. 

“Hundreds of Commercial Business and HOA [home owners associations] in GA [Georgia] are part of the network,” the Flock presentation stated. “Plus Nexar partnership which includes 350k Uber/Lyft and other delivery service devices.”

Expanding to rideshare drivers would’ve added a new dimension to Flock’s already far-reaching surveillance network. Most of its cameras are mounted on stationary poles that survey a fixed area. In addition to providing more coverage, using cars could help Flock keep a lower profile; one of the main problems Flock is facing amid overwhelming public backlash is that protestors are vandalizing or blocking its surveillance pylons.

Another murky area of Flock’s plans was that it’s unclear if the Uber and Lyft drivers would’ve been aware they were participating in surveillance efforts, 404 noted.

Flock didn’t discount the possibility of these plans being realized through another partnership in the future. If they do, it’ll only fuel the massive privacy and safety concerns already posed by its cameras. The footage it collects into a centralized network can be accessed by cops across the country, often with little accountability. At least 50 police officers have been accused of using the cameras to stalk women, according to a recent Washington Post tally.  

Several people have also been swarmed by cops after Flock wrongly identified them as suspects.

The dashcam partnership wasn’t the only plan that Flock had to deepen its surveillance. Last month, it officially cancelled a “human distress detection” pilot program, a network of always-on acoustic sensors that monitored public places for sounds of “screaming.”

More on surveillance: Public Offers Implausible Number of Alibis for Man Accused of Destroying Flock Cameras

The post Flock’s Secret Surveillance Plan Will Make You Gasp appeared first on Futurism.