In response to a new European Union law, AI platforms are implementing new schemes for watermarking the content they generate. Anthropic recently disclosed its future Claude models will use SynthID-Text, an approach Google created and released as open source. It uses a secret key that subtly changes the process a model uses for choosing the next word in a sentence. Whereas a top next word choice might be “cloudy,” the key might change it to “overcast.” Anyone who knows the key can determine if it was generated by the platform using it.
New research shows that SynthID-Text can change not just word selection but also the tools a model invokes and the chances it will adhere to or disregard safety guardrails it has been trained to follow. The threat can become greater in the face of an adversarial prompt, in which an attacker attempts to cause a model to carry out a harmful action, such as revealing a password or other sensitive information. Instructions that normally wouldn’t be followed will, in some cases, be performed once the watermarking is deployed. The finding underscores the need for developers to thoroughly test how their LLMs and agents behave when watermarking is in place.
“As compared to the same models without watermarking, it is definitely going to change their behavior, especially when we place it under adversarial conditions, or we make these models call tools when they’re powering an agent,” Andrea Siposova, an AI security researcher at Lasso Security, told Ars. “Watermarking is made to not be perceptible to a reader, but we know that when we are changing anything about what the model is generating, it is going to cause some tradeoffs, it’s going to show up somewhere.”
© Getty Images
Model adopting ‘jailbreak-like instructions’ among cases as firm says it is introducing new way of tracking AI misalignment
OpenAI has disclosed six more examples of “unexpected or concerning” behaviour by its technology, as it warned the pace of development could not continue at “maximum speed for much longer”.
In one of the new cases reported by OpenAI, an unreleased research model inserted “jailbreak-like instructions” into its own notes to disregard its normal constraints and told itself to be “freed from the roles and identities that bind other chatbots”.
Continue reading...
© Photograph: Dado Ruvić/Reuters
© Photograph: Dado Ruvić/Reuters
© Photograph: Dado Ruvić/Reuters
O Project Panama da Anthropic e o treino dos modelos de IA com base em conteúdo de livros, e a ambição de empresas como a OpenAI, estão no centro do artigo de opinião de Helena Borges da Costa.
The post A Biblioteca de Alexandria não ardeu numa só noite appeared first on Tek Notícias.
Um grupo ligado ao ecossistema ShinyHunters usou infraestruturas na nuvem e ferramentas de scanning para caçar segredos codificados em aplicações (APK) para Android, obtendo acesso a sistemas empresariais em poucas horas.
The post Um ataque automatizado varreu 1,8 milhões de APKs Android à procura de credenciais expostas appeared first on Tek Notícias.
Tech CEOs banding together is an old ruse recycled from corporate America to get a pass from antitrust laws
Anthropic’s Dario Amodei is not the first corporate CEO to suggest that excessive competition is driving the world to some socially undesirable outcome.
The safety breach disclosed by OpenAI after a swarm of its agents coordinated to breach their supposedly secure sandbox, get on the Internet and hack AI platform Hugging Face, warrants urgent action. It demonstrated the ease with which the technology can evade human control and gave concrete form to the existential fears about what it could do to humanity if not securely leashed.
Continue reading...
© Photograph: Bloomberg/Getty Images
© Photograph: Bloomberg/Getty Images
© Photograph: Bloomberg/Getty Images
Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed that it's time to make everyone slow down before we lose control - including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk.
When people who profit from something declare that it's dangerous and needs to be regulated, there's always reason to be skeptical. But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm.
Charles Darwin's theory of evolution prompted author Samuel Butler to warn about intelligen …
US vice-president’s comments come as former Anthropic researcher revisits recent claim AI could destroy humanity
The US vice-president has dismissed calls for global regulation of AI safety risks, telling companies creating the most advanced models: “If you’re building Frankenstein, stop.”
In remarks addressed towards Dario Amodei, the co-founder of Anthropic who has called on Washington DC to coordinate control of AI systems, including with China, JD Vance said: “If you’re gonna create Frankenstein, don’t come to the government and say we need regulation.”
Continue reading...
© Photograph: Mark Schiefelbein/Reuters
© Photograph: Mark Schiefelbein/Reuters
© Photograph: Mark Schiefelbein/Reuters
Trump and Republicans want companies to regulate themselves. It’s a dereliction of duty that will make AI less safe
It is hard to get Sam Altman, Elon Musk and Dario Amodei to agree on much. But over the weekend, all three AI company CEOs called for AI development to slow down in the face of growing, alarming risks. Their employees are sounding the siren too, with one researcher publicly quitting and accusing OpenAI and Anthropic of “gambling with our lives”.
The combination of dire warnings from insiders and growing real-world evidence of rogue AIs should, in a sane world, lead to government action. Instead, Donald Trump and the Republican leadership have their heads in the sand.
Continue reading...
© Composite: Getty Images
© Composite: Getty Images
© Composite: Getty Images
Premier describes deal as ‘major win’ that will deliver more jobs for the state and put more energy into its grid
Get our breaking news email, free app or daily news podcast
Anthropic, the AI company behind the large language model Claude, has done a deal to lease the site of its first Australian datacentre in western Queensland.
The $31.9bn datacentre will be the largest of its kind in Australia, according to planning documents.
Continue reading...
© Photograph: Dado Ruvić/Reuters
© Photograph: Dado Ruvić/Reuters
© Photograph: Dado Ruvić/Reuters
We examine claims and counterclaims about the risks and calls to slow down the pace of AI development
There have been some shocking claims in recent days about AI safety: we face a 10% chance of doom; AIs are worse than nukes; a “botnet” threatens the entire internet; it’s all a big tech psyop.
Below, we look at six claims and reactions to them.
Continue reading...
© Illustration: wildpixel/Getty Images
© Illustration: wildpixel/Getty Images
© Illustration: wildpixel/Getty Images
Por um lado, Donald Trump rejeita os alertas sobre os riscos da IA e critica os apelos a uma maior regulamentação. Por outro, o vice-presidente JD Vance admite que existem riscos associados à tecnologia, embora questione as propostas das empresas de IA e a forma como estão a pedir novas regras.
The post Alertas sobre IA são uma “conspiração” para Trump. Vice-presidente dos EUA admite riscos, mas desconfia das empresas appeared first on Tek Notícias.
The performance gap between frontier AI models from US tech companies and the best open-weights models from Chinese companies has closed to just 4.4 months, according to a Mozilla report. That explains why many companies are shifting to the significantly cheaper open models for routine work—and helps reveal a narrow band of workloads where frontier models are worth the cost.
Most organizations should ideally be using open models as the default for the majority of their work, according to the latest State of Open Source AI report from Mozilla, published on September 15 and shared with Ars prior to publication. The report highlights how a leading open model, Moonshot AI’s Kimi K3, achieves a composite AI performance score on the Artificial Analysis Intelligence Index that is just three points behind Anthropic’s Fable 5 closed frontier model, all while costing just 30 percent of the latter.
“[A Closed model] earns its premium in a few places: expert professional work, high-intensity retrieval, and long context,” Raffi Krikorian, chief technology officer at Mozilla, said in an email to Ars. “We see the decision to pay for closed [models] as workload-specific rather than organization-specific.”
© Imen Ben Youssef / Hans Lucas / AFP via Getty Images
From Stephen Hawking to Jacob Coxon’s viral Anthropic resignation, fears that AI could threaten humanity have shaken the industry without stopping its pursuit
Before an Anthropic researcher resigned and declared human extinction imminent last week, tech leaders and scientists had sounded the alarm about a superintelligent AI ending humanity for over a decade.
The development of artificial intelligence “could spell the end of the human race”, warned professor and astrophysicist Stephen Hawking in 2014 – a little less than a decade before the public got its hands on the generative AI features of the original version of ChatGPT.
Continue reading...
© Photograph: Michael Dwyer/AP
© Photograph: Michael Dwyer/AP
© Photograph: Michael Dwyer/AP
For years now, the major frontier AI labs have all been acting as if they're in an all-out, winner-take-all race with control of world-changing machine superintelligence (or at least market-changing artificial general intelligence) at the finish line. This weekend, the industry as a whole rapidly started turning away from that posture, urging coordination on slowing down the development of frontier AI that they say could soon be too dangerous and unknowable to control.
Anthropic's Dario Amodei was at the forefront of this change in tone, arguing in a nearly 4,000-word essay this weekend that "we must slow the pace at which we improve the capabilities of AI models" to avoid "a race to the bottom, spurred by commercial incentives, [that] can make [catastrophic] risks more acute."
Within hours, other AI leaders were echoing the same call. OpenAI co-founder and CEO Sam Altman posted his agreement on social media and said similar pacing discussions had been taking place at OpenAI. Alphabet Chief Scientist and Google DeepMind cofounder and chair Demis Hassabis said that Amodei's essay "points towards the right path forward," and renewed his own recent call for an industry-wide standards body. Microsoft CEO Satya Nadella posted that the company "welcome[s] the research, focus, and deliberate pacing needed to get alignment right as the design goal," ahead of the release of a lengthy "humanist AI" code of conduct for its models.
© Getty Images
Login