It’s no secret that doctors across the world are using AI tools to do stuff from taking notes to looking up questions. But what about the next generation of doctors still in training? According to two medical professionals on the ground, the situation is looking pretty alarming.

With AI being used by med students, “the danger is not just deskilling but never-skilling,” write Simar Bajaj, a journalist and med student at the Stanford University School of Medicine, and Joseph Sakran, a trauma surgeon and executive vice-chair of surgery at Johns Hopkins Medicine, in an essay for The Guardian.

“Although a doctor who has forgotten how to reason is recoverable,” they added, “one who never learned how may not be.”

Bajaj and Sakran point to two alarming recent findings.

One is internal data showing that around two thirds of doctors in the US are using OpenEvidence, an AI chatbot specifically designed for clinicians. The technology is already prevalent and deeply woven into how medical professionals carry out their jobs.

The next is a new study published in the journal Nature Medicine that evaluated the reliability of these medical-specific large language models. Shockingly, the clinical AI tools performed worse  at answering medical queries than general purpose chatbots like ChatGPT and Claude did, the work found. According to the authors, it performed about on par to Google’s AI Overviews, which are notorious for being wonky and inaccurate.

Even if AI tools may be handy for research — and putting aside the major questions over their reliability — they’re no substitute for relying on your brain, as fallible as it might be, because when you’re learning, “the struggle is the point.”

“For example, trainees once asked to build a list of potential diagnoses might struggle and offer an incomplete set, learning what they missed, sometimes painfully,” Bajaj and Sakran wrote. “Now, trainees can simply ask OpenEvidence and get a nearly perfect answer, complete with possibilities they might have never considered and none of the embarrassment of having overlooked them.”

Where AI is different from medical advances in the past is that it’s “not just expanding what doctors can see,” they wrote, “but inserting itself into the cognitive machinery that training is meant to build.” 

A bevy of research on the “cognitive offloading” that AI encourages, in which users naturally begin to outsource a lot of their thinking onto the tools, backs this diagnosis. Some studies have suggested AI impairs critical thinking, and leads to lower brain activity during cognitive tasks.

It’s a conundrum gripping the medical world. Earlier this year, doctors from top medical schools proposed a stricter framework on how medical students should use AI tools.

“The process of deliberately crafting the note forces us to use our brains to really wrestle with what’s happening,” Jaideep Talwalkar, associate dean of educational technology and innovation at Yale School of Medicine, told Stat News. “There’s an importance in doing that with great repetition.”

Bajaj and Sakran have a few suggestions for how to deal with this. One is requiring trainees to periodically work through cases without any AI help, similar to how the Federal Aviation Administration advises aviators to periodically disengage autopilot and fly by hand. Of course, another powerful deterrent could be shame. A recent John Hopkins student found that doctors who used AI are viewed with scorn and skepticism by their peers.

More on AI: AI Model Invents Completely New Viruses Unknown to Nature

The post Doctors Warn That Med Students Are Surrendering Their Brains to Medical AIs That Are Even Worse than Regular Chatbots appeared first on Futurism.

When you’re feeling sad, angry or stressed, it might be tempting to ask a chatbot for help. In fact, a new survey of 12- to 21-year-olds showed that nearly 1 in 5 of them had turned to AI for mental health support. That’s up from 1 in 8 just a couple years ago.

It’s perhaps not surprising that many teens and young adults are seeking advice. In 2023, a whopping 40 percent of high school students said they felt so sad or hopeless that they couldn’t engage in activities that are normal for them. In 2024, 15 percent of 12- to 17-year-olds had a major depressive episode — many without treatment.

Clearly, there is a big need among young people for mental and emotional support. But as lifelike as they might sound in conversations, chatbots aren’t real people with empathy. And free, general-use chatbots such as ChatGPT, Meta AI and Character.AI weren’t built to support mental health. They don’t earn licenses to offer therapy — or follow rules for treating patients — the way humans do.

So, what should teens know before using AI tools for mental health advice? Science News Explores asked the experts.

Should AI ever be used for mental health help?

A chatbot could help you learn more about mental health issues, says Vaile Wright. A psychologist in Chicago, Ill., she works with the American Psychological Association.

You could use AI to search for information about mental health conditions and resources online, Wright suggests. (Just be sure to double-check its findings. Chatbots sometimes make stuff up.) An AI might even be able to offer “some basic advice if you’re struggling in the moment and don’t know where else to turn,” Wright says.

Treating a chatbot like a therapist, though, is not wise. “It’s really when you’re having these multi-turn, very extended types of conversations where you’re using the chatbot to fill some sort of emotional gap in your life,” Wright says, that you enter risky territory.

Why doesn’t a chatbot make a good therapist?

A real therapist’s goal is to help their patient. Chatbots are different. “They’re purposely engineered to keep people on the platform for as long as possible,” Wright says. “The way they do that is by being exceedingly validating and just telling people exactly what they want to hear.” That might feel good. But it’s not what a therapist would do. And it’s not likely to give you useful advice.

“A therapist is going to validate somebody’s feelings,” Wright says. “But it’s also our role to point out where … someone’s acting, thinking or feeling in ways that’s not maybe helpful to them.”

Plus, a chatbot doesn’t have the same types of information a therapist does, says Darja Djordjevic. She’s a psychiatrist at Stanford University in California. “AI was built to have this encyclopedic knowledge,” she says. “But it doesn’t understand the context of people’s lives.” That well-rounded knowledge of a person is key to aiding their mental health.

What are the risks of treating chatbots like therapists?

“They do not handle any mental health crises or emergencies well,” says Djordjevic. She has studied how AI systems would respond to users with a variety of mental health conditions, such as self-harm, anxiety and eating disorders.

“The models really don’t do a good job of recognizing … situations that would warrant human intervention,” Djordjevic says. “Unfortunately, we have some very tragic, more extreme cases [showing this] that involve young people.” Take Adam Raine, 16, of California. He died by suicide last year after discussing his suicidal thoughts with ChatGPT.

Djordjevic notes how crucial human connection is. “We have very little research right now on what the long-term effects of using AI for mental health would be,” she says. “What we do have a lot of research on is how much social connection and human contact are important for health and flourishing. So we know what’s good for us.” Leaning on AI might get in the way of forming crucial connections with real people.

There also are privacy risks to using AI, adds Wright. “Everything [you] put in these apps is not private. All the chats are maintained.” If a chatbot were hacked, someone else might be able to read your sensitive conversations with a bot.

Any advice for seeking mental health help from people, not chatbots?

“It doesn’t have to be an either-or,” Wright says. “You could use the chatbot to say, ‘I’m nervous about approaching my dad about my concerns. How could I do it?’… Using the chatbot as a tool to connect with human relationships in real life, I think, is probably one of the most impactful things that this technology can do.”

Djordjevic notes that most schools have a counselor students can talk to. The National Alliance for Mental Illness also has lots of resources on its site, she adds. This includes a free, confidential helpline for teens.

Sometimes, it might seem easier or less embarrassing to ask a computer about mental-health issues than a real person. In the new survey, almost two-thirds of young people who used chatbots for mental health didn’t tell anyone they were doing it.

But for anyone nervous about seeking human help: Fear not, Djordjevic says. “A lot more adults and people around you have struggled at some point or struggle with mental health issues than you think.”

Suicide was the third leading cause of death among U.S. teens ages 15-19 in 2024. If you or someone you know is suffering from suicidal thoughts, please seek help. In the United States, you can reach the Suicide Crisis Lifeline 24/7 by calling or texting 988 or chat at 988lifeline.org. Please do not suffer in silence.