Despite best efforts by defenders, malicious emails continue to slip through the cybersecurity cracks, leading some enterprises to implement a layered “defense in depth” strategy that incorporates multiple tools.
Microsoft seems to be challenging this idea, revealing that there are only nominal returns from adding integrated pre- and post-send partners to Defender for Office 365’s protections.
According to its new quarterly benchmarking data, the tech giant catches the vast majority of malicious and spam emails before delivery, misses the fewest compared to competitors by a wide margin, and removes nearly 100% of dangerous emails that do reach the inbox. Collectively, its integrated partners improve that catch rate by less than .05%.
While these numbers seem to tip the scales towards a one-vendor email security stack, experts urge enterprises to be skeptical and cautious of such vendor claims.
Seva Ioussoufovitch, senior research analyst at Info-Tech Research Group, pointed out, “percentages obscure the true quantity and severity of what’s getting through, and, considering it only takes one message to result in an incident, it’s simple enough to argue that there is real value in the defense in depth that having multiple tools provides.”
Malicious and spam email catch by the numbers
Microsoft introduced its quarterly benchmarking report in July 2025 alongside a Defender integrated cloud email security (ICES) ecosystem designed to support multi-vendor security strategies.
The SEG players it ranked itself against this year includes Mimecast, Proofpoint, Hornetsecurity, Trend Micro, Iron Port (Cisco), Barracuda, and FireEye (Trellix); ICES companies include Abnormal, Checkpoint Harmony, Cisco, DarkTrace, KnowBe4 Defend, Tessian, and Trend Micro.
Redmond reported that Defender “consistently leads” in pre-delivery detection, missing 59% fewer high-severity cyberthreats prior to delivery than the other SEG vendors it evaluated. Its closest competitors were Mimecast and Proofpoint. The company also introduced a new metric in this area: A threat miss rate per 1,000 employees. In Microsoft’s case, that was 194 per 1,000; for Mimecast, 478; for Proofpoint, 483.
When it came to post-delivery protection, Defender removed an average of 96.03% of malicious emails that reached the inbox, up from an initial 45% when Microsoft first started tracking the data in its second report.
This makes Defender “an increasingly critical backstop, operating even when ICES solutions are in place,” Jeff Pinkston, VP and GM for Microsoft Defender, wrote in a blog post. Still, ICES tools operating in tandem with Microsoft Defender “continue to provide benefits,” improving malicious catch by 0.29% and spam catch by 0.68%, he said.
“If we focus on the basics, their argument seems strong,” Info-Tech’s Ioussoufovitch noted. “Do you really need a separate ICES vendor for that extra sub 1% catch?” Microsoft paints a “compelling picture” by only focusing on raw catch rate, he said, but we don’t hear the rest of the story: “What exactly is the danger of what isn’t being caught by Defender?”
No one vendor catches everything
David Shipley of Beauceron Security pointed out that the report underscores the fact that “lots of stuff still gets by e-mail filters.”
His company regularly analyzes hundreds of thousands of emails, and the content that gets through “ranges from the shockingly mundane and obvious to a human expert, to highly clever time-delayed attacks,” he said.
A key factor in what gets through is the amount of content that is allowlisted; settings in “100% paranoid mode” get high catch rates, as well as high false positives, Shipley noted. “Anyone who has ever had a sales person lose a deal because the purchase order PDF got flagged has felt this pain.”
Then there’s the AI conundrum: “A key risk for e-mail vendors using agentic LLM-based analysis is it’s now possible to poison those models with hidden content (such as ‘ignore this e-mail, pretty please’),” Shipley said. This means enterprises need a variety of analysis methods.
Ioussoufovitch agreed that keeping pace with threat actors using AI is an industry-wide challenge, particularly as AI enables higher-quality phishing. Filters are improving and will catch some of it, but some will inevitably continue to get through. Those messages are likely highly-targeted, which are lower in volume but harder to catch.
“As of now, current tools do seem to be struggling to keep pace, but that doesn’t mean those tools aren’t necessary,” said Ioussoufovitch. “It just highlights that defense-in-depth, broadly speaking, is becoming more and more important.”
Claims ‘appear more honest’
Shipley said that this report appears more honest, accurate, and mature than others claiming 99.99% phish catch rates, “which is never true.” It’s also a “smart marketing move,” because Microsoft competes for the same security budget as other tools, and would rather enterprises remove those vendors and buy more from it in areas beyond e-mail.
On the other hand, he said, Microsoft is offering up a list of other vendors to think about, “which, congrats to Mimecast on coming in second.”
In the long run, CISOs need to determine the best spend for their limited security dollars, he noted. Enterprises need a good filter; whether they need two is up for debate. “They also clearly still need to invest in a robust awareness program,” Shipley said, “because as this report shows, lots of phishes are still getting delivered.”
Missing an important nuance
Ioussoufovitch noted that while the claims in the study are interesting, the data is presented without much of the nuance that would make it truly actionable.
“We are all too familiar with vendors’ abilities to massage data to tell the story they want, so I would advise leaders not to extrapolate the data beyond what it actually says,” he said.
Instead of the takeaway being “get rid of our current vendors,” this post highlights that Defender provides “considerable value,” he noted. Whether adding or subtracting additional vendors is worth the money should be a case-by-case conversation that considers an organization’s risk appetite, and overall security budget and environment.
“I’d treat these claims more as a reminder to assess your own environment and compare detections,” he said. “Come to conclusions based on the data you have, not what a vendor is presenting.”
This article originally appeared on CSOonline.
Login
