GitHub and Microsoft's AI Scanner Cut 9,791 Findings to 99 Verified Reports: 55 Vulnerabilities Fixed in Caddy, ImageMagick, Express, Netty, FastMCP and AutoGPT

AI · 2026-11-18 · By H.A

GitHub and Microsoft's AI Scanner Cut 9,791 Findings to 99 Verified Reports: 55 Vulnerabilities Fixed in Caddy, ImageMagick, Express, Netty, FastMCP and AutoGPT

A GitHub–Microsoft pilot used the MDASH AI scanner, a triage agent and human verification to send six open-source projects 99 private, patch-ready reports, and 55 flaws are now fixed.

GitHub and Microsoft have published the results of a pilot that used an AI scanner on six widely used open-source projects. Unlike most AI bug-hunting campaigns, it did not leave maintainers with a pile of raw findings. Of 9,791 candidate issues, only 99 verified reports reached the maintainers, each with a proof of concept and a suggested fix. By 15 November, 55 of those flaws had been fixed and 14 fixes had already shipped in releases.

The six projects were AutoGPT, Caddy, Express, FastMCP, ImageMagick and Netty. Together they sit under a large share of modern web servers, Node.js applications, Java network stacks, image pipelines and AI agent tooling.

What Happened

GitHub announced the results on 17 November, the same day Microsoft introduced MAI Cyber 2 Flash, a new security model, at its Ignite conference. GitHub's title sums up the pilot's message: maintainers don't need more AI bug reports, they need fixes.

All six projects are alumni of the GitHub Secure Open Source Fund and agreed to be scanned. Maintainers kept full control over what to fix, how and when. GitHub says the first fixes shipped about a week after the first scan.

How the Pipeline Worked

The pilot ran in five stages, each narrowing the output of the one before:

That works out to roughly one report for every 100 raw findings. GitHub's point is that triage should be done by whoever sends the report, not by the volunteer who receives it.

What the Maintainers Saw

GitHub shared each project's experience:

GitHub did not publish the individual vulnerabilities, CVE IDs or advisory IDs behind the 55 fixes. It also did not break down their severity.

Why It Matters

Open-source maintainers are dealing with a flood of AI-generated security reports. Many are duplicates, misreadings of the threat model, or "vulnerabilities" that cannot actually be exploited. Every one still takes a volunteer's time to read and reject.

The pilot argues that the fix is a higher bar for the sender, not a ban on AI. GitHub proposes a standard for anyone reporting with AI tools:

The post quotes Log4j maintainer Christian Grobmeier: "AI defending open source has to be better than the AI attacking it." That matters because attackers are using the same capability. AI agents that read patches and write exploits are already being used in real intrusions. Bugs that defenders find and fix first are bugs attackers cannot weaponise.

The Caveats

GitHub is open about what doesn't scale yet:

The sample is also small and self-selected. All six projects are well funded by open-source standards, had security training through the fund, and opted in. Results for a one-person project with no security process could look very different.

What Comes Next

GitHub says it will offer AI security scans to every alumni project of the Secure Open Source Fund. Findings will be checked before private reporting, and alumni also get GitHub Copilot Max. Lessons from the pilot will go into private vulnerability reporting and security advisories. The fund says it has supported 216 projects and more than 300 maintainers so far.

On the scanning side, Microsoft says MAI Cyber 2 Flash will be available only inside its security products, including MDASH. From mid-December it will replace the current model in MDASH's MAI profile for eligible customers. Microsoft reports a 92.3% task success rate on the CyberGym-E2E benchmark after excluding 92 tasks that overlapped with its training data. These are vendor-reported figures and have not been independently reproduced.

What To Do Now

Sources

REEF SENTINEL