SANSAR: Cryptominers on Hijacked AI Clusters Forge NVIDIA's Telemetry Library So GPUs Report 0% While They Mine

High · 2026-11-16 · By H.A

SANSAR: Cryptominers on Hijacked AI Clusters Forge NVIDIA's Telemetry Library So GPUs Report 0% While They Mine

Attackers entered GPU hosts through an exposed Ray job agent, swapped libnvidia-ml.so.1 to fake idle GPUs, and hid behind a kernel rootkit, Tor C2 and an Ethereum dead drop.

Overview

Most cryptojacking is caught the same way: someone notices a GPU pinned at 100% when it should be idle, or a power bill that no longer fits the workload. SANSAR is built to beat exactly that check. In a Linux intrusion that researchers track as REF3364, the attackers replaced NVIDIA's own management library on compromised GPU servers. Every tool that asked the driver "how busy is this card?" got back the answer "0%", while the GPUs were in fact mining for someone else.

The victims were AI compute hosts. The way in was not an NVIDIA flaw but an internet-reachable Ray job agent, the distributed Python framework widely used for model training and inference. Once inside, the operators deployed a modular toolkit: a Rust orchestrator controlled over Tor, off-the-shelf CPU and GPU miners, a kernel rootkit, a falsified NVML library, and a second backdoor that finds its controller through an Ethereum smart contract. The toolkit reached three GPU hosts, and mining was confirmed on two.

The Campaign

The weak point was a familiar one in AI infrastructure. On the head node, the Ray dashboard listened only on loopback, which looks safe. But the Ray job agent was reachable from the internet and accepted job submissions without authentication. In Ray, submitting a job means running arbitrary Python on the cluster, so an exposed job endpoint amounts to remote code execution by design. Ray's own security guidance says it must be reachable only from trusted networks. A wave of attacks on exposed Ray clusters was already seen in 2024, and this case shows the lesson still has not fully landed.

The SANSAR toolkit is staged in /opt/vadi_/ and consists of:

Timeline

Tradecraft

Lying to the dashboard. The NVML trick is the most original part of the operation. It does not use LD_PRELOAD, which defenders already look for. Instead it overwrites the system's libnvidia-ml.so.1 and hides the original as /usr/lib/.libnvidia-ml.so.1.backup. The wrapper passes every call to the real library, then edits the results: utilisation is always 0%, temperature a random 36–46 °C, power a random 100–200 W. The numbers look healthy and slightly noisy. nvidia-smi, monitoring agents and exporters built on NVML all inherit the lie, because they read through the same library.

Hiding at the kernel level. The acpi_thermal_mgr rootkit hooks directory listing (getdents/getdents64), file opens, kill, and the kernel's TCP/UDP socket listings. Hidden processes (by default kurt and lyrebird) vanish from /proc, return "no such process" to PID lookups, and can only be signalled by the sansar manager itself. It is built through DKMS and wired into the initramfs, so it survives reboots and kernel updates. It can also remove itself from the module list, although that option was off by default.

C2 that is hard to take down. SANSAR registers over Tor with an HMAC-signed handshake and a custom binary protocol, and wipes shell history on every session (HISTFILE=/dev/null). The netns backdoor uses a different route: it makes read-only eth_call requests to public Ethereum RPC endpoints, decodes an IPv4 address stored in a smart contract, and connects to it on TCP 8080. To move their infrastructure, the operators only have to update the contract. There is no domain to seize.

Turkish fingerprints. The component names are Turkish words (kurt = wolf, tilki = fox, vadi = valley), and the rootkit source left in /usr/src/ has Turkish comments. That points to the developers' working language, not to a nation-state sponsor, and nothing here is tied to a named group. The motive is plainly financial, with a DDoS feature on the side.

Indicators of Compromise

SHA-256:

Network and blockchain:

Host artefacts:

Detection

The hard part is that the compromised host is the witness, and its testimony has been doctored. Detection has to rely on evidence the attacker does not control:

Recommended Actions

Analyst Assessment

SANSAR matters less for its miners, which are commodity, than for what it says about where attackers are going. GPU clusters are now worth enough that criminals invest in tooling built specifically to hide from GPU observability. Faking NVML output is a cheap trick with an outsized effect, because almost every GPU monitoring stack trusts that one library. We expect the technique to be reused, including on rented cloud GPUs where the victim sees only the dashboard and the invoice. Teams running AI infrastructure should assume that in-band telemetry can be forged and anchor their monitoring in signals the host cannot fake. Severity: High for organisations running internet-reachable Ray or similar job-execution frameworks on GPU hardware.

Sources

REEF SENTINEL