WhatsApp Tests On-Device AI to Detect Chat Scams
WhatsApp is testing an optional on-device machine learning feature called Scam Alert to identify fraudulent messages from unknown senders without compromising user privacy.
Meta has launched a limited beta of Scam Alert, an opt-in feature for WhatsApp that uses a compact, locally downloaded machine learning model to scan incoming messages from non-contacts. The system analyzes conversational structures and linguistic signals to flag potential scams. Because the classification happens entirely on the user's device, the content of the messages is never exposed to WhatsApp's servers. If a scam is suspected, the app displays a warning to the recipient, who can then block the sender, report the chat, or mark it as trusted. Users who trust a flagged chat can optionally share the last 5 messages to help refine the model.
To measure the model's performance without compromising privacy, the architecture separates local classification from telemetry collection. The system aggregates warning events locally and transmits these metrics through an Oblivious HTTP relay using anonymous credentials. Data is processed inside confidential virtual machines acting as trusted execution environments. Before any population-level statistics are shared with WhatsApp, the pipeline applies minimum cohort thresholds and differential privacy. This setup builds on Meta's PAPAYA federated analytics system, which was introduced at the USENIX NSDI 2025 conference.
The distribution of the machine learning model itself is treated as a security boundary. Meta publishes each model version and its SHA-256 hash to a third-party, append-only transparency ledger. Before loading a model, the client device verifies its signature, freshness, hash, and ledger entry. Downloads are routed through Oblivious HTTP, and experimental model assignments are handled locally to prevent servers from targeting specific users with custom model variants.
For security and AI practitioners, this implementation demonstrates how to deploy sensitive telemetry systems in consumer applications where user trust is paramount. Meta is expanding its Bug Bounty program to cover this confidential federated analytics pipeline and plans to publish the confidential virtual machine binary alongside privacy-relevant source code for external review. Stephanie Lio, head of AI product marketing at WhatsApp, noted on LinkedIn that the design is "on-device only" and ensures that "users remain in control."
This is our own summary of reporting by InfoQ AI
