Netflix Open-Sources Agentic Workflow for Causal Inference

Netflix has released the source code for its observational causal inference workflow, a system designed to automate repetitive and error-prone tasks in data analysis. The tool, called oci-agent, uses a dual-agent actor-critic loop to estimate causality, generate reports, and suggest subsequent analytical steps. By framing observational analysis as a target trial emulation, the system helps researchers identify the optimal A/B test structure to answer complex business questions.

The workflow operates through a collaborative process between a human analyst and two specialized AI agents. The human first establishes an analysis plan and a templated Jupyter notebook. An actor agent then translates this plan into a specification, populates the notebook parameters, and runs the execution. Following this, a critic agent reviews the results, assigns a rating of fully satisfactory, satisfactory with caveats, or not satisfactory, and recommends adjustments to the specification.

To evaluate the system, Netflix tested the workflow against the Atlantic Causal Inference Conference competition dataset, where it achieved results competitive with benchmark systems. In a practical case study measuring how engaging with new entertainment types like games affects two-month customer retention, the team compared oci-agent against a baseline Claude model. While Claude opted for a basic linear regression, the oci-agent workflow identified early adopter bias and a failed placebo test, ultimately producing an estimated effect that Netflix reported was "just 25% of the baseline" estimate.

For data practitioners, this open-source release shifts the focus from manual scripting to high-level oversight. Rather than relying on black-box AI outputs, the workflow generates transparent, reproducible artifacts including plans, plots, and executable notebooks. This process audit approach allows human experts to verify every step of the causal analysis, lowering the entry barrier for complex statistical tasks while maintaining rigorous quality control.