Staff+ Software Engineer, Safeguards Evals

Home > Other programming jobs

Staff+ Software Engineer, Safeguards Evals in USA new

Anthropic
Full time
Email

USA
11/09/2026
Report

Seattle, WA

Responsibilities

Design and run experiments to improve evaluation quality, test-data representativeness, grading accuracy, and evaluation coverage.
Build and own an evaluation harness for an agentic investigation system, including metrics, test cases, grading approaches, and end-to-end performance measurement.
Construct reinforcement-learning environments to improve Claude's safety investigation capabilities.
Create evaluation datasets representing real-world misuse using real traffic patterns and synthetic generation.
Collaborate with Policy and Enforcement to translate observed harm patterns into measurable evaluations.
Ship evaluation methods into regression and release pipelines used during model training, agent changes, prompt updates, model upgrades, and post-launch operations.
Build tooling that enables policy experts to author, run, and iterate on evaluations without engineering support.
Surface evaluation findings to research and training teams to drive model improvements.

Requirements

Experience building and maintaining data pipelines.
Experience working with LLMs and understanding their capabilities and failure modes, especially agentic systems with tool use and multi-step reasoning.
Strong data analysis skills and the ability to draw reliable insights from large datasets.
Ability to move between research prototyping and production-quality code.
Ability to translate ambiguous problems into concrete, testable experiments.
Strong interest in AI safety and meaningful societal impact.
Preferred: 8+ years of industry software engineering or ML engineering experience.
Preferred: expertise with LLM or agent evaluation frameworks, benchmarks, or automated grading systems.
Preferred: extensive experience in trust and safety, content moderation, abuse detection, red teaming, adversarial testing, or jailbreak research.
Preferred: experience with synthetic data generation, data augmentation, distributed systems, large-scale data processing, prompt engineering, or LLM-powered applications.
Bachelor's degree or an equivalent combination of education, training, and/or experience in a relevant field.

Benefits

Annual base salary range of $320,000-$485,000 USD.
Hybrid policy requiring staff to work from an office at least 25% of the time, with some roles requiring more office time.
Visa sponsorship is offered with reasonable efforts made after an offer, supported by an immigration lawyer.
Competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office collaboration space.

Salary: $320k - $485k/yr

Anthropic

Anthropic builds AI systems that are reliable, interpretable, and steerable, with a commitment to ensuring their safety and benefit to users and society. Our work is driven by a dedicated team of researchers, engineers, and policy experts focused on the development of beneficial AI solutions.

Similar positions

Software Engineering Specialist Commercial Intelli

IKEA
Full time

Others
11/10/2026

Malmö, Sweden

Software Automation & Platform Engineer

Rackner
Full time

Remote
11/10/2026

United States

Principal Software Engineer, Product Engineering

Dropbox
Full time

Canada
11/10/2026

Remote

Software Engineer, Host Assurance

OpenAI
Full time

USA
11/10/2026
Salary: $266k-$445k

Remote

Software and Documentation Support Engineer

Booz Allen Hamilton
Full time

USA
11/10/2026
Salary: $53k - $108k/yr

Port Hueneme, CA