Research Engineer / Scientist, Data & Evals

Home > Python programming jobs

Research Engineer / Scientist, Data & Evals in USA new

Harvey
Full time
Email

USA
10/31/2026
Report

Remote

Responsibilities

Build synthetic data pipelines that generate, filter, and curate training and evaluation samples for legal reasoning and agentic workflows.
Design evaluations and verifiers, calibrate them against expert judgment, and investigate their failure modes.
Build and maintain capability benchmarks such as the Legal Agent Benchmark and product-focused evaluations for customer workflows.
Work with legal domain experts to define quality metrics, rubrics, and grading mechanisms.
Own training and evaluation data quality, including provenance, coverage, and contamination safeguards.
Connect product usage and customer feedback to new evaluations and training data.
Investigate model failures and use findings to guide training and release decisions.

Requirements

Experience building data pipelines, ML evaluation systems, or experimentation tooling.
Strong Python skills and experience owning production-quality code.
Rigorous understanding of measurement, statistical reasoning, experimental design, and misleading metrics.
Ability to translate ambiguous goals and expert judgment into structured datasets, rubrics, and measurable criteria.
Experience with synthetic data, LLM evaluations, automated raters, or preference data is preferred.
Strong data and measurement engineering experience close to ML model development is valuable even without prior LLM experience.

Benefits

Full-time role with compensation of $230,000-$450,000 USD depending on location.

Salary: $230k - $340k/yr

Harvey

Harvey is domain-specific AI for legal and professional services. Adopted by Fortune 500 companies like AT&T, Verizon, Cox, Koch, KKR, Bridgewater, more than 100,000 lawyers across 2,400+ customers in 70 countries and over 75% of AmLaw 100 law firms rely on Harvey to advance legal expertise faste...

Similar positions

Software Engineer II, Full Stack

Affirm
Full time

Canada
10/31/2026

Remote

AI/ML Software Engineer

ASG
Full time

USA
10/31/2026
Salary: $200k - $250k/yr

Remote

Senior Security Engineer

Commonwealth Bank of Australia
Full time

Others
10/31/2026

Bengaluru, India

Software Engineer, Identity

Lyft
Full time

Canada
10/31/2026

Toronto

Senior Security Engineer, AI

Justworks
Full time

USA
10/31/2026
Salary: $188k - $242k/yr

New York, NY