Senior

Home > Python programming jobs

Senior in USA new

FurtherAI
Full time
Email

USA
10/15/2026
Report

San Francisco, CA

Responsibilities

Own evaluation frameworks and infrastructure measuring the accuracy, quality, and reliability of AI agents across workflows, assistants, document processing, and computer use.
Develop golden datasets, benchmarks, and regression suites with Forward Deployed Engineers and domain experts for real-world insurance tasks and edge cases.
Design evaluations for multi-step agent trajectories, tool use, reasoning, and end-to-end task completion.
Build LLM-as-judge evaluations, deterministic graders, human evaluation workflows, and quality rubrics.
Integrate offline and online evaluations into development and release processes to measure improvements and catch regressions.
Benchmark frontier and open-weight models across accuracy, latency, cost, and reliability to inform model selection and production releases.
Create continuous improvement loops using production failures, customer feedback, and agent traces.
Build reusable evaluation frameworks, benchmarking systems, and production pipelines for insurance workflows and AI skills.

Requirements

5+ years of software engineering experience with strong hands-on experience building production AI/ML systems.
Experience building evaluation frameworks, LLM-based applications, model benchmarking systems, or applied AI infrastructure.
Strong understanding of modern LLMs, agent architectures, evaluation methodologies, and model performance tradeoffs.
Strong Python engineering skills and ability to build reliable, production-grade systems.
Ability to independently own ambiguous technical problems from research and experimentation through production.
Clear written and verbal communication skills.
Preferred experience with LLM-as-judge systems, human evaluation, agent trajectory evaluations, or large-scale benchmarking.
Preferred experience with open-weight models, fine-tuning, post-training, or model routing.
Preferred B2B SaaS background and 0-to-1 experience.

Benefits

Fully covered health, dental, and vision benefits.
Competitive compensation, meaningful stock options, and company 401(k).
Unlimited PTO.
Fully in-office in San Francisco five days per week.
Lunch and dinner provided onsite.
Team events including happy hours and off-sites.
Pre-tax commuter benefits.

Salary: $300k - $350k/yr

FurtherAI

FurtherAI builds domain-specific AI software for insurers, reinsurers, MGAs, and brokers, offering an AI Workspace that automates submission intake, policy comparisons, and underwriting audits. It sells an enterprise platform that augments internal teams with AI coworkers to streamline underwriti...

Similar positions

Senior AI Engineer

Infiterra S.A
Full time

Remote
10/15/2026

Greece, UK

Senior Security Engineer

GitLab
Full time

Remote
10/15/2026

Israel, Poland

Senior Data Scientist

DuckDuckGo
Full time

Remote
10/15/2026
Salary: USD 178,500

Worldwide

Senior AI Controls Solutions Engineer

Phaidra
Full time

Remote
10/15/2026
Salary: $108k-$190k

USA, Canada

Senior Software Engineer

Nuro
Full time

USA
10/15/2026
Salary: $160k - $291k/yr

Mountain View, CA