Software Engineer, Inference

Home > Python programming jobs

Software Engineer, Inference in USA new

Thinking Machines Lab
Full time
Email

USA
10/03/2026
Report

San Francisco, CA

Responsibilities

Operate and scale production inference systems serving live traffic, including Tinker's multi-tenant serving platform.
Own safe, incremental rollouts for new models, model versions, and inference optimizations.
Build and improve observability, monitoring, alerting, and capacity-planning systems.
Partner with inference and research teams to productionize new serving techniques.
Lead incident response, root-cause analysis, and durable fixes for production inference issues.
Design graceful degradation, failover, and redundancy for resilient serving.
Manage capacity and cost tradeoffs as traffic and model sizes scale.

Requirements

Experience operating large-scale, latency-sensitive production systems.
Proficiency in Python and Go or another systems language.
Experience with observability, monitoring, and incident response for production services.
Strong understanding of distributed systems and failure at scale.
Preferred: experience running production inference for large language models or other large-scale ML systems.
Preferred: experience with canarying, blue/green deployments, feature flags, capacity planning, GPU or TPU cost optimization, batching, caching, or quantization.
Comfort with on-call responsibilities, leading critical incident response, and working autonomously in a fast-changing environment.

Benefits

Based in San Francisco, California.
Annual salary range of $300,000–$400,000 USD.
Visa sponsorship is available, with the company committed to working through the visa process for the right fit, though success is not guaranteed for every candidate or role.
Generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.

Salary: $300k - $400k/yr

Thinking Machines Lab

Thinking Machines Lab pioneers advanced AI research, blending cutting‑edge theory with practical applications to solve complex problems across industries. Our interdisciplinary team pushes the boundaries of machine learning, neural architecture, and cognitive modeling to deliver innovative soluti...

Similar positions

Software Engineer - AI (Senior/Staff)

Monarch Money
Full time

USA
10/03/2026
Salary: Competitive

Remote

Senior Software Engineer, Backend - Platform

Coinbase
Full time

USA
10/03/2026
Salary: $186k - $219k/yr

Remote

Senior Software Engineer - Cloud Platform

Fortinet
Full time

USA
10/03/2026
Salary: $140k - $160k/yr

Santa Clara, CA

Senior Software Engineer

CHAOS Industries
Full time

UK
10/03/2026
Salary: Competitive

London

Software Engineer Intern (Summer 2027)

Affirm
Full time

USA
10/03/2026

San Francisco, CA