Staff Software Engineer, Inference

Home > Golang programming jobs

Staff Software Engineer, Inference in UK

Together AI
Full time
Email

UK
08/21/2026
Report

London

Responsibilities

Design and implement versioned provisioning state machines for the full lifecycle of physical hosts and GPU clusters.
Build declarative self-service APIs and a control plane for requesting, scaling, and tearing down inference clusters.
Automate self-healing by detecting degraded nodes, draining them, triggering repair or replacement, and restoring healthy capacity.
Own pipeline reliability through idempotency, retries, rollback, and drift detection.
Partner with inference and ML platform teams to encode cluster shapes, topology, interconnects, and scheduling constraints.
Develop typed, tested, versioned infrastructure software and operate and support it in production.

Requirements

Strong software engineering background in Go, Python, Rust, or a similar language, including writing and testing production software.
Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
Experience building control planes or orchestration systems that model state and reconcile it over time, such as Kubernetes controllers or operators.
Experience designing event-driven systems using message queues, event streams, or pub/sub systems such as Kafka, NATS, or SQS.
Experience building internal platforms or APIs for other engineering teams with attention to developer experience.
Preferred exposure to bare-metal provisioning technologies such as PXE, iPXE, Redfish, IPMI, or BMC; networking fundamentals; or GPU and accelerator infrastructure.
Preferred experience with GPU cluster software stacks such as NCCL, CUDA, and InfiniBand or RoCE.
Prior experience at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization is preferred.
Systems programming experience in Rust or Go is preferred.

Benefits

The role is part of Together AI, a research-driven company focused on open and transparent AI systems and AI infrastructure.
The company offers equal employment opportunity regardless of protected characteristics.

Together AI

Together AI is the AI Native Cloud, purpose-built for AI engineers and researchers with a full suite of tooling across inference, model shaping, and pre-training. AI natives can use Together AI as a full-stack AI platform — from a high- performance inference engine built for reliable and fast sca...

Similar positions

Staff Engineer - Cloud Security

Illumio
Full time

USA
10/30/2026
Salary: $196k - $235k/yr

Sunnyvale, CA

Linux devices software engineer - snapd

Canonical
Full time

Remote
10/30/2026

EMEA, Americas

Software Engineer, Traffic Infrastructure

OpenAI
Full time

USA
10/30/2026
Salary: $266k - $445k/yr

San Francisco, CA

Software Engineer, Mobile Web

Figma
Full time

USA
10/30/2026
Salary: $153k-$376k

Remote

Sr. Software Development Engineer

Zscaler
Full time

Others
10/30/2026

Bengaluru, India