Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior Junior AI Inference Reliability

Home > Javascript/Typescript programming jobs

Senior Junior AI Inference Reliability in USA

  • Together AI
  • Full time
  • Email
  • San Francisco, CA

Responsibilities

  • Operate and improve the production provisioning state machine across host discovery, inference bring-up, GPU driver/CUDA validation, health checks, and decommissioning.
  • Diagnose production failures involving InfiniBand/RDMA, GPU Xid variants, model-serving faults, Kubernetes controllers, and fleet-scale state transitions.
  • Extend automated self-healing through new detection and remediation plugins.
  • Own provisioning and fleet-health SLOs and error budgets, lead incident response, and drive systemic fixes for idempotency, retries, rollbacks, and drift.
  • Partner with inference and ML platform teams to incorporate real-world cluster, interconnect, and scheduling failure modes into platform requirements.
  • Build tools and architectural improvements for inference performance, latency, throughput, capacity efficiency, observability, and stability.
  • Lead cross-functional reliability priorities across infrastructure, ML platform, and product teams.

Requirements

  • Production experience operating provisioning or host-lifecycle pipelines at fleet scale.
  • Deep production-reliability experience with large-scale distributed systems; GPU or ML-serving infrastructure experience is preferred.
  • Hands-on experience troubleshooting InfiniBand/RDMA issues, including IB port counters and subnet manager logs.
  • Familiarity with multi-tenant model serving, LoRA or adapter-based serving, and differentiated SLOs across model pools.
  • A track record of eliminating operational toil through automation and self-built tooling.
  • Experience with Kubernetes controller and CRD patterns, reconciliation loops, and node-level health or remediation systems.
  • Comfort with event-driven architectures, backpressure, circuit breaking, and built-in metrics, logs, and traces.
  • Strong software-engineering skills and experience shipping production code.

Benefits

  • Competitive compensation, startup equity, health insurance, and other competitive benefits.
  • Full-time position with a US base salary range of $160,000-$280,000 plus equity and benefits.

Salary: $160k - $280k/yr

Together AI

Together AI is the AI Native Cloud, purpose-built for AI engineers and researchers with a full suite of tooling across inference, model shaping, and pre-training. AI natives can use Together AI as a full-stack AI platform — from a high- performance inference engine built for reliable and fast sca...

Similar positions

Junior Web Developer

  • Storyteller
  • Contract
  • Remote
  • 10/30/2026
  • Salary: €23k
  • Romania

Senior Software Engineer - UI

  • AIM
  • Full time
  • USA
  • 10/30/2026
  • Salary: Competitive
  • Seattle, WA

Junior Web Developer

  • Storyteller
  • Contract
  • Remote
  • 10/29/2026
  • Bulgaria

Junior Web Developer

  • Storyteller
  • Contract
  • Remote
  • 10/29/2026
  • Hungary

Senior Front End Developer

  • Fidelity
  • Full time
  • Others
  • 10/29/2026
  • Bengaluru, India

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026