Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Staff Software Engineer, Inference

Home > Golang programming jobs

Staff Software Engineer, Inference in Others new

  • Together AI
  • Full time
  • Email
  • Amsterdam, Netherlands

Responsibilities

  • Design and implement a versioned provisioning state machine covering physical-host discovery, inference bring-up, GPU driver and CUDA setup, health validation, and decommissioning or RMA.
  • Build declarative self-service APIs and a Kubernetes-native control plane for requesting, scaling, and tearing down inference clusters.
  • Automate detection, draining, repair, replacement, and reintegration of degraded or failed nodes.
  • Ensure production reliability through idempotency, retries, rollback, drift detection, testing, code review, versioning, and CI/CD.
  • Partner with the inference and ML platform teams to encode topology, interconnect, and scheduling requirements as platform abstractions.
  • Develop defragmentation, rebalancing, scheduling, bin-packing, health, remediation, and utilization systems for efficient GPU capacity.

Requirements

  • Strong software engineering experience using Go, Python, Rust, or a similar language.
  • Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
  • Experience building software control planes or orchestration systems using state modeling and reconciliation, such as Kubernetes controllers, operators, custom reconciliation loops, or workflow engines.
  • Experience designing event-driven systems using message queues, event streams, or publish/subscribe systems rather than polling or cron-driven scripts.
  • Experience building internal platforms or APIs for other engineering teams with attention to developer experience.
  • Preferred experience with bare-metal provisioning technologies such as PXE, iPXE, Redfish, IPMI, or BMC.
  • Preferred networking fundamentals including VLANs, BGP, or fabric design, and/or GPU or accelerator infrastructure.
  • Preferred experience with GPU cluster software stacks including NCCL, CUDA, and InfiniBand or RoCE.
  • Experience at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization is preferred.
  • Systems programming experience in Rust or Go is preferred.

Together AI

Together AI is the AI Native Cloud, purpose-built for AI engineers and researchers with a full suite of tooling across inference, model shaping, and pre-training. AI natives can use Together AI as a full-stack AI platform — from a high- performance inference engine built for reliable and fast sca...

Similar positions

Staff Software Engineer, PostgreSQL Specialist

  • ID.me
  • Full time
  • USA
  • 10/26/2026
  • Salary: $174k - $253k/yr
  • Mountain View, CA

Sr. Software Engineer, Cloud Platform

  • Samsara
  • Full time
  • UK
  • 10/26/2026
  • Remote

Software Engineer L3

  • Twilio
  • Full time
  • USA
  • 10/26/2026
  • Salary: $139k-$204k
  • Remote

Fullstack Software Engineer, Notifications Lifecyc

  • Reddit
  • Full time
  • USA
  • 10/26/2026
  • Salary: $164k-$229k
  • Remote

Software Engineer, Ads

  • Reddit
  • Full time
  • Canada
  • 10/26/2026
  • Remote

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026