Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Senior ML Infrastructure Engineer

Home > Python programming jobs

Senior ML Infrastructure Engineer in USA

  • SambaNova Systems
  • Full time
  • Email
  • Austin, TX

Responsibilities

  • Design and productionize advanced inference techniques on RDU for performance and cost optimization.
  • Develop capabilities involving speculative decoding, constrained decoding, function/tool calling, prompt caching, and long-context inference.
  • Own SambaNova’s integration with vLLM and adjacent serving frameworks, adapting them to RDU architecture.
  • Own the public inference API surface exposed through SambaStack and SambaCloud.
  • Build and maintain accuracy verification and regression infrastructure for inference features.
  • Partner with ML, compiler, runtime, and product teams to take features from prototype to production.
  • Contribute to technical designs, code reviews, and architectural decisions as a senior individual contributor.

Requirements

  • Bachelor’s degree in Computer Science, Electrical Engineering, or a related field.
  • 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving.
  • Strong fundamentals in algorithms, data structures, concurrency, and systems design.
  • Experience designing and maintaining production services with strict latency, throughput, and availability requirements.
  • Working knowledge of modern LLM inference techniques.
  • Familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang.
  • Proficiency in Python.
  • Experience collaborating across teams to deliver complex, system-level engineering solutions.

Benefits

  • Base salary range of $200,000–$275,000 USD, plus equity for US-based full-time employment positions.
  • SambaNova covers 95% of employee medical insurance premiums and 77% of dependent premiums, with a Health Savings Account and employer contribution.
  • Dental, vision, short- and long-term disability, basic and voluntary life, AD&D, and flexible spending account options are offered.
  • Well-being benefits include Headspace, Gympass+ access, One Medical membership, counseling services, and an Employee Assistance Program.

Salary: $200k - $275k/yr

SambaNova Systems

Welcome to SambaNova: Revolutionizing AI Capacity At SambaNova, we're empowering developers, enterprises, governments, and data centers to unlock their full AI potential. Our full-stack infrastructure, from chips to models, enables lightning-fast performance, low power consumption, and high-effic...

Similar positions

Backend Engineer (Europe)

  • Kalepa
  • Full time
  • Remote
  • 10/26/2026
  • Salary: $80k-$105k
  • Europe

Sr. AI/ML Engineer II

  • MetroStar
  • Full time
  • USA
  • 10/26/2026
  • Salary: $200k - $240k/yr
  • Reston, VA

Staff Security Engineer

  • Quince
  • Full time
  • Others
  • 10/26/2026
  • Bengaluru, India

ChatGPT Performance Engineer

  • OpenAI
  • Full time
  • USA
  • 10/26/2026
  • Salary: $325k-$405k
  • Remote

Junior Backend Engineer (Europe)

  • Kalepa
  • Full time
  • Remote
  • 10/25/2026
  • Salary: $60k-$70k
  • Europe

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026