Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Sr. Software Engineer, Training/Inference

Home > C/C++ programming jobs

Sr. Software Engineer, Training/Inference in USA new

  • Palo Alto, CA

Responsibilities

  • Architect and implement distributed model-serving infrastructure for load balancing, auto-scaling, batch scheduling, and global KV caching.
  • Optimize model inference latency, throughput, concurrency, reliability, and tail latency at production scale.
  • Benchmark, fine-tune, and accelerate inference engines through GPU kernel work and code generation.
  • Develop tools to trace, replay, and resolve issues across orchestration and GPU-kernel layers.
  • Build CI/CD infrastructure for endpoint deployment, image publishing, and inference engine updates.
  • Advance research on test-time compute scaling, RL rollout, and model-hardware co-design.

Requirements

  • Bachelor’s degree in computer science, computer engineering, or a science discipline.
  • 5+ years of low-level systems programming experience in C/C++ or Rust.
  • Experience with large-scale, highly concurrent production serving.
  • Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.
  • Strong background in batching, caching, load balancing, parallelism, GPU kernels, and code generation.
  • Experience with quantization, speculative decoding, distillation, low-precision numerics, testing, benchmarking, reliability, and CI/CD for inference systems.

Benefits

  • $200,000–$375,000 base salary, plus equity.
  • Comprehensive medical, vision, and dental coverage.
  • 401(k) retirement plan.
  • Short- and long-term disability insurance and life insurance.
  • Various discounts and perks.

Salary: $200k - $375k/yr

xAI

xAI builds advanced AI systems designed to deeply understand the universe and support humanity's quest for knowledge. Targeting innovators and thinkers, our platform emphasizes engineering excellence and hands-on collaboration, fostering an environment where curiosity drives results.

Similar positions

Sr. Software Engineer, Linux Kernel

  • xAI
  • Full time
  • USA
  • 11/05/2026
  • Salary: $200k - $375k/yr
  • Palo Alto, CA

Principal Software Engineer

  • Microsoft
  • Full time
  • USA
  • 11/05/2026
  • Salary: $143k - $304k/yr
  • Redmond, WA

Senior Software Engineer, Chassis

  • Waymo
  • Full time
  • USA
  • 11/05/2026
  • Salary: $213k - $263k/yr
  • Mountain View, CA

Senior Software Engineer

  • Microsoft
  • Full time
  • USA
  • 11/05/2026
  • Salary: $120k - $261k/yr
  • Redmond, WA

Senior Software Engineer

  • SonicWall
  • Full time
  • Others
  • 11/05/2026
  • Pune, India

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026