Senior Machine Learning Engineer, Inference

Home > Javascript/Typescript programming jobs

Senior Machine Learning Engineer, Inference in USA new

Inworld
Full time
Email

USA
10/24/2026
Report

Mountain View, CA

Responsibilities

Optimize realtime machine learning inference using modern serving frameworks and model-acceleration techniques.
Profile and improve high-performance inference systems running on NVIDIA GPUs.
Build distributed, multi-GPU and multi-node serving infrastructure capable of handling thousands of concurrent connections.
Use Kubernetes, Ray, and custom load balancing to scale and operate inference workloads reliably.
Take models from research through containerization, serving optimization, and stable production deployment.
Own ambiguous performance, latency, throughput, and reliability problems from investigation through implementation.

Requirements

Deep understanding of inference-serving frameworks and techniques such as vLLM or TRT-LLM.
Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
Proficiency in C++, CUDA, Rust, or highly optimized Python, including the ability to profile and optimize code for NVIDIA GPUs.
Experience with Kubernetes, Ray, custom load balancing, multi-GPU or multi-node inference, and high-concurrency systems.
Non-trivial systems programming projects, major inference-engine open-source contributions, or deep-dive technical write-ups.
PhD in computer science, physics, or mathematics, or equivalent practical experience building backend or machine learning systems.
Ability to work independently in ambiguous environments and take systems from research through reliable production operation.

Benefits

Full-time position with a base salary of $250,000-$300,000 plus bonus, equity, and benefits.
Relocation assistance is offered.
In-person work in the Mountain View office.

Salary: $250k - $300k/yr

Inworld

Inworld builds real-time voice AI and agent infrastructure for developers, including text-to-speech, speech-to-speech, speech recognition, and LLM routing delivered via APIs and SDKs. Customers use it to create interactive characters and agentic experiences in games, apps, and virtual worlds; rev...

Similar positions

Senior/Staff/Principal Software Engineer

Skylight
Full time

Remote
10/26/2026
Salary: $90k-$125k

United States

Senior Software Developer - Applications

Learntastic
Full time

USA
10/26/2026

Remote

Senior Platform Engineer

Clera
Full time

Others
10/26/2026

Munich, Germany

Senior Software Engineer - Full Stack

Bankjoy
Full time

Canada
10/25/2026
Salary: Competitive

Remote

Staff Software Engineer, Site Reliability

Ironclad
Full time

USA
10/25/2026
Salary: $210k - $235k/yr

San Francisco, CA