Staff AI Platform Engineer

Home > Python programming jobs

Staff AI Platform Engineer in USA

Code Metal
Full time
Email

USA
10/07/2026
Report

Remote

Responsibilities

Set the technical direction and architecture for the AI platform, lead the four-person team, own design documents and RFCs, guide build-versus-buy decisions, and mentor engineers.
Deploy, benchmark, and tune production inference for open-weight models on vLLM, SGLang, and TensorRT-LLM.
Own the model gateway for self-hosted and commercial models, including authentication, routing, failover, quotas, and cost attribution.
Build reusable agent harnesses and orchestration primitives for reliable and verifiable workflows.
Develop context-engineering services for agent memory, retrieval, and data discovery.
Instrument services with OpenTelemetry traces and metrics and build experiment-tracking and artifact infrastructure.
Design the platform for multi-tenancy, versioned APIs, security, customer deployments, and air-gapped environments.
Partner with Applied AI Research, product teams, and DevOps to align the platform with organizational needs.

Requirements

Production-grade Python and strong platform engineering fundamentals, including API and service design, distributed systems, containers, Kubernetes, CI/CD, and testing.
Experience shipping production agentic systems and improving their reliability.
Experience with retrieval-augmented generation, embeddings, vector or hybrid search, and agent memory, ideally involving code or large technical corpora.
Experience instrumenting services with tracing and metrics and operating AI services against SLOs.
Data science and AI research fundamentals covering transformers, LLM inference, experiment design, benchmarking, and model evaluation.
Working familiarity with PyTorch and Hugging Face, plus experience fine-tuning, evaluating, or serving language models.
Staff-level technical leadership across multiple systems or teams, including architecture ownership, design documents and RFCs, roadmap development, and mentoring.
Typically 8+ years of software engineering experience, including 4+ years building and operating ML or LLM systems in production; equivalent depth of experience is valued.
Preferred experience with LLM gateways or proxies such as SMG or Bifrost, or equivalent API gateway development.
Preferred experience optimizing inference through speculative decoding, prefix caching, tensor/pipeline/expert parallelism, disaggregated prefill and decode, and GPU profiling.
Preferred experience building LLM and agent evaluation harnesses and experiment-tracking or artifact systems such as MLflow or Weights & Biases.
Preferred experience taking internal platforms to external products, deploying AI systems in on-premises, air-gapped, classified, regulated, defense, or aerospace environments.

Benefits

Health care plan with 100% premium coverage, including medical, dental, and vision.
401(k) with 5% matching.
Uncapped vacation, sick leave, and public holidays.
Flexible hybrid or remote work arrangement.
Relocation assistance for qualifying employees.
US citizenship may be required for certain security-clearance project assignments.

Salary: $205k - $250k/yr

Code Metal

Code Metal builds an AI-assisted code transpilation and verification platform that translates algorithms into optimized implementations for heterogeneous hardware (GPUs, FPGAs, ASICs, and edge SoCs). It serves defense, automotive, aerospace, and semiconductor teams deploying DSP, RF, communicatio...

Similar positions

Staff Machine Learning Engineer

Altana
Full time

USA
10/15/2026
Salary: $205k - $240k/yr

Boston, MA

Security Platform Automation Engineer

Apple
Full time

USA
10/15/2026

Seattle, WA

Analytics Engineer

Reddit
Full time

USA
10/15/2026
Salary: $164k-$230k

Remote

Senior AI Engineer

Infiterra S.A
Full time

Remote
10/15/2026

Greece, UK

Security Operations Engineer (SecOps)

Collibra
Full time

USA
10/15/2026
Salary: $116k - $145k/yr

Remote