Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Inference Systems Backend Engineer

Home > C/C++ programming jobs

Inference Systems Backend Engineer in Others

  • ByteDance
  • Full time
  • Email
  • Singapore, Singapore

Responsibilities

  • Develop and performance-optimize Volcano Engine large model training and inference systems.
  • Optimize model computation, tune thousand-GPU training clusters, build distributed LLM inference systems, and schedule large-scale inference traffic.
  • Solve high-concurrency, high-reliability, and high-scalability challenges for training and inference traffic at hundreds-of-billions-of-tokens scale.
  • Research and introduce architectures involving subgraph matching, compiler optimization, and model quantization.
  • Integrate GPUs, NPUs, and TPUs with training and inference frameworks.
  • Improve utilization across globally distributed GPU clusters through elastic scheduling, GPU oversubscription, and task orchestration.
  • Collaborate with algorithm teams to jointly optimize algorithms and systems.

Requirements

  • Proficiency in C/C++ and Python development in Linux environments.
  • Experience with large-scale machine learning systems or search, advertising, and recommendation systems.
  • Familiarity with at least one machine learning framework, such as TensorFlow, PyTorch, MXNet, or an in-house framework.
  • Familiarity with at least one large model training or inference framework, including vLLM, TensorRT-LLM, SGLang, or Megatron-LM.
  • Strong independent problem-solving, teamwork, communication, learning, and self-motivation skills.
  • Preferred: experience designing large-scale distributed-system architectures.
  • Preferred: understanding of GPU hardware architecture and software stacks such as CUDA and cuDNN, with GPU performance analysis and optimization experience.
  • Preferred: Master’s or PhD-level research background in distributed systems, parallel computing, programming languages and compilers, networking, or storage systems.

ByteDance

ByteDance is a global incubator of platforms at the cutting edge of commerce, content, entertainment and enterprise services - over 2.5bn people interact with ByteDance products including TikTok. Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This...

Similar positions

Senior Software Engineer

  • Zoox
  • Full time
  • USA
  • 10/30/2026
  • Salary: $235k - $285k/yr
  • Foster City, CA

Security Software Engineer

  • Canonical
  • Full time
  • Remote
  • 10/30/2026
  • Anywhere

Advisory Engineer, Middleware & App

  • Lenovo
  • Full time
  • Others
  • 10/30/2026
  • Shanghai, China

Principal Place and Route Engineer

  • SambaNova Systems
  • Full time
  • USA
  • 10/30/2026
  • Salary: $1 - $1000k/yr
  • Austin, TX

Software Engineer I - Middleware

  • Honeywell
  • Full time
  • USA
  • 10/29/2026
  • Phoenix, AZ

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026