Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Inference Systems Backend Engineer

Home > C/C++ programming jobs

Inference Systems Backend Engineer in Others

  • ByteDance
  • Full time
  • Email
  • Singapore, Singapore

Responsibilities

  • Develop and performance-optimize Volcano Engine large model training and inference systems.
  • Optimize model computation, tune thousand-GPU training clusters, build distributed LLM inference systems, and schedule large-scale inference traffic.
  • Solve high-concurrency, high-reliability, and high-scalability challenges for training and inference traffic at hundreds-of-billions-of-tokens scale.
  • Research and introduce architectures involving subgraph matching, compiler optimization, and model quantization.
  • Integrate GPUs, NPUs, and TPUs with training and inference frameworks.
  • Improve utilization across globally distributed GPU clusters through elastic scheduling, GPU oversubscription, and task orchestration.
  • Collaborate with algorithm teams to jointly optimize algorithms and systems.

Requirements

  • Proficiency in C/C++ and Python development in Linux environments.
  • Experience with large-scale machine learning systems or search, advertising, and recommendation systems.
  • Familiarity with at least one machine learning framework, such as TensorFlow, PyTorch, MXNet, or an in-house framework.
  • Familiarity with at least one large model training or inference framework, including vLLM, TensorRT-LLM, SGLang, or Megatron-LM.
  • Strong independent problem-solving, teamwork, communication, learning, and self-motivation skills.
  • Preferred: experience designing large-scale distributed-system architectures.
  • Preferred: understanding of GPU hardware architecture and software stacks such as CUDA and cuDNN, with GPU performance analysis and optimization experience.
  • Preferred: Master’s or PhD-level research background in distributed systems, parallel computing, programming languages and compilers, networking, or storage systems.

ByteDance

ByteDance is a global incubator of platforms at the cutting edge of commerce, content, entertainment and enterprise services - over 2.5bn people interact with ByteDance products including TikTok. Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This...

Similar positions

Senior Software Engineer, Simulation

  • ATOMS
  • Full time
  • USA
  • 09/13/2026
  • Salary: $195k - $246k/yr
  • San Francisco, CA

Firmware Engineer II

  • Varda Space Industries
  • Full time
  • USA
  • 09/13/2026
  • Salary: $144k - $184k/yr
  • El Segundo, CA

Senior Software Engineer, Simulation

  • CloudKitchens
  • Full time
  • USA
  • 09/13/2026
  • Salary: $195k - $246k/yr
  • San Francisco, CA

Software Engineer, Battlespace Awareness

  • Anduril Industries
  • Full time
  • USA
  • 09/13/2026
  • Salary: $191k - $253k/yr
  • Waltham, MA

Software Engineer, Battlespace Awareness

  • Anduril Industries
  • Full time
  • USA
  • 09/13/2026
  • Salary: $165k - $218k/yr
  • Broomfield, CO

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026