Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Software Engineer, Data Infrastructure

Home > Python programming jobs

Software Engineer, Data Infrastructure in USA new

  • Thinking Machines Lab
  • Full time
  • Email
  • San Francisco, CA

Responsibilities

  • Design, build, and operate scalable, fault-tolerant distributed compute, data orchestration, and multimodal storage infrastructure for LLM research.
  • Develop high-throughput systems for data ingestion, processing, transformation, training-data catalogs, deduplication, quality checks, and search.
  • Build traceability, reproducibility, and quality-control systems across the data lifecycle.
  • Implement and maintain monitoring and alerting for platform reliability and performance.
  • Collaborate with research teams to improve data quality, unlock features, and accelerate training cycles.
  • Own projects end-to-end and work across teams and technology stacks to deliver infrastructure.

Requirements

  • Bachelor’s degree or equivalent experience in computer science, engineering, or a similar field.
  • Proficiency in at least one backend language, specifically Python or Rust.
  • Fluency with distributed compute frameworks such as Apache Spark or Ray.
  • Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
  • Comfort operating across the stack and owning projects end-to-end.
  • Preferred: hands-on experience with Kafka, dbt, Terraform, and Airflow.
  • Preferred: experience building a web crawler and scaling deduplication, data mining, and search systems.
  • Preferred: strong knowledge of file formats and storage systems, including Parquet and Delta Lake.
  • Preferred: proactive documentation, testing, and development of tools that empower teammates.

Benefits

  • Based in San Francisco, California or New York, New York.
  • Generous health, dental, and vision benefits.
  • Unlimited paid time off.
  • Paid parental leave.
  • Relocation support as needed.

Salary: $300k - $400k/yr

Thinking Machines Lab

Thinking Machines Lab pioneers advanced AI research, blending cutting‑edge theory with practical applications to solve complex problems across industries. Our interdisciplinary team pushes the boundaries of machine learning, neural architecture, and cognitive modeling to deliver innovative soluti...

Similar positions

Lead Data Engineer (P3999)

  • 84.51˚
  • Full time
  • USA
  • 11/18/2026
  • Salary: $125k - $207k/yr
  • Chicago, IL

Software Engineer, Datacenter Server Lifecycle

  • Anthropic
  • Full time
  • USA
  • 11/18/2026
  • Salary: $405k - $485k/yr
  • Remote

Data Engineer II, Data Core

  • Yes Energy
  • Full time
  • USA
  • 11/18/2026
  • Salary: $90k - $136k/yr
  • Boston, MA

Solutions Engineer, AI Build Team

  • Stripe
  • Full time
  • USA
  • 11/18/2026
  • Salary: $240k-$360k
  • Remote

Infrastructure - Intermediate Software Engineer

  • Sony Pictures Imageworks
  • Full time
  • Canada
  • 11/18/2026
  • Vancouver