Your browser does not support javascript! Please enable it, otherwise web will not work for you.

Software Engineer, Agent Platform

Home > Other programming jobs

Software Engineer, Agent Platform in USA

  • San Francisco, CA

Responsibilities

  • Own AI-powered feature behavior across product surfaces, including quality, safety, variance, and failure modes.
  • Design prompting, retrieval, routing, and tool-use strategies for non-deterministic systems.
  • Build and maintain evaluation systems using statistical signals, distributions, trends, golden datasets, regression detection, and human feedback.
  • Detect and resolve hallucinations, partial correctness, instruction drift, and context sensitivity.
  • Implement guardrails, fallbacks, and degradation paths for unexpected model behavior.
  • Partner with product and infrastructure teams on quality, cost, latency, robustness, and shipping decisions.
  • Influence model selection, model behavior, and tool design for real user workflows.
  • Set quality standards and mentor engineers on non-deterministic system design and evaluation.

Requirements

  • 6+ years of professional engineering experience with ownership of complex production systems.
  • Experience owning AI/LLM behavior beyond basic integration, including variance and failure-mode mitigation.
  • Ability to reason about probabilistic systems and quality, cost, recall, precision, speed, and robustness tradeoffs.
  • Experience designing or maintaining evaluation frameworks, golden datasets, regression detection, or human-in-the-loop feedback loops.
  • Strong product intuition and ability to define quality for non-deterministic outputs.
  • Ability to work independently in ambiguous problem spaces and establish standards others rely on.
  • Bonus: production experience with RAG, agentic systems, or tool-using models.
  • Bonus: familiarity with vector databases, embeddings, or retrieval pipelines.
  • Bonus: exposure to fine-tuning, model routing, or post-training techniques.
  • Bonus: experience building shared AI infrastructure or mentoring engineers.

Retool

Retool builds a developer platform for quickly creating internal apps, workflows, and automations by combining visual components with code and connecting to existing databases and APIs. It sells cloud and self‑hosted enterprise subscriptions with governance, security, and permissions features for...

Similar positions

Associate Software Development Engineer - R0157139

  • Brillio
  • Full time
  • Others
  • 09/22/2026
  • Bengaluru, India

Custom Software Engineering Lead

  • Accenture
  • Full time
  • Others
  • 09/22/2026
  • Hyderābād, India

Platform Engineer

  • Catawiki
  • Full time
  • Others
  • 09/22/2026
  • Amsterdam, Netherlands

Principal Software Development Engineer

  • Autodesk
  • Full time
  • Remote
  • 09/22/2026
  • Salary: Competitive
  • Mexico

Custom Software Engineer

  • Accenture
  • Full time
  • Others
  • 09/22/2026
  • Hyderābād, India

Listings

by


Job Type

Language

Country

CodingJobBoard.com © 2026