Senior Software Engineer - Network Observability

Home > C/C++ programming jobs

Senior Software Engineer - Network Observability in USA

Clockwork.io
Full time
Email

USA
09/11/2026
Report

Palo Alto, CA

Responsibilities

Design, develop, and scale high-performance monitoring platforms for RDMA, RoCE, InfiniBand, and TCP/IP infrastructure.
Build backend telemetry services, observability dashboards, alerts, diagnostics, anomaly detection, SLA monitoring, and traffic analysis workflows.
Troubleshoot production issues across application, operating system, server, RDMA, and network layers.
Optimize low-latency collection, aggregation, and alerting systems.
Collaborate cross-functionally, improve engineering practices, automate operational workflows, and contribute to platform technical direction.

Requirements

Bachelor’s or master’s degree in computer science, computer engineering, electrical engineering, or a related technical field.
Strong hands-on programming experience in C++, Go, Python, Rust, or similar systems programming languages.
Experience delivering complex infrastructure projects and designing and implementing distributed systems.
Experience building distributed systems, backend services, telemetry pipelines, or observability platforms.
Hands-on experience with RDMA, RoCE, InfiniBand, or other high-performance network fabrics.
Familiarity with libibverbs, RDMA verbs, RDMA CM, queue pairs, completion queues, memory registration, and related RDMA concepts.
Strong knowledge of Linux networking, TCP/IP, DNS, HTTP, routing, MTU, congestion control, packet loss, latency, and performance tuning.
Experience with network diagnostics, path discovery, reachability checks, synthetic probes, or active network measurements.
Experience with Prometheus, Grafana, Datadog, Splunk, OpenTelemetry, or similar monitoring and visualization tools.
Strong debugging skills across software, operating system, server, and network layers.
Experience operating production systems in Linux-based environments.
Preferred experience includes AI/ML, HPC, storage, GPU cluster, large-scale RoCE or InfiniBand, NCCL, distributed training, eBPF, XDP, DPDK, perf, tcpdump, Wireshark, ethtool, iproute2, rdma-core, cloud infrastructure, Kubernetes, service discovery, configuration management, infrastructure automation, security, compliance, and time-series telemetry systems.

Benefits

Competitive compensation and a great benefits package.
Catered lunch.
Friendly and inclusive workplace culture.
Challenging projects.
Additional equity participation may include stock options or other equity awards, subject to the company’s equity program and applicable approvals.

Salary: $140k - $210k/yr

Clockwork.io

Clockwork.io pioneers Software-Driven AI Fabrics™, delivering a programmable software layer that makes large-scale AI clusters observable, deterministic, and resilient by design to drive continuous workload progress and peak cluster utilization. Its FleetIQ platform enables enterprises to train, ...

Similar positions

Senior Software Engineer

Zoox
Full time

USA
10/30/2026
Salary: $235k - $285k/yr

Foster City, CA

Security Software Engineer

Canonical
Full time

Remote
10/30/2026

Anywhere

Technical Manager - Automotive and Industrial

Canonical
Full time

Remote
10/30/2026
Salary: Competitive

Anywhere

Advisory Engineer, Middleware & App

Lenovo
Full time

Others
10/30/2026

Shanghai, China

Principal Place and Route Engineer

SambaNova Systems
Full time

USA
10/30/2026
Salary: $1 - $1000k/yr

Austin, TX