Senior QA Automation Engineer

Home > Python programming jobs

Senior QA Automation Engineer in Others

IDT
Full time
Email

Others
09/19/2026
Report

Bogotá, Colombia

Responsibilities

Own the end-to-end test strategy for an AI voice virtual agent across LLM, speech, and telephony layers.
Design and execute complex conversational, call-flow, failure-recovery, and exploratory test scenarios.
Build and maintain evaluation frameworks using golden datasets, semantic or rubric-based assertions, and regression detection.
Adversarially test for hallucinations, incorrect grounding, guardrail bypasses, prompt injection, and unauthorized data disclosure.
Prepare SIP devices and call scenarios, automate call origination, and diagnose issues across LLM, ASR, TTS, media, and routing layers.
Extend automated coverage and integrate test suites into release pipelines as quality gates.
Define quality metrics and communicate release risks using data and documented evidence.
Maintain traceable test coverage in the TMS and participate fully in Scrum ceremonies and team artifacts.

Requirements

5+ years of QA experience with recent hands-on ownership of a complex integrated product.
Practical SIP/VoIP testing experience covering call flows, SIP signaling, media handling, softphones, or SIP trunking, including troubleshooting with SIP traces or packet captures.
Proficiency in Python sufficient to build and maintain automation frameworks independently.
Experience testing non-deterministic or probabilistic systems and designing assertions for variable outputs.
Strong analytical skills for defining quality metrics and supporting release decisions with data.
Hands-on experience with formal TMS tools and Jira.
Professional fluency in spoken and written English with reliable overlap with US Eastern business hours.
Preferred experience includes LLM or conversational AI testing, ASR and TTS evaluation, RAG systems, WebRTC and media-layer testing, IVR or contact-center platforms, multilingual testing, and prompt engineering literacy.

Benefits

Remote work opportunity for candidates in LATAM.
B2B employment arrangement with gross compensation.
Long-term growth perspective, annual performance review, and continuous learning opportunities.
Company support for professional training, seminars, conferences, and English classes.
Good hardware, referral program, and work with a talented team.

IDT

IDT delivers innovative technology solutions that empower businesses to streamline operations and accelerate growth. By combining cutting‑edge software with expert consulting, we help clients unlock new efficiencies and stay ahead of market shifts. Our collaborative approach ensures every project...

Similar positions

Senior, Machine Learning Engineer

Torc Robotics
Full time

USA
09/22/2026
Salary: $177k - $213k/yr

Remote

Senior Software Engineer

Commonwealth Bank of Australia
Full time

Australia
09/22/2026

Sydney

Deployed Engineer, Professional Services

LangChain
Full time

Remote
09/22/2026
Salary: Competitive

APAC

Senior AI Engineer

Jeeves
Full time

Remote
09/22/2026

Brazil

Senior Product Security Engineer

AlphaSense
Full time

USA
09/22/2026
Salary: $182k - $228k/yr

Remote