What it is: The engineer who builds the testing, evaluation, monitoring, and compliance infrastructure that ensures AI systems — particularly GenAI and agentic systems — work correctly, safely, and within regulatory boundaries. This includes everything from designing evaluation harnesses and benchmark suites to implementing model monitoring, drift detection, bias auditing, and the specific compliance controls required by regulations like the EU AI Act.

Why it didn’t exist 18 months ago: In late 2024, AI evaluation was largely ad hoc. Teams ran their models against a few benchmarks, eyeballed the outputs, and called it quality assurance. Governance was a slide in a strategy deck, not a production engineering discipline. The EU AI Act had been passed but enforcement timelines felt distant. Nobody was hiring dedicated “AI evaluation and governance engineers” because most companies hadn’t yet reached the maturity level where evaluation and governance became engineering problems rather than policy discussions.

What changed: Two converging forces created this role almost overnight.

First, agentic AI made evaluation exponentially harder. When an AI system is answering questions in a chatbot, evaluation is relatively straightforward — you can check whether the answers are accurate, relevant, and safe. When an AI system is autonomously executing multi-step workflows, using tools, making decisions, and interacting with real production systems, evaluation becomes a full engineering discipline. You need to test not just individual outputs but entire decision chains. You need to verify that guardrails work under adversarial conditions. You need monitoring systems that can detect when an autonomous agent is drifting outside its intended operational boundaries.

Second, regulation caught up with reality. The EU AI Act’s compliance obligations begin phasing in from August 2026. For any company deploying AI systems in or serving European markets — which includes most global enterprises — the regulatory requirements are specific and demanding: risk assessments, transparency documentation, human oversight mechanisms, data governance frameworks, and ongoing monitoring obligations. These aren’t requirements you can satisfy with a policy document. They require engineering — purpose-built systems that log decisions, explain outputs, track data lineage, and provide audit trails.

The AI evaluation and governance engineer is the person who builds all of this. They design the evaluation frameworks that tell you whether your agentic system is actually working as intended. They build the monitoring infrastructure that catches model drift, performance degradation, and safety violations in real time. They implement the compliance controls that turn regulatory requirements into production code. And they do it in a way that doesn’t destroy the performance or usability of the AI systems they’re governing.

Why everyone’s fighting over them: Because the regulatory clock is ticking and the evaluation challenge is growing faster than most teams can handle. August 2026 is not a distant deadline — it’s four months away. Every enterprise deploying AI in European markets needs compliance-ready systems by then, and most don’t have a single engineer dedicated to AI governance. Meanwhile, every company deploying agentic systems is discovering that their existing testing and monitoring infrastructure was designed for deterministic software, not autonomous AI — and it’s completely inadequate.

The role requires a rare combination: deep AI/ML expertise, understanding of enterprise compliance frameworks, production engineering skills, and the ability to translate legal and regulatory language into technical specifications. The number of people with this combination of skills is — once again — measured in the low thousands globally.