Principal AI Engineer - Context - Agents and Context

Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale. The Elastic Search AI Platform brings together the precision of search and the intelligence of AI to help organizations deliver on the promise of AI.

What is The Role

The Context Engine team builds the knowledge layer that AI agents use to work with enterprise data in Elasticsearch. We extract knowledge from any data sources into a structured AI Index, serve it to agents through public APIs, MCP tools and framework integrations, and close the loop with agent traces so that what the engine knows improves from real usage.

As a Principal AI Engineer, you own the improvement loop of this product end to end: how agents, automations and skills behave in production, how we observe them, how we evaluate them, and how we ship changes to them safely. This is a hybrid/distributed role at the intersection of engineering, data science, and product. You will write production code, design evaluation and telemetry, and set the technical bar for how the team iterates on agentic behaviour.

The codebase is TypeScript and we build it in the open, so you'll be shipping code, designs, and discussions in public alongside the rest of the Elastic Stack.

What You Will Be Doing

• Own the production improvement loop for Context Engine: understand how extraction automations, retrieval tools and memory behave based on offline evaluations, customer conversations, and telemetry.
• Define how we iterate on agents and skills safely: versioning and rollout of prompts, regression coverage, staged and shadow evaluation, and guardrails.
• Design the telemetry needed for data-informed engineering decisions: what to capture from agent traces, tool calls, and knowledge retrieval.
• Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency, and cost.
• Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and write technical proposals.

What You Bring

• 10+ years of software engineering experience, with recent years spent shipping and operating AI-driven products on real production traffic.
• A track record of eval-driven product improvement: diagnosed agent/LLM behavior, designed evaluations, shipped fixes, and measured outcomes.
• Direct experience building agents with state and memory, and iterating on prompts, skills, and tool behavior safely in production.
• Familiarity with MCP, including exposing public MCP servers and tools.
• Experience designing telemetry for AI systems and running product experiments end-to-end.
• Strong backend engineering skills in either Python or TypeScript: APIs, stateful workflows, data pipelines, and production services.
• Comfort working with data scientists, engineers, and product managers as peers.
• A pragmatic, low-ego style suited to a distributed, async-first team.

Bonus Points

• Experience with agent frameworks such as LangGraph, CrewAI, Claude Agent SDK, or similar.
• Agentic retrieval experience, including knowledge representation.
• Practical Elasticsearch experience.