Staff Backend Engineer - Grafana Second Horizon

The Opportunity

At Grafana Labs, we build observability tools that help users understand, respond to, and improve their systems – regardless of scale, complexity, or tech stack. We recently started a skunkworks initiative with a mission to bring observability to the rest of the business. Our goal is to make Grafana the single best place where humans and AI agents understand and act on real-time data from across the enterprise.

As part of this initiative, we are building an AI-native data intelligence system that gives agents reliable, governed access to enterprise context. This system helps agents retrieve the right data, metadata, definitions, lineage, quality signals, and institutional knowledge without every agent builder having to maintain their own brittle context files.

We’re looking for a Staff-level Backend Engineer to help build the first production services for this context layer management system. You will design and ship the core backend architecture that powers ingestion, context storage, retrieval APIs, and agent-facing integrations.

What You'll Be Doing

• Build core backend services: Design, implement, test, and operate the first services for context ingestion, context indexing, retrieval orchestration, API access, source configuration, and system administration.
• Create a scalable SaaS foundation: Help define and build the architecture for a multi-tenant service, including tenant isolation, usage tracking, quotas, audit logs, background jobs, and reliable service boundaries.
• Power agent-facing retrieval workflows: Build APIs and service interfaces that allow AI agents, MCP tools, CLIs, and internal applications to retrieve relevant context, provenance, confidence signals, and warnings.
• Work across product and infrastructure: Partner with the team to make practical tradeoffs between fast experimentation and long-term reliability.
• Operate what you build: Instrument services with metrics, logs, traces, alerts, and dashboards.
• Contribute to technical direction: Help shape the architecture, service boundaries, storage choices, API contracts, deployment patterns, and engineering practices.
• Ownership and impact: Take full ownership of the AI solutions you develop, ensuring they are scalable, maintainable, and aligned with real user workflows.

What Makes You a Great Fit

• Solid experience building production-grade, user-facing software systems with minimal supervision.
• Familiarity with AI technologies, LLMs, prompt engineering, and building applications powered by GenAI.
• Comfortable releasing prototypes, collecting feedback, and iterating with a pragmatic mindset.
• Proven ability to deal with ambiguity and define scope where things are loosely defined.
• Experience working in cloud-native environments (e.g., AWS, GCP, Azure).
• Experience using observability tools to understand and troubleshoot system behavior.

Bonus Points For

• Experience building or working with agent frameworks or multi-agent workflows.
• Experience as a data analyst or working with data platforms (e.g., Looker, Tableau, PowerBI, Snowflake, DataBricks).
• Experience building tools for data engineering.