Staff Software Engineer - Databases, Tempo

The Opportunity

We build Tempo, the open-source distributed tracing backend behind Grafana Cloud Traces and Grafana Enterprise Traces (GET). As we shift from foundational work to product and operational excellence, we are evolving Tempo into a platform that powers Grafana’s next generation of observability products, including App Observability, Asserts, and AI-driven assistants.

What You'll Be Doing

As a Staff Engineer on Tempo, you will set technical direction on the hardest problems in our roadmap:

• Lead multi-quarter technical initiatives: Drive projects from problem framing through rollout, such as trace aggregation APIs and query engine improvements.
• Own the architecture: Manage core Tempo components (ingestion, storage, query, metrics generation) and drive design reviews.
• Design APIs for humans and agents: Shape the next generation of Tempo’s interfaces to support LLM-driven assistants and external integrators.
• Drive operational excellence: Own outcomes against SLOs (P99 write latency, incident recurrence, TCO) and push for Zero Ops through automation.
• Partner with cross-functional teams: Collaborate with PMs and sibling teams (App Observability, Asserts, Grafana Assistant) to unblock product development.
• Mentor and lead: Raise the engineering bar through code review, design feedback, and technical writing.
• Contribute to open source: Engage the community and steer the project in the open.

What Makes You a Great Fit

• Technical leadership: A track record of leading complex, multi-quarter initiatives.
• Deep systems experience: Substantial hands-on experience building and operating distributed data systems.
• Strong software craftsmanship: Ability to write clean, robust, performant software.
• Go proficiency: Deep experience in Go or other systems languages (Rust, C, C++).
• Operational mindset: Experience owning production services and treating SLOs as a product feature.
• Customer focus: Pragmatic approach to breaking complex problems into short feedback loops.

Bonus Points

• Experience with tracing, OpenTelemetry, or large-scale observability systems.
• Experience designing query languages or APIs for programmatic consumption.
• Experience with columnar storage formats (e.g., Parquet).
• Experience operating multi-tenant, multi-cell SaaS infrastructure on Kubernetes.
• Experience building for AI/LLM consumers.

How We Work

We are a remote-first team that values creativity, diverse perspectives, and clear communication. We rely on asynchronous, written collaboration and video meetings to stay aligned.