Senior Backend Software Engineer - Loki

About Grafana Labs

Grafana Labs is the company behind Grafana Cloud, the fully managed observability platform trusted by more than 10,000 organizations to ensure reliability, resolve incidents faster, and optimize telemetry at scale. Built on open source and open standards and designed for interoperability across any stack, Grafana Cloud brings AI to observability and observability to AI, giving teams unified visibility so they can see, understand, and act on all their disparate data.

We are a 100% remote company with team members across 40+ countries, backed by leading investors including Lightspeed Venture Partners, Sequoia Capital, GIC, Coatue, J.P. Morgan, CapitalG, and Lead Edge Capital.

This is a remote position. We are looking for candidates in the US and Canada.

About the Role & Team

Grafana Cloud is our composable observability platform that integrates metrics, logs, traces, and profiles with Grafana. It allows customers to leverage open source observability software—including Prometheus, Mimir, Loki, Tempo, and Pyroscope—without the overhead of maintaining and scaling their own stack.

The Databases team owns and operates multi-tenant distributed telemetry databases implemented in Go and operating at scale on Kubernetes across all major cloud providers (AWS, GCP, Azure).

What You Will Be Doing

• Take an active role in influencing the roadmap and technical direction
• Work with your team to deliver new features, iterate, and continuously improve
• Drive projects from initial concept through to production operations
• Embrace open-source culture and contribute to OSS projects
• Design, build, operate, and maintain critical distributed systems, owning reliability, performance, and availability
• Participate in team follow-the-sun on-call rotations and take operational ownership of services
• Support team members through design reviews, PR reviews, and technical mentorship

What We Are Looking For

• Pragmatism: Ability to tackle complex challenges, break them down into short feedback loops, build modular solutions, deliver MVPs, and iterate
• Collaboration & Communication: Strong communication skills within a fully remote, distributed squad setup
• Backend Engineering: Solid experience with at least one programming language (Go preferred, or Python, C, C++, Rust)
• End-to-End Ownership: Experience delivering projects from requirements gathering to shipping and running in production
• Cloud & Systems: Experience developing software that runs in the Cloud or hands-on systems engineering experience
• DevOps Mindset: Familiarity with on-call duties, operational ownership, and DevOps practices
• Code Quality: Track record of writing clean, robust, performant, and maintainable software
• Observability: Familiarity with metrics, logs, and traces to debug complex production issues

Nice to Haves

• Experience with Kubernetes
• Experience with queue/streaming systems (e.g., Kafka protocol)
• Prior experience using Grafana and Prometheus in operational or homelab settings
• Exposure to microservices architecture and large-scale distributed systems
• Familiarity with Infrastructure as Code (IaC)

Why You’ll Thrive at Grafana Labs

• 100% Remote, Global Culture: United by collaboration and shared purpose across 40+ countries
• Work-Life Balance: Global annual leave policy of 30 days per year (including 3 company shutdown days)
• Growth & Trust: High-trust, low-ego environment with clear career development pathways
• In-Person Onboarding: Cohort onboarding to connect with fellow Grafanistas from day one