Senior Software Engineer II - LLM Observability

What We’re Building

Honeycomb is defining observability and raising expectations of what developer tools can do. We work with well-known companies like HelloFresh, Slack, LaunchDarkly, and Vanguard across a range of industries. We’ve closed Series D funding, scaled past 200 people, and were named to Forbes’ America’s Best Startups of 2022 and 2023.

Who We Are & How We Work

We are a fully distributed company. We believe it is not where you sit, but how you deliver that matters most. We invest in our people, care about our culture, and imbue trust, autonomy, and accountability from Day 1.

About the LLM Observability Team

The LLM Observability (LLM O11y) team builds the foundational layer that makes AI workloads observable in Honeycomb. We define how AI systems are represented in telemetry (spans, traces, token usage, cost, eval signals) and ensure data from diverse ecosystems (e.g., LangChain, OpenLLMetry, Braintrust) is normalized into a consistent OpenTelemetry-based model.

This team owns the infrastructure and data model that all downstream AI observability experiences rely on, powering everything from cost analysis to anomaly detection.

What You'll Do

• Design, build, and deliver backend systems and APIs: Architect and implement APIs and backend services in Go, emphasizing clarity, maintainability, cost efficiency, scale, and backward compatibility.
• Build and maintain full-stack product features: Contribute frontend (React/TypeScript) and backend (Go) code to deliver impactful product features across the stack.
• Provide technical leadership: Define patterns and operational guidelines, lead complex multi-team projects, and mentor teammates. Tech lead responsibilities rotate with each feature to give everyone growth opportunities.
• Collaborate across disciplines: Partner within a product triad model (Design, Product, Engineering) from discovery through delivery, iterating quickly based on user feedback.
• Support and own software in production: Participate in on-call rotations to ensure reliable operations and address user pain points proactively.
• Communicate clearly and multiply impact: Mentor engineers, share knowledge, and foster team capability through clear written and verbal communication.

What You'll Bring

• Deep backend engineering experience: Substantial expertise building APIs and backend services in Go (or similar languages), with strong opinions on API design, error handling, and developer experience.
• Frontend proficiency: Professional experience delivering frontend features in React and TypeScript across the full development lifecycle.
• Technical leadership: Track record of planning and delivering complex projects spanning multiple teams or systems, balancing technical debt with business goals.
• Ownership and autonomy: Proactive approach to identifying problems, proposing solutions, debugging full-stack issues, and owning projects end-to-end.
• Mentorship: A history of mentoring engineers, conducting technical design reviews, and raising the quality bar for your team.
• Collaborative mindset: Comfort working cross-functionally and building small changes that iterate based on customer feedback.
• Production experience: Hands-on experience supporting software in production and participating in on-call rotations.
• Nice to have: Proficiency with or interest in AI-assisted development tools (Copilot, Claude Code, Cursor, etc.).

Pay & Benefits

• Base Salary: $183,340 — $206,000 USD
• Equity: Equity stake with employee-friendly stock program
• Time Off: Unlimited PTO
• Remote Setup: Home office, coworking, and internet stipends
• Health & Family: Full health benefits coverage for employees, plus up to 16 weeks of paid parental leave
• Growth: Annual professional development allowance