Senior AI Product Engineer, Backend - Arize AI

About Arize

AI is rapidly transforming the world. As generative AI reshapes industries, teams need powerful ways to monitor, troubleshoot, and optimize their AI systems. Arize AI is the leading AI & Agent Engineering observability and evaluation platform, empowering AI engineers to ship high-performing, reliable agents and applications. From first prototype to production scale, Arize AX unifies build, test, and run in a single workspace so teams can ship faster with confidence.

We are a Series C company backed by top-tier investors, with over $135M in funding and a rapidly growing customer base of 150+ leading enterprises and Fortune 500 companies, including Booking.com, Uber, Siemens, and PepsiCo.

The Opportunity

Our Backend Engineering team builds all of the highly scalable distributed services that power Arize’s ML observability platform. While Go is our primary language for these distributed systems, the team also maintains services and tools written in Python, Java, and TypeScript. The scope of every individual on this team is high, whether it’s finding the most efficient way to compute model evaluation metrics across billions of data points, designing the next generation of our OLAP database architecture, or researching and implementing the latest dimensionality reduction techniques.

You will be part of the core team that drives product innovation at Arize. You will be challenged with understanding how some of the most impactful engineering teams are developing AI and LLM-powered applications, and building the right tools to enable them to do their best work. Our product solutions range from clean APIs that magically instrument applications, interactive playgrounds for prompt engineering and agent development, to scaling up real-time evaluation infrastructure handling millions of annotations per second.

What You’ll Do

• Write maintainable, scalable, and performant backend code primarily in Go, Java, and Python, with opportunities to work in TypeScript.
• Build high-volume and highly available analytics systems.
• Design and build APIs specific to our customers’ Machine Learning and LLM workflows.
• Prototype, optimize, and maintain scalable backend services that power the Arize core platform.
• Extend and contribute back to open-source OLAP databases and distributed message queue frameworks.
• Develop and integrate collection tools for robust monitoring of ML and LLM pipelines.
• Research and implement cutting-edge visualization and dimensionality reduction algorithms in a distributed environment.
• Collaborate with product, design, and customer engineering teams to enhance and expand product offerings.
• Contribute to building our in-house AI Agents.

What We’re Looking For

• 5+ years of experience working with high-performance backend systems.
• Strong experience writing Go, Python, TypeScript/Node, Java, or similar server programming languages.
• Enthusiasm and interest in the AI and LLM ecosystem, with a desire to learn and stay updated on emerging technologies.
• Previous experience building and operating highly complex SaaS platforms/systems.
• Knowledge of working with public clouds and container orchestration (AWS, GCP, Azure, Kubernetes, etc.).

Bonus Points

• Experience with distributed stream processing (Kafka, Gazette, or similar).
• Experience with OLAP systems.
• Familiarity with system observability tooling like Prometheus.
• Working knowledge of Machine Learning and/or Data Science.
• First-hand experience working with large language models (LLMs) or developing AI products.

Compensation & Benefits

• Estimated Salary Range: $125,000 - $225,000 per year
• Competitive equity package
• Comprehensive health, dental, and vision insurance
• 401(k) plan
• Unlimited paid time off (PTO)
• Generous parental leave plan
• Mental health and wellness support
• Work-from-home monthly stipend for co-working spaces (or optional access to NYC and SF Bay Area offices)

Diversity & Inclusion

Arize’s mission is to make the world’s AI work—and work for people. We actively cultivate an inclusive work environment through regular discussions with ethicists and researchers, culturally conscious events, and dedicated internal resource groups like Lady Arizers.