Systems Architect AI/ML Infrastructure - Deepgram

Company Overview

Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More than 200,000 developers and 1,300+ organizations build voice offerings that are 'Powered by Deepgram', including Twilio, Cloudflare, Sierra, Decagon, Vapi, Daily, Cresta, Granola, and Jack in the Box.

Deepgram's voice-native foundation models are accessed through cloud APIs or as self-hosted and on-premises software, with unmatched accuracy, low latency, and cost efficiency. Backed by a Series C led by global investors, Deepgram has processed over 50,000 years of audio and transcribed more than 1 trillion words.

Company Operating Rhythm

At Deepgram, we expect an AI-first mindset—AI use and comfort are core to how we operate, innovate, and measure performance. Every team member is expected to actively use, experiment with, and integrate advanced AI tools into everyday workflows.

We move at the pace of AI. Change is rapid, and day-to-day work evolves quickly. Success here requires adaptability, rapid learning, and an eagerness to experiment rather than seeking a prescriptive 9-to-5 role.

The Opportunity

Deepgram's infrastructure spans bare metal GPU clusters, multi-cloud deployments, and global edge presence—all serving real-time voice AI at scale while powering large-scale model training.

As a Systems Architect, you will own the end-to-end infrastructure architecture. You will design compute, storage, and networking systems for production inference and research training workloads, build multi-cloud strategies balancing performance with cost, and create elastic infrastructure scaling with Deepgram's demands. This is a senior technical leadership role where your architectural decisions shape our underlying foundation.

What You'll Do

• Define and drive end-to-end infrastructure architecture for Deepgram's AI/ML workloads across production inference and research training.
• Design multi-cloud and hybrid infrastructure strategies balancing performance, reliability, cost, and vendor flexibility.
• Architect compute orchestration systems to efficiently schedule and manage GPU and CPU workloads across heterogeneous infrastructure.
• Design storage architectures handling massive audio ML datasets, from high-throughput training pipelines to low-latency model serving.
• Lead capacity planning across infrastructure dimensions, modeling growth to scale ahead of demand.
• Drive cost optimization and FinOps practices to reduce infrastructure spend without compromising performance or reliability.
• Design burstable, elastic training infrastructure scaling up for large training runs and down to minimize idle costs.
• Architect research compute infrastructure giving ML teams required resources while maintaining operational efficiency.
• Establish architectural standards, design review processes, and technical documentation practices.
• Collaborate with engineering leadership to align infrastructure strategy with product roadmaps and business objectives.
• Evaluate emerging hardware, cloud services, and infrastructure technologies for adoption.

You'll Love This Role If You

• Think in systems and see connections between compute, storage, and networking under load.
• Are motivated by designing systems at the intersection of real-time production serving and large-scale ML training.
• Enjoy architectural trade-offs across cost, performance, reliability, and velocity.
• Want to work across the full stack—from bare metal and GPUs to cloud services and container orchestration.
• Thrive when operating strategically while staying technically deep enough to validate designs and debug complex issues.

It's Important To Us That You Have

• 7+ years of experience in infrastructure engineering, systems architecture, or a senior technical role focused on large-scale infrastructure.
• Proven experience designing multi-cloud architectures spanning AWS and at least one other major cloud provider or on-premises environment.
• Deep expertise in storage system design (block, object, and file storage), including performance tuning for large-scale data workloads.
• Strong experience with compute orchestration using Kubernetes and efficient workload scheduling.
• Hands-on experience with GPU infrastructure—procurement considerations, cluster design, driver, and runtime management.
• Track record of capacity planning and infrastructure scaling for high-growth environments.
• Ability to communicate complex architectural decisions clearly to technical and non-technical stakeholders.
• Strong understanding of networking fundamentals as they relate to infrastructure architecture.

It Would Be Great If You Had

• Direct experience architecting infrastructure for ML training workloads (distributed training, large dataset management, experiment infrastructure).
• Background in cost optimization and FinOps practices for cloud and bare metal infrastructure.
• Experience operating and managing bare metal infrastructure in colocation facilities.
• Expertise in network architecture design, including high-bandwidth GPU interconnects and global traffic routing.
• Experience with infrastructure modeling and simulation for capacity planning.
• Familiarity with Slurm, Ray, or other HPC/ML job scheduling systems.
• Understanding of power, cooling, and physical infrastructure considerations for GPU-dense deployments.