Senior AI Product Engineer - Oyster

About Oyster

Oyster is a global employment platform that enables companies to hire, pay, and care for talent anywhere in the world. As a fully remote, B Corp-certified company distributed across 60+ countries, Oyster connects brilliant people with global opportunities.

The Opportunity

Oyster AI is an assistant for customers and team members designed to be accurate, fast, reliable, and safe with data. In this role, you will own complex improvements end-to-end—from defining problems and designing solutions through evaluation, rollout, and production monitoring.

Your work will span agent capabilities, tools, knowledge retrieval, answer quality, and infrastructure, balancing trade-offs between model quality, latency, reliability, and cost. This is a hands-on product engineering role where you will also help other product teams adopt AI patterns and reusable foundations.

Our backend is built with Ruby on Rails and our frontend with React. Strong AI engineers from other stacks are welcome, with Rails experience considered a bonus.

What You’ll Build & Own

• End-to-end AI improvements: Connect user needs to architecture, implementation, evaluation, and safe rollout for complex Oyster AI initiatives.
• Reliable agents & tools: Design clear tool contracts, robust error handling, and bounded execution with appropriate human oversight for consequential actions.
• Better knowledge access: Choose retrieval, embedding, indexing, and freshness strategies that improve answer quality while respecting data access boundaries.
• Evaluation frameworks: Build evaluation loops using representative datasets, experiments, regression checks, and calibrated automated judges.
• Production root-cause fixes: Diagnose failures across prompts, retrieval, tools, models, and infrastructure, fixing underlying causes and validating with evidence.
• Explicit trade-offs: Select models and system designs deliberately, documenting decisions and rollout criteria.
• AI capability enablement: Support product teams through technical guidance, code review, pairing, and practical mentoring to help them deliver AI features independently.

What You’ll Bring

• Senior-level engineering judgment with a track record of independently leading complex production systems from design through operation.
• Hands-on experience shipping LLM-powered product capabilities to production with direct responsibility for implementation decisions.
• Depth in key AI areas: retrieval/RAG, indexing, evaluation datasets, LLM-as-judge, or agents and tool calling.
• Ability to diagnose failures across model, retrieval, tooling, and application layers, validating improvements with empirical evidence.
• Strong fundamentals in software engineering, API integration, debugging, testing, and system design, plus willingness to work in Ruby/Rails and React.
• Sound judgment regarding sensitive data, access control, model limitations, and operational risk.
• Clear communication of trade-offs and effective async collaboration across engineering and product teams.

Nice to Have

• Production experience with Ruby on Rails, particularly integrating AI capabilities into established backends.
• Full-stack experience shipping conversational or assistant interfaces.
• Integrations with Slack or in-app assistants maintaining clear user and permission contexts.
• Experience operating retrieval pipelines, agent tools, evaluation infrastructure, or model routing at scale.

Benefits & Support

• Work from Anywhere: Fully remote environment with flexibility to work from anywhere you are authorized to live.
• Paid Time Off: 40 days off per year (including public holidays and vacation).
• Mental Health Support: Access to Plumm mental well-being services.
• Wellbeing Allowance: Monthly stipend via ThanksBen wallet for wellness expenses.
• Parental Leave: Minimum 3 months' paid leave for all new parents, with job protection up to 12 months.
• WFH Stipend: Equipment stipend for laptop and home office setup.