Senior Data Engineer - People Data Labs

About Us

People Data Labs (PDL) is the provider of people and company data. We do the heavy lifting of data collection and standardization so our customers can focus on building and scaling innovative, compliant data solutions. Our sole focus is on building the best data available by integrating thousands of compliantly sourced datasets into a single, developer-friendly source of truth. Leading companies across the world use PDL’s workforce data to enrich recruiting platforms, power AI models, create custom audiences, and more.

We are looking for individuals who can balance extreme ownership with a “one-team, one-dream” mindset. Our Data Engineering Team is the secret sauce behind all that we do, and we are looking for the best of the best.

If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this role is for you. We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things.

What You Get to Do

• Build infrastructure for ingestion, transformation, and loading an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks.
• Build an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into clean, consumable datasets.
• Develop CI/CD pipelines and anomaly detection systems capable of continuously improving the quality of data pushed into production.
• Architect solutions to largely undefined data engineering and data science problems.

Technical Requirements

• 5–7+ years of industry experience with clear examples of strategic technical problem-solving and implementation.
• Strong software development fundamentals.
• Proficiency with Python and SQL.
• Expertise with Apache Spark (Java, Scala, and/or Python-based).
• Experience building scalable data processing systems (cleaning, transformation) from the ground up.
• Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow preferred, dbt, Dagster, or similar).
• Knowledge of modern data design and storage patterns (incremental updating, partitioning, segmentation, rebuilds, backfills).
• Experience working in Databricks (including Delta Live Tables, data lakehouse patterns, etc.).
• Experience with cloud computing services (AWS preferred, GCP, Azure, or similar).
• Experience with data warehousing (Databricks, Snowflake, Redshift, BigQuery, etc.).
• Understanding of modern data storage formats and tools (Parquet, ORC, Avro, Delta Lake).

Team & Collaboration Skills

• Balance high ownership and autonomy with a strong ability to collaborate.
• Work effectively remotely (proactive about managing blockers, reaching out, and asking questions).
• Strong written communication skills on Slack/Chat and in documentation.
• Experience writing data design docs (pipeline design, dataflow, schema design).
• Ability to scope and break down projects and communicate progress and blockers effectively.

Nice to Haves

• Degree in a quantitative discipline such as Computer Science, Mathematics, Statistics, or Engineering.
• Experience working with entity data (entity resolution / record linkage).
• Experience working with data acquisition and data integration.
• Expertise with Python and the Python data stack (e.g., NumPy, Pandas).
• Experience with streaming platforms (e.g., Kafka).
• Experience evaluating data quality and maintaining consistently high data standards across releases.

Benefits

• Stock options / equity
• Competitive salary ($200,000 - $220,000)
• Unlimited paid time off (PTO)
• Medical, dental, & vision insurance
• Health, fitness, and office stipends
• Permanent remote flexibility to work wherever and however you want