Lead Data Engineer Job at WorkHQ, Los Angeles, CA

dGNNYzFMWkppR1d2ZUc3SnlXb280OEVKdXc9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Remote work, Shift work,

Similar Jobs

AdvoCare

Chief Financial Officer Job at AdvoCare

 ...collaborates cross-functionally to align financial strategies with corporate objectives and regulatory compliance. Strategy &...  ...Oversight Guide and mentor accounting and finance team leaders to ensure strong internal controls and reliable reporting... 

Insight Global

Sql database administrator Job at Insight Global

 ...Global is seeking a Sr. SQL DBA to join their IT team. The SQL Database Administrator (DBA) will be responsible for maintaining and ensuring...  ...necessary security updates and patches. Optimize and manage multiple database products to ensure high performance and... 

Sanco Equipment

Heavy Equipment Service Technician Job at Sanco Equipment

 ...SUMMARY We are a full-line Bobcat and XCMG dealer serving customers across Minnesota and Northern Iowa. The Heavy Equipment Service Technician is responsible for diagnosing, repairing, and maintaining a wide range of Bobcat and XCMG construction and agricultural equipment... 

GTN Technical Staffing

Epicor Analyst Job at GTN Technical Staffing

Our client is seeking a highly analytical and detail-oriented ERP Data Analyst to serve as the primary resource for enterprise reporting, analytics, and business intelligence. This individual will play a critical role in transforming Epicor ERP data into actionable insights...

Everest Technologies, Inc.

Scrum Master Job at Everest Technologies, Inc.

Summary: -Sr. level Scrum Master with proven track record leading agile teams to achieve sprint goals and delivery commitments Strong core skills:facilitation, communication, forecasting, dependency management, organization Attention to detail with excellent...