Job Description
Job Title: Lead Data Engineer
Location: Irving, TX - Hybrid
Must have:
- Expert in Databricks, PySpark and Azure data services like ADF, Synapse etc.
- Must have lead level experience
- 8+ years of professional data engineering experience, with proven technical leadership.
- Deep understanding of Azure tech stack (Databricks, ADF, Key Vault, ASQL, ADLS Gen2, Service Principles etc.)
- Deep experience with Databricks (design, optimization, notebooks, stored procedures, UDFs, etc).
- Hands-on experience with ETL tools such as SSIS and Azure Data Factory.
- Experience working with API calls and Ingesting data from external source systems
- Experience with SQL, Python and PySpark
- Experience with building dynamic and scalable solutions for data ingestion and curation
- Experience in data engineering and MS SQL Server and T-SQL development.
- Strong understanding of data lake/delta lake architectures and modern data pipelines.
- Strong understanding of medallion architectures and design patterns.
- Strong understanding of data modeling techniques.
- Experience handling large-scale data volumes (millions of records).
- Proficiency in building and maintaining CI/CD pipelines.
- Experience with IaC technologies such as Terraform or Pulumi.
- Exposure to big data tools (Spark, Hadoop, Kafka) preferred.
- Experience with reporting tools such as Power BI or Tableau a plus.
- Knowledge of .NET development beneficial for cross-team collaboration.
- Prior experience working in an Agile development environment required.
- Strong communication and collaboration skills with the ability to translate technical concepts to business partners.
What Success Looks Like:
- You're the go-to technical lead for all things related to data design, integration, and performance.
- You enable consistency and best practices across Azure, Databricks, SQL, ETL, and data engineering domains.
- You mentor others while remaining hands-on with architecture and solution delivery.
- You ensure the Internal Analytics platform continues to evolve as a reliable, scalable data source for the organization.
Job Tags