Job Description

Description We are looking for a highly skilled Lead Data Engineer to join our team. This role requires an individual with extensive experience in Databricks and a solid understanding of the insurance industry. You will play a pivotal role in designing and implementing scalable data architectures to support advanced analytics, reporting, and AI/BI/ML initiatives across various insurance functions. Responsibilities: - Develop and implement enterprise-level data architectures using Databricks Lakehouse, Delta Lake, and Azure cloud services. - Create and maintain secure, scalable data pipelines to process structured and unstructured data from multiple sources. - Define data integration, modeling, and governance frameworks tailored to insurance data domains such as policy, claims, underwriting, and billing. - Collaborate with actuarial, underwriting, and BI teams to design semantic layers and analytics-ready datasets. - Optimize Databricks cluster performance and workflows to ensure cost efficiency and reliability. - Establish and enforce data security protocols aligned with industry regulations and compliance standards. - Implement data quality, lineage, and metadata management frameworks using tools like Unity Catalog or Collibra. - Mentor teams on best practices for data modeling, Databricks optimization, and cloud architecture. - Review and validate solutions from divisional teams and vendors to ensure scalability, resilience, and cost-effectiveness. - Evaluate emerging technologies and develop blueprints for adoption to enhance data capabilities. Requirements - Bachelor's or master's degree in Computer Science, Data Engineering, Information Systems, or a related field. - At least 7 years of experience in data architecture or engineering, including 4 years working with Databricks. - Expertise in insurance data models, including policy, claims, underwriting, and accounting domains. - Proficiency in Python, PySpark, ETL processes, and Databricks workflows. - Strong knowledge of Azure Databricks and modern data lakehouse architecture. - Experience integrating data with BI tools such as Power BI, Tableau, or Looker. - Familiarity with data governance tools like Unity Catalog or Collibra. - Excellent communication and stakeholder management skills. Technology Doesn't Change the World, People Do.® Robert Half is the world's first and largest specialized talent solutions firm that connects highly qualified job seekers to opportunities at great companies. We offer contract, temporary and permanent placement solutions for finance and accounting, technology, marketing and creative, legal, and administrative and customer support roles. Robert Half works to put you in the best position to succeed. We provide access to top jobs, competitive compensation and benefits, and free online training. Stay on top of every opportunity - whenever you choose - even on the go. Download the Robert Half app ( and get 1-tap apply, notifications of AI-matched jobs, and much more. All applicants applying for U.S. job openings must be legally authorized to work in the United States. Benefits are available to contract/temporary professionals, including medical, vision, dental, and life and disability insurance. Hired contract/temporary professionals are also eligible to enroll in our company 401(k) plan. Visit roberthalf.gobenefits.net for more information. © 2025 Robert Half. An Equal Opportunity Employer. M/F/Disability/Veterans. By clicking "Apply Now," you're agreeing to Robert Half's Terms of Use ( .

Job Tags

Permanent employment, Contract work, Temporary work,

Similar Jobs