Lead Data Engineer

Position: Lead Data Engineer

Location: Denver, CO - Hybrid

Duration: 6+ months with possible extension

Position Summary:

We are seeking a Lead Data Engineer to join our dynamic team. The ideal candidate will have a robust background in data engineering, with a proven track record of designing and implementing efficient data pipelines and systems. This role will play a significant part in transforming and optimizing our data architecture to support our data-driven strategy.

Key Responsibilities:

• Develop and maintain scalable data pipelines and systems using Python, Spark, and Databricks to process and manage large-scale datasets.
• Design and implement unified data models with enterprise-wide consistency in key naming conventions, data types, and transformation functions.
• Optimize operations in Spark for handling large, unstructured, and nested datasets by implementing efficient data structuring and processing techniques.
• Collaborate with data science and analytics teams to create and refine data processes that enhance decision-making and align with business goals.
• Lead initiatives to streamline and enhance data quality monitoring systems.

Qualifications:

• Extensive experience as a Data Engineer, with a focus on Python programming in Spark/Databricks environments.
• Proven expertise in designing and optimizing large-scale data pipelines, ETL processes, and data lakes.
• Strong background in cloud platforms, ideally Azure, with hands-on experience managing cloud-based data infrastructure.
• Demonstrated ability to convert legacy systems into modern Spark/Databricks platforms, improving efficiency and performance.
• Excellent problem-solving skills with the ability to troubleshoot and resolve complex data issues.
• Strong communication and collaboration skills, with the ability to work effectively in a remote team environment.