Data Engineering Lead

Overview

On Site
Full Time

Skills

Data Engineering Lead

Job Details



Data Engineering Lead


Telework


We are seeking a Data Engineering Lead to support the VA HELM program. This is a great opportunity for a seasoned technical lead who excels at building scalable data systems and leading teams all while in the comfort of your own home!


Job Description: Seeking an experienced and highly skilled Data Engineering Technical Lead to spearhead the design, development, and optimization of robust and scalable data pipelines and data infrastructure. This role requires a deep understanding of data engineering principles, advanced proficiency in distributed data processing technologies, and a proven track record of leading technical teams in an agile environment. The Technical Lead will play a critical role in shaping our data architecture, ensuring data quality and governance, and enabling data-driven initiatives across the organization, particularly within the healthcare and supply chain logistics domains. This role is 100% remote.


Key Responsibilities:



  • Lead, mentor, and guide a team of data engineers in best practices for data pipeline development, data modeling, and infrastructure management. Provide technical oversight and ensure adherence to architectural standards.

  • Drive the design and implementation of highly scalable, reliable, and efficient data architectures, including data lakes, data warehouses, and streaming platforms. Define technical roadmaps and strategies for data engineering initiatives.

  • Design, develop, and maintain complex ETL/ELT processes and data pipelines using modern data engineering tools and frameworks (e.g., Apache Spark, Kafka, Airflow). Ensure data ingestion, transformation, and loading processes are optimized for performance and reliability.

  • Establish and enforce data governance policies, data quality standards, and data security protocols. Implement robust monitoring and alerting for data pipelines and data quality issues.

  • Leverage expertise in cloud-based data platforms (e.g., AWS, Azure, Google Cloud Platform) to design and implement cloud-native data solutions, optimizing for cost, performance, and scalability.

  • Collaborate closely with product owners, data scientists, data analysts, and other engineering teams to translate business requirements into technical specifications and deliver data solutions that align with strategic objectives. Act as a primary interface with customers, business stakeholders, and cross-functional teams to gather requirements, provide updates, and ensure alignment on data initiatives.

  • Identify and resolve complex data-related performance bottlenecks and architectural challenges. Implement strategies for query optimization and data storage efficiency.

  • Actively participate in agile ceremonies, including sprint planning, backlog refinement, daily stand-ups, and sprint reviews, contributing to a culture of continuous delivery and improvement. Lead and own the full Software Development Life Cycle (SDLC) for data solutions, from conceptualization and design through deployment, testing, and operational support.

  • Create and maintain comprehensive technical documentation, including data flow diagrams, architectural designs, data dictionaries, and operational runbooks.


Minimum Qualifications



  • Bachelor's Degree in a related field; Master's Degree preferred

  • Industry leading certification for technical expertise preferred

  • 12+ years of experience in a technical field with at least 2 years in a technical leadership or lead engineer role.


Other Job Specific Skills



  • Expert-level proficiency in SQL and experience with relational and NoSQL databases.

  • Extensive experience with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and stream processing technologies (e.g., Apache Kafka, Flink).

  • Strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures.

  • Proven experience with cloud-based data platforms (e.g., AWS Redshift, S3, Glue, EMR; Azure Data Lake, Synapse, Databricks; Google Cloud Platform BigQuery, Dataflow).

  • Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Scala, Java).

  • Demonstrated experience with CI/CD practices for data pipelines and infrastructure as code (IaC).

  • Solid understanding of data governance, data security, and compliance best practices (e.g., HIPAA, GDPR).




#cjpost

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.