Senior Data Engineer

Overview

On Site
Depends on Experience
Contract - W2
Contract - 24 week(s)

Skills

Data Architect
MS SQL Server

Job Details

Senior Data Engineer

Responsibilities:

  • Design and Implement Data Pipelines: Create robust data pipelines to seamlessly integrate data from various sources into a unified Data Lakehouse.
  • Ensure Data Quality: Develop and implement data quality pipelines to validate and maintain the accuracy of data, ensuring the creation of trusted datasets.
  • Data Integration: Utilize your expertise in various data integration patterns, including ETL, ELT, Pub/Sub, and Change Data Capture, to connect and transform data efficiently.
  • Programming: Apply your extensive programming skills in Python, ANSI SQL, PLSQL, and TSQL to develop and maintain data pipeline architectures.
  • Data Engineering Packages: Leverage common Python Data Engineering packages such as pandas, Numpy, Pyarrow, Pytest, Scikit-Learn, and Boto3 to enhance data processing.
  • Software Development Practices: Apply best software development practices, including Design Principles and Patterns, Testing, Refactoring, CI/CD, and version control, to ensure data solutions are reliable and maintainable.
  • Data Lakehouse Implementation: Utilize your experience to implement and manage a Data Lakehouse using technologies like Apache Iceberg or Delta Lake.
#11036

 

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.