Sr. Data Engineer

Overview

On Site
Accepts corp to corp applications
Contract - Long Term

Skills

SQL
AWS
Python
PYSPARK
Aurora Postgres

Job Details

Hi,

Our client is looking for a Sr.Data Engineer for Contract project in New York, NY (Hybrid) below is the detailed requirement.

If you are interested kindly share your updated resume to proceed further.

Job Role: Sr. Data Engineer

Location: New York, NY (Hybrid)

Mode of Hiring: Long Term

Job Description: The ideal candidate will have a strong background in data engineering and development, with expertise in various data skills and tools. The role involves working on complex data projects, collaborating with cross-functional teams, and ensuring the efficient processing and management of data.

Responsibilities:

  • Bachelor's degree in Computer science or equivalent, with minimum 12+ Years of relevant experience.
  • Develop and maintain data pipelines using Python and Pyspark.
  • Design and implement efficient SQL queries for data extraction and manipulation.
  • Work with Aurora Postgres and Redshift for data storage and management.
  • Utilize Informatica or other ETL tools for data integration and transformation.
  • Implement and manage AWS cloud services, including Athena, S3, ECS/Docker, EMR, EC2, Lambda, CloudWatch, and EventBridge.
  • Use Airflow or other orchestration tools for workflow management.
  • Ensure data quality and integrity through rigorous testing and validation.
  • Collaborate with other teams to understand data requirements and deliver solutions.

Required Skills:

  • Proficiency in Python and Pyspark.
  • Solid experience in AWS Glue
  • Strong SQL skills.
  • Experience with Aurora Postgres and Redshift.
  • Knowledge of Informatica or other ETL tools.
  • Experience with AWS cloud services, including Athena, S3, Lambda, CloudWatch, and EventBridge.

Preferred Skills:

  • ECS/ Docker (or any docker experience/concepts)
  • EMR, EC2 (or knowledge of any other distributed computing)
  • Familiarity with Airflow or other orchestration tools.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.