Looking for Lead Data Engineer (Need Only local Charlotte, NC Inperson Interview)

Overview

On Site
Depends on Experience
Accepts corp to corp applications
Contract - W2

Skills

Python
pyspark

Job Details

Job Summary

We are seeking a skilled Data Engineer to design, build, and maintain scalable and robust data platforms on the AWS ecosystem. The ideal candidate will have strong experience in RDBMS, modern data formats, and ETL/data integration tools such as DataStage and AWS Glue, along with hands-on expertise in PySpark and Python.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines for structured and semi-structured data.
  • Build and optimize data ingestion, transformation, and loading processes using AWS Glue, DataStage, and PySpark.
  • Work with RDBMS systems such as PostgreSQL and SQL Server for data modeling, querying, and performance tuning.
  • Process and manage data in multiple formats including JSON, Parquet, and other file types.
  • Develop data solutions leveraging AWS services such as S3, Glue, and related analytics services.
  • Implement data quality checks, monitoring, and error-handling mechanisms.
  • Optimize data storage and processing for performance, scalability, and cost efficiency.
  • Collaborate with data analysts, data scientists, and application teams to deliver reliable data solutions.
  • Follow best practices for data security, governance, and compliance.

Required Skills & Qualifications

  • Strong experience with RDBMS (PostgreSQL, SQL Server, or similar).
  • Hands-on expertise with ETL/Data Integration tools such as IBM DataStage and AWS Glue.
  • Proficiency in PySpark for large-scale data processing.
  • Strong programming skills in Python.
  • Experience working with AWS data platforms, especially Amazon S3.
  • Solid understanding of data formats like JSON and Parquet.
  • Experience designing and optimizing data pipelines and workflows.
  • Good knowledge of SQL and data modeling concepts.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.