Data Engineer

Data Engineer

Location: Chicago, IL
Position Type: Contract

Job Summary

We are seeking an experienced Data Engineer with strong hands-on experience in Databricks, Apache Spark, PySpark, Python, ETL/ELT, and AWS. The ideal candidate will design, develop, and optimize scalable data pipelines and cloud-based data processing solutions.

Must-Have Skills

• Databricks

• Apache Spark

• PySpark

• Python

• AWS / Amazon Web Services

• ETL / ELT

• SQL

• Data Pipeline Development

• Large-Scale Data Processing

Preferred Skills

• Delta Lake

• Databricks Workflows / Jobs

• Data Warehousing

• Cloud Data Engineering

• Spark Performance Optimization

• Data Quality and Validation

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.

• Build and optimize data processing solutions using Databricks, Apache Spark, and PySpark.

• Develop Python-based data transformation and processing applications.

• Integrate data from multiple sources into enterprise data platforms.

• Utilize AWS services to develop and support cloud-based data solutions.

• Perform data extraction, transformation, cleansing, validation, and quality checks.

• Optimize Spark jobs, Databricks workflows, and data pipelines for performance and reliability.

• Troubleshoot production data pipeline issues and ensure reliable operations.

• Collaborate with data architects, developers, analysts, and business teams.

• Follow established coding, security, governance, and deployment standards.

Required Qualifications

• Strong hands-on experience with Databricks and Apache Spark.

• Strong programming experience with Python and PySpark.

• Solid understanding of ETL/ELT processes and data integration.

• Experience designing, developing, and optimizing data pipelines.

• Strong experience with AWS cloud services, particularly data-related services.

• Experience working with large-scale data processing and transformation.

• Strong SQL skills for data extraction, transformation, and validation.

• Experience troubleshooting and optimizing data pipelines for performance and reliability.

• Good communication and collaboration skills.

Keywords: Data Engineer, Databricks, Apache Spark, Spark, PySpark, Python, AWS, Amazon Web Services, ETL, ELT, SQL, Delta Lake, Data Pipelines, Data Warehousing, Cloud Data Engineering.

Regards,

Sagar

Manager