Lead Python Engineer with Spark, AWS Glue

Overview

On Site
$50 - $60
Accepts corp to corp applications
Contract - W2
Contract - Independent
100% Travel

Skills

Amazon Web Services
Apache Spark
Collaboration
Continuous Delivery
Continuous Integration
Data Processing
Extract
Transform
Load
Management
Performance Tuning
Python
Real-time
Software Development
Streaming
Workflow

Job Details

  • Design & develop scalable data applications using Python and Apache Spark for batch and real-time (streaming) data processing.
  • Build, orchestrate, and optimize AWS Glue jobs to manage large-scale ETL workflows across heterogeneous data sources.
  • Collaborate with data engineers and analysts to understand requirements and efficiently implement business logic and transformations.
  • Implement robust error handling, logging, and performance tuning to support high-throughput, reliable data pipelines.
  • Ensure code quality, modularity, and reusability by adhering to software development best practices and CI/CD standards.
  • Contribute to deployment readiness and maintenance of production-grade data pipelines.

HIL USTAG-CHENNAI TEAM

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.