ETL Developer

Job Description :

Seeking a skilled ETL Developer specializing in Datastage ETL AWS Cloud data engineering to design build and optimize automated ETL pipelines Also enhance existing ETLs in on premise ETL tool Datastage and create new ETLs in python framework Key Responsibilities ETL Development Build scalable ETL processes using AWS Glue PySpark Python to transform raw data into optimized Parquet Avro formats Compute and Automation Write efficient Python scripts within AWS Lambda for lightweight data processing file validation and event driven triggers Data Lake Management Architect and maintain S3 bucket structure Bronze Silver Gold zones and manage data lifecycle policies Monitoring and Logging Set up comprehensive CloudWatch dashboards and alarms to monitor pipeline health performance bottlenecks and execution costs Performance Tuning Optimize Spark jobs in Glue and SQL queries to ensure high data throughput and minimal AWS spend Document incident recovery steps and preventive actions Debug and support issues caused by upstream system defects or downstream application failures Work cross functionally to ensure data lineage dependencies and integrations remain stable Support ETL execution across Dev QA Stage and Production environments Collaborate with Infrastructure teams to maintain IBM InfoSphere Information Server IIS platform health Manage environment level configurations parameter sets and shared containers Promote DataStage code DSX ISX across environments following change management process Validate ETL components during deployment and support post migration testing Assist in patching fix packs and version upgrades of the IIS DataStage platform Support approval workflows for DataStage project access and environment level privileges Coordinate with IAM teams for provisioning de provisioning and periodic access reviews Required Skills and Experience Strong SQL skills Snowflake and Oracle databases Experience supporting Finance or Regulatory Reporting ETL workloads Solid understanding of ETL performance tuning and parallelism concepts Experience in production support job monitoring and incident