Job Description
Skill: DATA Stage Engineer
Must Have Technical/Functional Skills:
• 8+ years of experience supporting, designing and developing applications using IBM DataStage ETL jobs to load into the Greenplum EDW Database to meet the business requirements
• Knowledge on Oracle, DB2, SQL Server, Teradata
• Knowledge on Pyspark
• Sound understanding of data modeling, data quality and data profiling.
• Familiar with IBM Infosphere DataStage
• Experience in writing Script using SQL and Python
• Knowledge on Cloudera data lake
• Ability and desire to learn new technologies quickly.
• Ability to work independently and collaborate with others at all levels of technical understanding.
Roles & Responsibilities:
• Analyze environment and data designs to assess impacts to requested changes by managing code base and environment usage for development, staging and testing and applying branch and code versioning to support different data sets [10%]
• Ensure processes are consistent with business requirements from Information Analysts (IAs) on how to process the upstream data [10%]
• Source data from Oracle, DB2, SQL Server, Teradata, and profile the data to understand the requirements [10%]
• Designing the IBM DataStage ETL jobs to load into the Greenplum EDW Database to meet the business requirements within the SLAs [10%]
• Perform unit tests on code [5%]
• Setup Control-M Schedules and order them per the business needs [5%]
• Keep track of the reported defects, and support other teams to resolve defects [5%]
• Monitor DataStage jobs, identify bugs, and resolve issues by fixing the data or rewriting the code [10%]
• Work and mentor team on use of IBM InfoSphere DataStage [5%]
• Design, build and maintenance of transform rules and processes to derive the correct data from the extracted data and to transform it into the required format [10%]
• Design code and write SQL, Python scripts, and Unix Shell Scripts based on ETL requirements [5%]
• Create mappings to assist Information analysts on the team [5%]
• Design and develop data pipeline to load into Cloudera data lake [10%]
• Other duties as assigned
