ETL AB initio Developer

Role: ETL AB initio Developer

Location: Jersey City, NJ

Mode of Hire: Contract

Mandatory Skills: Python, Java, Ansi SQL, Abinitio, Spark

• Experience in building data integration solutions using the AB initio ETL .
• Creating efficient and scalable data processing pipelines and applications using Apache spark
• Proficient in understanding distributed computing principles.
• Creating ,optimizing, and maintaining directed acyclic graphs(DAGs) in Python to define and orchestrate data pipelines and automated tasks.
• Implementing , scheduling, and monitoring complex data workflows, ensuring timely and accurate data processing.
• Identifying, diagnosing, and resolving issues within airflow workflows, and optimizing DAGs for performance, scalability, and resource efficiency.
• Good understanding of cloud technology. Must have strong technical experience in Desing Mapping Specifications HLD LLD.
• Must have the ability to relate to both business and technical members of the team and possess excellent communication skills.
• Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, EMR and integrate with our internal Archival Service Platform for efficient data purging and Life Cycle management.
• Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
• Building data integration solutions using the Ab initio ETL .
• Creating efficient and scalable data processing pipelines and applications using Apache spark.
• Implementing, scheduling, and monitoring complex data workflows, ensuring timely and accurate data processing.
• Implement and manage agents for monitoring, logging, and automation within aws environments
• Handling migration from Pyspark to AWS
  
  

  Equal opportunity Employer

  We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.