Data Engineer

Overview

Hybrid
Depends on Experience
Full Time

Skills

Data Engineer
SAP
AWS

Job Details

Title: Data Engineer

Location: Newark, NJ / Edison, NJ ( Hybrid 1 2 / Days per Week)

Mode: Fulltime Position

Required Skills:

  • Minimum of 8 years of hands-on experience in data engineering or related roles.
  • Experience extracting and integrating data from SAP systems (e.g., SAP ECC, S/4HANA).
  • Experience with data warehousing concepts and technologies.
  • Proficiency in SQL for data querying and manipulation. Good with Python, PySpark or Java.
  • Hands-on experience with AWS services such as S3, Redshift, DMS, Glue, and Lambda.
  • Understanding of data modelling principles and techniques.
  • Knowledge of data security best practices and compliance requirements.
  • Experience with ETL (Extract, Transform, Load) processes and tools.
  • Excellent problem-solving, communication, and collaboration skills.

Preferred Qualifications:

  • Experience of working in the Utility Industry
  • Experience integrating SAP IS-U (Industry Solution for Utilities) data.

Job Responsibilities:

  • Extract and transform data from SAP systems into cloud-based storage solutions.
  • Design, implement, and maintain data pipelines, data warehouses, and other data solutions using AWS services.
  • Create and manage data models to ensure data integrity and facilitate efficient data analysis.
  • Implement and maintain data security and compliance measures, including access controls, encryption, and data masking.
  • Ensure data quality, accuracy, and consistency through data validation, cleansing, and monitoring.
  • Collaborate with data scientists, business analysts, SAP functional SMEs and other stakeholders to understand requirements and deliver data solutions that meet business needs.
  • Manage and structure data within SQL-based Data Lakes for efficient querying and access.
  • Design and maintain ETL/ELT pipelines to ingest data into Amazon Redshift for analytics and reporting.
  • Develop and schedule AWS Glue jobs to automate data processing workflows using PySpark or Spark.
  • Optimize Redshift performance through efficient schema design, indexing, and data modelling.
  • Ensure data quality, consistency, and integrity across the entire data pipeline.
  • Collaborate with cross-functional teams to gather data requirements and support business intelligence needs.
  • Monitor, troubleshoot, and document data pipelines, ensuring reliability and scalability.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.