Senior Data Engineer

Overview

On Site
Depends on Experience
Contract - W2
Contract - 6 Month(s)
No Travel Required
Unable to Provide Sponsorship

Skills

Data Warehouse
Database
Microsoft Azure
Scripting
Microsoft SSIS
Big Data
Data Engineering
Data Analysis
Data Modeling
DevOps
JIRA
SQL Azure
ADF
Agile
Apache HTTP Server

Job Details

The following requirement is open with our client.                              

Title                                       : Senior Data Engineer

Location                              : Newark, NJ(Onsite)

Duration                              : 6+ Months

Relevant Experience (in Yrs.): 8+yrs.

Role Description:

Relevant Experience 8+ Years

Roles & Responsibilities

• Data Management and Storage:

o Design and implement data storage systems using Azure services like Azure SQL Database, Azure Data Lake Storage, and Azure Synapse.

o Ensure scalability, performance, and cost-effectiveness.

• Data Integration and ETL (Extract, Transform, Load):

o Develop and implement data integration processes using Azure Data Factory.

o Extract data from various sources, transform it, and load it into data warehouses or data lakes.

• Big Data and Analytics:

o Utilize big data technologies such as Apache Spark.

o Create data processing workflows and pipelines to support data analytics and machine learning applications.

• Build and maintain new and existing applications in preparation for a large-scale architectural migration within an Agile function.

• Monitor and optimize data pipelines and database performance to ensure data processing efficiency.

• Build interfaces for supporting evolving and new applications and accommodating new data sources and types of data.

• Document data engineering processes, data models, and pipelines to ensure transparency and maintainability.

Data Lake Proficiency in:

o Software development and scripting languages

o ETL tools (e.g., SSIS, Azure Data Factory, Power BI Dataflow).

o Database management (MSSQL, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse).

• Knowledge of data modeling and data warehousing concepts.

• Excellent problem-solving and troubleshooting abilities.

• Attention to detail and commitment to data accuracy.

• Experience with cloud-based data migration.

• Strong analytical and problem-solving skills, with ability to conduct root cause analysis on system, process or production problems and ability to provide viable solutions.

• Experience working in an Agile environment with Scrum Master/Product owner and ability to deliver.

• 3+ years of programming experience in Python/Pyspark.

• Knowledge of Jira, Confluence, SAFe development methodology & DevOps

Must Have Skills:

Azure Data Engineer, Microsoft Fabric

SQL Database, Azure Data Lake Storage, Azure Synapse.

ETL, ADF, Apache, Power BI, DevOps

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.