Data Engineer

Overview

On Site
Full Time
Part Time
Accepts corp to corp applications
Contract - Independent
Contract - W2

Skills

Legacy Systems
Agile
API
Analytics
Business Intelligence
Accessibility
Decision-making
Data Quality
Business Process
Wiki
Data Analysis
Apache Spark
SQL
Microsoft SSIS
Microsoft TFS
CA Workload Automation AE
Apache Hive
Python
PySpark
Data Lake
Documentation
Java
C#
Extract
Transform
Load

Job Details

Data Engineer

Charlotte, NC (Need Onsite day 1, hybrid 3 days from office).


Job Description:

We're looking for a Data Engineer with strong technical skills and a programmer's mindset to support both legacy systems and an upcoming data lake initiative.

Responsibilities

Use agile software development processes to make iterative improvements to our back-end systems.

Model front-end and back-end data sources to help draw a more comprehensive picture of user flows throughout the system and to enable powerful data analysis.

Build data pipelines that clean, transform, and aggregate data from disparate sources.

Develop models that can be used to make predictions and answer questions for the overall business.

Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.

Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.

Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.Writes unit/integration tests, contributes to engineering wiki, and documents work.

Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.

Works closely with a team of frontend and backend engineers, product managers, and analysts.

Defines company data assets (data models), spark, sparkSQL, and hiveSQL jobs to populate data models.

Requirements:

Legacy Pipeline (Immediate ramp-up):

SQL, Liquibase, SSIS, TFS, AutoSys

New Data Lake Pipeline (Upcoming project):

Hive, Dremio, Python, PySpark, Airflow, Data Lake Architecture

Ideal Candidate Should Have:

Strong hands-on experience in some or most of the above technologies

Excellent documentation skills

Ability to read and understand compiled codebases (Java, C#); ability to write in these languages is a plus

A programmer's mindset to solve complex data and integration challenges

Preferred, but not required:

Experience in building or maintaining ETL processes

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.