Senior Data Engineer

Overview

On Site

Full Time

Part Time

Accepts corp to corp applications

Contract - W2

Contract - Independent

Skills

Data Processing

Big Data

Unstructured Data

Collaboration

Workflow

Amazon Web Services

Cloud Computing

Electronic Health Record (EHR)

Amazon S3

Virtual Private Cloud

Databricks

Scala

Python

Java

Terraform

Data Engineering

Apache Spark

Distributed Computing

Orchestration

DevOps

Continuous Integration

Continuous Delivery

Git

Jenkins

Docker

Kubernetes

Problem Solving

Conflict Resolution

Debugging

Optimization

Apache Kafka

Amazon Kinesis

Data Governance

Agile

Job Details

LOCAL ONLY***

Role Overview

We are seeking a highly skilled Senior Data Engineer with strong expertise in building scalable, secure, and high-performance data solutions on AWS. The ideal candidate should have hands-on experience with Databricks (DBX), distributed data processing, cloud-native frameworks, and automation using Terraform.

Key Responsibilities

Design, develop, and optimize data pipelines on AWS leveraging EMR, EKS, and Databricks.
Implement Infrastructure as Code (IaC) using Terraform to automate cloud deployments.
Develop scalable solutions using Scala, Python, and Java for big data processing.
Integrate structured, semi-structured, and unstructured data sources into the data platform.
Collaborate with data scientists, analysts, and application teams to deliver end-to-end solutions.
Ensure platform security, reliability, cost optimization, and adherence to best practices.
Implement CI/CD pipelines and monitoring for data workflows.

Required Skills

AWS Cloud Services: EMR, EKS, S3, IAM, Lambda, Glue, VPC.
Databricks (DBX): Delta Lake, Spark, MLflow (nice to have).
Programming: Strong proficiency in Scala, Python, Java.
Infrastructure as Code (IaC): Terraform (must-have).
Data Engineering: Spark, distributed computing, data pipelines, orchestration.
DevOps/CI-CD: Git, Jenkins, Docker, Kubernetes (EKS).
Strong problem-solving, debugging, and optimization skills.

Nice-to-Have Skills

Experience with event-driven architectures (Kafka, Kinesis).
Familiarity with data governance, lineage, and security frameworks.
Knowledge of Agile methodologies and collaborative development practices.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.