Data Engineer - PySpark & Databricks (Locals Only)

Morris Plains, NJ, US • Posted 3 days ago • Updated 3 days ago

Contract W2

Contract Independent

Travel Required

On-site

Depends on Experience

Helishores Inc

Fitment

Dice Job Match Score™

🧠 Analyzing your skills...

Job Details

Skills

Amazon Web Services
PySpark
SQL Tuning
Stakeholder Management
Mechanics
Continuous Integration
Data Warehouse
Databricks
Decision-making
Collaboration
Communication
Conflict Resolution
Continuous Delivery
Agile
Amazon S3
Analytics
Apache Spark
Broadcasting
Caching
Design Patterns
ELT
Performance Tuning
Problem Solving
Extract, Transform, Load
Machine Learning (ML)
Mentorship
Optimization
Streaming
Team Leadership

Summary

Role: Data Engineer - PySpark & Databricks
Location: Morris Plains, NJ (Locals Only No Relocation)

Job Description:
Design, develop, and maintain ETL/ELT pipelines using PySpark on Databricks.
Build and optimize batch and streaming data pipelines.
Implement Delta Lake solutions (Delta tables, time travel, ACID transactions).
Collaborate with data scientists, analysts, and architects to deliver analytics-ready datasets.

Required Qualifications:
Overall, 8+ years of experience required.
Hands-on experience with Data Frames, RDDs, joins, transformations, and actions within PySpark.
Proven experience leading teams and mentoring engineers.
Job optimization, cluster configuration, repartitioning, and Shuffle mechanics in Databricks.
S3 buckets, IAM, CloudWatch, and integration with Databricks and AWS.
Strong query skills for analytics and ETL with SQL.
Performance tuning: Partitioning, caching, broadcast joins, and skew handling.
Delta Lake, Medallion Architecture, Spark Streaming, Spark ML, and CI/CD pipelines.
ETL/ELT design patterns.
Handling large-scale structured and semi-structured data.
Performance tuning (partitioning, caching, broadcast joins).
Understanding of data warehousing concepts.
Excellent communication and stakeholder management skills.
Ability to work in Agile delivery environments.
Ownership mindset and delivery-focused approach.
Strong technical decision-making and problem-solving skills.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91129309
Position Id: 8863469
Posted 3 days ago

Company Info

About Helishores Inc

We strive to be a trusted partner as an IT Solution provider. We are committed to achieve customer's business objectives by delivering professional services and adopting innovative technologies. Our goal is to provide you with the right resources, with the right IT competencies who fit seamlessly into our client's corporate culture and ambiance.

Go to company profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

It looks like there aren't any Similar Jobs for this job yet.

Search all similar jobs