Data Engineer - PySpark & Databricks (Locals Only)
Morris Plains, NJ, US • Posted 3 days ago • Updated 3 days ago

Helishores Inc
Dice Job Match Score™
🧠 Analyzing your skills...
Job Details
Skills
- Amazon Web Services
- PySpark
- SQL Tuning
- Stakeholder Management
- Mechanics
- Continuous Integration
- Data Warehouse
- Databricks
- Decision-making
- Collaboration
- Communication
- Conflict Resolution
- Continuous Delivery
- Agile
- Amazon S3
- Analytics
- Apache Spark
- Broadcasting
- Caching
- Design Patterns
- ELT
- Performance Tuning
- Problem Solving
- Extract, Transform, Load
- Machine Learning (ML)
- Mentorship
- Optimization
- Streaming
- Team Leadership
Summary
Role: Data Engineer - PySpark & Databricks
Location: Morris Plains, NJ (Locals Only No Relocation)
Job Description:
Design, develop, and maintain ETL/ELT pipelines using PySpark on Databricks.
Build and optimize batch and streaming data pipelines.
Implement Delta Lake solutions (Delta tables, time travel, ACID transactions).
Collaborate with data scientists, analysts, and architects to deliver analytics-ready datasets.
Required Qualifications:
Overall, 8+ years of experience required.
Hands-on experience with Data Frames, RDDs, joins, transformations, and actions within PySpark.
Proven experience leading teams and mentoring engineers.
Job optimization, cluster configuration, repartitioning, and Shuffle mechanics in Databricks.
S3 buckets, IAM, CloudWatch, and integration with Databricks and AWS.
Strong query skills for analytics and ETL with SQL.
Performance tuning: Partitioning, caching, broadcast joins, and skew handling.
Delta Lake, Medallion Architecture, Spark Streaming, Spark ML, and CI/CD pipelines.
ETL/ELT design patterns.
Handling large-scale structured and semi-structured data.
Performance tuning (partitioning, caching, broadcast joins).
Understanding of data warehousing concepts.
Excellent communication and stakeholder management skills.
Ability to work in Agile delivery environments.
Ownership mindset and delivery-focused approach.
Strong technical decision-making and problem-solving skills.
- Dice Id: 91129309
- Position Id: 8863469
- Posted 3 days ago
Company Info
We strive to be a trusted partner as an IT Solution provider. We are committed to achieve customer's business objectives by delivering professional services and adopting innovative technologies. Our goal is to provide you with the right resources, with the right IT competencies who fit seamlessly into our client's corporate culture and ambiance.

Similar Jobs
It looks like there aren't any Similar Jobs for this job yet.
Search all similar jobs