Overview
On Site
Full Time
Contract - W2
Contract - 1 day((s))
Skills
NOSQL
MongoDB
PySpark
data management
MySQL
Git
data pipelines
workflows
Microsoft SQL Server
cloud computing
big data
Manufacturing
Python (Programming Language)
Leadership
Apache Spark
Extract Transform Load (ETL)
Information Engineering
Microsoft Azure
Amazon Web Services
Analytical Thinking
Information Technology
Systems Development Life Cycle
Problem Solving
Agile Methodology
Oracle Applications
Data Systems
Apache Hadoop
Ecosystems
Financial Data Analysis
Apache Hive
Moulding Machines
Apache HBase
Job Details
Duration: 6 Months Contract with possible extension
Title: Lead Data Engineer Big Data (Cloud & Palantir)
Location: Remote
Start Date: ASAP! Apply Now
Position Overview:
The Lead Data Engineer will architect, build, and manage scalable cloud-based data pipelines supporting enterprise analytics and reporting. This role requires deep big data expertise, hands-on experience with Palantir Foundry, and strong collaboration with business and analytics teams. The engineer will serve as a Subject Matter Expert (SME) and lead end-to-end delivery of data solutions across manufacturing and financial data domains.
Key Responsibilities:
Data Engineering & Architecture
- Design, build, and maintain large-scale data pipelines in cloud environments.
- Develop best practices for data ingestion, transformation, curation, and reporting.
- Architect scalable pipelines for enterprise-wide financial and manufacturing data.
- Implement efficient ETL/ELT workflows using big data and cloud technologies.
Collaboration & Leadership
- Act as a liaison between business users, analysts, and engineering teams.
- Lead the delivery of data-driven solutions from requirements through deployment.
- Guide and mentor data engineers and analysts on best practices and technical patterns.
- Support business stakeholders by translating their needs into technical solutions.
Analytics & Optimization
- Collect and analyze large datasets to support efficiency improvements and predictive analytics.
- Ensure availability of clean, reliable, and well-modeled data for enterprise reporting.
- Recommend strategies for optimized data storage, performance, and cost management in the cloud.
Required Qualifications:
Education
- Bachelor's Degree in Computer Science, Engineering, or related field.
Experience
- 8 10 years of strong Data Engineering experience.
- Proven experience in cloud-based data platforms and big data ecosystems.
- Strong background designing and building end-to-end data pipelines at enterprise scale.
Technical Skills:
- Hands-on experience with Palantir Foundry for building scalable data workflows.
- Strong knowledge of Hadoop ecosystem: Hive, PySpark, Spark, etc.
- Proficiency in Python or Java for data engineering solutions.
- Experience with ETL tools, concepts, and pipelines.
- Expertise with NoSQL databases (HBase, MongoDB) and RDBMS (SQL Server, Oracle, MySQL).
- Experience working in cloud environments (AWS, Azure, or Google Cloud Platform).
- Solid understanding of Git and Agile SDLC processes.
Soft Skills:
- Strong analytical and problem-solving abilities.
- Excellent communication skills across technical and non-technical audiences.
- Ability to lead, mentor, and collaborate within cross-functional teams.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.