Lead Data Engineer Palantir & PySpark (MUST)

Overview

Remote
$Depending upon Experience
Full Time
Accepts corp to corp applications
Contract - W2

Skills

Azure
GCP
etl
sql
AWS
S3
Python
Git
agile
Data Modeling
DATA GOVERNANCE
Snowflake
CI/CD
PySpark
databricks
REINSURANCE
DATA INTEGRATION
metadata management
Data Pipelines
Palantir
dataframes
Resilient Distributed Datasets / RDDs

Job Details

Lead Data Engineer Palantir & PySpark (Lead Data engineer role where experience should be on both hands on and leading the team.)

Candidate preference: Should be in the United States and from East coast.

Experience: 8-15 Years

Location : Remote

Client Industry: Reinsurance / Insurance

Lead Data engineer role where experience should be on both hands on and leading the team.

Key Responsibilities:

Hands-on Lead experience working in Reinsurance/ Insurance domain.

Work within the Palantir Foundry platform to design robust workflows, manage datasets, and ensure efficient data lineage and governance.

Ensure data security, compliance, and governance in line with industry and client standards.

Design, develop, and maintain data pipelines and ETL workflows using PySpark, SQL, and Palantir Foundry.

Required Skills & Qualifications:

8 15 years of overall experience in data engineering roles.

Strong hands-on expertise in PySpark (DataFrame, RDDs, performance optimization).

Proven experience working with Palantir Foundry or similar data integration platforms.

Good understanding of reinsurance including exposure, claims, and policy data structures.

Should be proficient in SQL, Python, and working with large datasets in distributed environments.

Experience with cloud platforms (AWS, Azure, or Google Cloud Platform) and related data services (e.g., S3, Snowflake, Databricks).

Should be familiar with CI/CD pipelines, version control (Git), and Agile delivery methodologies.

The successful candidate who meets the above skills criteria can email their resume to:

Thank you.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.