Apply Now

Senior Data Engineer

Dallas, TX, US • Posted 1 hour ago • Updated 1 hour ago

Full Time

Travel Required

Able to Sponsor

On-site

$80,000 - $120,000/yr

Fitment

Dice Job Match Score™

🤯 Applying directly to the forehead...

Job Details

Skills

Amazon Web Services
Amazon S3
Apache Spark
Data Modeling
Databricks
Extract
Transform
Load
PySpark
Python
SQL
Microsoft Azure
Artificial Intelligence
Data Lake
Data Masking

Summary

Job Title: Senior Data Engineer (Data Federation & Lakehouse)

Location: Dallas, NJ, NY, Chicago (onsite for 5 days)

Full time on W2

Interview process:

HRT -> internal Interview -> Client interview

Responsibilities:

As a Senior Data Engineer, you will be responsible for breaking down data silos. This role focuses on building a unified, high-performance data layer using Data Federation techniques. You won't just move data; you will architect a Data Lakehouse environment where disparate sources feel like a single, cohesive database for our analytics and AI teams.

Core Responsibilities:

Data Federation Architecture: Design and implement federated query layers (e.g., Starburst/Trino) to allow high-speed analytics across distributed data sources without unnecessary data movement.
ETL/ELT Pipeline Development: Build scalable, distributed data processing pipelines using Python and Apache Spark (PySpark).
Lakehouse Implementation: Manage and optimize modern table formats like Delta Lake, Apache Iceberg, or Hudi to bring ACID transactions to our data lake.
Performance Tuning: Optimize Spark jobs and SQL queries across the federation layer to minimize latency and manage compute costs.
Governance & Security: Implement fine-grained access control and data masking within the federation engine to ensure data privacy across all connected platforms.

Technical Requirements

Python & Spark: 5+ years of experience with Python and deep expertise in Apache Spark tuning (partitioning, shuffling, caching).
Data Federation Tools: Hands-on experience with Starburst Enterprise, Trino (Presto), or Dremio.
Lakehouse Ecosystem: Proven track record working with Delta Lake or Iceberg architectures.
Cloud Platforms: Extensive experience with AWS (EMR, S3, Glue), Azure (Databricks, ADLS), or Google Cloud Platform.
SQL Mastery: Expert-level SQL skills for complex analytical queries and query plan analysis.
Data Modeling: Proficiency in designing Star/Snowflake schemas and understanding "Medallion Architecture" (Bronze, Silver, Gold layers).

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10462843
Position Id: 8965615
Posted 1 hour ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Lead Data Engineer

Dallas, Texas

•

Today

Day to Day Job Duties: Perform end-to-end datastore migration from on-prem Data Lake to AWS-hosted Lakehouse as part of the migration factory teamRefactor and migrate data pipelines including extraction logic and job scheduling.Execute data transfer ensuring integrity and reliability.Translate legacy SQL and Spark logic to Snowflake and Iceberg.Analyze data usage patterns to deliver data products.Build data reconciliation frameworks.Collaborate with stakeholders for validation and sign-off.Act a

Easy Apply

Full-time

Depends on Experience

Applications Development Senior Programmer Analyst - Assistant Vice President

Remote or Irving, Texas

•

Today

The Applications Development Senior Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities. Responsibilities: Develop and maintain scalable ETL/ELT pipelines using PySpark, Spark SQL, and Delta Lake on Databricks.Support imple

Full-time

USD 107,120.00 - 160,680.00 per year

Senior Big Data Engineer

Remote or Irving, Texas

•

Today

The Senior Big Data Engineer is a hands on seasoned professional role. Applies in-depth disciplinary knowledge, contributing to the development of new techniques and the improvement of processes and work-flow for the area or function. Integrates subject matter and industry expertise within a defined area. Requires in-depth understanding of how areas collectively integrate within the sub-function as well as coordinate and contribute to the objectives of the function and overall business. Evaluate

Full-time

USD 96,400.00 - 144,600.00 per year

Data Engineer (Databricks, Spark, Python)

Irving, Texas

•

Yesterday

Job Title: Data Engineer (Databricks, Spark, Python) Employment Type: W2 Only Work Model: Hybrid - 3 Days Onsite / Week Locations: Irving, TX Tampa, FL Jersey City, NJ Wilmington Delaware Job Description We are looking for an experienced Data Engineer with strong hands-on expertise in Databricks, Apache Spark, and Python to support enterprise-scale data engineering initiatives for Infosys/Citi. The ideal candidate should have solid experience building scalable data pipelines, working with la

Easy Apply

Full-time, Part-time, Third Party, Contract

USD 55-60

Search all similar jobs