Apply Now

DataBricks Engineer

Hybrid in New York, NY, US • Posted 16 hours ago • Updated 16 hours ago

Contract Independent

Contract W2

No Travel Required

On-site

$55/hr

Fitment

Dice Job Match Score™

⏳ Almost there, hang tight...

Job Details

Skills

DataBrick
Quick Sense
.Net
API
Apache Spark
C#
Python
PySpark
SQL
Scala
Terraform
Microsoft Azure
Git
ELT
Data Lake
DevOps
Databricks
Microservices
PostgreSQL
Extract, Transform, Load

Summary

Experience: 8-10 Years

Design and implement end-to-end data solutions by leveraging Azure Databricks for scalable data processing, .NET for custom integration/API layers, and Qlik Sense for enterprise-grade analytics. You will be responsible for ensuring that data flows seamlessly from raw sources into a refined Lakehouse architecture and is surfaced through high-performance dashboards.

Key Responsibilities

< data-path-to-node="8">1. Data Engineering (Databricks & Spark)

Pipeline Development: Build and optimize batch and streaming ETL/ELT pipelines using PySpark, Spark SQL, or Scala within Databricks.
Lakehouse Architecture: Implement Delta Lake to ensure ACID compliance, schema enforcement, and data versioning (Time Travel).
Performance Tuning: Optimize Spark jobs by managing partitions, caching, and broadcast joins to reduce DBU consumption and latency.

< data-path-to-node="10">2. Custom Integration & Backend (.NET)

API Development: Develop ASP.NET Core Web APIs to bridge Databricks/Data Lake data with external applications.
Custom Extensions: Use C# / .NET to build custom Qlik Sense extensions or automation scripts using the Qlik Engine API.
Data Ingestion: Create custom .NET workers or microservices to handle unique data ingestion scenarios that standard ETL tools cannot address.

< data-path-to-node="12">3. Analytics & Visualization (Qlik Sense)

Data Modeling: Design complex Associative Data Models in Qlik Sense, ensuring optimal use of QVDs and avoiding synthetic keys.
Dashboarding: Create intuitive, interactive dashboards using Set Analysis and advanced visualization techniques.
Integration: Embed Qlik Sense analytics into .NET applications using Qlik Sense Mashups or the Capability API.

Technical Skills Required

Category	Must-Have Skills
Data Platform	Azure Databricks, Apache Spark, Delta Lake, Unity Catalog
Programming	C# (.NET 6/8), Python (PySpark), SQL
BI & Viz	Qlik Sense (Scripting, Set Analysis, QMC, NPrinting)
Databases	SQL Server, PostgreSQL, or Snowflake
DevOps	Azure DevOps, CI/CD pipelines, Git, Terraform (optional)

Skills: DataBrick, Quick Sense, .Net

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91163556
Position Id: 8948347
Posted 16 hours ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Software Engineer III

New York, New York

•

Today

RESPONSIBILITIES: Kforce has a client seeking an onsite Software Engineer to join their team in New York. Summary: The Engineer will support and develop their Databricks-based data platform during the development phase. This role focuses on building, improving, and validating reliability patterns early, rather than production firefighting. The ideal candidate is hands-on with Databricks, PySpark, Python, and has working knowledge of Azure cloud services. This role partners closely with Data Eng

Contract

$45 - $55 hourly

W2- New York, NY (Onsite) :: Databricks Solution Architect (Only G.C / U.S.C)

New York, New York

•

22d ago

Databricks Solution Architect (Only G.C / U.S.C) 6+Months New York, NY (Onsite) **Company Overview:** We are a leading technology company specializing in data analytics and cloud solutions. Our innovative approach helps businesses harness the power of data to drive growth and efficiency. We are seeking a skilled Databricks Architect with extensive experience to join our dynamic team and contribute to building cutting-edge Lakehouse solutions. **Key Responsibilities:** - Design and implement sca

Easy Apply

Contract

Depends on Experience

AWS Data Engineer with Databricks and DBT : Lebanon, NJ : Long-Term Contract

Jersey City, New Jersey

•

Today

Hi,I''d like you totakea look at a great new position we nowhaveavailable! Your opinion would be valued with regards to this opening. I really appreciate sending an updated resume & the best number with the best time to reach you.Role:AWSDataEngineerwith Databricks and DBTLocation:Lebanon, NJ Duration:Long-Term Contract Responsibilities Design and Development ofDataPipelines: Design, build, and optimize robust ETL/ELT pipelines usingAWSservices(S3, Glue, Lambda) and the Databricks platform (Spa

Easy Apply

Third Party, Contract

Depends on Experience

Databricks Architect

New York, New York

•

Today

Role: Databricks Architect Experience: 10+ years Location: NYC Required Skills: Platform architect designing the governed MRM Lakehouse foundation on Databricks (Strong understanding of Azure Cloud, Snowflake, Databricks and Data Engineering & MLFlow)"

Easy Apply

Contract, Third Party

Depends on Experience

Search all similar jobs