Apply Now

Azure databricks

Wilmington, DE, US • Posted 1 day ago • Updated 1 day ago

Full Time

On-site

Up to $120,000/yr

Fitment

Dice Job Match Score™

⭐ Evaluating experience...

Job Details

Skills

Databricks & Apache Spark (PySpark/Scala)
Cloud Data Engineering (Azure/AWS/GCP)
ETL/ELT Pipeline Design & Optimization
SQL & Delta Lake Expertise
Data Engineering Production Support & Governance

Summary

Key Responsibilities

Design and implement ETL/ELT pipelines using Databricks and Apache Spark.
Optimize data workflows for performance, scalability, and cost efficiency.
Collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions.
Manage and monitor Databricks clusters, jobs, and workflows.
Integrate Databricks with Azure/AWS/Google Cloud Platform services (depending on organization s cloud strategy).
Ensure data quality, governance, and security compliance across all solutions.
Troubleshoot and resolve issues related to data ingestion, transformation, and processing.
Mentor junior engineers and contribute to best practices and standards.

Required Skills & Experience

6 10 years of experience in data engineering or big data development.
Strong expertise in Databricks and Apache Spark (PySpark/Scala).
Hands-on experience with cloud platforms (Azure Data Lake, AWS S3, Google Cloud Platform BigQuery).
Proficiency in SQL and working with relational and NoSQL databases.
Experience with Delta Lake, MLflow, and Databricks notebooks.
Solid understanding of data warehousing concepts, ETL frameworks, and distributed computing.
Familiarity with CI/CD pipelines, Git, and DevOps practices.
Strong problem-solving, communication, and collaboration skills.

Experience with machine learning workflows in Databricks.

Knowledge of data governance frameworks (e.g., GDPR, HIPAA).
Exposure to streaming technologies (Kafka, Event Hub, Kinesis).
Certification in Databricks, Azure/AWS/Google Cloud Platform is a plus.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10430883
Position Id: 8944759
Posted 1 day ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Databricks AWS Lead Software Engineer

Wilmington, Delaware

•

Today

Job Description We are seeking a highly skilled Lead Data Engineer with proven experience in Databricks and AWS to join our data engineering team. As a Lead Software Engineer at JPMorganChase within the Corporate Technology Finance team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutio

Full-time

Azure data bricks admin

Wilmington, Delaware

•

Yesterday

Minimum 4+ years of handson experience administering Azure Databricks platforms in an enterprise environment. Proven experience configuring and managing Azure Databricks platform components, including: Clusters: autoscaling configurations, instance pools, cluster policies, and performance optimizationJobs and Workflows: scheduling, concurrency controls, retries, alerts, and notificationsWorkspace assets: notebooks, Git repos, libraries, init scripts, and secret scopes Strong experience with Unit

Easy Apply

Full-time

Up to $115,000

Sr. Lead Software Engineer - Pyspark/Python, AWS, Databricks

Wilmington, Delaware

•

Today

Job Description Description As a Lead Software Engineer at JPMorganChase within Corporate & Enterprise Technology, you will drive the design, development, and deployment of robust technology solutions leveraging PySpark/Python, AWS, and Databricks. In this leadership role, you will manage a local team of developers, architect scalable systems, and uphold best practices across coding standards, testing, and cloud integration. You will partner closely with cross-functional teams to deliver high-

Full-time

Lead Data Architect, Consumer Card Technology

Wilmington, Delaware

•

Today

Job Description Join a diverse team of visionaries dedicated to making a lasting impact in the Card Technology industry. As a Lead Data Architect at JPMorgan Chase within the Card Technology Team, you will have the opportunity to design and implement high-quality data architecture solutions for a range of software applications and platform products. As part of this diverse team of innovators, you will play a crucial role in advancing the Card Technology industry. Job responsibilities Drive da

Full-time

USD 152,000.00 - 215,000.00 per year

Search all similar jobs