Azure Data Engineering Specialist (Medallion, Databricks) - Reliable Software Resources - Greenfield, IN, US | Dice.com

Reliable Software Resources

Apply Now

Azure Data Engineering Specialist (Medallion, Databricks)

Greenfield, IN, US • Posted 15 days ago • Updated 15 days ago

Contract Independent

Contract W2

On-site

Depends on Experience

Apply Now

Fitment

Dice Job Match Score™

To see how well you match this job, please log in or create an account.

Once logged in, be sure to complete your profile to get the most accurate match score.

Fitment

Dice Job Match Score™

🔗 Matching skills to job...

Job Details

Skills

Azure Databricks
Azure SQL
DataLake

Summary

Azure Data Engineering Specialist (Medallion, Databricks)

Location :: Onsite Greenfields, IN

Duration :: FTE

Job Description ::

We are seeking a highly skilled Data Engineering Specialist with above mentioned Primary Skills to join our dynamic team who are at the forefront of enabling enterprises in healthcare sectors. The ideal candidate should be passionate about working on Data Engineering on Azure Cloud with strong focus on DevOps practices in building product for our customers. Effectively Communicate and Collaborate with internal teams and customer to build code leveraging or building low level design documents aligning to standard coding principles and guidelines.

Skills/Experience

4+ years of experience in Azure Databricks with PySpark and 5+ years of experience in Azure Cloud platform
3+ years of experience in ADF (Azure Data Factory), ADLS Gen 2 and Azure SQL
2+ years of experience in Databricks workflow & Unity catalog
2+ years of experience in Python programming & package builds
Prior experience in working on Agile/Scrum projects with exposure to tools like Jira/Azure DevOps
Provide constructive feedback during code reviews and be open to receiving feedback on your own code
Problem-Solving and Analytical Thinking, Capability to troubleshoot and resolve issues efficiently
Provides regular updates, proactive and due diligent to carry out responsibilities
Communicate effectively with internal and customer stakeholders
Communication approach: verbal, emails and instant messages
Strong interpersonal skills to build and maintain productive relationships with team members

Secondary Skills

Good to have Azure Entra/AD skills and GitHub Actions
Good to have orchestration experience using Airflow, Dagster, LogicApp
Good to have experience working on event-driven architectures using Kafka, Azure Event Hub
Good to have experience in managing Cloud storage solutions on Azure Data Lake Storage (ADLS)
Good to have exposure on Google Cloud Pub/Sub; Experience with Google Cloud Storage will be an advantage
Good to have experience developing and maintaining Change Data Capture (CDC) solutions preferably using Debezium
Good to have hands-on experience on data migration projects specifically involving Azure Synapse and Databricks Lakehouse

Job / Role Description

Data management experience handling Analytics workload covering design, development, and maintenance of Lakehouse solutions sourcing data from platforms such as ERP sources, API sources, Relational stores, NoSQL and on-prem sources using Databricks/PySpark as distributed /Big Data management service, supporting batch and near-real-time ingestion, transformation, and processing
Strong experience in implementing secure, hierarchical namespace-based Data Lake Storage for structured/semi-structured data, aligned to bronze-silver-gold layers with ADLS Gen2. Hands-on experience with lifecycle policies, access control (RBAC/ACLs), and folder-level security. Understanding of best practices in file partitioning, retention management, and storage performance optimization
Comprehensive experience working across the Azure ecosystem, including networking, security, monitoring, and cost management relevant to data engineering workloads. Understanding of VNets, Private Endpoints, Key Vaults, Managed Identities, and Azure Monitor. Exposure to DevOps tools for deployment automation (e.g., Azure DevOps, ARM/Bicep/Terraform)
Skilled in governing and manage data access for Azure Data Lakehouses with Unity Catalog. Experience in configuring data permissions, object lineage, and access policies with Unity Catalog. Understanding of integrating Unity Catalog with Azure AD, external metastores, and audit trails
Lead solution design discussions, mentor juniors, and ensure adherence to coding guidelines, design patterns, and peer review processes. Able to prepare Design documents for development and guiding the team technically. Experience preparing technical design documents, HLD/LLDs, and architecture diagrams. Familiarity with code quality tools (e.g., SonarQube, pylint), and version control workflows (Git)
Ability to optimize Spark jobs and manage large-scale data processing using RDD/Dataframe APIs. Demonstrated expertise in partitioning strategies, file format optimization (Parquet/Delta), and Spark SQL tuning. Familiarity with Databricks runtime versions, cluster policies, libraries, and workspace management
Experience in building efficient orchestration solutions using Azure data factory, Databricks Workflows. Ability to design modular, reusable workflows using tasks, triggers, and dependencies. Skilled in using dynamic expressions, parameterized pipelines, custom activities, and triggers; Familiarity with integration runtime configurations, pipeline performance tuning, and error handling strategies
Capable of developing T-SQL queries, stored procedures, and managing metadata layers on Azure SQL; Experience in writing modular, testable Python code used in data transformations, utility functions, and packaging reusable components. Familiarity with Python environments, dependency management (pip/Poetry/Conda), and packaging libraries. Ability to write unit tests using Pytest/unit test and integrate with CI/CD pipelines.
Demonstrates strong verbal and written communication, proactive stakeholder engagement, and a collaborative attitude in cross-functional teams. Ability to articulate technical concepts clearly to both technical and business audiences. Experience in working with product owners, QA, and business analysts to translate requirements into deliverables

Best Regards,

Thanks,

Abdul Samad

IT Talent Acquisition

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10122703
Position Id: 8818526
Posted 15 days ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Remote

•

13d ago

Our client is unable to hire H1-B candidates at this time. Client is a small, growing consulting company, with a focus on AI and Data Solutions. They are seeking a part-time Data Engineer (20 hours/week) with extensive experience on Databricks and Azure. As a Data Engineer: You will be a critical player in building the foundational data infrastructure for a leading firm's data and AI strategy.Working primarily with Databricks on the Azure platform, you will design, develop, and maintain robust d

Easy Apply

Contract

Depends on Experience

Azure Databricks Data Engineer

Remote

•

8d ago

Azure Databricks Data EngineerPosition OverviewContract Duration: 6 months Location: Remote or Hybrid (Kansas City area) Rate: Hourly (contract position) Work Authorization: Must be legally authorized to work in the US without sponsorship About the ProjectJoin CFA s initiative to extend our industry renowned lending platform through Azure Databricks-powered data integration. You ll architect and integrate with a centralized data lake that unifies disparate systems (Salesforce, loan servicing pla

Easy Apply

Contract

$60 - $100

Data Architect (with strong Azure & DevOps experience) - Remote. Candidate must be expertise in Azure Storage, Azure Purview, Python, Azure Databricks, Azure Service Fabric, and Azure DevOps.

Remote

•

Yesterday

Job Summary: We are seeking a highly skilled Sr. Data Architect with 12 to 16 years of experience to join our team in a hybrid work model. The ideal candidate will have expertise in Azure Storage, Azure Purview, Python, Azure Databricks, Azure Service Fabric, and Azure DevOps. This role involves designing and implementing cutting-edge solutions that align with our companys goals and drive innovation. The position operates during the day shift and does not require travel. Experience: 12 - 16 yea

Easy Apply

Third Party, Contract

Depends on Experience

Azure Databricks Lead/Architect - Modern Data

Detroit, Michigan

•

18d ago

Job Title: Azure Databricks Lead/Architect - Modern Data Location : Detroit MI Onsite Duration : 24+ Months Domain : Property and Casualty (P&C) is preferred Responsibilities: Develop modern data solutions and architecture for cloud-native data platforms.Build cost-effective infrastructure in Databricks and orchestrate workflows using Databricks/ADF.Lead data strategy sessions focused on scalability, performance, and flexibility.Collaborate with customers to implement solutions for data moderni

Easy Apply

Contract

Depends on Experience

Azure Data Engineering Specialist (Medallion, Databricks)

Dice Job Match Score™

Dice Job Match Score™

Job Details

Skills

Summary

Similar Jobs

Dice Job Match Score™

Vamshi Krishna