Databricks Data Engineer with Azure
Kainos Innovative Solutions Inc
Dice Job Match Score™
🔢 Crunching numbers...
Job Details
Skills
- Databricks
- Azure
Summary
Location: Houston, TX (4 days hybrid onsite)
Job Summary:
We are looking for an experienced Azure + Databricks Data Engineer to design, develop, and maintain scalable data pipelines and data solutions using Microsoft Azure and Databricks. The ideal candidate should have strong hands-on experience with PySpark, SQL, Azure Data Factory, and Delta Lake.
Key Responsibilities:
- Design and develop scalable ETL/ELT data pipelines using Azure and Databricks.
- Build data processing solutions using Databricks, Apache Spark, and PySpark.
- Develop and optimize SQL queries and data transformations.
- Create and manage pipelines using Azure Data Factory (ADF).
- Work with Azure Data Lake Storage (ADLS/ADLS Gen2) for data ingestion and storage.
- Implement data solutions using Delta Lake and Databricks.
- Perform data integration from multiple sources and ensure data quality and reliability.
- Optimize Spark jobs, Databricks notebooks, and data pipelines for performance.
- Work with Unity Catalog for data governance and access management.
- Support deployment and CI/CD processes using Azure DevOps.
- Troubleshoot data pipeline failures and resolve production issues.
- Collaborate with data architects, analysts, developers, and business teams.
Required Skills:
- Strong experience as a Data Engineer with Azure and Databricks.
- Hands-on experience with Databricks, Apache Spark, and PySpark.
- Strong SQL skills.
- Experience with Azure Data Factory (ADF).
- Experience with Azure Data Lake Storage (ADLS/ADLS Gen2).
- Strong understanding of Delta Lake and data lake architecture.
- Experience developing ETL/ELT pipelines.
- Knowledge of Databricks Unity Catalog and data governance.
- Experience with Azure DevOps and CI/CD.
- Good understanding of data modeling, data integration, and data quality.
Preferred Skills:
- Experience with Terraform or Infrastructure as Code.
- Knowledge of Azure Event Hubs/Kafka.
- Experience with Delta Live Tables (DLT).
- Familiarity with cloud security and Azure IAM.
- Experience working in Agile development environments.
- Dice Id: 91138973
- Position Id: 9065033
- Posted 2 hours ago
Company Info
About Kainos Innovative Solutions Inc
Kainos was established in early 2019 and is headquartered in Falls Church, Virginia. Though a humble beginning, we bring proven expertise in-depth knowledge gained through working in key roles in multinational companies serving global customers in the IT industry over the last few decades.
With our expertise, we understand the complexity of today’s and next generation technologies. This enables us to deliver Innovative solutions for global challenges that are scalable, optimal and secure for Government Agencies and Commercial Clients delivered on-time customized to fit the budget and meeting the business needs. Our areas of expertise include providing Digital Services, Application, Data & Infrastructure Services, Cyber Security Services, Professional Services, and Customer Support Services.
Our focus is to offer customized solutions and service with cost effective solutions. We faithfully strive to be a customer-centric organization.
Never miss an opportunity! Create an alert based on the job you applied for.Similar Jobs
It looks like there aren't any Similar Jobs for this job yet.
Search all similar jobs