Hi ,
Greetings from Healthcare Triangle !!
We do have opening for our client,
Role : Data Bricks Engineer
Location : Minneapolis, MN
Duration : Long-term Contract
About Role :
Azure Cloud, Databricks workflows, DLT, Notebooks, Python, SQL, PySpark, ADLS Gen2, ADF, Github, PL/SQL, ORACLE, ETL experience, Understanding of Delta Lake architecture, CDC patterns, and Lakehouse.
Job Description :
* Ability to design and orchestrate data pipelines using Databricks Workflows and DLT and strong Understanding of Medallion Architecture.
* Expertise in developing Databricks notebooks for scalable solutions using Python, SQL, and PySpark.
* Understanding of Delta Lake architecture, CDC patterns, and Lakehouse.
* Strong understanding of Delta table key features like ACID-compliant Delta tables, time travel, schema enforcement, Deep and shallow Clones.
* Performance tuning using Liquid clustering, partitioning, Z-ordering, and data skipping in delta tables.
* Knowledge on data governance (unity catalog),Data Security (RBAC, Fine grain access control), Data Sharing (Delta Sharing).
* Proficient in working with Azure Data Lake Storage Gen2 (ADLS Gen2),Azure Data Factory (ADF) and terraform for provisioning and management of Azure resources.
* Knowledge on dealing with Spark streaming and Auto loaders in Data bricks.
* Strong experience in analyzing and understanding legacy Informatica ETL workflows, including mappings, transformations, and data flow logic, to support seamless migration to Databricks-based data pipelines.
* Hands-on experience in implementing CI/CD pipelines using Jenkins to automate deployment of Databricks notebooks, jobs, and data workflows.
* Integrating GitHub with Databricks Repos to enable seamless code synchronization, change tracking, and automated deployment workflows.
* Knowledge of Snowflake, Oracle, MySQL, and Shell scripting for diverse data integration.
* Knowledge of Power BI and Azure Synapse Analytics for data analytics dashboards and reports.