Overview
Skills
Job Details
Position: Databricks Developer
Location: Richardon, Hybrid
Long Term Contract
Job Description:
As a Senior Databricks Developer, you will lead the development and implementation of advanced data processing systems on the Azure platform. You will optimize and manage large datasets, ensuring their effective integration and utilization while adhering to best practices in the cloud data architecture. You will also be expected to provide technical leadership and mentorship to junior team members.
Key Responsibilities:
● Lead the development and deployment data processing solutions using Azure Databricks.
● Understanding of ETL/ELT processes from various sources, including streaming data.
● Collaborate with data engineers, architects and business analysts to gather and analyze requirements.
● Develop, optimize, and manage complex data pipelines and workflows.
● Ensure data quality, integrity, security and compliance measures are adhered to across various data sources.
● Troubleshoot and resolve performance issues related to Databricks solutions.
● Document technical specifications and maintain comprehensive project documentation.
● Provide technical guidance and mentorship to junior developers.
Qualifications & Skills:
● Bachelor's degree in Computer Science, Information Technology or related field.
● Minimum of 5 years of experience in data engineering or similar roles.
● Proven hand-on experience with Azure Databricks and data processing frameworks.
● Strong understanding of data warehousing, ETL processes and data pipeline design.
● Experience with SPQ, Python and Spark.
● Experience using Jira, Confluence, GitHub, UCD, etc.
● Excellent problem-solving and analytical skills.
● Effective communication, teamwork, teaching/mentorship abilities.
● Technical Documentation.