Job Summary:
We are seeking an experienced and highly skilled Data Engineering Lead to spearhead our data initiatives, with a primary focus on Azure Data Lake and its associated ecosystem. The ideal candidate will possess deep architectural knowledge of Azure Data Lake and Databricks, a solid understanding of security concepts within Azure, and a proven track record of leading teams in delivering scalable and secure data solutions.
Major Responsibilities:
Architectural Design and Implementation:
Design and implement robust, scalable, and efficient data architectures leveraging Azure Data Lake and Databricks.
Define and enforce best practices for data ingestion, storage, processing, and retrieval.
Optimize data workflows to ensure high performance and cost efficiency.
Data Governance and Security:
Develop and implement security measures for Azure Data Lake, ensuring compliance with organizational and regulatory standards.
Manage role-based access control (RBAC), encryption, and other security protocols within Azure Subscriptions.
Collaborate with security teams to perform regular audits and vulnerability assessments.
Team Leadership and Collaboration:
Lead and mentor a team of data engineers, providing technical guidance and fostering professional development.
Collaborate with cross-functional teams, including data scientists, business analysts, and IT teams, to deliver data-driven solutions.
Drive agile practices and ensure timely delivery of projects.
Platform Optimization and Monitoring:
Oversee the deployment and management of Azure Data Lake and Databricks environments.
Implement monitoring and alerting systems to ensure system reliability and performance.
Evaluate and incorporate new Azure services and technologies to enhance the data platform.
Strategic Planning and Roadmap Development:
Develop and execute a roadmap for data engineering aligned with business objectives.
Stay abreast of industry trends and advancements in data engineering and Azure technologies.
Provide recommendations for long-term data strategy, including data lakehouse adoption and cloud optimization.
Education and Experience Requirements:
Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
8+ years of experience in data engineering, with at least 3 years in a leadership role.
Extensive hands-on experience with Azure Data Lake, Databricks, and other Azure services.
Proven expertise in architecting and implementing large-scale data solutions.
ship is required for this position.
Required Knowledge and Skills:
Technical Skills:
Proficiency in SQL.
Deep understanding of Azure security concepts, including subscription management, RBAC, and data encryption.
Experience with data modeling, ETL pipelines, and big data technologies.
Familiarity with CI/CD pipelines and DevOps practices in a data engineering context.
Soft Skills:
Strong leadership and team management abilities.
Excellent communication and stakeholder management skills.
Analytical mindset with a proactive approach to problem-solving.
Preferred Qualifications:
Azure certifications, such as Azure Data Engineer Associate or Azure Solutions Architect Expert.
Experience with implementing data lakehouse architectures.
Familiarity with data governance frameworks like GDPR, CCPA, or HIPAA.
Never miss an opportunity! Create an alert based on the job you applied for.
