Tech Lead (Python + Databricks + Streamlit)

Position Summary

We are looking for a Tech Lead with strong hands-on expertise in Python and Databricks, along with exposure to Streamlit (optional) for data-driven application development. The ideal candidate will lead technical design, architecture, and delivery of scalable data and application platforms while mentoring development teams and driving engineering best practices.

Key Responsibilities

• Lead and mentor a team of developers and data engineers across the full software development lifecycle.
• Design and develop scalable backend solutions, APIs, and data processing frameworks using Python.
• Proficient in creating HLD, LLD, and detailed design documentation using visual artifacts.
• Build and optimize enterprise-grade data pipelines using Databricks, PySpark, and Spark SQL.
• Good understanding of Medallion Architecture, Data Quality rules, exception handling/correction processes, and reprocessing of correction DAGs.
• Collaborate with Business analysts, technical and business stakeholders to translate requirements into technical solutions.
• Drive code quality through design reviews, best practices, testing, and CI/CD implementation.
• Ensure application reliability, scalability, security, and performance in cloud environments.
• Support DevOps processes, deployment automation, monitoring, and production support.

Required Skills & Experience

• 10 12+ years of experience in software development and data engineering.
• Strong hands-on experience in Python
• Extensive experience with Databricks, PySpark,
• Good to have understanding of Streamlit or other modern UI frameworks for data applications Optional
• Experience building scalable ETL/ELT pipelines and distributed systems.
• Strong knowledge of Azure (preferred) or AWS cloud platforms.
• Experience with Git, CI/CD pipelines, Docker, and containerized deployments.
• Proficiency in SQL and database technologies (PostgreSQL, MySQL, MongoDB, CosmosDB, etc.).
• Strong problem-solving, leadership, and stakeholder management skills.

Preferred Qualifications

• Experience in Data Quality, Exception Handling, Metadata Management, Data Governance solutions.
• Exposure to Agile/Scrum delivery models.
• Experience leading teams in enterprise-scale data platform implementations.