Azure Data Tech Lead with Java

Role: Azure Data Tech Lead with Java

Location: Alpharetta GA (or) Berkley Hills NJ

Must Have Skills –

Skill 1 – 5Yrs of Exp – Azure Databricks

Skill 2 – 5Yrs of Exp – SQL

Skill 3 – 5Yrs of Exp – Spark

Skill 4 – 5Yrs of Exp – Java

Skill 5 – 3Yrs of Exp – Python

1. Role Overview

•         Experienced Data Technical Lead to design, develop, and support scalable cloud-based data platforms and streaming data pipelines.

•         The ideal candidate should possess strong expertise in Databricks, PySpark, Python, Java-based streaming technologies, GitLab CI/CD pipelines, and cloud migration initiatives.

2. Key Responsibilities

•         Design and develop scalable Databricks ETL/ELT pipelines (Lakeflow & LakeBase) using Azure Databricks, PySpark, and Python.

•         Implement real-time and batch data ingestion frameworks using Kafka and Java-based streaming solutions.

•         Develop and optimize data processing workflows in Azure Databricks.

•         Integrate and manage data movement between PostgreSQL, YugabyteDB (Cassandra-based NoSQL), and cloud platforms.

•         Build reusable frameworks for data ingestion, transformation, validation, and orchestration.

•         Develop SQL-based data transformations, reporting datasets, and performance optimization solutions.

•         Design and implement GitLab CI/CD pipelines for automated deployment, testing, and release management of Databricks notebooks, jobs, and data pipelines.

•         Support Snowflake on-premises to Azure cloud migration initiatives.

•         Ensure coding standards, performance tuning, monitoring, and operational stability of data pipelines.

•         Develop Power BI dashboards and reports for business intelligence and analytics reporting.

•         Develop API automation and integration solutions for data exchange between enterprise systems.

3. Required Skills.

•         Strong experience in Python and PySpark development.

•         Hands-on experience with Azure Databricks and Databricks SQL.

•         Experience in Java-based streaming and ingestion frameworks.

•         Strong knowledge of Apache Kafka streaming concepts.

•         Experience working with PostgreSQL databases.

•         Experience with YugabyteDB or Cassandra-based NoSQL databases.

•         Strong SQL development and query optimization skills.

•         Hands-on experience with GitLab CI/CD pipeline development and deployment automation.

•         Understanding cloud-based data engineering and distributed processing concepts.

•         Experience in data migration projects, especially Snowflake on-prem to Azure cloud migration.

•         Experience designing enterprise-scale data lake or lakehouse architectures.

•         Knowledge of streaming architectures and real-time analytics.

•         Familiarity with cloud monitoring and observability tools.

📩 Interested candidates, please connect or share resumes at: .