Databricks Lead - W2 Position

Only on W2

The following requirement is open with our client. 

Client                                     : TCS

Title                                       : Databricks Lead

Location                               : Washington, DC( Hybrid)

Duration                              : 12 Months

Rate                                       : $65/hr on W2

Relevant Experience      :

Detailed Job Description:

We are looking for an experienced Databricks Lead.

Job Responsibilities:       

·         Design and develop scalable ETL/ELT pipelines using Databricks including Delta Lake, Auto Loader, and DLT.

·         Develop batch, streaming, and CDC ingestion pipelines to enable efficient and near real-time data processing.

·         Prior experience with data migration from various data sources like Redshift, DB2, Oracle, Mysql, PLSSql in to databricks

·         Implement ingestion patterns using Auto Loader with checkpointing and schema evolution for structured and semi-structured data

·         Develop and maintain reusable DBX functions for data transformation, data quality checks, and validation logic, ensuring consistent implementation across DLT and notebook-based pipelines.

·         Leverage Unity Catalog to establish secure data governance, enabling controlled access, lineage tracking, and management of External Locations and Volumes across workspaces.

·         Partner with cross-functional teams to enable secure data sharing using Delta Sharing across internal domains and external partners.

·         Integrate Power BI/Tableau/Looker with Databricks using optimized connectors (ODBC/JDBC) and Unity Catalog security controls

·         Implement Medallion architecture (Bronze, Silver, Gold layers).

·         Develop incremental and CDC-based ingestion pipelines

·         Design and implement real-time streaming pipelines using Kafka and Structured Streaming.

·         Optimize Spark jobs, SQL queries, and streaming pipelines.

·         Strong SQL and data modeling skills.

·         Experience with cloud platforms and distributed systems.

·         Build, package, and deploy Databricks data pipelines using Databricks Asset Bundles (DAB), with automated CI/CD processes managed through GitHub or GitLab.

Must Have Skills:

Databricks, ETL, Databricks Asset Bundles (DAB), Data Migration, DB2, Oracle, CDC-based ingestion pipelines, Data Governance, BI, Tableau, SQL, GitHub or GitLab.

Thanks and Regards, 

Goutham Eluri

Technical Recruiter 

ASCII Group LLC. 

38345 W. 10 Mile Rd, Ste.#365; Farmington, MI  48335 

Office:

Email:  Website: