Databricks Expert

Prabhav Services, Inc. originated in the year 2005 and developed into a Service and Solutions company with multiple locations. Prabhav provides a plethora of software solutions for a variety of users. Its Project Development Division meets the needs of its clients with unmatched quality and efficacy, either onsite, offsite, or offshore.

Prabhav makes technology work for the client s business. We see technology as an engine for growth, helping businesses become more adaptive, competitive, and profitable.

Our client is looking for SR. DATABRICKS SME for Onsite Role in Middlesex, NJ

We are seeking a skilled Databricks Data Engineer to join our data platform team. You will be responsible for designing, developing, and maintaining scalable, robust data pipelines using the Databricks Lakehouse architecture. You will work with data scientists, analysts, and other stakeholders to unlock the value of our data through high-performance computing, streaming, and advanced analytics.

• Lakehouse Pipeline Design: Develop end-to-end ingestion, transformation, and modeling pipelines (Bronze, Silver, Gold layers) on Delta Lake.
• Spark Optimization: Tune Spark jobs, managing partitioning, caching, and file sizes to ensure cost-efficient, high-performance processing.
• Data Governance & Quality: Implement ACID transactions, time travel, and schema enforcement using Unity Catalog, while managing data retention and security.
• Workflow Orchestration: Build, automate, and monitor production jobs using Databricks Jobs, Workflows, and Airflow.
• Collaboration: Work with data scientists to implement machine learning models using MLflow and support data modeling needs.
• DataOps & CI/CD: Implement DevOps best practices (Git, Terraform) for deploying notebook-based code and infrastructure across environments.
  Digiqt +4

• Experience: 2+ years of hands-on experience in data engineering, with a specific focus on Databricks.
• Technical Skills: Proficiency in PySpark (Python) and SQL is essential.
• Databricks Ecosystem: Strong experience with Delta Lake, DLT (Delta Live Tables), Spark SQL, and Databricks Jobs.
• Cloud Proficiency: Experience with cloud data platforms (Azure Databricks, AWS Databricks, or Google Cloud Platform).
• CI/CD: Experience with Git for version control and infrastructure-as-code principles.

• Experience with Machine Learning Lifecycle Management (MLflow).
• Databricks Certifications (e.g., Databricks Certified Data Engineer Professional).