Job Title: Agentic AI Data bricks
Location: Raritan, NJ
Duration: 12 Months + Extension
Bill Rate: $97/hr
Job Type: W2 Contract
Client: To Be Discussed Later
Work Authorization: US-Citizen, H-1B, OPT-EAD, GC-EAD
Job description:
Experience 10 years Location Flexible Function Data Engineering AIML Engineering Cloud Data Platforms
Role Summary:
We are looking for an experienced Databricks Azure professional with strong expertise in cloud scale data engineering lakehouse architecture Agentic AI and Data bricks Agent Bricks The ideal candidate will have 7 or more years of experience designing building and operationalizing enterprise data and AI platforms on Azure Databricks with handson exposure to generative AI solutions retrieval augmented generation vector search AI agents model serving MLflow Unity Catalog and production grade governance practices
Roles Responsibilities:
Design develop and maintain scalable data engineering and AIML solutions on Azure Databricks using Delta Lake Unity Catalog Databricks Workflows Databricks SQL and MLflow
Build and deploy Agentic AI applications using Databricks Agent Bricks Mosaic AI Agent Framework Knowledge Assistants Supervisor Agents toolcalling agents and multiagent orchestration patterns
Develop retrievalaugmented generation solutions using Databricks Vector Search embeddings model serving endpoints foundation models and governed enterprise knowledge sources
Architect lakehouse solutions across ingestion transformation orchestration governance security performance optimization and monitoring layers
Integrate Azurenative services such as Azure Data Lake Storage Azure Data Factory Azure Key Vault Azure DevOps Microsoft Entra ID Azure Functions and Azure OpenAI where applicable
Implement robust CICD pipelines for Databricks notebooks jobs bundles ML models and AI agent deployments using Azure DevOps or equivalent tools
Establish data and AI governance standards using Unity Catalog access controls lineage audit logging model registry and secure serving patterns
Optimize Spark workloads Delta tables SQL warehouses serverless compute cluster policies job performance and cost utilization
Collaborate with data engineers ML engineers architects product teams and business stakeholders to convert business requirements into reliable data and AI capabilities
Define best practices reusable frameworks coding standards deployment patterns and operational runbooks for Databricks and Agentic AI platforms
Support production operations incident resolution root cause analysis monitoring SLA management and continuous improvement for data and AI workloads
Mentor junior engineers and contribute to knowledge sharing technical design reviews proofofconcepts and platform modernization initiatives
Must Have Skills:
7 years of overall experience in data engineering analytics engineering AIML engineering or cloud data platform engineering
Strong handson experience with Azure Databricks Apache Spark PySpark Spark SQL Delta Lake Delta Live Tables Databricks Workflows Databricks SQL and Lakehouse architecture
Practical experience with Databricks Agent Bricks Agentic AI patterns AI agents tool calling multiagent systems Knowledge Assistants Supervisor Agents and governed AI application development
Handson experience building generative AI solutions such as RAG applications LLM chains prompt engineering workflows vector search implementations embeddings pipelines and model serving solutions
Strong understanding of MLflow for tracking model lifecycle management tracing evaluation monitoring and deployment of ML and GenAI workloads
Experience with Unity Catalog data governance access control catalogschematable permissions lineage auditability and secure AIdata sharing patterns
Proficiency in Python SQL PySpark data modeling performance tuning API integration and productiongrade software engineering practices
Experience with Azure services including ADLS Gen2 Azure Data Factory Azure DevOps Azure Key Vault Microsoft Entra ID Azure Monitor and Azure OpenAI or comparable LLM services
Strong knowledge of CICD infrastructureascode concepts Gitbased development Databricks Asset Bundles environment promotion and release management
Ability to troubleshoot complex data pipelines Spark performance issues job failures security constraints and AI model or agent behavior in production environments
Good understanding of Responsible AI data privacy model evaluation hallucination mitigation guardrails prompt safety and enterprise AI governance
Excellent communication stakeholder management documentation and technical leadership skills
GoodtoHave Skills
Experience with LangChain LangGraph OpenAI Agents SDK LlamaIndex MCP servers AI Gateway or custom agent frameworks
Exposure to Databricks Apps Genie Foundation Model APIs Model Context Protocol integrations and enterprise chatbot or copilot implementations
Experience with streaming data pipelines using Structured Streaming Event Hu
Mandatory Skills : Azure Data Factory, Databricks
Equal opportunity employer: We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.
Never miss an opportunity! Create an alert based on the job you applied for.
