Senior Data Scientist Agentic AI & MCP Protocol (Azure Databricks, NLP)

Responsibilities:

Design, implement, and optimize Agentic AI systems using multi-agent frameworks (LangGraph, CrewAI, Semantic Kernel, AutoGen, or custom orchestration layers).

Develop enterprise-scale NLP and Q&A pipelines leveraging Azure OpenAI (GPT-4/5), Hugging Face models, and Databricks MLflow for end-to-end lifecycle management.

Build retrieval-augmented generation (RAG) systems that integrate structured (SQL/Delta Lake) and unstructured data (documents, PDFs, emails) stored in Databricks, Cosmos DB, or Azure Data Lake.

Build knowledge-aware conversational agents capable of executing multi-step reasoning, SQL generation, and context persistence across sessions. Fine-tune and evaluate LLMs for domain-specific dialogue, intent recognition, and context-driven summarization. Integrate Databricks pipelines with Azure OpenAI endpoints, Cognitive Search, and Azure ML for scalable serving. Develop evaluation metrics and continuous feedback loops to measure chatbot accuracy, response relevance, and hallucination control. Apply Responsible AI and data governance principles through Unity Catalog integration and secure prompt-respo se logging. Collaborate cross-functionally with data engineers, software developers, and UX designers to deliver intelligent, human-like virtual assistants.

Required Qualifications
Bachelor's or Master s in Computer Science, Computational Linguistics, Data Science, or a related quantitative field.

7+ years of experience in Data Science / NLP / conversational AI

3+ years in Azure Databricks or equivalent distributed compute environment.

Proven experience in designing Multi-Agent orchestration system, autonomous reasoning, MCP protocol, LLM-driven systems using OpenAI, Azure OpenAI, Hugging Face, or similar APIs.

Knowledge in LAngGraph/ CrewAI / Auto Gen / Semantic Kernel.

Expertise in Python, PySpark, LangChain/LangGraph, Transformers, and vector databases (e.g., Milvus, Weaviate, FAISS, or CosmosDB vector search). Strong background in RAG pipeline development, prompt engineering, and LLM fine-tuning / PEFT / LoRA.

Experience integrating models through MLflow, Azure ML, and AKS for deployment.

Understanding of semantic search, embedding models, knowledge graphs, and retrieval fusion.

Excellent communication and documentation skills; proven ability to translate AI concepts into production systems.