Gen AI Architect (Eval Framework)

10+ years of experience in software engineering, machine learning, data science, or artificial intelligence.

• Sound experience with Retrieval-Augmented Generation (RAG), fine-tuning, and multi-agent orchestration.
• Experienced in developing GenAI applications leveraging multi-agent frameworks and/or graph-based GenAI approaches (e.g., GraphRAG).
• Proficient in using common NLP and/or ML Python frameworks, such as PyTorch, TensorFlow, Transformers/Hugging Face, and NumPy.
• LLM skills including fine-tuning, LLMOps, function-calling, and retrieval augmented generation (RAG).
• Familiarity with data governance, AI ethics, and responsible AI practices.
• Strong proficiency in Python.
• Experience following software best practices in team settings, including version control (Git), CI/CD, documentation, & unit testing.
• Exposure to Microsoft Azure or similar cloud computing ecosystem.
• Ability to design scalable solutions and optimize performance for business impact.
• Strong problem-solving skills and the ability to work in a fast-paced, dynamic environment.
• Familiarity with vector databases, RAG pipelines, and agentic frameworks.
• Excellent communication and documentation skills.

• Advanced GenAI Expertise: Experience developing applications using multi-agent frameworks and/or graph-based approaches such as GraphRAG and LangGraph.
• Cloud & MLOps Proficiency: Hands-on experience with Azure AI services, containerization (Docker/Kubernetes), and ML pipelines.

• Design and develop GenAI-based applications using advanced techniques such as Retrieval-Augmented Generation (RAG), text-to-SQL, function calling, and agentic architectures.
• Implement multi-agent frameworks and explore graph-based GenAI approaches (e.g., GraphRAG) for complex problem-solving.
• Define and enforce evaluation standards and best practices for GenAI agents, RAG pipelines, and multi-agent orchestration.
• Performance evaluations to optimize ML and GenAI models for accuracy, scalability, and business impact.
• Engage with business stakeholders to understand requirements, gather feedback, and tailor solutions to meet strategic goals.
• Translate business needs into technical specifications and actionable plans.
• Ensure adherence to software engineering best practices, including version control (Git), CI/CD pipelines, documentation, and unit testing.
• Stay current with emerging GenAI evaluation tools, frameworks, and methodologies.
• Provide technical leadership and mentor team members on best practices and emerging GenAI technologies.