Senior Data Scientist (AI & Databricks) - Job Description
Introduction
Join an amazing company where you can work with cutting-edge technologies and platforms. Give your career an Infinite edge, with a stimulating environment and a global work culture. Be a part of an organization where we celebrate integrity, innovation, collaboration, teamwork, and passion. A culture where every employee is a leader delivering ideas that make a difference to this world we live in.
In the Senior Data Scientist (AI & Databricks) responsibilities include, although not limited to:
- Design, develop, and deploy advanced AI, machine learning, and Generative AI solutions using the Databricks Lakehouse Platform and cloud-native architectures.
- Build, tune, and operationalize LLM-, SLM-, and RAG-based models to solve complex business problems.
- Identify, evaluate, and select best-fit LLM frameworks and models (open-source and popular) based on accuracy, performance, cost, and scalability.
- Develop and optimize Retrieval-Augmented Generation (RAG) pipelines, including embeddings, vector search, and retrieval strategies.
- Apply prompt engineering, prompt chaining, and orchestration patterns to improve reasoning and response quality.
- Perform model tuning and optimization, including fine-tuning, hyperparameter tuning, and benchmarking.
- Implement model observability, monitoring accuracy, drift, latency, hallucinations, and cost, enabling feedback loops.
- Ensure high model accuracy, robustness, and performance using rigorous evaluation techniques.
- Apply statistical techniques to solve complex business problems.
- Use Python, SQL, PySpark, and modern ML libraries to build scalable AI solutions.
- Manage experimentation and model lifecycle using MLflow.
- Deploy and operate models in cloud environments with CI/CD, scaling, and retraining.
- Communicate insights and mentor junior data scientists.
In addition to the qualifications listed below, the ideal candidate will demonstrate the following traits:
- Deep curiosity for Generative AI and LLM technologies.
- Strong analytical and statistical mindset.
- Ownership-driven approach to model quality.
- Clear communication and collaboration skills.
- Ability to balance experimentation with production readiness.
Minimum Qualifications:
- Bachelor s degree in Data Science, Computer Science, Statistics, Mathematics, or related field.
- 3+ years of hands-on Databricks experience (Spark, Delta Lake, MLflow).
- Proven experience with LLM, SLM, and RAG-based models.
- Strong skills in Python, SQL, and modern ML libraries.
- Strong background in statistics and experimentation.
- Experience with model tuning, observability, and feedback loops.
- Experience building and deploying models in cloud environments.
- Strong English communication skills.
Preferred Qualifications:
- Experience with prompt engineering, prompt chaining, and LLM orchestration frameworks.
- Experience with vector databases and embeddings.
- Familiarity with LLMOps and MLOps practices.
- Experience integrating models with APIs and enterprise systems.
- Exposure to Responsible AI practices.
- Relevant Databricks or cloud certifications.
Never miss an opportunity! Create an alert based on the job you applied for.
