Senior Data Scientist (AI & Databricks)

Introduction

Join an amazing company where you can work with cutting-edge technologies and platforms. Give your career an Infinite edge, with a stimulating environment and a global work culture. Be a part of an organization where we celebrate integrity, innovation, collaboration, teamwork, and passion. A culture where every employee is a leader delivering ideas that make a difference to this world we live in.

In the Senior Data Scientist (AI & Databricks) responsibilities include, although not limited to:

- Design, develop, and deploy advanced AI, machine learning, and Generative AI solutions using the Databricks Lakehouse Platform and cloud-native architectures.

- Build, tune, and operationalize LLM-, SLM-, and RAG-based models to solve complex business problems.

- Identify, evaluate, and select best-fit LLM frameworks and models (open-source and popular) based on accuracy, performance, cost, and scalability.

- Develop and optimize Retrieval-Augmented Generation (RAG) pipelines, including embeddings, vector search, and retrieval strategies.

- Apply prompt engineering, prompt chaining, and orchestration patterns to improve reasoning and response quality.

- Perform model tuning and optimization, including fine-tuning, hyperparameter tuning, and benchmarking.

- Implement model observability, monitoring accuracy, drift, latency, hallucinations, and cost, enabling feedback loops.

- Ensure high model accuracy, robustness, and performance using rigorous evaluation techniques.

- Apply statistical techniques to solve complex business problems.

- Use Python, SQL, PySpark, and modern ML libraries to build scalable AI solutions.

- Manage experimentation and model lifecycle using MLflow.

- Deploy and operate models in cloud environments with CI/CD, scaling, and retraining.

- Communicate insights and mentor junior data scientists.

In addition to the qualifications listed below, the ideal candidate will demonstrate the following traits:

- Deep curiosity for Generative AI and LLM technologies.

- Strong analytical and statistical mindset.

- Ownership-driven approach to model quality.

- Clear communication and collaboration skills.

- Ability to balance experimentation with production readiness.

Minimum Qualifications:

- Bachelor’s degree in Data Science, Computer Science, Statistics, Mathematics, or related field.

- 10+ years of experience as a Data Scientist.

- 3+ years of hands-on Databricks experience (Spark, Delta Lake, MLflow).

- Proven experience with LLM, SLM, and RAG-based models.

- Strong skills in Python, SQL, and modern ML libraries.

- Strong background in statistics and experimentation.

- Experience with model tuning, observability, and feedback loops.

- Experience building and deploying models in cloud environments.

- Strong English communication skills.

Preferred Qualifications:

- Experience with prompt engineering, prompt chaining, and LLM orchestration frameworks.

- Experience with vector databases and embeddings.

- Familiarity with LLMOps and MLOps practices.

- Experience integrating models with APIs and enterprise systems.

- Exposure to Responsible AI practices.

- Relevant Databricks or cloud certifications.