We are seeking a highly technical Data Scientist with deep cloud experience, primarily AWS, to design, build, and operationalize machine learning and AI/LLM solutions. This role requires strong engineering discipline, current knowledge of generative and agentic AI, front-end delivery of model outputs, and a rigorous approach to governance, security, monitoring, and measurable model quality across the full model lifecycle.
Key Responsibilities:
Model Development & AI/ML Engineering
• Design, build, train, and validate machine learning models with strong understanding of data, feature engineering, and model behavior.
• Develop solutions using LLMs and generative AI, including OpenAI models/APIs.
• Design and implement agentic AI solutions, including multi-step, tool-using, autonomosemi-autonomous agents.
• Build and evaluate RAG solutions, including embeddings, vector search, semantic chunking, and retrieval strategies.
• Determine when to use AI/LLM solutions versus traditional deterministic or statistical approaches.
• Design human-in-the-loop (HITL) and human-on-the-loop (HOTL) workflows.
• Establish measurable testing and evaluation criteria such as accuracy, precision/recall, drift, latency, cost, hallucination rate, and bias metrics.
• Write and maintain automated test cases for model validation, including AI-assisted approaches for test coverage.
Operational Support & Model Lifecycle
• Provide operational support for deployed ML/AI models, including monitoring, incident triage, and troubleshooting.
• Implement governance and monitoring frameworks to track performance, drift, bias, and usage.
• Own model updates, including retraining, fine-tuning, versioning, and re-validation.
• Implement logging, tracing, audit trails, model decision tracking, and lineage.
Cloud, Engineering & Front-End
• Build and deploy solutions primarily on AWS, including SageMaker, Lambda, S3, ECS/EKS, and Bedrock.
• Strong coding skills in Python and Java.
• Build interactive front-end applications using React, TypeScript, or Java-based frameworks.
• Containerize workloads using Docker and manage GPU-based compute.
• Build and maintain CI/CD pipelines for ML/AI workloads.
• Apply DevSecOps principles, including security scanning, secrets management, infrastructure as code, and automated compliance checks.
Governance, Security & Responsible AI
• Ensure AI/ML solutions meet governance, legal, security, and regulatory requirements.
• Implement guardrails for data privacy, bias mitigation, content safety, access controls, and prompt-injection defenses.
• Ensure models and pipelines meet compliance requirements before production release.
