Java AI Engineer / AI Engineer (Java backend)

Job Description: AI Engineer (Java Backend)

Experience: 11+ Years
Employment Type: Full-Time (Contract W2 only)

About the Role

We are looking for a seasoned AI Engineer with a strong Java backend foundation to design, build, and scale production-grade AI/ML-powered systems. This role sits at the intersection of enterprise Java engineering and applied AI — you'll be responsible for integrating LLMs, building intelligent services, and ensuring these systems are robust, scalable, and production-ready within a Java-centric ecosystem.

Key Responsibilities

• Design and develop scalable backend services in Java (Spring Boot/Spring Framework) that integrate AI/ML and LLM-based capabilities into enterprise applications.
• Architect and implement RAG (Retrieval-Augmented Generation) pipelines, vector search, and semantic retrieval systems.
• Integrate with LLM providers (OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, etc.) via APIs, and build robust prompt-engineering and orchestration layers.
• Build and maintain microservices architectures exposing AI capabilities (REST/gRPC APIs) for internal and external consumption.
• Own end-to-end MLOps/LLMOps practices — model versioning, deployment pipelines, monitoring, and observability for AI services.
• Collaborate with Data Science/ML teams to productionize models, embeddings, and inference pipelines at scale.
• Design for performance, low latency, and high throughput, applying caching, async processing, and message queue patterns (Kafka, RabbitMQ).
• Implement vector databases (Pinecone, Weaviate, Milvus, pgvector, Elasticsearch) for semantic search and knowledge retrieval use cases.
• Drive best practices around security, data privacy, and responsible AI (PII handling, guardrails, hallucination mitigation).
• Mentor junior engineers, conduct code/design reviews, and contribute to architectural decision-making as a technical leader.
• Partner with product and business stakeholders to translate requirements into scalable technical solutions.
• Evaluate and prototype emerging frameworks/tools (LangChain4j, Spring AI, LlamaIndex, Semantic Kernel) for enterprise fit.

Required Skills & Experience

• 11+ years of hands-on software engineering experience, with deep expertise in Core Java, Java 11/17+, Spring Boot, Spring Framework.
• Proven experience (2+ years) building AI/LLM-integrated applications — prompt engineering, embeddings, RAG, agentic workflows, or ML model serving.
• Strong understanding of microservices, distributed systems, and API design (REST, GraphQL, gRPC).
• Hands-on experience with vector databases and semantic search implementations.
• Experience with frameworks like Spring AI, LangChain4j, or equivalent Python-based orchestration (LangChain, LlamaIndex) with ability to bridge into Java services.
• Solid grasp of cloud platforms (AWS/Azure/Google Cloud Platform) — particularly AI/ML services (Bedrock, SageMaker, Azure OpenAI, Vertex AI).
• Experience with containerization and orchestration (Docker, Kubernetes).
• Strong database fundamentals — SQL (PostgreSQL/MySQL) and NoSQL (MongoDB, Redis).
• Familiarity with CI/CD pipelines, Infrastructure as Code (Terraform), and observability tooling (Prometheus, Grafana, ELK).
• Understanding of LLM concepts — tokenization, embeddings, fine-tuning vs. prompt engineering, context windows, function/tool calling.
• Excellent problem-solving skills with a track record of leading complex, high-scale engineering initiatives.
• Strong communication skills; experience working directly with cross-functional and leadership stakeholders.

Nice to Have

• Experience with Python for ML/AI prototyping alongside Java backend development.
• Exposure to agentic AI frameworks (multi-agent orchestration, tool-calling agents).
• Familiarity with MCP (Model Context Protocol) or similar AI integration standards.
• Prior experience in a technical lead or architect capacity.
• Contributions to open-source AI/Java tooling.
• Domain experience in [Finance/Healthcare/Retail/etc. — customize as needed].

Education

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field (or equivalent practical experience).