Job Title: Senior Java Developer with AI/ML (LLM Integration) Expertise

Location: Boston, MA (4 days/week onsite – Local candidates only)
Experience: 13+ years

Interview Process: Final round – In-person (F2F mandatory)

Position Overview

We are seeking a highly experienced Senior Individual Contributor with deep expertise in Java-based backend systems and hands-on experience integrating Large Language Models (LLMs) into enterprise applications. The ideal candidate will architect and build scalable, event-driven microservices on AWS while enabling intelligent capabilities such as chatbots, generative AI workflows, semantic search, and AI-assisted automation.

This role requires strong system design skills, performance optimization expertise, and practical knowledge of deploying AI-powered features into production-grade systems.

Key Responsibilities

1. Architecture & System Design

·        Architect and design modular, scalable microservices using Java and Node.js.

·        Build event-driven systems leveraging AWS services such as SNS, SQS, Lambda, ECS, and AWS Batch.

·        Design resilient distributed systems with high availability, observability, and fault tolerance.

·        Define best practices for API design, versioning, and service-to-service communication.

2. LLM & AI/ML Integration (Deep Dive)

Integrating Large Language Models (LLMs) into Java applications involves enabling AI-powered intelligence within backend services. The candidate will:

·        Integrate LLM APIs (e.g., OpenAI, Anthropic, Bedrock, etc.) into Java microservices.

·        Build Retrieval-Augmented Generation (RAG) pipelines using vector databases.

·        Design prompt engineering frameworks for domain-specific use cases.

·        Implement conversational AI systems and enterprise-grade chatbots.

·        Develop document ingestion pipelines for semantic search and contextual retrieval.

·        Optimize token usage, latency, and cost for production LLM workloads.

·        Implement guardrails for AI safety, hallucination mitigation, and content moderation.

·        Design caching, async processing, and streaming responses for scalable AI workloads.

·        Secure AI services with proper authentication, rate limiting, and monitoring.

·        Work with embeddings and similarity search for knowledge-based AI systems.

·        Integrate LLM workflows with event-driven architectures (e.g., triggering AI pipelines via SQS/SNS events).

3. Performance Optimization & Reliability

·        Profile and tune JVM performance (tuning, memory optimization, threading).

·        Conduct performance benchmarking and load testing.

·        Improve latency and throughput in distributed systems.

·        Implement observability (logging, tracing, monitoring).

·        Establish quality gates, code reviews, and production reliability standards.

4. AWS & Cloud Engineering

·        Design cloud-native solutions using AWS best practices.

·        Deploy containerized applications using ECS/Fargate.

·        Build serverless event processing pipelines using Lambda.

·        Implement CI/CD pipelines and infrastructure-as-code.

·        Ensure cost optimization and security compliance.

Required Qualifications:

·        13+ years of backend development experience.

·        10+ years of hands-on Java development experience.

·        Strong experience with Node.js in distributed systems.

·        Proven expertise in designing large-scale distributed microservices architectures.

·        Deep knowledge of AWS (SNS, SQS, Lambda, ECS, Batch, IAM, CloudWatch).

·        Hands-on experience integrating LLMs or AI APIs into backend systems.

·        Strong understanding of event-driven architecture patterns.

·        Experience with performance profiling and optimization in production environments.

·        Experience building modular, testable, maintainable systems.

·        Strong understanding of REST APIs and asynchronous processing.

·        Bachelor’s or Master’s degree in Computer Science or related field.

Preferred Qualifications:

·        Experience with vector databases (Pinecone, OpenSearch, FAISS, etc.).

·        Knowledge of RAG architectures and embeddings.

·        Experience with Spring Boot ecosystem.

·        Experience implementing enterprise chatbots.

·        Familiarity with AI governance, explainability, and monitoring.

·        Experience in financial, healthcare, or large enterprise domains.

Technical Stack (Indicative):

·        Languages: Java (Spring Boot), Node.js

·        Cloud: AWS (SNS, SQS, Lambda, ECS, Batch)

·        Architecture: Microservices, Event-Driven Systems

·        AI/ML: LLM APIs, Prompt Engineering, RAG, Vector Search

·        DevOps: CI/CD, Docker, CloudWatch

·        Databases: SQL/NoSQL, Vector Databases