GenAI/LLM Platform Engineer

Job Title - GenAI/LLM Platform Engineer

Location- Memphis, TN
Contract - 12+ Months Contract

LLM Platform Engineer for GenAI Infrastructure Overview is seeking a highly skilled engineer with deep expertise in large language models (LLMs), information retrieval, and distributed systems to design and build the foundation for GenAI applications at enterprise scale. This role is not about building end-user AI apps it is about creating the centralized AI infrastructure that will enable 50+ teams across the organization to innovate securely, efficiently, and consistently.

ESSENTIAL JOB FUNCTIONS

• LLM Gateway Design & Development
• Architect and implement an enterprise-grade LLM Gateway serving as the single-entry point for all application teams.
• Build mediation pipelines for prompt cleansing, PII detection/masking, guardrails, linting, throttling, and quota enforcement.
• Ensure system supports multi-tenant, policy-driven routing and governance. Governance, Security, and Observability
• Design centralized controls to enforce organizational compliance and security policies across all LLM usage.
• Enable logging, monitoring, and reporting for auditability, cost tracking, and usage insights.
• Implement safeguards to mitigate brand and messaging risks, ensuring LLM outputs remain aligned with enterprise communication standards, especially in consumer-facing applications.
• Future RAG & Vector Database Integration
• Architect a shared, hybrid retrieval-augmented generation layer to support enterprise-wide use cases.
• Design abstraction layers to minimize coupling with any one vector database provider, enabling portability and cost optimization.
• Optimize information retrieval pipelines: semantic search, embeddings, metadata filtering, hybrid retrieval.
• Research & Systems Thinking: Apply formal methods, distributed systems design, and computer science principles to ensure correctness, scalability, and resilience.
• Stay current with advancements in embeddings, search algorithms, vector stores, and LLM serving platforms.
• Advise teams on tradeoffs of embedding models, vector DB choices, and migration strategies.

REQUIREMENTS

• Strong background in Computer Science, Machine Learning, or related field (M.S. preferred).
• Deep expertise in LLM integration, prompt/response mediation, and GenAI system design.
• Hands-on experience with vector databases (e.g., Pinecone, Weaviate, Milvus, FAISS, Vespa, pgvector) and embedding models.
• Knowledge of information retrieval methods: dense retrieval, semantic search, filtering, hybrid approaches.
• Experience building distributed, high-scale backend systems with strong governance/security requirements.
• Hands-on experience with cloud-native development such as AWS, including containerization technologies (e.g., Docker, Kubernetes) and CI/CD pipelines for scalable deployment and automation.
• Proficiency in at least one modern programming language used in AI/ML systems (Python, Java, Go, etc.).
• Familiarity with frameworks such as LangChain, LlamaIndex, LiteLLM, or equivalent orchestration layers.
• Familiarity with Microsoft M365 Copilot (strongly preferred) and Salesforce Agentforce (nice to have), particularly in the context of LLM integration and governance.
• Strong systems thinking: ability to design abstractions, modularity, and portability across providers.
• Excellent communication skills able to collaborate with both technical and non-technical stakeholders.
• 3-5 years experience.