Overview
Skills
Job Details
Job Title: Lead Data Engineer – Python, AWS, GenAI, FastAPI
Location: Raleigh, NC (Onsite Only)
Type: Full-Time / W2
About the Role:
We are hiring a Lead Data Engineer with a strong foundation in Python, AWS, FastAPI, and Generative AI (GenAI) technologies. This is an onsite, client-facing role based in Raleigh, NC, offering the opportunity to work on cutting-edge AI-driven platforms and cloud-native architectures.
This position is ideal for a highly skilled technologist who can lead the development of scalable microservices, make architecture-level decisions, and drive innovation using GenAI frameworks and best practices.
Must-Have Technical Skills:
- AWS Cloud Services, including IAM, S3, Lambda, etc.
- Amazon EKS (Elastic Kubernetes Service)
- AWS Bedrock for GenAI integration
- Python (Advanced scripting and backend development)
- FastAPI (Design and deployment of RESTful services)
- LangChain / LangGraph for LLM integration
- OIDC / SAML – Authentication and Single Sign-On (SSO) protocols
Nice-to-Have Technical Skills:
- Databricks (Notebooks, Model Serving, Unity Catalog, Delta Tables)
- MLflow (Model lifecycle management)
- Apache Spark (Data processing)
- Databricks SQL
Core Concepts – Required:
- Microservices Architecture & Design
- API Design and Best Practices
- Authentication, Authorization, and IAM
- Prompt Engineering for LLMs / GenAI workflows
- Database Technologies:
- Relational (RDBMS)
- NoSQL (Document, Key-Value stores)
- Kubernetes (K8s) – Deployment and orchestration
- Strong decision-making ability on tech stack selection and architectural trade-offs
Nice-to-Have Concepts & Domain Exposure:
- Prompt Management & Optimization
- Machine-Communication Protocols (MCP), A2A (Application-to-Application) Protocols
Responsibilities:
- Lead end-to-end design and implementation of data services and AI integrations
- Architect and deploy scalable microservices using FastAPI and Python
- Build and optimize GenAI-powered applications using LangChain, Bedrock, etc.
- Manage authentication and authorization workflows using OIDC/SAML
- Work closely with business stakeholders to gather requirements and deliver impactful solutions
- Own deployment pipelines and infrastructure management on AWS + EKS
- Contribute to architecture reviews, tech evaluations, and GenAI Proof of Concepts (POCs)