The Data Engineering Manager is a hands-on role and will be the primary architect and technical lead for the data infrastructure powering our next-generation Agentic AI products. Acting as a hands-on leader, you are responsible for the team’s overall delivery, translating complex product requirements into actionable technical tasks for a small engineering squad. You will design the multi-modal data stores (Vector and Graph) that serve as the Active Memory for autonomous agents while remaining deeply embedded in the codebase to drive execution.
Key Responsibilities
Technical Execution: Lead the technical delivery by translating high-level product roadmaps into actionable development cycles. You will own the task breakdown and manage the workflow to ensure high-quality output from the team
Strategic Data Architecture: Architect and directly implement multi-modal data pipelines that process structured parts catalogs and unstructured sources (PDFs, Word, PNG/SVG diagrams) into specialized Vector and Graph data stores.
Knowledge Layer Development: Design and optimize the storage of embeddings in Vector Databases (e.g., Pinecone, ChromaDB, Vertex AI Search) and Graph Databases (e.g., FalkorDB, Neo4j) to enable multi-step agentic reasoning across disparate data sources.
Agent-Driven Development: Deeply integrate autonomous coding agents into your daily workflow to plan, generate, and refactor data infrastructure and microservices.
Evaluation Pipelines: Collaborate with AI Engineers to build Gold Dataset pipelines used for the automated verification, retrieval quality measurement, and confidence scoring of AI outputs.
Microservices: Develop and maintain scalable data services and RESTful APIs using Python (FastAPI/Django) to provide structured, validated data.
Cloud Operations: Deploy and monitor high-scale data workloads on Google Cloud Platform (Vertex AI, BigQuery), ensuring system reliability, security, and cost-effectiveness
Skills and Experience
10+ years of Software/Data Engineering experience, with a proven history of leading technical teams.
Proven track record of designing and building production-grade AI data systems.
Technical Stack: Advanced Python (FastAPI, Pydantic, SQLAlchemy) and SQL mastery for building scalable microservices.
AI: Hands-on experience with Vector Databases (Pinecone, ChromaDB), RAG pipelines, and GraphRAG patterns.
Data Tooling: Deep experience with Prefect (preferred) or Apache Airflow or Cloud Composer, BigQuery, and DataProc.
Cloud Infrastructure: Experienced in working with cloud platforms (Google Cloud Platform, AWS) and deploying data workloads and pipelines at scale.
Coding Agent: Demonstrated proficiency in using coding agents to accelerate the SDLC and plan and code complex engineering tasks.
