AI Data Engineer

Hybrid in Charlotte, NC or Hartford, CT

Top Skills' Details

***This is a Hybrid on-site role in Charlotte, NC or Hartford, CT
- 6+ years of Data Engineering experience with large databases (i.e. Snowflake, Terradata, etc.)
- Strong experience with Python
- 1-2 years of experience with Google Cloud Platform AI - especially Vertex AI, Workbench, Model Garden

Secondary Skills - Nice to Haves

• Snowflake
• Terradata

Job Description

We are seeking a talented and motivated AI Data Engineer to join our innovative team. The ideal candidate will gain strong expertise in generative AI technologies, experience in implementing AI pipelines, and knowledge of vector and graph databases. We're looking for someone with some level of hands-on experience in prompt engineering, unstructured data processing, and agentic workflow implementation. As a AI Data Engineer, you will contribute to the development of advanced AI systems that leverage state-of-the-art generative models, implement efficient RAG (Retrieval-Augmented Generation) architectures, and integrate with our data infrastructure. Familiarity with Snowflake integration and insurance industry use cases is a plus.

Primary Job Responsibilities
Design, develop, and implement complex data pipelines for AI/ML, including those supporting RAG architectures, using technologies such as Python, Snowflake, AWS, Google Cloud Platform, and Vertex AI.
Implement on end-to-end generative AI pipelines, from data ingestion to pipeline deployment and monitoring.
Build and maintain data pipelines that ingest, transform, and load data from various sources (structured, unstructured, and semi-structured) into data warehouses, data lakes, vector databases (e.g., Pinecone, Weaviate, Faiss - consider specifying which ones you use or are exploring), and graph databases (e.g., Neo4j, Amazon Neptune - same consideration as above).
Develop and implement data quality checks, validation processes, and monitoring solutions to ensure data accuracy, consistency, and reliability.
Implement end-to-end generative AI data pipelines, from data ingestion to pipeline deployment and monitoring.
Develop complex AI systems, adhering to best practices in software engineering and AI development.
Work with cross-functional teams to integrate AI solutions into existing products and services.
Keep up-to-date with AI advancements and apply new technologies and methodologies to our systems.
Assist in mentoring junior AI/data engineers in AI development best practices.
Implement and optimize RAG architectures and pipelines.
Develop solutions for handling unstructured data in AI pipelines.
Implement agentic workflows for autonomous AI systems.
Develop graph database solutions for complex data relationships in AI systems.
Integrate AI pipelines with Snowflake data warehouse for efficient data processing and storage.
Apply GenAI solutions to insurance-specific use cases and challenges.

Required Qualifications:
Bachelor's in Computer Science, Artificial Intelligence, or a related field.
6+ years of experience in data engineering
Awareness of data engineering, with at least some hands on with generative AI technologies.
Ability to showcase implementation of production-ready enterprise-grade GenAI pipelines.
Experience & awareness of prompt engineering techniques for large language models.
Experience & awareness in implementing Retrieval-Augmented Generation (RAG) pipelines, integrating retrieval mechanisms with language models.
Knowledge of vector databases and graph databases, including implementation and optimization.
Experience & awareness in processing and leveraging unstructured data for GenAI applications.
Proficiency in implementing agentic workflows for AI systems.

Additional Skills & Qualifications

Experience & awareness in implementing Retrieval-Augmented Generation (RAG) pipelines, integrating retrieval mechanisms with language models.
Knowledge of vector databases and graph databases, including implementation and optimization.
Experience & awareness in processing and leveraging unstructured data for GenAI applications.
Proficiency in implementing agentic workflows for AI systems.