Overview
On Site
Depends on Experience
Full Time
Accepts corp to corp applications
Skills
GraphRAG
Snowflake
Airflow
Job Details
Role: Lead Data Engineer: Snowflake, Airflow and Graph RAG
Project duration: 12 months
Location: (New York, NY; Westport, CT) - Hybrid
Project duration: 12 months
Location: (New York, NY; Westport, CT) - Hybrid
Must Have: Snowflake, Airflow and Graph RAG
Responsibilities:
* Identify and integrate structured, unstructured, and semi-structured data sources.
* Implement RBAC and develop logical data models, ensuring compliance and data lineage.
* Document business transformations and establish data quality rules.
* Create a comprehensive data catalog and register data assets with metadata.
* Translate and implement the architectural blueprint for the Data Factory.
* Ensure robust data management and quality to support automated investment processes and AI-driven analytics.
* Implement Neo4j-powered Knowledge Graph and ensure investment decision traceability.
* Develop and maintain multi-dimensional asset ontology and temporal versioning.
* Incorporate GraphRAG for transforming unstructured documents into queryable graph entities.
* Implement property-based access control and oversee real-time data ingestion pipelines.
1. Data Architecture and Management:
* Oversee scalable data warehousing solutions using Snowflake.
* Develop frameworks for data quality and governance, including metadata registration and sensitivity/security considerations.
Requirements:
* Strong proficiency in Snowflake and Airflow (mandatory).
* Deep knowledge of GraphRAG and Graph Data Structures.
* Strong background in Data Architecture/Design with experience in market data sources.
* Experience integrating heavy AI-driven architectures.
* Ability to share knowledge with project and client teams on Cypher, Graph, and RAG traversals.
* AWS experience is a plus.
* Identify and integrate structured, unstructured, and semi-structured data sources.
* Implement RBAC and develop logical data models, ensuring compliance and data lineage.
* Document business transformations and establish data quality rules.
* Create a comprehensive data catalog and register data assets with metadata.
* Translate and implement the architectural blueprint for the Data Factory.
* Ensure robust data management and quality to support automated investment processes and AI-driven analytics.
* Implement Neo4j-powered Knowledge Graph and ensure investment decision traceability.
* Develop and maintain multi-dimensional asset ontology and temporal versioning.
* Incorporate GraphRAG for transforming unstructured documents into queryable graph entities.
* Implement property-based access control and oversee real-time data ingestion pipelines.
1. Data Architecture and Management:
* Oversee scalable data warehousing solutions using Snowflake.
* Develop frameworks for data quality and governance, including metadata registration and sensitivity/security considerations.
Requirements:
* Strong proficiency in Snowflake and Airflow (mandatory).
* Deep knowledge of GraphRAG and Graph Data Structures.
* Strong background in Data Architecture/Design with experience in market data sources.
* Experience integrating heavy AI-driven architectures.
* Ability to share knowledge with project and client teams on Cypher, Graph, and RAG traversals.
* AWS experience is a plus.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.