Lead Data (Snowflake, Airflow)

Position: Lead Data (Snowflake, Airflow)

Location: New York, NY; Westport, CT

Duration: 12+Months with possible extensions

Main skill: Snowflake, Airflow and Graph RAG

Responsibilities:

1. Data Integration and Management:

• Identify and integrate structured, unstructured, and semi-structured data sources.
• Implement RBAC and develop logical data models, ensuring compliance and data lineage.
• Document business transformations and establish data quality rules.
• Create a comprehensive data catalog and register data assets with metadata.

1. Design Implementation:

• Translate and implement the architectural blueprint for the Data Factory.
• Ensure robust data management and quality to support automated investment processes and AI-driven analytics.
• Implement Neo4j-powered Knowledge Graph and ensure investment decision traceability.

1. Advanced Data Solutions:

• Develop and maintain multi-dimensional asset ontology and temporal versioning.
• Incorporate GraphRAG for transforming unstructured documents into queryable graph entities.

1. Security and Access Control:

• Implement property-based access control and oversee real-time data ingestion pipelines.

1. Data Architecture and Management:

• Oversee scalable data warehousing solutions using Snowflake.
• Develop frameworks for data quality and governance, including metadata registration and sensitivity/security considerations.

Requirements:

1. Technical Expertise:

• Strong proficiency in Snowflake and Airflow (mandatory).
• Deep knowledge of GraphRAG and Graph Data Structures.
• Strong background in Data Architecture/Design with experience in market data sources.

1. AI and Integration Experience:

• Experience integrating heavy AI-driven architectures.
• Ability to share knowledge with project and client teams on Cypher, Graph, and RAG traversals.

1. Additional Skills:

• AWS experience is a plus.