Data Platform engineer

Data Platform Engineer

Position Overview

We are seeking an experienced AI Data Platform Consultant to architect and implement an AI-ready dataset layer for the authoritative federated data catalog for research and education. This is a consulting engagement focused on extending platform capabilities to support reproducible, versioned AI workflows while maintaining strict governance and federation principles.​

The ideal candidate will design reference architecture, develop persistent data pipelines, enable semantic retrieval capabilities, and provide comprehensive knowledge transfer to internal teams and academic stakeholders

Required Capabilities & Qualifications Core Technical Requirements

• Data Platform Architecture: Proven experience with data catalogs (CKAN preferred) and downstream execution layers.
• Reproducible AI Data Engineering: Expertise in dataset versioning, branching, and experiment tracking (Deep Lake, DVC, or LakeFS patterns).
• Federated Data Access: Experience managing distributed datasets at institutional endpoints without centralizing raw data.
• Cloud-Native Research Environments: Containerized platforms (Kubernetes, Docker) and JupyterHub workflows.
• Identity-Aware Security: OIDC/CILogon, API token management, and secure access patterns.

Preferred Experience

• Academic/research platforms with reproducible workflows.
• Handling multimodal research data (text, tabular, geospatial, images, sensors).
• Translating complex infrastructure into student- and faculty-friendly documentation