Senior Platform Engineer

Role: Senior Platform Engineer

Location: Dublin, OH

Duration: 12+ months

Role Summary

Senior Data Platform Engineer responsible for designing and operating scalable data lake ingestion frameworks and lakehouse architectures using Apache Iceberg (DLX). The role focuses on high-volume data ingestion, table format optimization, and reliable platform engineering to support advanced analytics and digital experience initiatives.

Key Responsibilities

• Architect and build data lake ingestion frameworks for batch and streaming workloads
• Design and manage Apache Iceberg table structures and lifecycle
• Develop scalable ingestion pipelines into DLX / Lakehouse platform
• Implement schema evolution, partitioning, and compaction strategies
• Optimize data ingestion performance, cost, and reliability
• Ensure data quality, lineage, and governance across the lake
• Integrate ingestion pipelines with orchestration and CI/CD pipelines
• Support CDC, incremental loads, and near real-time ingestion patterns
• Monitor platform health, SLAs, and data freshness metrics
• Collaborate with analytics, BI, and data science teams
• Troubleshoot large-scale distributed data issues
• Mentor engineers and enforce platform best practices

Required Technical Skills

Core Platform

• Strong experience in data lake ingestion architecture
• Hands-on expertise with Apache Iceberg (tables, snapshots, schema evolution)
• Experience building DLX / Lakehouse platforms
• Strong SQL and distributed data processing knowledge

Data Engineering

• Batch and streaming ingestion patterns
• CDC and incremental processing
• Partitioning and file size optimization
• Metadata and catalog integration

Cloud & Processing (one or more preferred)

• Spark / PySpark
• Kafka or streaming platforms
• Cloud storage (S3 / ADLS / GCS)
• Orchestration tools (Airflow or similar)

Platform & Engineering Practices

• Lakehouse architecture design
• Iceberg table maintenance (compaction, vacuum, snapshot mgmt)
• Data quality and observability
• CI/CD for data pipelines
• Performance and cost optimization
• Security and access governance

Experience & Qualifications

• 12+ years in Data Engineering / Data Platform Engineering
• 5+ years working with modern data lake/lakehouse technologies
• Hands-on experience with Apache Iceberg or similar table formats
• Experience handling large-scale distributed data platforms
• BFSI or digital platform experience preferred
• Bachelor s/Master s in Computer Science, Engineering, or related field

Preferred / Good to Have

• Experience with Delta Lake or Hudi
• Exposure to dbt or semantic modelling layers
• Knowledge of data catalog/governance tools
• Experience with Kubernetes-based data platforms
• Understanding of real-time analytics architectures

Key Competencies

• Strong platform and systems thinking
• Performance and reliability focus
• Data architecture mind-set
• Mentoring and technical leadership
• Effective cross-team collaboration