Role: L3 Senior Iceberg DBA / Lakehouse Operations Engineer
Work location: Remote work accepted from anywhere in US
Job Description:
10+ years of experience in Big Data / Data Engineering / DBA / Data Operations roles
Minimum 2+ years of hands-on experience with Apache Iceberg in production environments
6+ years of experience working with Cloudera ecosystem (CDP Ecosystem)
Strong expertise in:
o Iceberg table optimization (compaction, metadata management, partition evolution)
o Multi-engine performance tuning (Spark, Hive, Impala)
o Troubleshooting complex data and query performance issues
Proven experience handling:
o P1/P2 production incidents
o Large-scale environments (TB/PB scale)
o Data migration initiatives (Hive/Teradata Iceberg)
Lead enforcement of data modeling and Lakehouse standards across applications
Guide teams on:
o Medallion architecture implementation
o Balancing normalization vs performance
Review and resolve complex data modeling and performance trade-offs
Ensure consistency of data structures across domains and workloads
Mentor and guide L2 resources in operational best practices and troubleshooting
Required Skills
Strong hands-on experience with Apache Iceberg and/or Hive-based data lakes
Understanding of data modeling concepts (normal forms) and modern Lakehouse patterns (Medallion architecture)
Expertise in:
o Table-level optimization and performance tuning
o Large-scale data management (TB/PB scale)
Experience with:
o Spark SQL, Hive, Impala, NiFI, Trino
Strong understanding of:
o Partitioning strategies
o File formats (Parquet/ORC)
o Distributed query processing
Preferred Skills
Experience with:
o Hive-to-Iceberg or Teradata-to-Iceberg migration
o Cloudera CDP (CDE/CDW)
Familiarity with:
o Cloud platforms (AWS, Azure)
Scripting/automation (Python, Shell)
What You'll Work On
Enterprise-scale Iceberg Lakehouse platform supporting multiple applications
Large-scale data modernization initiatives
Performance optimization and stability of mission-critical analytical workloads
Why This Role Matters
Ensures data correctness and performance for downstream analytics and business-critical reporting
Enables successful modernization from legacy platforms to Iceberg
Maintains high availability and reliability of the enterprise data layer
Job Summary
We are seeking a highly skilled Iceberg DBA / Lakehouse Operations Engineer to own the reliability, performance, and operational integrity of the Iceberg data layer powering enterprise analytics and business-critical applications.
This role operates in a large-scale, multi-engine Lakehouse environment, supporting workloads across Spark, Hive, and Impala, and plays a key role in enterprise data modernization initiatives (Hive and Teradata Iceberg).
The ideal candidate brings deep expertise in Iceberg table operations, metadata management, and query performance optimization, ensuring consistent, high-performance data access across platforms in a cloud-based environment.
This role is critical to ensuring data accuracy and performance-any degradation directly impacts downstream reporting, analytics, and business-critical decision-making.
Key Responsibilities:
Iceberg Data Layer Ownership & Operations
Own day-to-day operations of Apache Iceberg tables supporting multiple enterprise applications
Ensure data reliability, consistency, and availability across all Lakehouse workloads
Maintain operational integrity for datasets at multi-terabyte to petabyte scale
Advanced Table Management & Optimization
Execute advanced Iceberg table maintenance and optimization strategies:
o Compaction (minor/major) and small file mitigation
o Snapshot expiration and metadata compaction to control metadata growth
o Orphan file cleanup (vacuum) to maintain storage efficiency
Optimize data layout and performance through:
o File size tuning and distribution strategies
o Partition evolution and pruning optimization
o Clustering and ordering techniques (e.g., Z-ordering or similar patterns)
Data Modeling Standards & Lakehouse Design Alignment
Support and enforce data modeling best practices aligned with:
o Normalized data structures (3NF) for source-aligned datasets
o Medallion architecture (Bronze / Silver / Gold layers) for curated data flows
Ensure Iceberg table design aligns with:
o Data ingestion patterns (raw vs curated layers)
o Downstream consumption and performance requirements
Assist in structuring datasets to balance:
o Data integrity and normalization
o Query performance and analytical efficiency
Work with data engineering teams to ensure consistent implementation of layered data architecture across multiple applications
Multi-Engine Query Performance & Consistency
Ensure consistent and performant query behavior across:
o Spark (CDE)
o Hive / Impala (CDW)
Troubleshoot and resolve:
o Query performance bottlenecks
o Metadata inconsistencies across engines
o Inefficient execution plans and scan patterns
Hive & Teradata Modernization Support
Play a key role in enterprise data platform modernization (Hive and Teradata Iceberg)
Support:
o Schema alignment and data type mapping
o Data validation and reconciliation
Troubleshoot migration-related issues and ensure post-migration stability and performance
Metadata & Data Lifecycle Management
Manage Iceberg metadata to ensure:
o Efficient scaling and performance
o Consistent table state across engines
Execute lifecycle operations:
o Data retention and archival policies
o Snapshot lifecycle management and cleanup
o Time-travel optimization and maintenance
Production Support, Incident Resolution & On-Call
Provide L2/L3 support for data-related production issues across Iceberg-based Lakehouse workloads
Participate in on-call rotation to support critical data platforms and ensure timely response to incidents
Respond to and resolve P1/P2 production incidents within defined SLAs, minimizing impact to downstream applications and reporting
Troubleshoot:
o Data inconsistencies and reporting discrepancies
o Query failures and performance degradation
Perform root cause analysis (RCA) and implement preventive measures to avoid recurring issues
Collaborate with platform and application teams during incident triage and resolution
Security & Data Governance Support
Support fine-grained access control using:
o Ranger policies and RBAC
Own and ensure data validation, reconciliation, and accuracy between source and Iceberg datasets
Ensure secure and compliant access to data across applications
Never miss an opportunity! Create an alert based on the job you applied for.
