L2 Senior Iceberg DBA \/ Lakehouse Operations Engineer || Remote

TECHNOGEN, Inc. is a Proven Leader in providing full IT Services, Software Development and Solutions for 15 years.

TECHNOGEN is a Small & Woman Owned Minority Business with GSA Advantage Certification. We have offices in VA; MD & Offshore development centers in India. We have successfully executed 100+ projects for clients ranging from small business and non-profits to Fortune 50 companies and federal, state and local agencies.

Hi,

Greetings of the day!

We are looking to Hire a Talented Professional for the below Job opportunity with one of our clients,
If you're interested, please share your updated resume at your earliest convenience, and I'll be happy to provide more details about the role.

Position: L2 Senior Iceberg DBA \/ Lakehouse Operations Engineer

Location: Remote work accepted from anywhere in US

Duration: Long Term

Job Description:

4 6 years of experience in Big Data / Data Operations / DBA roles
Minimum 1+ year of experience with Apache Iceberg or similar table formats (Hive/Delta/Hudi)
4+ years of experience with Cloudera ecosystem (CDP)

Hands-on experience with:
o Iceberg table operations and maintenance
o Spark SQL, Hive, or Impala

Experience in:
o Production support and incident handling
o Monitoring, troubleshooting, and operational support

Apply established data modeling and Lakehouse standards in day-to-day operations

Support:
o Table structuring
o Partition alignment with ingestion patterns
Assist in maintaining consistency of datasets across Bronze/Silver/Gold layers

Required Skills
Strong hands-on experience with Apache Iceberg and/or Hive-based data lakes
Understanding of data modeling concepts (normal forms) and modern Lakehouse patterns (Medallion architecture)
Expertise in:
o Table-level optimization and performance tuning
o Large-scale data management (TB/PB scale)
Experience with:
o Spark SQL, Hive, Impala, NiFI, Trino
Strong understanding of:
o Partitioning strategies
o File formats (Parquet/ORC)
o Distributed query processing

Preferred Skills
Experience with:
o Hive-to-Iceberg or Teradata-to-Iceberg migration
o Cloudera CDP (CDE/CDW)
Familiarity with:
o Cloud platforms (AWS, Azure)
Scripting/automation (Python, Shell)

What You'll Work On
Enterprise-scale Iceberg Lakehouse platform supporting multiple applications
Large-scale data modernization initiatives
Performance optimization and stability of mission-critical analytical workloads

Why This Role Matters
Ensures data correctness and performance for downstream analytics and business-critical reporting
Enables successful modernization from legacy platforms to Iceberg
Maintains high availability and reliability of the enterprise data layer

Job Summary
We are seeking a highly skilled Iceberg DBA / Lakehouse Operations Engineer to own the reliability, performance, and operational integrity of the Iceberg data layer powering enterprise analytics and business-critical applications.
This role operates in a large-scale, multi-engine Lakehouse environment, supporting workloads across Spark, Hive, and Impala, and plays a key role in enterprise data modernization initiatives (Hive and Teradata Iceberg).
The ideal candidate brings deep expertise in Iceberg table operations, metadata management, and query performance optimization, ensuring consistent, high-performance data access across platforms in a cloud-based environment.
This role is critical to ensuring data accuracy and performance-any degradation directly impacts downstream reporting, analytics, and business-critical decision-making.

Key Responsibilities:
Iceberg Data Layer Ownership & Operations
Own day-to-day operations of Apache Iceberg tables supporting multiple enterprise applications
Ensure data reliability, consistency, and availability across all Lakehouse workloads
Maintain operational integrity for datasets at multi-terabyte to petabyte scale

Advanced Table Management & Optimization
Execute advanced Iceberg table maintenance and optimization strategies:
o Compaction (minor/major) and small file mitigation
o Snapshot expiration and metadata compaction to control metadata growth
o Orphan file cleanup (vacuum) to maintain storage efficiency
Optimize data layout and performance through:
o File size tuning and distribution strategies
o Partition evolution and pruning optimization
o Clustering and ordering techniques (e.g., Z-ordering or similar patterns)

Data Modeling Standards & Lakehouse Design Alignment
Support and enforce data modeling best practices aligned with:
o Normalized data structures (3NF) for source-aligned datasets
o Medallion architecture (Bronze / Silver / Gold layers) for curated data flows
Ensure Iceberg table design aligns with:
o Data ingestion patterns (raw vs curated layers)
o Downstream consumption and performance requirements
Assist in structuring datasets to balance:
o Data integrity and normalization
o Query performance and analytical efficiency
Work with data engineering teams to ensure consistent implementation of layered data architecture across multiple applications

Multi-Engine Query Performance & Consistency
Ensure consistent and performant query behavior across:
o Spark (CDE)
o Hive / Impala (CDW)
Troubleshoot and resolve:
o Query performance bottlenecks
o Metadata inconsistencies across engines
o Inefficient execution plans and scan patterns

Hive & Teradata Modernization Support
Play a key role in enterprise data platform modernization (Hive and Teradata Iceberg)
Support:
o Schema alignment and data type mapping
o Data validation and reconciliation
Troubleshoot migration-related issues and ensure post-migration stability and performance

Metadata & Data Lifecycle Management
Manage Iceberg metadata to ensure:
o Efficient scaling and performance
o Consistent table state across engines
Execute lifecycle operations:
o Data retention and archival policies
o Snapshot lifecycle management and cleanup
o Time-travel optimization and maintenance

Production Support, Incident Resolution & On-Call
Provide L2/L3 support for data-related production issues across Iceberg-based Lakehouse workloads
Participate in on-call rotation to support critical data platforms and ensure timely response to incidents
Respond to and resolve P1/P2 production incidents within defined SLAs, minimizing impact to downstream applications and reporting
Troubleshoot:
o Data inconsistencies and reporting discrepancies
o Query failures and performance degradation
Perform root cause analysis (RCA) and implement preventive measures to avoid recurring issues
Collaborate with platform and application teams during incident triage and resolution
Security & Data Governance Support
Support fine-grained access control using:
o Ranger policies and RBAC
Own and ensure data validation, reconciliation, and accuracy between source and Iceberg datasets
Ensure secure and compliant access to data across applications

Ranjitha P | Sr. IT Recruiter