TECHNOGEN, Inc. is a Proven Leader in providing full IT Services, Software Development and Solutions for 15 years.
TECHNOGEN is a Small & Woman Owned Minority Business with GSA Advantage Certification. We have offices in VA; MD & Offshore development centers in India. We have successfully executed 100+ projects for clients ranging from small business and non-profits to Fortune 50 companies and federal, state and local agencies.
Role: L3 Cloudera Public Cloud Platform Engineer
Work location: Remote work accepted from anywhere in US
12+ years of experience in Big Data Platform Engineering / Cloud Platform Operations / Infrastructure roles
6+ years of hands-on experience with Cloudera ecosystem (CDH/CDP/ Cloudera Public Cloud)
Demonstrated ability to quickly learn and adapt to new technologies and evolving platform capabilities, beyond the currently defined CDP stack
Strong expertise in:
o End-to-end CDP platform operations (CDE, CDW, CDF, CDL, CAI)
o Advanced troubleshooting across multi-cluster, multi-environment deployments
o Kubernetes-based runtime environments (troubleshooting and diagnostics)
o Observability frameworks, including SLIs/SLOs, alerting, and performance tuning
Proven experience in:
o Leading P1/P2 incident response, triage, and resolution
o Managing platform upgrades, patching, and lifecycle events
o Supporting large-scale environments (TB/PB scale, high concurrency workloads)
Strong understanding of:
o Cloud infrastructure (IAM, VPC, networking, storage)
o Security and governance (Ranger, Kerberos, TLS/SSL, SDX)
Expected to:
o Lead complex troubleshooting and drive root cause resolution across platform layers
o Mentor and guide L2 engineers
o Coordinate with Cloudera support and infrastructure teams for critical issues
Hands-on experience in developing and troubleshooting NiFi (CDF) data flows, including:
o Flow design and configuration
o Processor-level debugging and performance tuning
o Handling backpressure, throughput optimization, and failure recovery
Required Skills
Strong experience with Cloudera CDP Public Cloud
Expertise in:
o Cloud platforms (AWS/Azure/Google Cloud Platform)
o Kubernetes concepts (troubleshooting-focused)
Hands-on with:
o CDE, CDW, CDF (NiFi), CAI
knowledge of:
o IAM, networking, observability tools
Platforms operating at multi-terabyte to petabyte scale with high concurrency workloads
Hands-on experience with:
o Kafka (or similar streaming platforms) including monitoring, troubleshooting, and performance tuning
Experience with Cloudera CDP CLI (Command Line Interface) for:
o Platform operations and administration
o Job execution and service management (CDE/CDW/CDL)
o Automation of routine operational tasks
Strong working knowledge of:
o Cloud IAM (AWS IAM / Azure AD) including roles, policies, and cross-service access
o User and group mapping across CDP, cloud IAM, and Ranger policies
o Troubleshooting access issues across storage (S3/ADLS), CDP services, and data access layers
Preferred Skills
Experience with:
o Modernization of legacy data platforms/applications to Cloudera CDP Public Cloud
o Migration and onboarding of workloads to CDE, CDW, and CAI environments
o Supporting hybrid or multi-environment transitions (on-prem cloud)
Familiarity with:
o Cloud platforms (AWS, Azure, Google Cloud Platform) including storage, IAM, and networking concepts
o Kubernetes-based runtime environments (troubleshooting-focused)
Strong scripting and automation skills (Python, Shell, Terraform) for platform operations
What You'll Work On
Enterprise-scale Cloudera CDP platform supporting data engineering, analytics, and AI workloads across multiple applications
Modernization of legacy platforms and applications into cloud-native CDP services
Operational support and scaling of:
o Data services (CDE, CDW, CDF, CDL)
o AI/ML platforms (CAI, inference, workbenches)
Platform performance optimization, observability, and reliability engineering for mission-critical workloads
Why This Role Matters
Ensures availability, stability, and performance of the CDP platform supporting all data and AI workloads
Enables successful modernization of legacy applications into scalable, cloud-native services
Maintains high availability, observability, and operational excellence across enterprise platforms
Acts as the backbone for data engineering, analytics, and AI initiatives
This role focuses on platform reliability and infrastructure operations and does not include data-layer ownership (e.g., Iceberg table management or data validation).
Job Summary
We are seeking a highly skilled Cloudera Public Cloud Platform Engineer to operate and manage the end-to-end CDP platform ecosystem, including data services, NiFI, Kafka, AI/ML platforms, and enterprise observability.
This role is responsible for ensuring availability, scalability, security, and performance of all platform services supporting data, analytics, and AI workloads across environments.
The ideal candidate brings strong expertise in CDP on-prem, public cloud services, cloud infrastructure, Kubernetes-based runtime environments, and platform observability, supporting high-concurrency, mission-critical workloads at multi-terabyte to petabyte scale
This role is critical to ensuring uninterrupted operation of data, analytics, and AI platforms-any degradation directly impacts downstream business reporting, data pipelines, and model execution.
Key Responsibilities
CDP Platform & Multi-Service Operations
Own end-to-end operational responsibility for Cloudera Public Cloud services across Dev / Stage / UAT / Prod:
o CDE, CDW, COD, CDL, CDF (NiFi), CDV, CAI, Kafka
Ensure multi-cluster stability, workload isolation, and SLA adherence
Support onboarding and operations of multiple applications across environments
Manage and support multi-environment, multi-cluster deployments with strict isolation, governance, and release coordination across Dev/UAT/Prod
AI/ML Platform Operations
Operate and support Cloudera AI (CAI) environments:
o AI Workbenches, AI Studios
o Model training and development environments
o AI inference endpoints and model serving
Troubleshoot:
o Resource contention (CPU/GPU)
o Model deployment/runtime failures
CDP Runtime & Kubernetes-Aware Operations
Operate CDP services running on Cloudera-managed Kubernetes infrastructure
Apply strong understanding of containerized workloads and Kubernetes concepts for troubleshooting
Diagnose and resolve:
o Pod failures, restarts, and resource contention
o Spark job failures in containerized environments (CDE)
o Service-to-service communication issues
Analyze logs and metrics to identify runtime failures and performance issues
Collaborate with Cloudera support for managed service-level issues
Data Integration & Platform Services
Operate and support:
o CDF (NiFi) for ingestion pipelines
o CDV (Data Visualization) for reporting workloads
o Octopai for data lineage and catalog integration
Ensure reliability and performance of data pipelines and integrations
Monitor and troubleshoot Kafka environments:
o Topic configurations, partitions, and replication
o Consumer lag and throughput issues
o Broker connectivity and performance bottlenecks
Security, Governance & SDX Administration
Implement and manage:
o Kerberos, TLS/SSL, Ranger policies
Administer SDX for:
o Centralized security
o Metadata and policy enforcement
Support Atlas and Octopai integration
Manage and troubleshoot user access and identity mapping across layers, including:
o Cloud IAM roles and permissions
o CDP users/groups and identity providers
o Ranger policies for fine-grained data access
Resolve access-related issues impacting:
o Data access (S3/ADLS)
o Query execution (CDW/CDE)
o Application and service-level permissions
Cloud Infrastructure & Networking
Troubleshoot:
o S3 / ADLS storage issues
o IAM roles and permissions
o VPC, subnets, routing, security groups
o Bastion host access and connectivity
Ensure secure and reliable connectivity across services
Understand and troubleshoot S3-based data lake patterns, including:
o Bucket structure, prefix design, and access patterns
o Performance issues related to small files, request rates, and throughput limits
o Encryption (SSE-S3, SSE-KMS) and access policies
Manage and troubleshoot cross-account IAM roles and access patterns for CDP environments
Ensure secure access between:
o CDP environments and cloud resources
o Multiple AWS accounts (dev/prod separation)
Disaster Recovery & Resiliency
Support and validate disaster recovery and failover strategies across CDP environments
Ensure backup, recovery, and environment resiliency for critical workloads
Participate in DR drills and recovery validation
Observability, Monitoring & Alerting (Critical)
Implement and manage end-to-end observability:
o Metrics, logs, and alerting
Use:
o Cloudera observability, Cloudera Manager, Prometheus, Grafana
Monitor:
o Cluster health
o Workload performance
o AI inference endpoints
Enable proactive issue detection and prevention
Define and implement SLIs/SLOs and alerting thresholds to ensure platform reliability and performance
Support high-severity (P1/P2) incident response, triage, and resolution within defined SLAs
Operational Support & On-Call
Participate in on-call rotation to support 24/7 platform operations
Respond to production incidents, alerts, and service disruptions within defined SLAs
Handle P1/P2 incidents, including triage, troubleshooting, and resolution
Perform root cause analysis (RCA) and implement preventive measures
Upgrades, Patching & Platform Lifecycle
Execute:
o CDP upgrades and version management
o Security patches and hotfixes
Perform:
o Rolling upgrades
o Validation and rollback strategies
Performance Optimization & Cost Efficiency
Optimize:
o Platform-level performance (Spark, Hive, Impala workloads)
o Cluster utilization and workload distribution
Drive:
o Autoscaling strategies
o Cost optimization (FinOps practices)
Best Regards
Govinda rajulu. M| Sr. Talent Acquisition Specialist
Never miss an opportunity! Create an alert based on the job you applied for.
