Core Platform Engineer

Role : Core Platform Engineer

Location : Charlotte-Concord-Gastonia, NC-SC  (Hybrid)

Rate: $60/hr

Core Platform Engineer

• Primary Role: Build and maintain secure, scalable infrastructure and services.
• Responsibilities:
• Support a highly available and scalable infrastructure containing Object storage, Openshift, Spark, Iceberg, Yunikorn, Trino
• Monitor for configuration drift and enforce infrastructure policies.
• Configure and monitor Big Data ecosystem components with various BI tools, observability tools etc
• Build automated regression and performance test suite to ensure health checks of all components of the platform
• Tasks:
• Monitor system health and enforce runtime policies.
• Implement and manage security protocols, including Oauth authentication, TLS encryption, and role-based access control (RBAC).
• Conduct regular maintenance, including cluster scaling, perform regular security audits.

Programming & Scripting

• Languages: Python, Bash, Shell, SQL, Java (basic), Scala (for big data, good to have)
• Automation & Scripting: Python scripting for automation, Linux shell scripting

Operating Systems & Containers

• System programing, performance tuning, networking
• OCP, Kubernetes (K8s), Helm, Terraform, container orchestration and deployment

Big Data & Data Engineering

• Frameworks: NexusOne, Apache Spark, Hadoop, Hive, Trino, Iceberg
• ETL Tools: Apache Airflow, NiFi (good to have)
• Data Pipelines: Batch and streaming (Kafka, Flink)
• Object Storage: S3, NetApp StorageGrid
• Data Formats: Parquet/Avro, ORC, JSON, CSV

AI/ML & MTC (Model Training & Consumption) (Nice to have)

• Frameworks or LLM modeling
• Model Ops: MLflow, Kubeflow, SageMaker
• Data Science: Feature engineering, model deployment, inference pipelines

Security & Access Control

• Access Models: RBAC (Role-Based Access Control), ABAC (Attribute-Based Access Control)
• Data Protection: Encryption at rest and in transit, TLS/SSL, KMS (Key Management Services)
• Compliance: GDPR, HIPAA (if applicable), IAM policies

System Design & Architecture (good to have, at least at a conceptual level)

• Design Principles: Microservices, Event-driven architecture, Serverless
• Scalability: Load balancing, caching (Redis, Memcached), horizontal scaling
• High Availability: Failover strategies, disaster recovery, monitoring (Prometheus, Grafana)