Senior Data Platform Engineer

New York City, NY, US • Posted 15 hours ago • Updated 46 minutes ago
Contract W2
12 Months
On-site
USD65.0/HOURLY - USD75.0/HOURLY
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Spark
  • Airflow
  • Terraform
  • AWS S3
  • Trino & GCS
  • Python/Java and SQL
  • BigQuery
  • Kafka
  • AWS to GCP
  • CI/CD
  • Kubernetes
  • Lambda

Summary

Role: Senior Data Platform Engineer

Location: New York City, NY (3 days Onsite Hybrid)

Prioritization - We need a senior data platform engineer. This engineer must have hands-on experience in Cloud & Data Platform. We prefer experience with Google Cloud Platform and AWS. This engineer builds and operates the platform infrastructure for the data, ML, and analytics teams. The scope includes data governance, security, data pipeline design, CI/CD, and related tools. Platform skills are the first priority: Terraform or a similar IaC tool, IAM and security policies, Spark compute platforms, BigQuery, Trino, job orchestration, and networking. Pipeline engineering and analytics engineering skills are the second priority.

Experience level - This Engineer should have 8+ years of total experience, with 5+ years of hands-on Cloud experience in either AWS or Google Cloud Platform and infrastructure ownership. Please screen out profiles that only design and do not implement the code.

Key Responsibilities:

Must have

Data Processing : Hands-on experience in Spark Jobs, Streaming ingestion, Data Migration and performance tuning (skew, partitioning, autoscaling, cost).

Job Orchestration : Manage and optimize core data platform infrastructure, workflow orchestration (Airflow / Cloud Composer), and automated data quality, observability, and validation frameworks.

Terraform or similar IaC : Infrastructure Design, implementation and CI/CD Process.

On-Call : Own platform SLAs, automated alerting and compute/storage cost optimization.

Data Governance : Dataset access control, policy tags , column-level security, row-level security, cost and quota management.

Hands-on experience in AWS S3, Trino & GCS.

Strong Skills in Python/Java and SQL.

Production ownership: troubleshooting, root-cause analysis and on-call for shared platform services.

Added advantage

Experience with the data migration from AWS to Google Cloud Platform. This includes access between the two clouds, for example Workload Identity Federation and BigQuery Omni.

Data formats: Apache Iceberg, Avro, and Parquet.

Streaming: Kafka, Spark Structured Streaming.

Data governance: PII masking, Cloud DLP, PII Scanner, Apache Ranger.

Kubernetes basics, AWS Lambda, Google Cloud Platform Cloud Run and dbt.

Experience building self-service internal platforms tools for the data users

Google Cloud Platform Professional Cloud Architect or Data Engineer certification.

Collaboration and communication (must have)

Coordinate daily with an offshore engineering team: break down work, write clear tickets and hand-offs, review their code and unblock them across time zones.

Communicate with stakeholders: explain platform decisions, trade-offs and timelines to engineering leads, analytics teams and product partners.

Write clear design docs, runbooks and guides.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10365788
  • Position Id: W3GEXT-64732
  • Posted 15 hours ago
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.