Senior Data Platform Engineer

Hybrid in New York, NY, US • Posted 3 hours ago • Updated 3 hours ago
Contract Independent
Contract W2
12 Months
No Travel Required
Hybrid
Depends on Experience
Company Branding Image
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • migration from AWS to GCP

Summary

Senior Data Platform Engineer | Location: New York City (3 days work from office) | 
 

Prioritization -  We need a senior data platform engineer. This engineer must have hands-on experience in Cloud & Data Platform. We prefer experience with Google Cloud Platform and AWS. This engineer builds and operates the platform infrastructure for the data, ML, and analytics teams. The scope includes data governance, security, data pipeline design, CI/CD, and related tools. Platform skills are the first priority: Terraform or a similar IaC tool, IAM and security policies, Spark compute platforms, BigQuery, Trino, job orchestration, and networking. Pipeline engineering and analytics engineering skills are the second priority.

 

Experience level -  This Engineer should have 8+ years of total experience, with 5+ years of hands-on Cloud experience in either AWS or Google Cloud Platform and infrastructure ownership. Please screen out profiles that only design and do not implement the code.

Key Responsibilities:

Must have

·       Data Processing : Hands-on experience in Spark Jobs, Streaming ingestion, Data Migration and performance tuning (skew, partitioning, autoscaling, cost).

·       Job Orchestration : Manage and optimize core data platform infrastructure, workflow orchestration (Airflow / Cloud Composer), and automated data quality, observability, and validation frameworks. 

·       Terraform or similar IaC : Infrastructure Design, implementation and CI/CD Process.

·       On-Call : Own platform SLAs, automated alerting and compute/storage cost optimization.

·       Data Governance : Dataset access control, policy tags , column-level security, row-level security, cost and quota management.

·       Hands-on experience in AWS S3, Trino & GCS.

·       Strong Skills in Python/Java and SQL.

·       Production ownership: troubleshooting, root-cause analysis and on-call for shared platform services.

 

Added advantage

·       Experience with the data migration from AWS to Google Cloud Platform. This includes access between the two clouds, for example Workload Identity Federation and BigQuery Omni.

·       Data formats: Apache Iceberg, Avro, and Parquet.

·       Streaming: Kafka, Spark Structured Streaming.

·       Data governance: PII masking, Cloud DLP, PII Scanner, Apache Ranger.

·       Kubernetes basics, AWS Lambda, Google Cloud Platform Cloud Run and dbt.

·       Experience building self-service internal platforms tools for the data users

·       Google Cloud Platform Professional Cloud Architect or Data Engineer certification.

Collaboration and communication (must have)

·       Coordinate daily with an offshore engineering team: break down work, write clear tickets and hand-offs, review their code and unblock them across time zones.

·       Communicate with stakeholders: explain platform decisions, trade-offs and timelines to engineering leads, analytics teams and product partners.

·       Write clear design docs, runbooks and guides.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91138214
  • Position Id: 9148759
  • Posted 3 hours ago

Company Info

About Marici Solutions

MARICI Solutions is a global business consulting organization, where inspired & visionary people with a shared passion for innovation come together to make a difference. We take a holistic approach from various perspectives to deliver measurable results along the entire value chain. We work closely with our clients and provide distinct advantages to sustainably transform their business processes, which eventually grow their businesses.

Driven by the passion of offering state of the art, customized and high-quality information technology services, our experts from Germany & India came together in 2017 and formed MARICI Solutions GmbH. The company is registered in Germany & India and well funded for long term sustenance. The team at MARCI comprises of experts from various industry sectors who have more than a decade of diverse & international consulting experience. MARICI is expanding its business across multiple continents and has secured various projects from some of the eminent market giants.

We believe that Excellence, Collaboration, Commitment and Innovation are foundations for success.

By Collaborating with our stakeholders, we identify opportunities for growth and Innovation. Ongoing innovation results from a combination of strategy, processes, systems, and culture. Through our Commitment to Excellence, we accomplish complex assignments and inspire people to do great things together. 

Contact the job poster
AT

Anish Tarte

Recruiter @ Marici Solutions

Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.