Data engineer with Google Cloud Platform and Pyspark

Position Overview

We are seeking an experienced Google Cloud Platform Data Engineer with heavy expertise in Google Cloud Platform, PySpark, and large-scale data warehousing to design, build, and optimize enterprise-grade cloud data architectures. In this role, you will architect resilient data pipelines, handle massive data processing workloads, and collaborate with analytics teams to deliver high-performance data solutions for major enterprise clients.

Key Responsibilities

• Data Pipeline & Architecture Development:

  • Design, build, and optimize scalable batch and streaming data pipelines using PySpark, Spark SQL, and Google BigQuery.

  • Leverage Google Cloud Platform-native services including Cloud Storage, Cloud Dataproc, Cloud Composer, Pub/Sub, and Dataform to streamline data integration workflows.

• Cloud Data Management & Optimization:

  • Build, partition, and tune large datasets within BigQuery for optimal query performance and cost efficiency.

  • Implement data validation, error handling, schema evolution, and automated data quality checks across ingestion layers.

• Infrastructure & CI/CD Automation:

  • Automate deployment and resource provisioning using Infrastructure as Code tools like Terraform alongside version control management with Git.

  • Implement and maintain continuous integration and continuous deployment (CI/CD) pipelines to ensure rapid, reliable release cycles.

• Collaboration & Governance:

  • Partner with enterprise architecture, security, and analytics stakeholders to adhere to cloud security best practices, compliance standards, and data governance frameworks.