Position: Data Engineer - Google Cloud Platform
Location: Phoenix, AZ | New York City, NY | Sunrise, FL(Onsite - Local Candidates Only)
Interview Mode: 1st round virtual; last round face-to-face interview required.
Job Summary
We are looking for an experienced Data Engineer with strong Google Cloud Platform (Google Cloud Platform) expertise to design, develop, and maintain scalable data pipelines and data-processing solutions. The ideal candidate will have strong hands-on experience with Python, SQL, BigQuery, Dataproc, Pub/Sub, Apache Airflow, and DAG development, along with a solid understanding of cloud-based data engineering and ETL/ELT architectures.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on Google Cloud Platform.
- Develop data-processing solutions using Python and SQL.
- Build and optimize data models, queries, and large-scale datasets in Google BigQuery.
- Develop distributed data-processing workloads using Google Cloud Platform Dataproc and related technologies.
- Build event-driven and real-time data pipelines using Google Cloud Pub/Sub.
- Develop, schedule, and maintain workflows using Apache Airflow and DAGs.
- Monitor data pipelines and troubleshoot performance, reliability, and data-quality issues.
- Optimize BigQuery queries and data-processing workloads for performance and cost.
- Integrate data from multiple structured and unstructured sources.
- Implement appropriate data validation, logging, monitoring, and error-handling mechanisms.
- Collaborate with data architects, analysts, application teams, and business stakeholders to understand data requirements.
- Follow engineering best practices for source control, testing, CI/CD, security, and production deployment.
Required Technical Skills
Must Have:
- Strong hands-on Python development experience
- Advanced SQL skills
- Strong hands-on experience with Google Cloud Platform (Google Cloud Platform)
- BigQuery data modeling, complex SQL, performance optimization, and large-scale data processing
- Dataproc experience developing and running distributed data-processing workloads
- Pub/Sub experience building event-driven or streaming data pipelines
- Apache Airflow
- Strong experience developing and maintaining DAGs
- ETL/ELT pipeline development
- Data integration and transformation
- Experience working with large-volume datasets
- Understanding of cloud security, IAM, monitoring, and production support
Preferred Skills
- Experience with PySpark / Apache Spark
- Google Cloud Platform Cloud Storage (GCS)
- Cloud Composer
- Dataflow
- CI/CD and automated deployment
- Git/GitHub
- Experience with APIs and data ingestion frameworks
- Data quality and observability tools
Experience working in Agile/Scrum environments
