Apply Now

Senior Google Cloud Platform Data Engineer Python, PySpark & BigQuery

Phoenix, AZ, US • Posted 5 hours ago • Updated 21 minutes ago

Full Time

Part Time

On-site

Fitment

Dice Job Match Score™

To see how well you match this job, please log in or create an account.

Once logged in, be sure to complete your profile to get the most accurate match score.

🔢 Crunching numbers...

Job Details

Skills

Database
Scalability
Scheduling
Git
Version Control
Data Validation
Performance Tuning
Software Design
Testing
Continuous Improvement
Technical Writing
Apache Spark
Apache Hadoop
Apache Hive
Relational Databases
Object-Oriented Programming
Design Patterns
Cloud Computing
Apache Airflow
Workflow
Orchestration
Data Engineering
ELT
Data Processing
Extract
Transform
Load
Optimization
Data Warehouse
Data Analysis
Apache Kafka
DevOps
GitHub
GitLab
Bitbucket
Apache Subversion
Jenkins
Continuous Integration
Continuous Delivery
Management
Shell Scripting
Algorithms
Automated Testing
Agile
Scrum
Software Development Methodology
Software Development
Big Data
Analytics
Python
PySpark
SQL
Google Cloud Platform
Google Cloud
Analytical Skill
Conflict Resolution
Problem Solving
Communication
Collaboration

Database
Scalability
Scheduling
Git
Version Control
Data Validation
Performance Tuning
Software Design
Testing
Continuous Improvement
Technical Writing
Apache Spark
Apache Hadoop
Apache Hive
Relational Databases
Object-Oriented Programming
Design Patterns
Cloud Computing
Apache Airflow
Workflow
Orchestration
Data Engineering
ELT
Data Processing
Extract
Transform
Load
Optimization
Data Warehouse
Data Analysis
Apache Kafka
DevOps
GitHub
GitLab
Bitbucket
Apache Subversion
Jenkins
Continuous Integration
Continuous Delivery
Management
Shell Scripting
Algorithms
Automated Testing
Agile
Scrum
Software Development Methodology
Software Development
Big Data
Analytics
Python
PySpark
SQL
Google Cloud Platform
Google Cloud
Analytical Skill
Conflict Resolution
Problem Solving
Communication
Collaboration

Summary

Senior Google Cloud Platform Data Engineer - Python, PySpark & BigQuery

Experience: 6+ Years

Job Location: As per project requirement

Job Summary

We are looking for an experienced Google Cloud Platform Data Engineer with strong expertise in Python, PySpark, SQL, and Google Cloud Platform (Google Cloud Platform). The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and big data solutions for enterprise data processing, analytics, and data warehousing.

The candidate should have hands-on experience with Google Cloud Platform services such as BigQuery, Cloud Composer (Airflow), Dataproc, and Pub/Sub, along with strong knowledge of Apache Spark, Hadoop, Hive, ETL processes, and data engineering best practices.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines using Python, PySpark, and Google Cloud Platform services.
Build and optimize large-scale data processing solutions using Apache Spark, Hadoop, Hive, and Big Data technologies.
Develop and manage data warehousing solutions using Google BigQuery.
Implement data ingestion, transformation, and processing workflows using Google Cloud Platform services such as BigQuery, Dataproc, Pub/Sub, and Cloud Composer (Airflow).
Design and implement event-driven data processing pipelines using Kafka or Google Cloud Pub/Sub.
Develop and optimize complex SQL queries, data models, and database structures for performance and scalability.
Implement workflow orchestration and scheduling using Apache Airflow.
Build and maintain CI/CD pipelines using Jenkins and Git-based source control tools.
Perform data validation, troubleshooting, performance tuning, and optimization of data pipelines.
Follow coding standards, object-oriented programming principles, and software design patterns.
Collaborate with product owners, architects, and cross-functional teams to understand requirements and deliver effective data solutions.
Participate in Agile/Scrum ceremonies, code reviews, testing, deployment, and continuous improvement initiatives.
Maintain technical documentation and ensure adherence to enterprise development standards.

Required Technical Skills

Programming & Big Data

Strong programming experience in Python.
Hands-on experience with PySpark and Apache Spark.
Good understanding of Hadoop and Hive.
Strong knowledge of SQL and relational databases.
Understanding of object-oriented programming and design patterns.

Google Cloud Platform (Google Cloud Platform)

Hands-on experience with Google Cloud Platform.
Google BigQuery for data warehousing and analytics.
Google Cloud Dataproc for distributed data processing.
Google Cloud Pub/Sub for event-driven data ingestion and processing.
Cloud Composer / Apache Airflow for workflow orchestration.

Data Engineering & Integration

Strong experience in ETL/ELT development and large-scale data processing.
Data pipeline design, development, transformation, and optimization.
Experience with data warehousing and data analytics.
Experience building event-processing pipelines using Kafka or Pub/Sub.

DevOps & Development Tools

Experience with GitHub, GitLab, Bitbucket, or SVN.
Knowledge of Jenkins and CI/CD pipeline implementation.
Familiarity with automated testing frameworks and source code management.
Shell scripting experience.

Preferred Qualifications

Google Cloud Professional Data Engineer certification.
Experience with enterprise-scale Big Data and analytics solutions.
Knowledge of distributed systems, multi-tier architectures, and algorithms.
Experience with automated testing and deployment frameworks.
Understanding of Agile/Scrum and SDLC methodologies.

Required Experience

6+ years of hands-on software development experience in Big Data and Analytics.
Strong practical experience with Python, PySpark, SQL, and Google Cloud Platform.
Experience designing and optimizing enterprise data pipelines.

Key Competencies

Strong analytical and problem-solving skills.
Excellent communication and collaboration abilities.
Ability to work independently and within cross-functional teams.
Proactive approach to identifying technical improvements.
Willingness to learn and adopt enterprise technologies and frameworks.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91018020
Position Id: PDT - 12285-13420-1793802976
Posted 5 hours ago

Company Info

About Purple Drive Technologies LLC

Founded in 2007, Purple Drive started as a tech solutions firm and has grown into a full-service consulting and talent partner. We help businesses navigate complex technology challenges while connecting top professionals with career-defining opportunities.

We believe in transforming businesses through smart IT solutions and empowering technologists to grow their expertise through challenging projects and meaningful partnerships. Built on over 20 years of trusted relationships, we create success stories for both our clients and the talented professionals who drive innovation forward.

Go to company profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Senior Data Architect

Dallas, Texas

•

Today

Job Description Must Have Technical/Functional Skills Key Responsibilities & Skills 7-12 years of experience in Data Engineering, Data Platforms, or Solution Architecture. Experience in Python, PySpark, Spark SQL, and distributed data processing. Experience working with Hadoop ecosystem and large-scale data platforms. Hands-on expertise with Databricks Lakehouse and modern cloud-native architectures. Experience with one or more cloud platforms: Azure, AWS, or Google Cloud Platform. Collaborate

Easy Apply

Full-time, Part-time, Contract, Third Party

Search all similar jobs

More jobs at Purple Drive Technologies LLC in Phoenix, AZ