Google Cloud Platform Data Engineer

Overview

On Site
$60,000 - $80,000
Full Time

Skills

GCP
Data Engineer
BigQuery & Cloud Dataflow
Data Warehousing
ETL
Data Pipeline Management
Data Modeling
SQL/NoSQL
Big Data
Apache Kafka
Python
Java & Scala
Data Security & Governance
Data Visualization
DevOps
CI/CD
ML & AI

Job Details

Google Cloud Platform Cloud Platform Expertise:
    • Core Services:
      A deep understanding of Google Cloud services like BigQuery, Cloud Storage, Cloud Dataflow, Cloud Pub/Sub, Cloud Composer (Airflow), and Cloud Dataproc is crucial.
  • Data Warehousing:
    Proficiency in building and optimizing data warehouses using BigQuery for analytics and data storage.
  • ETL Processes:
    Experience with Extract, Transform, Load (ETL) processes to move data between different sources, using tools like Cloud Dataflow, Apache Beam, and Cloud Composer.
  • Data Pipeline Management:
    Ability to design, build, and manage data pipelines for both batch and real-time data processing using various Google Cloud Platform services.
Data Management and Processing:
  • Data Modeling:
    Knowledge of data modeling techniques to design efficient and scalable data storage solutions.
  • SQL and NoSQL Databases:
    Proficiency in SQL for querying and manipulating data, and experience with NoSQL databases if required.
  • Big Data Technologies:
    Familiarity with big data frameworks like Apache Spark and Hadoop, especially when working with large datasets.
  • Real-time Data Processing:
    Understanding of real-time data processing concepts and tools like Apache Kafka and Cloud Pub/Sub.
Programming and Scripting:
  • Python:
    A widely used language in data engineering for scripting, data manipulation, and automation.
  • Java:
    Another popular language for building data processing applications and pipelines.
  • Other Languages:
    Depending on the specific project, knowledge of other languages like Go or Scala may also be required.
Other Essential Skills:
  • Data Security and Governance:
    Understanding of data security best practices and compliance requirements.
  • Data Visualization:
    Experience with data visualization tools like Tableau or Power BI to present data insights effectively.
  • DevOps and CI/CD:
    Familiarity with DevOps principles and Continuous Integration/Continuous Deployment (CI/CD) pipelines for automating deployments and managing infrastructure as code.
  • Machine Learning and AI:
    Increasingly, data engineers are expected to integrate machine learning models into data pipelines, requiring knowledge of tools like TensorFlow and AI Platform.
  • Problem-Solving and Analytical Skills:
    Strong analytical and problem-solving skills are crucial for designing efficient data solutions and troubleshooting issues.
  • Communication and Collaboration:
    Effective communication skills are essential for collaborating with data scientists, business analysts, and other stakeholders.
  • Cloud Architecture:
    Understanding of cloud architecture principles to design scalable, secure, and cost-effective solutions.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.