Big Data Engineer with Google Cloud Platform

Spark Frame Work – 4 Years Minimum

Kafka – 3 Years

Google Cloud Platform – 3 years

Airflow – 3 years

Python – 3 years

 

Nice to have

Java – 3 years

 

Micro Services – 2years

SQL – 4 Years

 

Daily Responsibilities

Attending Scrum calls to provide daily status updates

Pickup assigned JIRA stories and work with Analysts or Product Owners to understand

Responsible for the development, maintenance, and enhancements of Data Engineering solutions of varying

complexity levels across different data sources like DBMS, File systems (structured and unstructured)

on-prem and cloud infrastructure; creates level metrics and other complex metrics.

Demonstrates Data Engineering skills (Spark Data Frame, Big Queries) by writing pipelines for business

requirements.

Responsible for building, testing and enhancement of Data Pipelines solutions from a wide variety of

sources like Kafka streaming, Google Big Query and File systems; develops solutions with optimized data

performance and data security