Sr. Engineer, Data Architecture (Snowflake, Cloud, Big Data, ETL, normalization, DW Design, Azure, HDFS, S3, Python, Spark, SQL, Power BI, Tableau) - 100% Remote

Data Engineer, Cloud based PAAS, Big Data, Snowflake, Synapse, Big Query, Redshift, EMR, Data bricks, cloud based ETL, Prefect, Airflow, KubeFlow, stream-based processing, batch processing, Database Design, normalization, de-normalization, data warehouse design, Distributed File Systems, HDFS, ADLS, S3, On-prem to Cloud Migration, Python, Spark, SQL
Full Time
Depends on Experience
Travel not required

Job Description

Sr. Engineer, Data Architecture (Snowflake, Cloud, Big Data, ETL, normalization, DW Design, Azure, HDFS, S3, Python, Spark, SQL, Power BI, Tableau) - 100 % remote

POSITION: Sr. Engineer, Data Architecture (Snowflake, Cloud, Big Data, ETL, normalization, DW Design, Azure, HDFS, S3, Python, Spark, SQL, Power BI, Tableau)
LOCATION: 100% Remote
DURATION: Full-Time Remote position
SALARY: Excellent Compensation with benefits + 401K
SKILLS: Data Engineer, Cloud based PAAS, Big Data, Snowflake, Synapse, Big Query, Redshift, EMR, Data bricks, cloud based ETL, Prefect, Airflow, KubeFlow, stream-based processing, batch processing, Database Design, normalization, de-normalization, data warehouse design, Distributed File Systems, HDFS, ADLS, S3, On-prem to Cloud Migration, Python, Spark, SQL

DESCRIPTION:

ROLE:
For one of our prestigious clients, we are looking for Sr. Engineer, Data Architecture (100 % Remote) with a focus on Data Engineering who who will be responsible for the design and implementation our data strategies across multiple platform and services. This position requires extensive hands-on coding experience developing modern data pipelines (Prefect, Airflow, Luigi, Python, Spark, SQL) and associated code in cloud/on-prem Linux/Windows environments. This is highly collaborative position that will be partnering and advising multiple teams by providing guidance throughout the creation and consumption of our data pipelines.

KEY RSPONSIBILITIES:

  • Create and maintain optimal data pipeline architecture, design and implement ingress / egress data pipelines for applications, data layers and Data Lake; determine optimal data movement strategies across a hybrid cloud
  • Design and implement conceptual, logical, and physical data workflows that support business needs
  • Propose architecture that enables integration of disparate enterprise data
  • Lead cross functional design sessions with functional experts to understand and document data requirements and use cases
  • Develop and document the data movement standards, best practices, and evangelize them across department
  • Drive long-term data architecture roadmaps in alignment with corporate strategic objectives
  • Conduct code and design reviews to ensure data related standards and best practices are met
  • Proactively educate others on modern data engineering concepts and design
  • Mentor Junior Members of the team

OUR Ideal Candidate will have:

  • Production delivery experience in Cloud based PAAS Big Data related technologies (Snowflake, Synapse, Big Query, Redshift, EMR, Data bricks etc.)
  • Experience in multiple Cloud PAAS persistence technologies, and in-depth knowledge of cloud based ETL offerings and orchestration technologies (Prefect, Airflow, KubeFlow, etc.)
  • Experience in stream-based and batch processing utilizing modern technologies
  • Database design skills including normalization/de-normalization and data warehouse design
  • Strong analytical / Debugging / Troubleshooting skills
  • Understanding of Distributed File Systems (HDFS, ADLS, S3)
  • Experience transitioning from on-prem big data installations to cloud is a plus
  • Strong Programming Experience - Python / Spark / SQL etc.
  • Collaborative and informative mindset is a must!

REQUIRED Experience:

  • Spark / Python / SQL
  • Databricks / Snowflake / Synapse / MS SQL Server
  • ETL / Orchestration Tools (DBT etc.)
  • Azure /Cosmos / ADLS Gen 2
  • git
  • Power BI/Tableau
  • ML / Notebooks
  • Hadoop 2.0 / Impala / Hive

REQUIRED EDUCATION and EXPERIENCE:

  • Bachelors or master s in computer science, Information Systems, or an engineering discipline or relevant experience
  • 10+ to 15 years of related experience in developing data solutions and data movement

--------------------------------------------

Dice Id : 10216777
Position Id : B_DataEngArkit
Originally Posted : 2 months ago
Have a Job? Post it

Similar Positions

BigData/Hadoop Developer
  • Noralogic Inc
  • Richmond, VA, USA
Bigdata Developer
  • Vedainfo Inc.
  • Bethesda, MD, USA
Data Engineer : Spark + Python + Java + AWS
  • Cloudious LLC
  • Richardson, TX, USA
Cloud/Data Engineer
  • Randstad Technologies
  • Merrimack, NH, USA
Big Data ETL Developer
  • Sparksoft
  • Columbia, MD, USA
Big Data Architect
  • Visionet Systems
  • Tarrytown, NY, USA
Data Engineer
  • AGM Tech Solutions, LLC
  • Alpharetta, GA, USA
AWS Data Engineer - BigData
  • STS Consulting
  • Newark, NJ, USA
Senior Data Engineer
  • The Astor Group
  • New York, NY, USA