Azure Data Engineer

Overview

Hybrid
Depends on Experience
Contract - W2
Contract - Independent

Skills

Azure
Data Engineer
pyspark
Python

Job Details

Position Summary

Accountable for developing and delivering technological responses to targeted business outcomes. Analyze, design and develop enterprise data and information architecture deliverables, focusing on data as an asset for the enterprise. Understand and follow reusable standards, design patterns, guidelines, and configurations to deliver valuable data and information across the enterprise, including direct collaboration , where needed. Demonstrate the company's core values of respect, honesty, integrity, diversity, inclusion and safety.

Required:

4+ years of Python experience

Strong experience with SQL, querying with extensive join conditions and updating based on conditions

DevOps: Linux, GitHub, Bash

Experience with CI/CD lifecycles,

Experience writing Unit, integration, and regression tests.

Experience with static code analysis tools, Linters, and PEP 8 standards

Experience with Microsoft Azure and/or Google Cloud Platform

Experience with Agile SCRUM projects.

Highly Desired:
Data Management: Databricks, PySpark, Spark Structured Streaming, Delta Live Tables, Delta Sharing,
Expert knowledge of SQL
Experience with Streaming Technologies: Apache Kafka, Azure EventHubs, Avro
DevOps: GitHub Actions, Terraform, Artifactory
Desired:
Power BI, Tableau, Java, SpringBoot, Kubernetes, Docker, NoSQL

Key Responsibilities

Key Responsibilities

Utilize enterprise standards for data domains and data solutions, focusing on simplified integration and streamlined operational and analytical uses

Ensure there is clarity between ongoing projects, escalating when necessary, including direct collaboration .

Leverage innovative new technologies and approaches to renovate, extend, and transform the existing core data assets, including SQL-based, NoSQL-based, and Cloud-based data platforms

Define high-level migration plans to address the gaps between the current and future state

Contribute to the development of cost/benefit analysis for leadership to shape sound architectural decisions

Analyze technology environments to detect critical deficiencies and recommend solutions for improvement

Promote the reuse of data assets, including the management of the data catalog for reference

Draft architectural diagrams, interface specifications and other design documents