Jr Data Engineer

JOB SUMMARY:

This role is part of a broader data engineering team who develop data pipelines, integrations, and data products to meet internal operational and analytic use cases. The Sr. Software Engineer - Data will partner with other engineers, architects, analysts and product managers to implement scalable, sustainable solutions. They work on technical assignments that are moderately difficult and require considerable judgment and initiative. They must understand implications of work and make recommendations for solutions. They normally receive little instruction on daily work and only require general instructions on newly introduced assignments.

MAIN RESPONSIBILITIES:

Technical Engineering Services - this team operates in an Azure + Databricks Lakehouse. The role requires the following technical skill set

Cloud data engineering experience, including:

* data orchestration with tools such as Azure Data Factory

* data storage in cloud storage such as Azure Data Lake

* SDLC + CI/CD through systems such as Azure DevOps

* SQL proficiency (TSQL, PostGresSQL, ) and familiarity with Spark, including the ability to develop ETL processes using PySpark and SparkSQL

* Data streaming experience (Kafka, event hub, etc)

* Experience with parquet and other data exchange/storage format

* Experience with agile development and standard DevOps practices.

* Should understand principles of indexes and partitioning

* Should have experience in optimizing data engineering code for performance

* Proficiency with parquet and other data exchange/storage formats

* Experience with agile development and standard DevOps practices.

* Should have a deep understanding of indexes and partitioning

* Should be proficient optimizing data engineering code for performance (able to read a DAG, determine where CBO is using most resources)

* Should be proficient in common distributed cloud computing concepts like fault tolerance, idempotency, SOA, etc.

MAIN RESPONSIBILITIES:

* Approximate Percent of Time* Leadership & Relationship Skills - this team is part of a broader, cross-functional organization with a variety of role types and experience levels. This individual needs to be able to:

* Collaborate with technical teams and utilizes system expertise to deliver technical solutions.

* Continuously learns and teaches others existing and new technologies.

* Contributes to the development of others through mentoring, in house workshops, and learning sessions.

* Drive engineering projects by developing data solutions; conducting tests and inspections; preparing reports and calculations. Expected to supervise base and associate level engineers as needed.

* Understand system protocols, how systems operate and data flows. Aware of current technology benefits. Expected to independently develop a full software stack. Understands the building blocks, interactions, dependencies, and tools required to complete software and automation work. Independent study of current technology is expected. Interact with system engineers to define system requirements and/or necessary requirements for automation.

Communication:

This team leads the enterprise in platform and process. Communication of architecture, process, and implementation is expected to elevate all teams. This individual needs to be able to:

* Write basic documentation on how technology works. Create clear documentation for new code and systems used.

* Documenting systems designs, presentations, and business requirements for consumption and consideration at the manager and director level.

Technology Strategy

Contributes to emerging technologies to deliver business goals and understands current technology that supports business goals.

Understands system protocols, how systems operate and data flows. Expected to independently develop a full software stack.

Understands the building blocks, interactions, dependencies, and tools required to complete software and automation work. Aware of current technology benefits.

Identify and design solutions to potential data processes that can be improved with automation or optimized with current technical best practices.

Identify use cases and design technical solutions to include Agentic AI application

MAIN RESPONSIBILITIES: Innovation - this group leverages industry leading platforms and processes. This individual is expected to:

* Deal with the ambiguity involved in developing new systems/platforms

* Design and implement new ideas which utilize novel frameworks to improve an existing or new system/process/service.

* Review existing designs and processes to highlight more efficient ways to complete existing workload more effectively through industry perspectives.

Contribute to new and existing technology options that support business goals and understand how the current technology supports existing/future business goals.