Platform Engineer

Overview

Hybrid
$100,000 - $120,000
Full Time

Skills

Data Engineer
ETL
Python
Java
SQL exp
Data governance

Job Details

Role : Platform Engineer

Locations : NYC , NY

Job Type: Full Time

No visa restrictions OPT /H1B Workable


JOB DESCRIPTION

Major Roles & Responsibilities:
Analysis of business processes to understand current setup, navigating through multiple systems, products and coordinate with multiple stakeholders.
Build and implement data lineage frameworks that track the flow of data across multiple systems, databases, and pipelines.
Work with stakeholders to define and map the end-to-end flow of data, identifying sources, transformations, destinations, and dependencies.
Implement solutions to catalog metadata from diverse data sources, including MongoDB, Snowflake, and relational databases (e.g., SQL Server, DB2).
Collaborate with firmwide data governance teams to ensure that lineage metadata is integrated with Collibra for auditing and compliance purposes.
Develop processes to monitor data quality and ensure that data lineage tracks and reflects accurate data transformations and business rules.
Design and implement automation tools and scripts to capture and maintain data lineage information and catalog updates on a scheduled basis.
Create clear, understandable, and accurate documentation of data lineage for business users and technical teams.
Identify and resolve issues related to data inconsistencies, lineage discrepancies, or errors in data transformation processes.
Document technical requirements, specifications, and solutions.
Required skills:
3+ years of development experience.
Experience/exposure in data engineering, data governance, or related fields, with a focus on data lineage and metadata cataloging.
Strong expertise in data modeling, SQL, Stored Procedures/triggers, etc.
Should possess good knowledge of OO, data structures and algorithms.
Experience with data modeling, ETL pipelines, and data transformation.
Experience with SQL and relational databases (e.g., Sybase, DB2, SQL Server).
Proficiency in Python, Java, or other scripting languages for data integration and automation tasks.
Strong analytical and troubleshooting skills with the ability to solve complex data flow problems.
Excellent written and verbal communication skills, with the ability to explain technical concepts to non-technical stakeholders.
Ability to work in a fast-paced, collaborative environment with cross-functional teams.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.