Data Architect/ Sr./ Lead Data Engineer

Overview

On Site
Depends on Experience
Accepts corp to corp applications
Contract - W2
Able to Provide Sponsorship

Skills

Apache Spark
Amazon Neptune
Amazon Web Services
Apache Flink
ArangoDB
Attention To Detail
Cloud Computing
Computerized System Validation
Conflict Resolution
Data Engineering
Data Governance
Data Modeling
Data Processing
Data Security
Data Structure
ELT
Engineering Support
Fraud
Extract
Transform
Load
Google Cloud Platform
JSON
Java
Modeling
Neo4j
Microsoft Azure
Machine Learning (ML)
Research and Development
Scala
Python
System Integration
Storage

Job Details

Job Title: Mid to Senior Data Engineer (Contract)

Project Focus: ID Resolution Co-Innovation Project

Location: New York City, NY (On-site, 5 days/week)

Duration: Minimum 6-month contract

About the Project:

We are seeking a talented Mid to Senior Data Engineer to join our team for an exciting ID Resolution Co-Innovation Project. This project involves a collaborative effort between a Leading Service Provider and an Innovative Client to explore, develop, and operationalize novel methods for entity and identity resolution within graph-based data systems. You will play a crucial role in integrating client graph data with advanced graph models, testing analytical approaches, and evaluating probabilistic identity resolution within a production environment.

Key Responsibilities:

  • Data Ingestion & Transformation: Design, build, and maintain robust data pipelines to ingest, transform, and prepare large-scale graph datasets (potentially involving millions of user entities) from various sources (e.g., CSV, JSON) into a format suitable for advanced graph modeling.
  • Graph Data Engineering: Work extensively with graph data structures and technologies, ensuring efficient storage, retrieval, and processing of complex interconnected data.
  • System Integration: Collaborate closely with Client and Service Provider teams to gain access to key systems and infrastructure, confirm data availability, and integrate developed solutions into existing online software systems.
  • Data Security & Governance: Implement and enforce data security requirements, privacy regulations, and PII handling best practices throughout the data lifecycle.
  • Methodology Support: Provide engineering support for the development and evaluation of ID resolution methodologies, including graph learning, spectral graph theory, and other machine learning approaches.
  • Operationalization: Support the deployment and operationalization of selected ID resolution approaches within test and production environments, including configuration and integration.
  • Documentation: Contribute to the clear and concise documentation of data mappings, ingestion processes, transformation logic, and system configurations.

Required Skills & Experience:

  • 5+ years of experience in data engineering, with a strong focus on building and maintaining scalable data pipelines (ETL/ELT).
  • Proficiency in graph databases/technologies (e.g., Neo4j, AWS Neptune, ArangoDB, or similar) and deep understanding of graph data modeling principles.
  • Experience with large-scale data processing frameworks (e.g., Apache Spark, Flink).
  • Strong programming skills in languages commonly used for data engineering (e.g., Python, Scala, Java).
  • Familiarity with cloud platforms (AWS, Azure, Google Cloud Platform) and their data services.
  • Solid understanding of data security, privacy, and data governance best practices.
  • Experience with API integration for connecting disparate systems.
  • Ability to work effectively in a collaborative, fast-paced environment.
  • Excellent problem-solving skills and attention to detail.

Nice-to-Have Skills:

  • Exposure to machine learning concepts, particularly in the context of identity resolution, recommendation systems, or fraud detection.
  • Experience working on co-innovation or R&D focused projects

Sincerely,

Tamana Nair

Digitek Software, Inc.

650 Radio Drive, Lewis

Center, OH 43035 Tel No ext. 3105/ Fax

Email

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.