Mid/Senior Data Engineer AI Data Pipelines

Hybrid in New York, NY, US • Posted 9 days ago • Updated 1 day ago
Contract W2
6 Months
No Travel Required
Hybrid
Depends on Experience
Fitment

Dice Job Match Score™

🧠 Analyzing your skills...

Job Details

Skills

  • Analytics
  • Apache Flink
  • Apache Kafka
  • Artificial Intelligence
  • Big Data
  • Cloud Computing
  • Apache Hadoop
  • Data Engineering
  • Generative Artificial Intelligence (AI)
  • Python
  • JIRA
  • Customer Support
  • Claude
  • Codex

Summary

<>Mid/Senior Data Engineer – AI Data Pipelines <>NYC, NY<>Contract 6+ Months with possible extension

We’re hiring a strong Data Engineer to turn messy customer support data into clean, AIready intelligence. You’ll build highscale pipelines, vector DBs, and reliable data systems that power our nextgen AI agents.

Role Highlights

  • Clean, process & structure huge volumes of unstructured support data (Salesforce, Sprinklr, Bliss, JIRA).
  • Build AI memory systems using vector databases (Pinecone, Milvus, Weaviate, pgvector).
  • Create a central metrics hub for analytics & AI insights.
  • Own highreliability pipelines with monitoring & zerodowntime standards.
  • Work closely with product/ops and push for better data logging at the source.

Tech Stack

  • Python, PySpark
  • Spark, Kafka, Flink, Hadoop, Hudi, Presto, Pinot
  • Vector DBs (Pinecone/MilvWeaviate/pgvector)
  • Cloud data warehouses
  • AI tools (Claude, Codex, etc.) for automation & parsing

Ideal Candidate

  • 5+ years in data engineering with big data systems
  • Strong Python + PySpark
  • Experience building AIready data layers
  • Proactive, independent, businessfocused
  • Comfortable working in ambiguity and driving clarity
  • Honest communicator who flags risks early
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10110049
  • Position Id: 9024800
  • Posted 9 days ago
Contact the job poster
Vivek Mishra

Vivek Mishra

Pegasus Knowledge Solutions Recruiter @ Pegasus Knowledge Solutions
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Rutherford, New Jersey

Today

Full-time

USD 121,200.00 - 181,800.00 per year

Rutherford, New Jersey

Today

Full-time

USD 121,200.00 - 181,800.00 per year

New York, New York

2d ago

Easy Apply

Contract

$60 - $70

New York, New York

Today

Full-time

Search all similar jobs