Overview
Hybrid
Depends on Experience
Accepts corp to corp applications
Contract - Independent
Contract - W2
Skills
Artificial Intelligence
Big Data
Apache Spark
Cloud Computing
Machine Learning (ML)
Extract
Transform
Load
ELT
Databricks
Microsoft Azure
Python
Scripting
Graph Databases
Geographic Information System
Forecasting
Vector Databases
Data Partitioning
Data conflation
GIS spatial data
LLMs
Job Details
Agentic Data Engineer to design, develop, and deploy data pipelines that leverage agentic AI that solve real-world problems Client is seeking a highly skilled Agentic Data Engineer to design, develop, and deploy data pipelines that leverage agentic AI that solve real-world problems. The ideal candidate will have experience in designing data process to support agentic systems, ensure data quality and facilitating interaction between agents and data.
| Understanding the Big data Technologies | Required | 1 | Years |
| Experience developing ETL and ELT pipelines | Required | 1 | Years |
| Experience with Spark, GraphDB, Azure Databricks | Required | 1 | Years |
| Expertise in Data Partitioning | Required | 1 | Years |
| Experience with Data conflation | Required | 3 | Years |
| Experience developing Python Scripts | Required | 3 | Years |
| Experience training LLMs with structured and unstructured data sets | Required | 2 | Years |
| Experience with GIS spatial data | Required | 3 | Years |
Qualifications:
- Strong Data engineering fundamentals
- Utilize Big data frameworks like Spark/Databricks
- Training LLMs with structed and unstructured data sets.
- Understanding of Graph DB
- Experience with Azure Blob Storage, Azure Data Lakes, Azure Databricks
- Experience implementing Azure Machine Learning, Azure Computer Vision, Azure Video Indexer, Azure OpenAI models, Azure Media Services, Azure AI Search
- Determine effective data partitioning criteria
- Utilize data storage system spark to implement partition schemes
- Understanding core machine learning concepts and algorithms
- Familiarity with Cloud computing skills
- Strong programming skills in Python and experience with AI/ML frameworks.
- Proficiency in vector databases and embedding models for retrieval tasks.
- Expertise in integrating with AI agent frameworks.
- Experience with cloud AI services (Azure AI).
- Experience with GIS spatial data to create markers on maps ( lat long nearest topology of road, geo-locate between datasets, correlation etc.).
- Experience with Department of Transportation Data Domains developing an AI Composite Agentic Solution designed to identify and analyzedata models, connect & correlateinformation to validatehypotheses, forecast, predict and recommendpotential strategies and conduct What-if analysis.
- Bachelor's or master's degree in computer science, AI, Data Science, or a related field.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.