AI Architect — Agentic Data Modernization (IDM Platform) - Remote

Position: AI Architect — Agentic Data Modernization (IDM Platform)

Location- Remote

Duration: Long term Contract

Responsibility:

• Design multi-agent orchestration frameworks using Lang Graph — stateful graphs, conditional routing, agent handoff, retry and fallback logic
• Build agent harnesses coordinating Discovery, Parsing, Mapping, Code Generation, Validation, and Review agents across a shared execution context
• Develop the IDM prompt library — system prompts, few-shot templates, structured output schemas, and reflection loops for each conversion workstream
• Build LLM-powered code conversion pipelines ex: DataStage → Databricks PySpark, Dremio SQL → Snowflake SQL, legacy ETL → cloud-native equivalents
• Lead AICH–IDM platform integration — capability merger, MCP server design, shared tool registry, unified agentic execution surface
• Architect and operate conversion pipelines for 50,000–80,000+ legacy objects with parallelism, batching, resumability, and audit logging
• Build metadata frameworks for conversion traceability — extraction run tracking, job dependency graphs, column-level lineage, confidence scoring
• Implement LLM routing layers that select models (Claude, OpenAI, Azure OAI) based on task type and quality requirements
• Build and maintain IDM backend services — FastAPI, Celery/Redis, LangGraph runtime, CI/CD integration
• Surface agent observability — token usage, latency per hop, model selection audit trail, output quality metrics

Experience

• 10 - 15 years total experience
• 1-2 years building production LLM systems beyond chatbots
• One enterprise-scale legacy data migration program preferred
• Consulting background preferred
• Agile based delivery experience

Technical Skills Expected

• LangGraph — production-grade stateful graph design, interrupt/resume, shared memory, conditional edges
• LLM APIs — Anthropic Claude, OpenAI, Azure OpenAI; tool use, structured outputs, prompt construction at scale
• Python — async, FastAPI, Pydantic, Celery, Redis
• Prompt engineering — few-shot design, chain-of-thought, output parsers, self-consistency, reflection loops; not just RAG chat patterns
• Metadata-driven architecture — YAML config-driven generation, schema inference, column-level lineage design
• MCP server design and tool registration
• Vector stores and RAG
• Claude Code — experience using Claude Code as an agentic coding harness
• SKILL.md / prompt library design — ability to design and maintain skill files that encode conversion rules, output constraints, and few-shot patterns as versioned, reusable assets loaded by the harness at runtime

Technical Skills Good to Have

• Legacy ETL platforms — DataStage, Informatica etc; enough depth to understand what the agent is converting
• Databricks — PySpark, notebooks, Unity Catalog, Delta Lake
• Snowflake — SQL, Snowpark, DDL generation patterns
• AWS — S3, Glue, Lambda; IAM and data lake patterns
• Apache Iceberg — table format internals, catalog integration