Search Jobs | Dice.com

Job Details

Skills

RAG
RETRIEVAL AUGMENTED
LLM
GENERATIVE AI
GENAI
AGENTS
AGENTIC
MCP
MODEL CONTEXT PROTOCOL
PYTHON
EVALS
LLM EVALUATION
AI EVALUATION
LLM-AS-JUDGE
EVALUATION FRAMEWORK
HUMAN-IN-THE-LOOP

Summary

We are looking for an Applied AI Engineer

The Human Experience team is building a connected ecosystem of AI-powered research, content, and design capabilities that can be used across product and development workflows. We are looking for a hands-on applied AI engineer to establish the knowledge and technical foundations for this ecosystem, strengthen the way HX skills are built and operated, and integrate priority capabilities into the application development lifecycle. This person will design and implement knowledge systems for MESH and broader HX assets; build reliable ways for generators, evaluators, and product tools to use that knowledge; and establish the evaluation, governance, and lifecycle practices required for responsible reuse at scale.

Responsibilities:

Design and implement a structured, machine-readable representation of MESH principles, attributes, surface-specific guidance, observable signals, evaluation criteria, and supporting evidence.
Define the ontology, taxonomy, metadata, entities, relationships, and schemas needed to make MESH usable by people, AI skills, generators, evaluators, and product tools.
Identify the users and systems that will consume MESH knowledge and define appropriate access and retrieval patterns for each.
Enable tools to retrieve and apply the appropriate MESH guidance based on product surface, workflow, user context, and stage of the ADLC.
Establish processes for reviewing, validating, updating, versioning, and maintaining MESH knowledge.
Preserve traceability between MESH guidance, its source evidence, approved changes, and downstream applications.
Design a connected knowledge model for HX assets, including prior research, archetypes, user context, user flows, jobs to be done, content guidance, design guidance, and experience standards.
Evaluate and recommend the appropriate architecture, including graph, relational, semantic retrieval, vector search, and document-based approaches, based on the relationships, queries, governance requirements, and use cases the system must support.
Define how knowledge from different HX disciplines and repositories will be classified, related, retrieved, and reused.
Establish requirements for metadata, provenance, freshness, confidence, permissions, quality, and appropriate-use limitations.
Connect HX knowledge to relevant users, products, journeys, workflows, business domains, evidence, and ADLC activities.
Develop retrieval and grounding mechanisms that allow approved AI skills and internal tools to use HX knowledge accurately and contextually.
Partner with HX subject-matter experts to validate methodological fidelity and appropriate use of domain knowledge.
Define how MESH knowledge and guidance are consumed by AI generators, evaluators, and workflow tools.
Build reusable services, APIs, plugins, MCP tools, or other integration components that make MESH available within product and development workflows.
Integrate MESH into relevant stages of the ADLC, including discovery, definition, design, development, evaluation, release, and continuous learning.
Establish mechanisms for approved MESH updates to propagate reliably to dependent skills, evaluators, tools, and workflows.
Implement dependency tracking, compatibility checks, regression testing, release controls, and change notifications for downstream applications.
Partner with product and engineering teams to prototype, test, document, and hand off production integrations.
Assess the current AI skills ecosystem and recommend a target architecture and phased improvement plan that preserves viable work while addressing duplication, gaps, dependencies, and technical debt.
Establish and maintain repository structures, information architecture, templates, contribution standards, and publishing workflows for HX skills.
Develop, package, version, publish, document, and maintain reusable research, content, design, and evaluation skills.
Create maintainable patterns that allow HX contributors to extend the ecosystem without requiring deep AI engineering expertise.
Track dependencies, versions, usage, user feedback, failure patterns, known issues, and improvement opportunities.
Create onboarding, contribution, technical, and user documentation and facilitate working sessions with contributors and partner teams.
Design integrations between the HX knowledge and skills ecosystem and platforms such as Signals Hub and Product Studio.
Define how evidence from sources such as Pendo, Google Analytics, Salesforce, app reviews, user feedback, and research repositories should be ingested, normalized, classified, connected, and made available for approved uses.
Build prototypes, APIs, pipelines, connectors, repository automations, or other integration components within the agreed scope.
Preserve source attribution, data lineage, access permissions, freshness, and research context when evidence is used by AI workflows.
Support a release-to-discovery learning loop in which product outcomes and experience signals can inform future HX guidance, MESH updates, and skill priorities.
Translate intended outcomes, representative users, and real workflow scenarios into practical evaluation plans.
Build and maintain evaluation datasets, test cases, rubrics, automated checks, regression suites, and human-review workflows appropriate to each capability.
Evaluate retrieval quality, grounding, methodological fidelity, reliability, usability, and failure behavior across skills, generators, evaluators, and integrations.
Document limitations, failure modes, confidence, release decisions, and improvement opportunities.
Define quality thresholds and release criteria, and ensure that automated evaluators are calibrated against expert human judgment before broader use.
Implement monitoring and feedback loops that support ongoing improvement after release.
Define proportionate standards for testing, review, approval, versioning, release, distribution, deprecation, and maintenance.
Design for role-based access, source permissions, privacy, secure retrieval, auditability, retention, and responsible handling of research and product data.
Establish human oversight and approval points for consequential or uncertain AI-supported workflows.
Ensure that generated outputs can be traced to the relevant approved knowledge, evidence, version, and evaluation results.
Raise technical, security, feasibility, and experience risks early and recommend practical tradeoffs.

Requirements:

Demonstrated experience designing and building production-oriented LLM applications, agents, automations, or reusable AI workflows.
Strong software-engineering skills in Python, TypeScript, or a comparable language, with experience building maintainable APIs, services, integrations, or developer tools.
Experience with retrieval-augmented generation, structured outputs, tool or function calling, context management, and grounding AI outputs in approved knowledge sources.
Experience designing knowledge architectures, metadata schemas, ontologies, semantic models, graph-based systems, or comparable structured knowledge solutions.
Experience evaluating AI-assisted workflows through representative scenarios, structured rubrics, automated tests, regression testing, or human review.
Proficiency with Git-based version control, repository automation, and collaborative development workflows.
Ability to map end-to-end workflows and translate domain-expert methods into clear technical requirements and maintainable implementations.
Strong technical judgment regarding LLM capabilities, limitations, security risks, reliability, observability, and appropriate human oversight.
Ability to work iteratively in an ambiguous, cross-functional environment and communicate decisions, dependencies, and tradeoffs clearly.
Demonstrated ability to produce concise technical, contribution, and user-facing documentation.

Required Skills:

Experience with graph databases, knowledge graphs, ontology tooling, semantic search, vector databases, or hybrid retrieval architectures.
Experience with agent orchestration frameworks, AI evaluation frameworks, observability tools, or Model Context Protocol and comparable integration standards.
Experience with enterprise AI environments, internal developer platforms, governed data systems, or complex product integrations.
Experience in research, content, design, ResearchOps, ContentOps, DesignOps, developer experience, or another domain where expert methods are translated into repeatable workflows.
Experience integrating analytics, CRM, feedback, review, or research data from systems such as Pendo, Google Analytics, Salesforce, or research repositories.
Experience administering repositories and defining contribution, publishing, release, and maintenance processes; Bitbucket experience is helpful but not required.
Familiarity with human-centered AI principles, UX-quality frameworks such as MESH, or evaluation of agentic experiences.
Familiarity with Shape Up or another product shaping and delivery approach.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91132164
Position Id: 26-00301
Posted 8 hours ago

Job Details

Skills

RAG
RETRIEVAL AUGMENTED
LLM
GENERATIVE AI
GENAI
AGENTS
AGENTIC
MCP
MODEL CONTEXT PROTOCOL
PYTHON
EVALS
LLM EVALUATION
AI EVALUATION
LLM-AS-JUDGE
EVALUATION FRAMEWORK
HUMAN-IN-THE-LOOP

Summary

We are looking for an Applied AI Engineer

Responsibilities:

Design and implement a structured, machine-readable representation of MESH principles, attributes, surface-specific guidance, observable signals, evaluation criteria, and supporting evidence.
Define the ontology, taxonomy, metadata, entities, relationships, and schemas needed to make MESH usable by people, AI skills, generators, evaluators, and product tools.
Identify the users and systems that will consume MESH knowledge and define appropriate access and retrieval patterns for each.
Enable tools to retrieve and apply the appropriate MESH guidance based on product surface, workflow, user context, and stage of the ADLC.
Establish processes for reviewing, validating, updating, versioning, and maintaining MESH knowledge.
Preserve traceability between MESH guidance, its source evidence, approved changes, and downstream applications.
Design a connected knowledge model for HX assets, including prior research, archetypes, user context, user flows, jobs to be done, content guidance, design guidance, and experience standards.
Evaluate and recommend the appropriate architecture, including graph, relational, semantic retrieval, vector search, and document-based approaches, based on the relationships, queries, governance requirements, and use cases the system must support.
Define how knowledge from different HX disciplines and repositories will be classified, related, retrieved, and reused.
Establish requirements for metadata, provenance, freshness, confidence, permissions, quality, and appropriate-use limitations.
Connect HX knowledge to relevant users, products, journeys, workflows, business domains, evidence, and ADLC activities.
Develop retrieval and grounding mechanisms that allow approved AI skills and internal tools to use HX knowledge accurately and contextually.
Partner with HX subject-matter experts to validate methodological fidelity and appropriate use of domain knowledge.
Define how MESH knowledge and guidance are consumed by AI generators, evaluators, and workflow tools.
Build reusable services, APIs, plugins, MCP tools, or other integration components that make MESH available within product and development workflows.
Integrate MESH into relevant stages of the ADLC, including discovery, definition, design, development, evaluation, release, and continuous learning.
Establish mechanisms for approved MESH updates to propagate reliably to dependent skills, evaluators, tools, and workflows.
Implement dependency tracking, compatibility checks, regression testing, release controls, and change notifications for downstream applications.
Partner with product and engineering teams to prototype, test, document, and hand off production integrations.
Assess the current AI skills ecosystem and recommend a target architecture and phased improvement plan that preserves viable work while addressing duplication, gaps, dependencies, and technical debt.
Establish and maintain repository structures, information architecture, templates, contribution standards, and publishing workflows for HX skills.
Develop, package, version, publish, document, and maintain reusable research, content, design, and evaluation skills.
Create maintainable patterns that allow HX contributors to extend the ecosystem without requiring deep AI engineering expertise.
Track dependencies, versions, usage, user feedback, failure patterns, known issues, and improvement opportunities.
Create onboarding, contribution, technical, and user documentation and facilitate working sessions with contributors and partner teams.
Design integrations between the HX knowledge and skills ecosystem and platforms such as Signals Hub and Product Studio.
Define how evidence from sources such as Pendo, Google Analytics, Salesforce, app reviews, user feedback, and research repositories should be ingested, normalized, classified, connected, and made available for approved uses.
Build prototypes, APIs, pipelines, connectors, repository automations, or other integration components within the agreed scope.
Preserve source attribution, data lineage, access permissions, freshness, and research context when evidence is used by AI workflows.
Support a release-to-discovery learning loop in which product outcomes and experience signals can inform future HX guidance, MESH updates, and skill priorities.
Translate intended outcomes, representative users, and real workflow scenarios into practical evaluation plans.
Build and maintain evaluation datasets, test cases, rubrics, automated checks, regression suites, and human-review workflows appropriate to each capability.
Evaluate retrieval quality, grounding, methodological fidelity, reliability, usability, and failure behavior across skills, generators, evaluators, and integrations.
Document limitations, failure modes, confidence, release decisions, and improvement opportunities.
Define quality thresholds and release criteria, and ensure that automated evaluators are calibrated against expert human judgment before broader use.
Implement monitoring and feedback loops that support ongoing improvement after release.
Define proportionate standards for testing, review, approval, versioning, release, distribution, deprecation, and maintenance.
Design for role-based access, source permissions, privacy, secure retrieval, auditability, retention, and responsible handling of research and product data.
Establish human oversight and approval points for consequential or uncertain AI-supported workflows.
Ensure that generated outputs can be traced to the relevant approved knowledge, evidence, version, and evaluation results.
Raise technical, security, feasibility, and experience risks early and recommend practical tradeoffs.

Requirements:

Demonstrated experience designing and building production-oriented LLM applications, agents, automations, or reusable AI workflows.
Strong software-engineering skills in Python, TypeScript, or a comparable language, with experience building maintainable APIs, services, integrations, or developer tools.
Experience with retrieval-augmented generation, structured outputs, tool or function calling, context management, and grounding AI outputs in approved knowledge sources.
Experience designing knowledge architectures, metadata schemas, ontologies, semantic models, graph-based systems, or comparable structured knowledge solutions.
Experience evaluating AI-assisted workflows through representative scenarios, structured rubrics, automated tests, regression testing, or human review.
Proficiency with Git-based version control, repository automation, and collaborative development workflows.
Ability to map end-to-end workflows and translate domain-expert methods into clear technical requirements and maintainable implementations.
Strong technical judgment regarding LLM capabilities, limitations, security risks, reliability, observability, and appropriate human oversight.
Ability to work iteratively in an ambiguous, cross-functional environment and communicate decisions, dependencies, and tradeoffs clearly.
Demonstrated ability to produce concise technical, contribution, and user-facing documentation.

Required Skills:

Experience with graph databases, knowledge graphs, ontology tooling, semantic search, vector databases, or hybrid retrieval architectures.
Experience with agent orchestration frameworks, AI evaluation frameworks, observability tools, or Model Context Protocol and comparable integration standards.
Experience with enterprise AI environments, internal developer platforms, governed data systems, or complex product integrations.
Experience in research, content, design, ResearchOps, ContentOps, DesignOps, developer experience, or another domain where expert methods are translated into repeatable workflows.
Experience integrating analytics, CRM, feedback, review, or research data from systems such as Pendo, Google Analytics, Salesforce, or research repositories.
Experience administering repositories and defining contribution, publishing, release, and maintenance processes; Bitbucket experience is helpful but not required.
Familiarity with human-centered AI principles, UX-quality frameworks such as MESH, or evaluation of agentic experiences.
Familiarity with Shape Up or another product shaping and delivery approach.

Dice Id: 91132164
Position Id: 26-00301
Posted 8 hours ago

Senior Machine Learning Engineer

New York, New York

•

Today

Job Description: About the Team: Our Technology team drives the evolution of our Technology, Engineering, Data, Product and User Experience functions. With a keen focus on delivering cutting-edge solutions, we shape the digital landscape for our customers, readers and users. From revolutionizing visuals to optimizing tools and harnessing the power of data, mobile, video and social platforms, our team is committed to providing a seamless and immersive experience across all touchpoints. Collabora

Full-time

USD 125,000.00 - 155,000.00 per year

Sr. Applied Knowledge Engineer (UX)

Skills

Summary

We are looking for an Applied AI Engineer

Responsibilities:

Requirements:

Required Skills:

Dice Job Match Score™

Similar Jobs

Sr. Applied Knowledge Engineer (UX)

Job Details

Skills

Summary

We are looking for an Applied AI Engineer

Responsibilities:

Requirements:

Required Skills:

Dice Job Match Score™

Similar Jobs