We are seeking a Lead AI Engineer with expertise in Databricks and Databricks Agentic AI processes to join our team. The successful candidate will play a crucial role in delivering agentic solutions within our data platform for the PLM use cases. This position requires a seasoned professional with a strong background in AI engineering and a track record of leveraging Databricks for building agents. Two AI Engineers are required to design, build and operate the core intelligence layer of the Product Lifecycle Management (PLM) solution. Working within an approved AI pattern on Databricks (Azure), you will be fully embedded within the PLM team to develop a deep understanding of the domain, processes and data before building the end-to-end pipeline from raw supplier document ingestion through to structured AI-generated recommendations surfaced for human review. You will also build and maintain the Databricks Apps front-end (React/Python) that exposes this capability to approximately 30 business users. Responsibilities Embed fully within the PLM team from day one to understand supplier specification workflows, test method structures and properties management before writing a line of code Design and implement the document ingestion pipeline using Databricks ai_parse_document for PDF, Excel and Word supplier specification files, storing artifacts in Unity Catalog Volumes Build and maintain Agent Bricks (Information Extraction) pipelines to extract structured fields from supplier documents against the canonical PLM data model Develop and optimize LLM-powered grouping and recommendation logic using ai_query and Databricks Model Serving, ensuring outputs are explainable and auditable Create the Databricks Apps front-end (React + Python backend): file upload, metadata grids, dropdown-driven forms, human-in-the-loop review/approval screens and write-back to Lakebase Integrate the Graph Database layer to store and traverse relationships between specifications, properties, test methods and AI recommendations Configure and maintain Azure Private Link / Private Endpoint connectivity, including DNS forwarding for databricksapps.com, to ensure all traffic stays within the corporate network perimeter Implement structured logging, lineage tracking in Unity Catalog, and audit trails to support Responsible AI governance requirements Contribute to ARB artifacts and technical design documentation as required Collaborate daily with the Business Analyst to translate PLM domain requirements into working data models and pipeline logic, and with the Delivery Manager to surface risks and effort estimates Requirements 5+ years of experience in AI/data engineering roles At least 1 year of relevant leadership experience Expertise in Databricks (Unity Catalog, Volumes, Model Serving) Proficiency in ai_parse_document and Agent Bricks Skills in Python (PySpark, pandas, FastAPI) Background in React front-end development (hooks, REST integration) Competency in Azure (Private Link, Private Endpoints, VNet) Knowledge of LLM prompt engineering and ai_query Familiarity with Graph Databases (Neo4j or equivalent) Understanding of Delta Lake and Lakebase Capability to design and integrate REST APIs Experience with Azure DevOps (ADO) Proficiency in English at an Upper-Intermediate level (B2) or higher Nice to have Experience with GxP or regulated manufacturing data environments Exposure to PLM or ERP systems (SAP, Teamcenter, Enovia) Databricks certification (Data Engineer Professional or ML Professional) Background in graph data modeling and lineage use cases
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 10330481
- Position Id: 1fe39f744c027e53992f4bbb23b79380
- Posted 2 days ago
Never miss an opportunity! Create an alert based on the job you applied for.