AI/ML Software Engineer- State of Maryland experienced

SUMMARY

The AOC is seeking proposals from prospective Offerors to provide one (1) AI/ML Software Engineer. The

AI/ML Software Engineer will build software tools that incorporate AI/ML techniques to automate narrowly defined tasks with high accuracy, assist internal users with their job functions, and improve the experience external users have when interacting with the Maryland Judiciary. This includes, but is not limited to, RPA work, building or refining chatbots, incorporating AI/ML into reporting tools, building llm agents for knowledge retrieval, deep research, translation, transcription, redaction, document analysis, document generation, agentic coding, and data processing.

B. OFFEROR RESOURCE QUALIFICATIONS

1. The Offeror shall propose resource(s) that meet the following minimum qualifications:

a. Bachelor of Science in Engineering, Computer Science, Data Science, or Mathematics, or a

related field (as determined by the AOC).

2. The AOC prefers Offeror proposed resource(s) to have the following qualifications:

a. At least three (3) years experience in data science, machine learning, or applied AI

development.

b. At least three (3) years experience in software engineering, architecture, or web

development.

C. SCOPE OF WORK

Offeror proposed resource(s) shall be responsible for the following:

1. System Design & Collaboration:

a. Work within established constraints regarding infrastructure, programming languages, and

model selection

b. Contribute to technical decision-making related to data processing, retrieval strategies, and

system integration

c. Collaborate with team members to define agent architectures, workflows, and system design

decisions

d. Evaluate and select appropriate approaches for given tasks, including determining when to

use LLM-based versus non-LLM techniques

e. Designing and building software systems that integrate AI/ML techniques to automate tasks,

assist internal users, and improve user-facing services.

2. Testing, Evaluation, and Quality Assurance:

a. Assist in the design and implementation of testing and evaluation pipelines for AI/ML

systems

b. Develop unit and integration tests for AI-enabled workflows and data pipelines

c. Generate and utilize synthetic data to support evaluation and benchmarking efforts

d. Contribute to improving system performance, including accuracy, latency, and cost

efficiency

3. Deployment & Operations:

a. Support deployment of AI/ML applications within a hybrid cloud environment

b. Work with containerized applications to ensure reliable deployment and updates.

c. Optimize systems for environments with limited computational resources, including minimal

GPU availability

4. General Responsibilities:

a. Deliver production-grade systems aligned with defined requirements, while supporting

iterative improvement of evolving tools

b. Document system designs, workflows, and technical decisions as required

c. Stay informed on relevant advancements in AI/ML and apply them where appropriate within

project constraints

5. In addition to the overall responsibilities described in Section III.C.1-4, the Offeror s proposed

resource(s) will complete the deliverables listed below by Purchase Order year. The estimated level

of effort for each deliverable may vary based on its complexity and may be adjusted as needed,

including extension beyond the app

E. OFFEROR RESOURCE(S) SKILLS, EXPERIENCE, & CAPABILITIES

1. Offeror shall propose resource(s) possessing the following preferred skills, experience, and

capabilities:

a. Experience with:

(1) SQL and relational database systems (e.g., PostgreSQL)

(2) Fine-tuning small language models or embedding models

(3) Contributing to or maintaining open-source software projects

(4) Graph databases or graph extensions (e.g., Neo4j, Apache AGE)

(5) Designing and implementing multi-agent or task-oriented AI systems

(6) Embedding models, vector similarity, re-ranking, and graph retrieval techniques in

RAG systems

(7) Version control systems (e.g., Git), containerization technologies (e.g., Docker), and

service-oriented architectures

(8) Collaborating with large language models (LLMs), including both API-based

integration and local deployment

(9) Validating AI-generated outputs, mitigating hallucinations, and integrating AI tools

into production service pipelines

b. Ability to:

(1) Understand data structures, algorithms, and clean coding principles

(2) Select and apply appropriate techniques (LLM and non-LLM) based on task

requirements

(3) Develop and improve testing and evaluation pipelines for AI systems, including use

of synthetic data

(4) Demonstrate proficiency in Python, including the ability to develop productiongrade backend services, APIs, middleware, and data pipelines.

(5) Design and implement AI/ML systems that operate effectively on complex,

inconsistent, or evolving datasets while balancing accuracy, latency, and cost (token

consumption)

(6) Collaborate with team members to define system architecture, agent workflows, and

data pipelines while working in constrained environments, including limited GPU

availability and predefined infrastructure

c. Knowledge of:

(1) Hybrid cloud environments and distributed system considerations

(2) Threading, asynchronous processing, and queues in backend servers

(3) React and Microsoft Teams Toolkit for developing chatbot user interfaces

(4) Non-llm data analysis techniques for structured, semi-structured, and unstructured data

(5) Classical natural language processing (NLP) techniques in addition to LLM-based approaches

(6) Data science and LLM-related libraries in Rust or other performance-oriented programming languages into production service pipelines