Overview
On Site
USD 122,400.00 - 228,000.00 per year
Full Time
Skills
Data Analysis
Reporting
SAFE
Bridging
Continuous Improvement
Scalability
Banking
Research
Regulatory Compliance
Data Science
Python
Java
Machine Learning (ML)
PyTorch
TensorFlow
Evaluation
Large Language Models (LLMs)
Workflow
Deep Learning
Machine Learning Operations (ML Ops)
Continuous Integration
Continuous Delivery
Cloud Computing
Artificial Intelligence
Analytical Skill
Conflict Resolution
Problem Solving
Collaboration
Innovation
Finance
Health Insurance
Life Insurance
Training
Coaching
Network
Law
Recruiting
Job Details
Application Deadline:
11/29/2025
Address:
320 S Canal Street
Job Family Group:
Data Analytics & Reporting
About the Team
BMO's Applied AI team sets the standard for safe, high-performing AI systems across the bank, building BMO's leading-edge AI products and capabilities. Our work enables BMO's strategic outcomes, bridging business needs with scalable and responsible AI. If you thrive on technical challenges and want to make a strategic impact, this role is for you.
About the Role
As an Engineering & Data Science Specialist in Applied AI Evaluations, you will design, implement, and optimize evaluation frameworks and pipelines that measure the quality, safety, and performance of advanced AI systems at BMO. You'll work closely with the Product Owner and collaborate with engineering, research, and business teams to translate evaluation standards into robust, scalable solutions.
Your work will directly impact model development, deployment, and continuous improvement cycles-ensuring that evaluation signals are deeply integrated into experimental and production systems. You'll contribute to the development of reusable tools, agents, and harnesses that support responsible AI practices and regulatory compliance.
Key Responsibilities
Qualifications
Salary :
$122,400.00 - $228,000.00
Pay Type:
Salaried
The above represents BMO Financial Group's pay range and type.
Salaries will vary based on factors such as location, skills, experience, education, and qualifications for the role, and may include a commission structure. Salaries for part-time roles will be pro-rated based on number of hours regularly worked. For commission roles, the salary listed above represents BMO Financial Group's expected target for the first year in this position.
BMO Financial Group's total compensation package will vary based on the pay type of the position and may include performance-based incentives, discretionary bonuses, as well as other perks and rewards. BMO also offers health insurance, tuition reimbursement, accident and life insurance, and retirement savings plans. To view more details of our benefits, please visit:
About Us
At BMO we are driven by a shared Purpose: Boldly Grow the Good in business and life. It calls on us to create lasting, positive change for our customers, our communities and our people. By working together, innovating and pushing boundaries, we transform lives and businesses, and power economic growth around the world.
As a member of the BMO team you are valued, respected and heard, and you have more ways to grow and make an impact. We strive to help you make an impact from day one - for yourself and our customers. We'll support you with the tools and resources you need to reach new milestones, as you help our customers reach theirs. From in-depth training and coaching, to manager support and network-building opportunities, we'll help you gain valuable experience, and broaden your skillset.
To find out more visit us at
BMO is proud to be an equal employment opportunity employer. We evaluate applicants without regard to race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or any other legally protected characteristics. We also consider applicants with criminal histories, consistent with applicable federal, state and local law.
BMO is committed to working with and providing reasonable accommodations to individuals with disabilities. If you need a reasonable accommodation because of a disability for any part of the employment process, please send an e-mail to and let us know the nature of your request and your contact information.
Note to Recruiters: BMO does not accept unsolicited resumes from any source other than directly from a candidate. Any unsolicited resumes sent to BMO, directly or indirectly, will be considered BMO property. BMO will not pay a fee for any placement resulting from the receipt of an unsolicited resume. A recruiting agency must first have a valid, written and fully executed agency agreement contract for service to submit resumes.
11/29/2025
Address:
320 S Canal Street
Job Family Group:
Data Analytics & Reporting
About the Team
BMO's Applied AI team sets the standard for safe, high-performing AI systems across the bank, building BMO's leading-edge AI products and capabilities. Our work enables BMO's strategic outcomes, bridging business needs with scalable and responsible AI. If you thrive on technical challenges and want to make a strategic impact, this role is for you.
About the Role
As an Engineering & Data Science Specialist in Applied AI Evaluations, you will design, implement, and optimize evaluation frameworks and pipelines that measure the quality, safety, and performance of advanced AI systems at BMO. You'll work closely with the Product Owner and collaborate with engineering, research, and business teams to translate evaluation standards into robust, scalable solutions.
Your work will directly impact model development, deployment, and continuous improvement cycles-ensuring that evaluation signals are deeply integrated into experimental and production systems. You'll contribute to the development of reusable tools, agents, and harnesses that support responsible AI practices and regulatory compliance.
Key Responsibilities
- Develop, implement, and maintain evaluation pipelines and harnesses for AI models, ensuring reliability, reproducibility, and scalability for banking applications.
- Collaborate with the Product Owner to translate evaluation standards and business requirements into technical solutions.
- Prototype and validate evaluation methods using real banking workflows, integrating feedback into model training and deployment.
- Integrate evaluation metrics and signals into research, training, and production systems to drive measurable improvements in model performance and customer outcomes.
- Partner with engineering, research, and product teams to shape model interaction paradigms and deployment strategies.
- Build reusable tools and systems that enable contributions from across BMO and raise the quality bar for AI solutions.
- Ensure all evaluation frameworks adhere to regulatory, compliance, and Responsible AI requirements.
- Document processes, share knowledge, and contribute to a culture of continuous learning and responsible innovation.
Qualifications
- 3+ years of experience in engineering or data science roles, preferably in AI, machine learning, or data-driven product environments.
- Strong programming skills (e.g., Python, Java, or similar) and experience with ML/AI frameworks (e.g., PyTorch, TensorFlow).
- Experience building and maintaining evaluation pipelines for large language models (LLMs) and/or other AI systems, including multi-agent workflows, tool use, or long-context scenarios.
- Solid understanding of deep learning concepts, model training, and deployment practices.
- Familiarity with MLOps, CI/CD, and cloud-based AI infrastructure is a plus.
- Strong analytical and problem-solving skills, with the ability to communicate technical concepts to both technical and non-technical audiences.
- Motivated by high-impact collaboration and able to thrive in fast-paced, ambiguous environments.
- Commitment to BMO's values of inclusion, integrity, and responsible innovation.
Salary :
$122,400.00 - $228,000.00
Pay Type:
Salaried
The above represents BMO Financial Group's pay range and type.
Salaries will vary based on factors such as location, skills, experience, education, and qualifications for the role, and may include a commission structure. Salaries for part-time roles will be pro-rated based on number of hours regularly worked. For commission roles, the salary listed above represents BMO Financial Group's expected target for the first year in this position.
BMO Financial Group's total compensation package will vary based on the pay type of the position and may include performance-based incentives, discretionary bonuses, as well as other perks and rewards. BMO also offers health insurance, tuition reimbursement, accident and life insurance, and retirement savings plans. To view more details of our benefits, please visit:
About Us
At BMO we are driven by a shared Purpose: Boldly Grow the Good in business and life. It calls on us to create lasting, positive change for our customers, our communities and our people. By working together, innovating and pushing boundaries, we transform lives and businesses, and power economic growth around the world.
As a member of the BMO team you are valued, respected and heard, and you have more ways to grow and make an impact. We strive to help you make an impact from day one - for yourself and our customers. We'll support you with the tools and resources you need to reach new milestones, as you help our customers reach theirs. From in-depth training and coaching, to manager support and network-building opportunities, we'll help you gain valuable experience, and broaden your skillset.
To find out more visit us at
BMO is proud to be an equal employment opportunity employer. We evaluate applicants without regard to race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or any other legally protected characteristics. We also consider applicants with criminal histories, consistent with applicable federal, state and local law.
BMO is committed to working with and providing reasonable accommodations to individuals with disabilities. If you need a reasonable accommodation because of a disability for any part of the employment process, please send an e-mail to and let us know the nature of your request and your contact information.
Note to Recruiters: BMO does not accept unsolicited resumes from any source other than directly from a candidate. Any unsolicited resumes sent to BMO, directly or indirectly, will be considered BMO property. BMO will not pay a fee for any placement resulting from the receipt of an unsolicited resume. A recruiting agency must first have a valid, written and fully executed agency agreement contract for service to submit resumes.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.