Team Red Dog is hiring a
Content Automation Engineer for our client, a global productivity and collaboration leader and AI-driven technology organization. In this highly technical role, you'll help define how Microsoft 365 and Copilot evaluate AI-generated content by designing scalable automation frameworks, building intelligent testing systems, and developing human-in-the-loop quality strategies. You'll partner with engineering, content, and AI teams to create automated evaluation workflows that improve efficiency, strengthen quality standards, and directly influence experiences used by millions of customers worldwide. This is an outstanding opportunity for an automation-focused engineer who enjoys solving complex quality challenges at the intersection of AI, content, and software engineering.
Top Required Skills (Must Haves):- Python & Test Automation - 2-4 years of experience developing automation workflows, scripting with Python, and building scalable testing and validation frameworks.
- AI Tooling & Evaluation Systems - Experience using Copilot, VSCode, or similar AI development tools to support automated testing, AI evaluation, and quality engineering initiatives.
- Quality Engineering & Framework Design - Experience designing testing frameworks, quality systems, workflow automation, and scalable evaluation methodologies that improve operational efficiency.
- Data Analysis & Technical Recommendations - Demonstrated ability to analyze testing results, interpret operational data, identify trends, and translate findings into strategic recommendations and measurable improvements.
Opportunity Overview:Help shape the future of AI-powered productivity by designing the systems that evaluate and improve Microsoft 365 and Copilot experiences at scale. Rather than focusing on manual testing, this role centers on building automation, intelligent evaluation frameworks, and AI-assisted quality processes that make content validation faster, smarter, and more reliable. You'll collaborate with cross-functional teams to influence long-term quality strategy while creating scalable solutions that improve both customer experience and operational efficiency.
How you will make an impact:- Design scalable testing and evaluation frameworks for templates, prompt templates, AI-generated content, and Microsoft 365 experiences.
- Define quality measurement approaches and establish testing standards across content ecosystems.
- Develop automated testing workflows, validation systems, and reusable quality frameworks.
- Build tools and processes that reduce manual effort while maintaining high-quality content standards.
- Identify opportunities to leverage AI for content validation, defect detection, and quality assurance.
- Design human-in-the-loop evaluation frameworks for assessing AI-generated outputs.
- Develop quality rubrics, benchmarking criteria, and repeatable testing methodologies.
- Analyze testing metrics and operational data to identify opportunities for continuous improvement.
- Measure automation effectiveness and optimize testing workflows over time.
- Partner with cross-functional teams to recommend automation strategies, evaluate quality risks, and drive adoption of scalable testing standards.
The expertise you bring:- Bachelor's degree in Computer Science, Engineering, Data Science, Information Systems, AI, or a related field.
- 2-4 years of experience in test automation, quality engineering, AI evaluation, or systems development.
- Strong Python programming skills and experience with automation tooling.
- Experience using Copilot, VSCode, or comparable AI development tools and infrastructure.
- Experience building testing frameworks, workflow automation solutions, or quality systems.
- Strong analytical skills with the ability to interpret data and translate findings into strategic recommendations.
- Excellent technical problem-solving and systems-thinking abilities.
- Experience evaluating generative AI systems or LLM-powered products is preferred.
- Experience designing human evaluation programs or human-in-the-loop workflows is preferred.
- Familiarity with content quality measurement, content operations, or AI quality frameworks is a plus.
- Experience building internal tools, dashboards, or quality monitoring systems is preferred.
- Knowledge of Microsoft 365, Copilot, or productivity software ecosystems is desirable.
What makes a candidate highly successful in this role:Successful candidates combine strong automation engineering skills with strategic systems thinking. They understand how to design scalable testing frameworks, identify opportunities to automate manual processes, and balance AI-driven evaluation with human judgment where it delivers the greatest value. Candidates who have hands-on experience with Python, AI tooling, quality engineering, and translating technical findings into actionable recommendations will stand out, along with those who demonstrate stable work history and ownership of long-term automation initiatives.
Why Work with Team Red Dog?At Team Red Dog, people are at the heart of everything we do. Our commitment to personalized service and our deep experience in matching talented professionals with meaningful roles at some of the world's most inspiring companies is what sets us apart. We take the time to understand your unique skills, strengths, and passions-because we believe your career should reflect who you are.
Whether you're looking to grow, pivot, or simply find a place where your work truly matters, we offer opportunities that empower you to make a positive impact. With excellent benefits, a supportive team, and a role where you can thrive while doing what you love, we're here to help you take the next step with confidence. Join us-and discover what it means to be genuinely valued in your career.
Generous benefits package for qualified employees includes:- Health insurance (medical, dental, vision, and life)
- Employer-matched 401K plan
- Paid time off
- Paid holidays
Estimated Start Date: Immediately
Location: Remote (U.S.) - Candidates must be able to work Pacific Time business hours.
Job #: 2564Job Type and Estimated Duration: W2 contract opportunity through December 31, 2026, with the possibility of extension up to 18 months' total time based on performance, budget, and client discretion.
Rate: $36-$38/hour
Team Red Dog is committed to providing equal opportunities to everyone, regardless of race, ethnicity, gender, age, religion, sexual orientation, disability, or any other characteristic. If you need accommodation during the recruitment process, reach out to , and we will work to ensure an accessible experience. We strictly adhere to federal, state, and local laws to maintain a workplace free from discrimination and harassment.
We offer competitive compensation aligned with U.S. industry standards, and our final offer will reflect the candidate's location, job-specific skills, experience, and knowledge.
- All applicants must be authorized to work in the U.S. without the need for sponsorship.
- Team Red Dog is an E-Verify employer.
- Employment is contingent upon the successful completion of a reference and background check.
- Please no solicitations from C2C or recruiting firms.
Never miss an opportunity! Create an alert based on the job you applied for.