Overview
Remote
On Site
$$45 - $75
Part Time
Contract - Independent
Skills
Software
Job Details
Job description
Compnay is partnering with an AI innovation initiative seeking Professional Software Engineers to support the evaluation and improvement of software engineering-focused AI tasks.
In this role, you'll use your engineering and systems-level reasoning to assess model competency in real-world technical domains, debugging, architecture decisions, and system behavior analysis.
What You'll Do
- Create challenging SWE-related prompts based on visuals, datasets, or model outputs.
- Evaluate AI-generated responses using provided rubrics.
- Develop or improve rubrics to ensure they reflect accurate reasoning for complex tasks.
- Review and stress-test model outputs for accuracy and logical consistency within your domain.
- Provide clear, structured feedback based on your subject-matter knowledge.
Ideal candidate profile
Experience using an LLM is ideal. They want someone who has been working in their actual field most recently, rather than AI training, but experience working with LLMs is a plus.
Daily tasks
Design problems and evaluate AI prompts based on the domain expertise. Provide thorough feedback.
Required skills
Need a bachelors degree or higher, experience in related field.
Full-time/part-time
variable
Additional context on remote hires
US Based
Timezone preferences
Any US timezone
Work schedule
5-7 hours per day worked expected. Must commit to at least 10+ hours per week. 2 days per week
Interview process
Two assessments then an offer
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.