Project Overview:
Join a growing community of professionals advancing the next wave of AI. As an AI Trainer, you'll play a hands-on role by analyzing and providing feedback on data to improve LLM performance, helping ensure that the next generation of AI technology is accurate and trustworthy.
We are seeking a skilled Senior Software Engineer to work as a project consultant in our AI Labor Marketplace. This is not a full-time employment position - you will be engaged as an expert project consultant on a contract basis.
Location: U.S.-based experts only
Engagement: Part-time, project-based expert evaluation work
Work Type: Remote
Project Summary:
Consultants will design realistic, senior-level software engineering tasks, evaluate GPT-5.2 model reasoning using structured rubrics, and produce gold-standard responses that reflect real-world engineering best practices.
Consultant Engagement Terms:
This is a project-based consultant role. Consultants will be paid on a per-project basis; hourly rates are estimates based on anticipated completion time. Consultants control their own schedule, provide their own tools, and may simultaneously provide services to other vendors/employers (subject to those vendors' allowances).
Responsibilities:
Contributors will:
• Create realistic, complex software engineering prompts based on professional experience
• Design clear, criteria-based evaluation rubrics
• Evaluate GPT-5.2 model outputs against defined success criteria
• Author high-quality gold responses demonstrating senior-level reasoning
• Ensure all work is synthetic, non-proprietary, and objectively evaluable
Expected Outcomes:
• High-quality prompts that reflect real engineering challenges
• Clear, defensible evaluation rubrics
• Consistent, well-reasoned model evaluations
• Gold-standard responses suitable for model training and benchmarking
Qualifications:
• 10+ years of professional software engineering experience
• Senior, Staff, Principal, Tech Lead, or Architect-level background
• Strong CS fundamentals and production system experience
• Experience at top SaaS companies or scaled startups preferred
• Prior data annotation or model evaluation experience is a strong plus