Note: The job is a remote job and is open to candidates in USA. Productive Playhouse is a global data company specializing in language and data services. They are seeking a Korean AI Model Rater to evaluate AI-generated responses and their integration with digital services, focusing on the accuracy and quality of these interactions.
Responsibilities
- Analyze and rate AI-generated responses and their associated tool outputs against a detailed set of quality guidelines
- Verify that the AI successfully integrates information from different digital services (APIs) as required by the prompt
- Evaluate the accuracy, completeness, and overall quality of the AI's response and tool execution
- Follow provided instructions to achieve task goals, with all evaluations recorded in English
Skills
- Advanced/Expert fluency in Korean language, as spoken in the South Korea region (ISO: [ko_kr])
- Strong comprehension of English (read and write)
- Strong conceptual knowledge of APIs and digital services (ability to understand how AI tools interact with external apps like search, maps, or calendar, without needing to write code)
- Ability to follow complex technical guidelines precisely
- All participants must have, or be willing to create, an Upwork account
- While participating in this project, adherence to the confidentiality terms outlined in the Upwork User Agreement is required
- Applicants must pass a quick skill verification assessment (MC, duration ~10 minutes)
- Work must be performed on participant's own devices
- You must use your own laptop, and smartphone (if required)
- A Gmail account may be required to access certain tools. If you do not have a Gmail account, you must be willing to create one
- Using AI during the assessment and work is STRICTLY forbidden, and results/inputs will be monitored to detect AI usage
Benefits
Company Overview
Productive Playhouse provides international language services including transcription, linguistics, translation, and field data gathering. It was founded in 2009, and is headquartered in Los Angeles, California, USA, with a workforce of 201-500 employees. Its website is https://productiveplayhouse.com/.
Apply Now