Note: The job is a remote job and is open to candidates in USA. Uber AI Solutions is a marketplace connecting freelancers with generative AI researchers, seeking analytical thinkers and Junior Linguists for high-level evaluation of GenAI models. The role involves linguistic analysis, safety auditing, and model stress-testing to ensure AI systems are safe, accurate, and natural.
Responsibilities
- Linguistic Analysis: Reviewing prompt context and model responses to analyze the relationship between user intent and AI output
- Safety & Severity Auditing: Evaluating outputs for potential risks (toxicity, bias, misinformation) and categorizing the severity of violations according to strict safety rubrics
- Side-by-Side (SxS) Evaluation: Comparing multiple outputs and using editorial judgment to choose the highest quality response
- Written Justification: Writing short, structured, and technically sound explanations for your ratings based on linguistic and safety principles
- Model Stress-Testing: Identifying model gaps and performing "Red Teaming" to ensure logic and factual accuracy
- Multimodal Evaluation: Assessing AI responses across various formats, including text, audio, and images, to ensure cross-functional model accuracy
Skills
- Bachelor's degree (or currently pursuing one) in Linguistics, English, Journalism, Communications, or a related research-heavy field
- Exceptional writing skills with the ability to explain complex and nuanced reasoning
- A strong pulse on content safety and the ability to identify subtle biases or harmful model behaviors
- Native-level English proficiency is a must
Company Overview
- Uber AI Solutions is the one-stop partner for enterprises aiming to innovate and scale. It was founded in undefined, and is headquartered in San Francisco, US, with a workforce of 5001-10000 employees. Its website is https://www.uber.com/ai-solutions.
Company H1B Sponsorship
- Uber AI Solutions has a track record of offering H1B sponsorships, with 4 in 2023, 2 in 2022, 8 in 2021, 85 in 2020. Please note that this does not guarantee sponsorship for this specific role.