Cohere
Mid
Evaluate and debug AI coding-agent outputs as a contract data science annotator for Cohere
Cohere is hiring a part-time independent contractor (16 hrs/week minimum, 12-month contract) to evaluate data science and coding tasks — reviewing/debugging code, assessing agent trajectories, and labeling model outputs for accuracy. Expect questions on hands-on Python/SQL data science work, code review rigor, and judgment calls when grading ambiguous or imperfect AI-generated outputs.
Step into this interview
Free · a live voice mock calibrated to this exact role
Likely format
Initial resume/writing-sample screening, then a virtual annotation test (data science take-home + writing sample), then a video screen with the Operations team.
What this interview tests
- Hands-on Python data science (numpy, pandas, matplotlib, sqlite)
- SQL syntax and query correctness
- Code review and debugging judgment
- Evaluating AI/agent-generated outputs for correctness
- Clear, structured technical feedback and proofreading
- Familiarity with coding agents and agent trajectories
Common question themes
Walk me through a data science task you solved end-to-end in Python
Describe a time you found and fixed a subtle bug in someone else's code
How do you verify the correctness of a SQL query or generated script?
How would you evaluate whether an AI coding agent's output actually solved the task?
Tell me about giving structured, technical feedback on someone else's work
What have you noticed about where coding agents like Claude Code or Codex tend to go wrong?
Related interviews
Cohere
Mid
Forward Deployed Engineer, Agentic Platform
Cohere
Mid
Member of Technical Staff, Applied ML
Cohere
Senior
Data Engineer, Data Foundations
Replit
Senior
Data Scientist, Product
Stripe
Mid
Data Analyst - Economic Insights & Communications
Affirm
Mid