OpenAI
Mid
Turn ambiguous agentic-model failures in API/power-user workflows into training data, evals, and shipped model behavior
This role sits on OpenAI's Agent Post-Training team, focused specifically on how agentic models behave for API developers and power users — tool use, function calling, long-horizon execution, and error recovery. The work spans the full loop from qualitative failure analysis to designing evals/graders/training environments to integrating fixes into major model training runs.
Step into this interview
Free · a live voice mock calibrated to this exact role
What this interview tests
- Post-training / RLHF-RLAIF for agentic models
- Eval and grader design from real developer workflows
- Tool use, function calling, and multi-step task coherence
- Qualitative failure analysis into training interventions
- Cross-functional ownership across research/product/infra
Common question themes
Describe a time you diagnosed a model failure from a transcript and turned it into training data or an eval
How do you design an eval or grader that reflects real API/power-user workflows rather than synthetic tests
Tell me about an agent tool-use or error-recovery failure mode you've addressed
How do you decide when a behavioral fix is ready to integrate into a major model training run
Describe a project you owned end to end across research, data, and product boundaries
Related interviews
OpenAI
Senior
Forward Deployed Engineer
OpenAI
Senior
Data Scientist, Identity
OpenAI
Senior
AI Deployment Engineer
Notion
New grad
Software Engineer, New Grad (AI)
Cohere
Mid
Forward Deployed Engineer, Agentic Platform
Airbnb
Senior