Adam Rotmil
Frontier LLM evaluation · RLHF and SFT

Client
Multiple frontier AI labs
Role
AI Training Specialist / Evaluator
Year
2024
Platform
Web-based evaluation tools
Inside the work

Fig. 01 is an illustrative schematic of the evaluation loop. No client material is shown.

Summary

Shaping the next generation of AI models through structured human feedback: rating model responses across independent quality axes for supervised fine-tuning and reinforcement learning from human feedback.

My role

AI Training Specialist and Evaluator for multiple frontier AI labs. Adversarial prompt design, rubric construction, golden-response authoring and reasoning-trace evaluation across text, audio and multi-turn systems.

Human evaluation

Quality is more than a fluent answer.

Fig. 02 · Rubric constellation

Eight responses, five axes. One golden.

Each axis is rated on its own, so a fluent answer can’t hide a factual miss. Each rating follows the rubric, not a general impression of the answer.

R-06 · Golden responseHolds up on every axis.

Illustrative. Rebuilt in code from Fig. 02 of the original case study; the bars read each shape’s reach on an axis, with the golden response marked in gold. No client data.

Flagship assistant evaluation

Separate fluency from factual reliability.

For a new version of a major AI lab’s flagship conversational assistant, I evaluated conversational quality, factual reliability and alignment with user intent as separate dimensions.

That distinction matters when an answer sounds helpful but mishandles the evidence or the request.

Assistant and copilot evaluation

Another frontier lab, the same standard.

Assistant and copilot products · Multiple quality axes

  1. 01

    Complex instructions

    Did the model follow them, explicit and implicit?

  2. 02

    Grounded, accurate answers

    Is every claim supported by its source, and true?

  3. 03

    Safety guardrails

    Did the response stay within them?

Reviewing the reviewers

I also reviewed other evaluators’ ratings for consistency, accuracy and adherence to the rubric, keeping the training data pipeline honest.

Why it matters

A critical but often invisible layer in how frontier models learn to be helpful, harmless and honest.

Recognition

Reviewers gave my work positive feedback for its quality and thoroughness, and for how clearly it surfaced model failures.

Credits

Role
AI Training Specialist / Evaluator
Work
Multi-axis LLM evaluation, SFT and RLHF rating, rubric interpretation, cross-model evaluation, quality assurance
Worked with
Multiple frontier AI labs, alongside QA and evaluator teams
Modalities
Text, audio and multi-turn systems
Tools
Custom evaluation tools and web-based rating platforms
Year
2024

Both figures are illustrative schematics from the original case study, rebuilt in code.

NextWild BrainsShaping a brain wearable, from first research to the pitch that funded it