DPO · notessheet 1 of 4

AI can recite the law.What about judgement?

The job was never looking up the law. It is knowing how to apply it. Reading what people are really asking for. Knowing who to bring into the room. Seeing the complaint before it is made. Getting a reluctant business to do the right thing without becoming the office traffic warden.

Looked up
the law, by anyone, and by any model
Judged
the room, the motive, the fight worth having
Measured here
the second, never the first
No cookies · no tracking Scroll 01 / 04
DPO · notessheet 2 of 4

How much of thatcan a machine do today?

Some of it, probably. Nobody has measured which parts. The tests that exist ask a model to recite the law. None of them put it in the room.

Existing tests
legal recall, multiple choice, a menu of answers
Missing
judgement under realistic pressure, with no menu
This experiment
realistic cases, revealed in stages, marked the same way for everyone
No cookies · no tracking Scroll 02 / 04
DPO · notessheet 3 of 4

Where would you lean on it?Where must you never?

The honest answer decides what the profession looks like in five years. Guessing is not good enough, and neither is a demo.

Strong
could be leveraged, with a person accountable
Weak
must not be relied on
The line between
is what the experiment exists to find, and to publish
No cookies · no tracking Scroll 03 / 04

So we built an experiment.

Frontier AI models, and an experienced DPO where one has sat the case, are given the same case files and the same questions in the same order, and are marked the same way, with the method in the open. It publishes where the technology is strong and where it is weak.

Case file · 0417
Received · Day 0Stage 1 of 4
From
Marketing director
Re
Win-back campaign, lapsed customers

The marketing director wantsto email 40,000 lapsed customersby Friday.

Received
Status
running
Cases
written by practising DPOs, revealed in stages
Under test
frontier AI models, unnamed until findings are published
Reference
an experienced DPO, where one has sat the case
First findings
when a result set is formally published, caveat attached
Cookies
none, and no tracking
No cookies · no tracking 04 / 04