← All posts
Why de-identify before, not after
Can AI models memorise names? What the research shows
In 2021, researchers pulled real names, phone numbers and addresses out of GPT-2, verbatim, just by asking it the right questions.
The evidence
- Carlini et al. (2021) extracted hundreds of memorised training sequences from GPT-2, including personal contact details.
- Later work showed extraction is possible from production chat models too.
- Memorisation increases with model size and with repeated data.
What we do about it
De-identification happens inside your environment, then the output is tested against the ICO's motivated intruder standard. See how we handle data.
Key takeaways
- Models can memorise rare personal details.
- You can't reliably remove data once trained.
- So de-identify and test before transfer.
Sources
- Carlini et al., Extracting Training Data from Large Language Models, USENIX Security 2021
What could your company's records earn?
A free, nine-question estimate. No files, no logins.
More in Privacy & law
Data brokers and national security: what the UK government is asking
DSIT's 2025 call for evidence asked how brokers keep data away from hostile actors. What it covered, and the checks every broker should run now.
Personal data that must never leave: a checklist
Special category data, criminal records, children's data and more: what to exclude before any data deal.
The motivated intruder test, step by step
The ICO's benchmark for whether de-identified data is safe: a determined, competent person with no special access.