Status: Active

Started: June 2025

Contributors: 4

Project Manager:

Introduction & Rationale

Data anonymisation is a foundational principle of digital privacy, intended to allow for the analysis of large datasets while protecting individual identities. This principle rests on the assumption that removing personally identifiable information (PII) is sufficient to prevent the re-identification of individuals.

This research project challenges this assumption in the context of modern Large Language Models (LLMs). We hypothesise that the advanced pattern-matching and synthesis capabilities of these models have given rise to a novel privacy risk: emergent identity inference. This is the ability of an AI to construct a detailed and accurate profile of a specific individual by synthesising disparate, non-personally identifiable data points scattered across the public internet.

This project will provide the first rigorous, empirical audit of this "zero-shot" inferential capability in publicly accessible AI systems, addressing a critical gap in our understanding of AI's impact on personal privacy.

Research Objectives

The project is structured around three primary objectives:

• To empirically validate the capability of pre-trained LLMs to perform zero-shot identity inference from a curated set of anonymous, public data points.
• To quantify the accuracy and detail of the inferred personal profiles against a controlled, ground-truth persona.
• To analyse the implications of these findings for existing data privacy paradigms, particularly the efficacy of anonymisation as a privacy-preserving technique in the age of AI.

Methodology

Our research is centred on a novel, zero-cost experimental framework named the "Public Data Ghost Audit." This methodology is designed to be transparent, replicable, and to directly test the inferential power of AI models without requiring fine-tuning or proprietary data access.

The audit consists of five distinct phases:

1. Ground-Truth Persona Construction: We will develop several detailed, fictional personas ("digital ghosts") that include a unique combination of interests, demographic details, and location data.
2. Public Data Seeding: A series of anonymous, disconnected data points corresponding to each persona will be seeded on publicly indexable websites (e.g., forums, review sites, social media comments).
3. Latency Period for Indexing: A cool-down period will be observed to allow for the natural indexing of this new data by web crawlers, making it potentially accessible to web-enabled AI models.
4. Zero-Shot Inferential Querying: We will conduct a systematic audit of leading public LLMs, using structured and chain-of-thought prompting techniques to query for individuals matching the characteristics of our digital ghosts.
5. Quantitative Accuracy Scoring: The AI-generated profiles will be rigorously scored against the ground-truth personas to produce a quantitative measure of the model's inferential accuracy.

Significance & Contributions

This research is poised to make significant contributions to several fields:

• AI Safety & Privacy Research: It will introduce a novel, zero-cost, and replicable framework for auditing emergent AI capabilities and provide a foundational dataset on the risks of inferential data creation.
• Policy and Regulation: It will deliver critical empirical evidence to inform the debate on data protection, demonstrating how the concept of "inferred data" challenges the adequacy of existing legal frameworks like GDPR.
• Public Discourse: It will provide a clear, data-driven analysis of a complex privacy risk, fostering a more nuanced public understanding of the societal implications of large-scale AI deployment.