What the independent licence audits establish
Independent evidence on AI training data provenance is concentrated in the Data Provenance Initiative (DPI) audit of dataset licensing and attribution. It appears as an arXiv preprint (arXiv:2310.16787) and as a peer-reviewed paper in Nature Machine Intelligence. The audit covers text fine-tuning and alignment datasets. Its headline finding is licence omission of roughly 70%. The arXiv versions disagree on the exact figure: one gives “70%+” and another “72%+”. Quote the number with its version, or as “roughly 70%”.
The authors also re-annotated Hugging Face licences and report that 66% of the analysed licences fell into a different use category, often one more permissive than the licence the dataset author intended. This is the authors’ own result. No independent replication appears in the sources reviewed, so treat it as one team’s finding rather than an established rate. The Nature version also attributes part of the problem to readers mistaking a licence on a GitHub repository’s code for a licence on its data.
What the audits do not cover
The DPI audit centres on text fine-tuning and alignment data. A secondary summary by Lacuna says it excludes large pre-training crawls such as Common Crawl. That scope statement comes from the summary, not the paper, and should be checked against the paper before you rely on it. Nothing in the sources reviewed shows how closed, proprietary training pipelines record provenance. Do not generalise from open dataset repositories to frontier model developers. Whether evaluation sets overlap training data is a separate verification problem; see benchmark contamination detection.
Documentation completeness: what the measurements show
The DPI multimodal paper, “Bridging the Data Provenance Gap Across Text, Speech, and Video”, reports that 36% of text, speech and video datasets lack licence or intended-use documentation. It also reports that most collections do not aggregate or expose that documentation. The DPI publications page describes the multimodal audit as covering about 4,000 datasets. The full methodology was not available in the sources reviewed, so the 36% figure carries that limit.
On Hugging Face, one study (arXiv:2312.15058) found documentation for 21,902 models (39.62%) and 1,925 datasets (28.48%). A later arXiv preprint (arXiv:2607.17242) reports dataset information in 38,542 artifacts (39.35%) and notes that a field being present can overstate documentation quality. That preprint is not peer-reviewed as far as the sources show, and its venue status could not be verified.
A 2024 study of Hugging Face model cards, reached only through a secondary aggregator citing Liang et al. (2024), reports training sections as most complete at 74.3%. Environmental impact (2.0%), evaluation (15.4%) and limitation sections (17.4%) are described as consistently underreported. Treat these figures as low confidence until the original is checked. Representational gaps in the same data are a different question; see hidden biases in AI training data.
Audit tools and what they can flag
The DPI publishes the Data Provenance Explorer and a GitHub repository, Data-Provenance-Collection. The repository describes 44 data collections spanning more than 1,800 fine-tuning text-to-text datasets. These tools cover the DPI’s collections and so inherit its text fine-tuning focus. An audit of the OpenML repository, a different system, found that 11% of datasets had one or more compliance problems. Its authors concluded that the system should flag possible discrepancies for human review rather than deliver a legal verdict. That is a sound statement of what automated checking can establish. It is not evidence about any AI developer.
Can provenance be reconstructed after training?
The sources do not answer this directly, so what follows is an inference. The DPI multimodal work finds that re-packaging datasets drops their documentation when it is not carried along. Provenance lost at one hand-off is therefore likely to be hard to recover later. The DPI paper also uses the term “documentation debt” and points to Datasheets for Datasets and Data Statements as theoretical guidance. I read that term on a mirror page, not the original. Vendor guidance describes reconstructing chain of custody from collection to training snapshot as achievable. That is commercial opinion, not an established standard.
What the EU AI Act requires, and what remains unverified
Article 53(1)(d) of Regulation (EU) 2024/1689 requires providers of general-purpose AI models to publish a “sufficiently detailed summary” of the content used for training, following an AI Office template. WilmerHale reports that the template was released on 24 July 2025. The Commission’s template text and explanatory notice were not retrieved, so the account below rests on secondary descriptions.
According to the regulations.ai description, the template has three parts: general model and provider metadata, an itemised listing of data sources, and processing and governance aspects. Secondary commentary adds a narrative description of dataset categories and, where applicable, the top domain names of online sources crawled or scraped. Check each element against the Commission text before relying on it.
One commentary states that the template does not require naming the specific works used, and that licensed data receives lighter disclosure because rights holders are already party to those deals. Neither point has been checked against the Commission text, so whether named works must be disclosed remains unresolved here. Sources also conflict on enforcement timing. One gives a 2025 applicability date with a later deadline for models already on the market, and another gives a 2026 enforcement date. This article gives no enforcement date. Check the regulation itself before citing one.
Open questions and what would settle them
An independent replication of the DPI licence error and re-annotation results would show whether they hold beyond the original samples. Pre-training web corpora are unmeasured in the sources reviewed; settling that needs audits of those corpora or disclosures from their builders. Whether providers have published summaries under Article 53(1)(d), and how those summaries compare, is unknown from this review. The Commission’s template text would settle most of the template questions. Until then, if you are assessing a specific model, check its dataset licence against the original dataset’s licence page, not only the host’s metadata.