Does the Wayback Machine keep everything? No, and the gap is wider than most readers assume. Two pieces from 2026 have renewed the question: a DC Report article dated 2 September 2026 titled “Broken links are erasing the public record”, and an Internet Archive blog post from 23 April 2026, “Gone but Not Forgotten: Recovering the Dead Web”. Most of the quantified evidence predates both.
What is new, and what is older data
The 2026 pieces are mostly secondary reporting or commentary from parties with a stake. The Archive’s post is one of them: it estimates how much the Wayback Machine rescues, and the Archive has an obvious interest in that estimate. The DC Report article reports on other research rather than conducting its own. The quantified figures on the open web come from a Pew Research Center study published on 17 May 2024.
How much of the web is actually gone
Pew sampled pages from Common Crawl for each year from 2013 to 2023. It found that 38% of webpages from 2013 were no longer accessible. Across the decade, a quarter of pages that existed at some point from 2013 to 2023 were gone as of October 2023. Most of those losses were single pages removed from sites that otherwise still worked.
Why the figures differ
The Internet Archive’s post estimates that the Wayback Machine rescued roughly 15% of the otherwise dead pages in the Pew sample. The same post presents an Old Dominion University finding that about 65% of sampled URLs were dead when checked in 2023. These are not the same measurement. The samples differ, the checks differ, and the Archive’s own comparison table warns against lining the studies up directly. Any single “percentage of the web lost” should be treated as unsettled.
What the Wayback Machine recovers
The 15% figure is the Archive’s own estimate, so it is an interested-party number rather than an independent audit. For an individual page, the practical question is simpler than the statistics: is there a capture at all? A page that was never crawled, or that sat behind a login or form, may have no copy to recover.
Gone, moved, archived, or excluded
Coverage often lumps four different states together. A page can be gone, with no live copy at its address. It can have moved, with the material still online at a new address. A copy can exist in the Wayback Machine. Or a copy can exist but have been excluded from public view. Each calls for a different response, and a “not found” error does not tell you which one you are facing.
A First Monday case study, which is older and small and whose date we could not confirm, found that many apparent disappearances were moved content. If that pattern holds more widely, real losses are smaller than they look. That is one study’s finding, not a settled rule.
What archives deliberately discard
Yes, archives throw material away on purpose. The International Council on Archives defines appraisal as deciding which records will be kept and which can be disposed of, and its guidance describes archivists examining collections item by item to decide what warrants continuing retention. A Peel Archives blog post makes the related point that archival records are a small subset of the records that survive, and that most archivists think saving every record would be neither wise nor possible.
Appraisal and reappraisal
Removing material after it has been acquired, known as deaccessioning, is a sensitive subject in the profession, because archivists have been wary of accidentally discarding things of value. A Journal of Western Archives article argues for written policies and for transferring material to other repositories. Reappraisal can also remove large amounts. A Margot Note blog summary of a published article reports that the Wisconsin Historical Society cut its government-records collection by 40% through a four-year reappraisal project. We could not retrieve the original article, so treat that figure as secondary. We also found no published national rejection rates, so none appear here.
What archives never collected, and removal requests
Some pages never reach an archive. Crawlers may not have found them, the content may sit behind a password or form, robots.txt may have blocked the crawler, or the site owner may have asked for exclusion. The Wayback Machine’s official help documentation lists these reasons for absence.
Removal is where sources disagree. One commercial guide says the Archive no longer retroactively honours robots.txt and relies on explicit requests. Another says it honours the robots.txt route and removes existing snapshots. Forum reports describe both outcomes: pages excluded after a Wayback-specific robots.txt rule appeared, and archived pages reappearing once that rule was removed. We found no current official Internet Archive policy page that settles the question.
Not every archive honours exclusions. A Hacker News comment quotes the Danish national web archive’s FAQ as saying it does not respect robots.txt because it collects the national web under legal mandate. We have not verified that FAQ, so check it before relying on the claim.
Digital formats and organisational failure
Digital objects are harder to keep whole. Databases, GIS data and websites need all their parts intact, and migrating a file to a new format can lose content or functionality, according to an essay by a director of the US National Archives and Records Administration published in a SAGE journal. Separately, platforms shut down, funding runs out, contracts lapse and identifiers stop resolving. An arXiv preprint on the ephemeral web and a vendor article from ar.io list causes of this kind. The arXiv item is a preprint, and the ar.io piece is promotional.
What to do about a missing page or record
- Check for a moved URL. Search the publisher’s site for the title and look for redirects or a new address.
- Search the Wayback Machine for captures of the exact address, then for close variants.
- Check the publisher’s or agency’s own records office, which may hold the document under another reference.
- Contact the originating repository or library directly.
A missing copy does not prove the original was never published. Government and local records look most exposed. A DC Report article attributes to Pew a finding that at least one broken link appeared on 21% of sampled government webpages, with local pages especially vulnerable. We could not retrieve the Pew text containing that figure, so check it at Pew before citing it. If you cite a web page yourself, save your own copy and note the date you checked it.