Vint Cerf, Google’s vice-president and one of the fathers of the internet, has warned that “bit rot” could lead to a “forgotten century” as our masses of digital files are lost to progress and become unreadable as technology evolves.
Cerf said that the applications to read files are being lost because they will no longer run on newer computers, rendering some files unintelligible and the data, memories and important happenings they contain lost to future generations.
But is this really a problem, and if so, what is the solution?
An age-old problem
Loss of important memories and content is not a new problem. Since the inception of video, recorded memories have been lost because the equipment to play them has either broken or been lost.
The VHS tape is a good example. Many people may still have family movies recorded on tape. While they are still likely to works if kept away from magnets, when was the last time you saw a VHS tape player, or even a TV with a scart socket to take the video input?
This loss of machines capable of playing back our recorded memories is essentially what Cerf is talking about in a digital, file-driven age. But what his comments fail to take into account is that once safely archived, files are a lot easier to play back decades later than something physical that needs a specific machine.
What is ‘bit rot’?
What Cerf coined as “bit rot” is a process by which the mechanisms for accessing a digital file are lost, rending that file useless junk. A big part of the problem is the use of closed file formats that require specific software to read those files.
If, for instance, you have memoirs written over the last decade stored in a Microsoft Office .doc file from Word or a similar program, that file is easily readable today with a multitude of programs - not all of them made by Microsoft. But the .doc file is a proprietary file format made and licensed by Microsoft.
Should Microsoft choose to stop supporting it and prevented other software from using the format, all those documents would be unreadable once the last version of the old software that could read them no longer runs on newer computers.
That is the danger of closed, proprietary formats and something consumers should be aware of. However, it is much less of an issue for most people because the majority of the content they collect as they move through life will be documented in widely supported, more open formats.
Photos, videos, emails and text documents
Photos, for instance, are invariably stored as jpeg files. Any image application worth using today supports jpeg files and that is highly unlikely to change for the foreseeable future.
Even if new, improved image formats are adopted, hundreds of applications will still be able to access jpegs.
A similar situation exists for video files. The current video standards, although relying on patented technology, are open and available to use in any program should developers see fit. Licensing fees may apply for some programs, but because the video format is standardised and not tied to one company, applications will be able to play those video files for decades.
Email, text documents and other simpler file formats consumers use beyond the Microsoft office files are also less likely to suffer from bit rot. One issue they may face is that the majority of documents and media stored online are locked on company servers. Some like Facebook and Google allow users to export those files, but there may come a time when those companies fold and with them go a user’s digital memories.
The bigger issues with digital files stored locally are faced by institutions rather than individuals.
Many organisations, from government and council services to small business and multinational conglomerates, all use proprietary software to create and their documents. It is these files, when archived for posterity, that could become an issue as the software that created them is no longer supported and the computer systems that ran them need to be replaced.
What can be done?
The solution to most of these issues is to adopt open standards for files. Almost all the important ways of storing information have open formats that anyone can use and build software to use freely.
The difficulty is that commercial software developers, such as Microsoft and its Office suite, have no incentive to use open formats because they allow companies, users and its customers to switch systems wholesale and still be able to operate.
The same solution that will allow generations to open files for posterity is the same solution that is most difficult to convince software vendors to adopt.
Beyond the file formats, ageing software and bit rot, there is a bigger problem that historians of the future will face: what is actually important?
Apart from the issues of physical preservation, translation and interpretation, looking back at letters and manuscripts from the time of Archimedes is made simpler by the fact that the volume of those letters is several orders of magnitude smaller than the electronic conveyances we fire off without thought today.
Sifting through the millions of emails, photos and videos that are easy and essentially free to create today will be the biggest challenge, and not one with an obvious solution. Our lives have benefitted greatly from the freedom of communication, but historians are going to have a hard job making sense of it all.
Comments (…)
Sign in or create your Guardian account to join the discussion