How the Library of Congress is Trying to Archive Twitter

Main Reading Room of the Library of Congress, Library of Congress, Prints and Photographs Division, 2009, via Wikimedia.

Losing the remnants of the past is one of the nightmares of historians. There are so many fragments that simply don’t survive the march of time, and as they disintegrate so too does our ability to understand the past.

The modern era and its technologies raise new questions about how archivists, historians, and other interested parties can preserve historical objects that are created every day. How can archivists archive the internet?

Let me take a brief pause here to talk about historical objects. Keep in mind that this is a gross simplification of the term. In the past, historians focused on the writings of great men, military tactics, and in the case of ancient civilizations, material objects. Over the past 100 years or so, historians have expanded the types of questions they ask of the past and what objects they use to explore it. This means that letters from ordinary people, advertisements, music, film, and clothing have all gained a new importance in the field.

That also means that when people study the 2010’s, they will probably want to study what is on the internet including trends in memes, hashtags, blogs, and, yes, tweets. Finding those pieces of the digital past, however, could be difficult.

Traditional library archives are filled with books, papers, and sometimes a few physical objects. Staff often digitize these items, but if a digital file is corrupted, there is a tangible item for scholars to turn to. The internet lacks that physicality, and as a result, librarians are working to find a way to store the information that exists on it, especially in public forums like Twitter.

Back in April of 2010, the Library of Congress announced that it would work with Twitter to archive public tweets from 2006 onward. The Library declared this initial stage a success in 2013 when it had archived around 170 billion tweets. The Library then turned its attention to figuring out how they could make that archive accessible to the people who wanted it.

And then the project stalled.

The Library of Congress still collects tweets and stores them in a server, but those tweets aren’t accessible or searchable to anyone yet. As The Atlantic reports, no tech engineers are currently assigned to the project and developments are slow. Library employees are understaffed, and Twitter has no incentive to help streamline the data transfer. In fact, Twitter frequently sells public tweeting data to marketing and research firms. And the company is making a tidy sum off of those sales. Making that information freely available would cut into Twitter’s profit.

Still the historian in me holds out hope that this project moves forward. Having a public database of tweets will make the lives of future scholars so much easier. Having worked on digital archives myself, however, I know that creating a database like that isn’t so simple. Librarians and staff have to decide how to tag tweets, how to group them, how to store them, and they have to create a usable interface.

But hopefully they’ll manage to make the Twitter archive a reality someday. We would all benefit from it.