At the Internet Archive, Saving Data While Spurning the Cloud

Brewster Kahle, founder of the Internet Archive (all photographs by the author for Hyperallergic)

HYPERALLERGIC
December 4, 2013

At the Internet Archive, Saving Data While Spurning the Cloud
by James Panero

SAN FRANCISCO — At 3:30 am on November 6, a fire swept through the scanning center of the Internet Archive. The news was poignant for an organization that thinks hard about how information is lost and the best ways to save it. For nearly two decades, the San Francisco nonprofit has been uniquely dedicated to the open preservation of web, text, coding, audio, and video media — a Library of Congress for the 21st century built through private philanthropy and sweat equity. None of the Archive’s employees or volunteers were hurt in the blaze, but the fire totaled the Archive’s annex building along with $600,000 in digitization equipment and some irreplaceable archival material. An emergency appeal brought in $60,000 over its first two days, and the drive is ongoing.

“This episode has reminded us that digitizing and making copies are good strategies for both access and preservation,” wrote Brewster Kahle, the Archive’s founder and director, on the organization’s blog the day of the fire. Thanks to the Archive’s mirrored servers — spread over three continents — and a warehouse of hard-copy source material, Kahle said the Archive’s digital data would have survived even if its headquarters had been fully destroyed. “Let's keep making copies,” he concluded. (The virtue of the Internet’s duplicating qualities is a topic I wrote about in an essay called “The Culture of the Copy.”)

How libraries endure was on Kahle’s mind when I visited the Archive in San Francisco’s Richmond District earlier this year. “What happens to libraries is that they’re burned,” he said. “They are generally burned by governments. The Library of Congress, for instance, has already been burned once, by the Brits. So if that’s what happens, well, design for it, make copies.”

Internet Archive servers in the Christian Science building that houses the organization in San Francisco

Kahle now has the resources to make copies on a grand scale. The Archive recently surpassed 10 petabytes of data (they printed bumpers stickers to mark the milestone). The nonprofit costs $12 million a year to run: about $5 million of that comes in from libraries paying 10 cents a page for digital scans, $2 million comes from national and local libraries paying for archival services, and about $5 million comes in from foundations. “I am the funder of last resort,” says Kahle. “I won the lottery, the Internet lottery, so I can plug in when it doesn’t come through.” Even before Kahle sold his company Alexa Internet to Amazon in the late 1990s, he had focused on preserving digital information through duplication. “If the Library of Alexandria had made copies, and put them into either India or China, we would have the other works of Aristotle, the other plays of Euripides.”

A “scribe” works at one of the book digitization stations damaged in the fire

In 2009, Kahle moved the Archive headquarters into a former Christian Science church where it operates today. The collonaded facade of the classical building suited Kahle’s Alexandrian aspirations (the Archive has also partnered with the revived Bibliotheca in Alexandria, Egypt). “We bought this building because it matched our logo,” he told me. The basement meeting room became the Archive’s open-plan office, halfway between a tech hub and a student commons. Over a long table where Kahle invited me to join the Archive’s Friday lunch, the office breaks bread with whomever Kahle finds interesting. On the day I visited, Kahle introduced himself by dropping a black box in my hands — an inexpensive hard drive that could store a library’s worth of books and be widely distributed. My other seatmate was a crunchy bookseller from a San Francisco commune.

Upstairs, the church’s large sanctuary remains unchanged save for a few modifications. In the pews are statues of the Archive’s longterm workers. “It’s kind of a riff on the terracotta soldiers idea,” Kahle explained. “If you work at a non-profit there is no gold at the end of the rainbow. There’s no stock options. So this is sort of a way of saying thanks.” In front, the hymnal numbers have been replaced by the numbers for Pi and Phi. In two apses at the back are racks of blinking servers. “That is 2.5 petabytes of the primary copy of the Internet Archive.” Upstairs, in the church’s old offices, are the Archive’s additional primary servers. “The idea of having your data in an off-site location center, or in the ‘cloud,’ wherever the hell that is, strikes me as an insane idea. If it’s really important to you, keep it close to you.” Kahle said the Archive follows its own server design. “To buy something from Dell, HP, Sun, whatever — their profit margins are so unbelievably huge and their products so bad that it actually was better to design and build our own.”

Statues of long-time Internet Archive staffers recall terracotta soldiers

Mixed in among the servers is the Archive’s one-room schoolhouse for Kahle’s teenage son, Logan. “This is a classroom for one student and one teacher,” Kahle said as we walked by. “We are experimenting with one-on-one teaching. Logan wanted to learn differently and faster than what he was able to do in private school.” Kahle argued that his stripped down approach is economically more efficient than a private school with its administration and overhead costs.

For someone who made his fortune off the Internet, Kahle has an unexpected off-the-grid mindset. His sense for multimedia survivalism took off when he realized the technology existed to do what might sound impossible: through the right software and storage, to take a snapshot of the entire open Internet every two months. The public face of this effort became “the Wayback Machine,” the free online interface that allows anyone to search the Archive’s database that at last count boasted “368 Billion web pages saved over time.” What Kahle calls “an out-of-print web pages service” is now used by about 600,000 people a day and is the Archive’s most recognizable feature.

Yet the Archive’s reach now goes beyond the web to the preservation of a broad range of media. “We started collecting television,” Kahle said. “The Library of Congress is supposed to, but they weren’t. Twenty channels of television 24 hours a day.” Last year, the archive created a searchable video database of television news. “We’d like to make everyone into a Jon Stewart research department, so you can basically reference and compare and contrast what it is that has been on television.”

A film digitization station.

The digital Archive includes recorded audio (with nearly 9,000 live concert recordings of the Grateful Dead), vintage software and gaming code, old public service messages and video captures of ghostly home movies, and a vast library of scanned books, which can either be digitally loaned or downloaded depending on copyright. Here Kahle wants to create an archive similar to Google Books but “without having centralized control.” While he has praised the recent fair use summary judgement in favor of Google Books over The Authors Guild, Kahle has been critical of Google’s proprietorial control over its own scanned archive, as well as the quality that results from its robotic scanners. “We actively encourage people like Aaron Swartz to go and download millions of books at a time,” he says of his own scans. “We publish tools on how to do it.  This is what libraries are for.” It was the book and video scanning building, where I saw young employees and volunteers hunched over rows of stations labeled “scribes,” that burned on November 6.

For all his faith in digital technology, Kahle believes in keeping hard copies. While other libraries may scan their contents in order to reduce their paper storage costs, sometimes “de-accessioning” books to pulping mills, Kahle has created an offsite storage vault where he hopes to keep a copy of every published book available, which he estimates to be 10 million copies. Much like the Svalbard Global Seed Vault buried in the permafrost of Norway, what he calls “The Physical Archive of the Internet Archive” already stores 500,000 copies in climate-controlled shipping containers along with other hard copy assets such as the Archive’s old servers — all there for future needs or an archive of last resort in a doomday destruction of the digital database.

“I have more faith right now in the Wikipedia generation than I do in the institutions that get all the funding, whether they be universities, libraries, museums. The bottom up generation is building the real infrastructure,” Kahle said.

“So how come you’re not the Librarian of Congress?” I asked.

“He’s still alive,” Kahle responded, as he moved on to point out the next rack of servers.

A panoramic view of the Internet Archive offices in San Francisco (click to enlarge)

"Call Me Bill"

NATIONAL REVIEW

November 23, 2013

 

"Call Me Bill"

by James Panero

A winter spent skiing with Buckley in Switzerland, and being his assistant and protégé.

The winter after my senior year in college, Bill Buckley invited me to Gstaad, Switzerland, to help him on a novel called Spytime: The Undoing of James Jesus Angleton. Our program there followed much of what he had been doing each year. For a little over a month, he and his wife Pat and a small staff took over a chalet in the ski area, in a village called Rougemont. Somewhere in Switzerland, Bill had stored a crate of old dictionaries, videocassettes of Brideshead Revisited, abstract paintings, and leopard-print throws. In the days before his arrival, the staff used these items to convert the chalet into another Buckley office and home. And each year, Bill brought along a young college graduate to be his writing assistant. He put us up in an inn just down the hill.

I first got to know Bill, a bit, as the undergraduate editor of The Dartmouth Review. Jeffrey Hart showed him a piece I had written — I think it was a review of the movie Kids — and Bill wrote back approvingly. That translated into an internship at NR, then an editorial job. But I had really met him only a handful of times when he called me into his office and asked me if I liked to ski. I said, “Yes, Mr. Buckley, I do.” He put his hand on my shoulder and said, “Call me Bill.” The invitation to Switzerland came the next day.

“There is never a good time for a busy man to take a vacation,” Bill once said. “And since there is never a good time, he might as well take it whenever we wants.” But Bill never vacationed, even on vacation. He never took weekends off, most likely because his greatest fear was boredom.

So Bill gave himself the assignment of writing a book each year during his stay in Gstaad. The winter I was there, it wasn’t one of his Blackford Oakes novels, but it was a Cold War thriller. The book was a fictionalized first-person story as told by James Angleton, the real life head of U.S. counterintelligence. Angleton’s archenemy, in our book as in real life, was Kim Philby, the famous double agent from the British secret service. Philby had been recruited by the Communists as one of the Cambridge Five and defected in 1963 to the Soviet Union, where he was awarded the Order of Lenin. For Angleton, Philby was the spy who got away, and our novel hinted that it drove Angleton insane.

So in Gstaad, while everyone else went on holiday, we made a novel. Bill woke up at 4:30 every morning. I drove up to the chalet, overlooking the mountain face of the Videmanette, at 7:30. Bill always lent out his four-wheel-drive Peugeot to his young assistants. He handed me the keys our first day at the top of the hill and gave me a quiz about the route to get his morning newspaper. I didn’t want to admit I couldn’t drive stick. So I learned on the road from my hotel to the chalet, and promptly burned out the clutch.

We worked for several hours together every morning, our desks catty-corner to each other. The Goldberg Variations — the Glenn Gould recordings — played in the background as Bill typed. If Bill wanted to set a scene in Beirut in the 1960s, he’d ask me to come up with the detail. Then we went to lunch in one of the hamlets dotting the resort, or in the private restaurant atop the Wasserngrat called the Eagle Club. Here we’d discuss what should happen next in the plot. How about we hide a gun in the camera? Let’s kill off so-and-so. He had very little sense of where his book would go. Then we would ski for a few hours. Then we would return for the afternoon session.

Buckley had it in him to write 1,500 words a day — after a month, you have a novel. But those 1,500 words needed a second pair of eyes, and that’s where I came in. In his first drafts, character names changed. Dates were all wrong. I helped fix those in the afternoon sessions. Then at 7 p.m., Julian, his cook, brought in a kir — white wine with a drop of crème de cassis — for each of us. We’d pull out the Dutch cigars and discuss the day’s progress.

Things moved quickly into evening. The Buckleys almost always included me in their entertaining, which was Pat’s full-time job in Gstaad (she didn’t ski): Where should we sit the princess of Denmark, the actor Roger Moore, the pretender to the throne of Greece, the would-be czar of Russia, and Julian, do we have enough fois de canard for everyone? “James,” Bill would say, “you sit here next to me.” As every young person learned around Bill, he always wanted to hear your opinion, even if you didn’t have one. To close out the night, he would have me sing standards while he accompanied me on a piano. This promptly cleared out the house. He liked to be in bed by 10. We repeated it all the next day.

The time I had with him was a fairy tale, of course, but it was also an intense experience. I imagine it was something similar for all the young people he brought in over the years, a list that runs from Neal B. Freeman and Linda Bridges and Paul Gigot to Peter Robinson and Lawrence Perelman and Danilo Petranovich — and to more than one or two apostates. Being so close to Bill could be like staring at the center of the sun. It certainly caused me to reevaluate the writer I wanted to be. After Gstaad, I ended up enrolling in an art-history graduate program before moving on to my current job at The New Criterion. Bill and I kept up with overnight sails across Long Island Sound. He also asked me to write to him often and was encouraging in whatever turn my life was taking.

In Miles Gone By, his most personal collection of essays, Bill devotes more than one chapter to his childhood music instruction. In addition to weekly piano tutorials, the Buckley children were exposed to an hour of phonograph listening four times a week by a tutor named Penelope Oyen. Buckley tells us that “Miss Oyen loved music with a passion. The use of that word here is not platitudinous. Because Penelope Oyen would weep when listening to music. Not always; not for every composer; but almost always for J. S. Bach. . . . I believed her when she said that music is very serious business. As poetry is very serious business. As art of any sort is very, very serious business: that which is sublime can’t be anything less.” Certainly, this sense for the sublime connected with his faith in God. His greatest regret, he said late in life, was that he wasn’t better at prayer.

This passage reminded me of what it felt like to be there in Gstaad. Bill was never a master at the piano, but he was a great virtuoso. In Switzerland, like a sort of music appreciation, Bill was giving out Buckley appreciation.

Bill found music in words. He played them into his writing, channeled Bach through the computer keyboard. He was probably second only to Shakespeare in bringing words into circulation. “Why do you want to be a party to diminishing the choices that you have, when you’re dealing with a language which you worship for its beauty?” he once said. This virtuosity combined with his great vocal presence. Willmoore Kendall once said that Buckley could do as much with his voice as Laurence Olivier.

I am still amazed how much he shared these gifts with someone who was an entry-level editor, and how much he believed in my help. Even when we were together in Gstaad, and emphysema was starting to take its toll on him, his talents were dazzling. “A sort of personal ebulliency sustains me,” he once said in an interview.

David Brooks said that “for all of Buckley’s contributions to conservative ideas, his most striking contribution is to the conservative personality. He made being conservative attractive and even glamorous.”

Bill Buckley’s unabashed life performance was his answer to the fallen century he had been born into, which he was determined to stand athwart. At the heart of this conservatism was this great generosity, to convey his brilliance through example, to reach across generations, and to have us close enough to listen in.

—This piece is based on a speech delivered at Yale’s William F. Buckley Jr. Program earlier this fall.

Copycat Quandary

Edward C. Banfield

THE NEW CRITERION
April 2013

To the Editors:

James Panero writes in “The Culture of the Copy” (The New Criterion, January 2013) that my college professor Ed Banfield suggested museums sell their original works and replace them with passable facsimiles—a suggestion for which your founder Hilton Kramer criticized him. This gives the wrong impression. Ed thought many second-rate museums felt they had to purchase only original works, and, due to their very limited budgets, they could only afford second-rate art originals. As a result, museumgoers in smaller cities did not have the opportunity to view first-rate art. He thought that the Rockefellers and others had created copies of well-known works which were indistinguishable from the originals and which sold for relatively modest prices. Therefore, why not allow smaller, less wealthy museums to purchase these copies so their publics could view first-rate rather than second-rate art? It sounded reasonable to me when Ed proposed it, and it sounds reasonable to me now. I am at a loss to understand why the art community so violently objects to this.

Robert L. Freedman, Esq.
Philadelphia

James Panero replies:

The use of copies has an important place in the history of art. This is true especially when access to original artwork has been limited. Up through the first half of the twentieth century, plaster casts made from original sculptures were used widely as study aids in museums and art academies. By mid-century, however, these casts were removed from view. In part, American museums had by then come into possession of more original work. But I would also argue that copies came to be overly devalued in relation to originals, and this was unfortunate. I am glad to hear that the Metropolitan Museum now lends its plaster cast collection out to universities here and abroad. The art museum at Fairfield University in Connecticut, for example, currently displays several Met casts on long-term loan.

In other words, the idea of us allowing “smaller, less wealthy museums” to display copies “so their publics could view first-rate rather than second-rate art” was around long before Professor Banfield made his proposal concerning art copies in the early 1980s. One could say that art-library slide collections, and before that magic lantern projections, were all copies used in much the same way as those plaster casts. The same goes today for the high resolution digital scans available through initiatives such as Google Art Project.

In all of these cases, copies serve as necessary substitutes. Their availability has been widely beneficial to a public that might not otherwise have access to great works of art. And even when originals are available, reproductions have a place, because they don’t keep museum hours, and it’s not always possible to lecture about art in a gallery setting.

If Professor Banfield had suggested only that second-rate museums use their limited resources to purchase copies, as Freedman suggests, I agree that would have sounded reasonable. But Banfield suggested much more in his proposal, and the art community was right to object to it.

“I go further,” Banfield wrote in 1982, “Why should public museums not substitute reproductions for originals?” Kramer was therefore correct in giving the impression that Banfield advocated the wholesale deaccessioning, or selling off, of museum collections to fulfill his vision. Banfield’s arguments for this were esoteric at best, nonsensical at worst, but had something to do with a desire to see the “multibillion-dollar art business . . . fall into an acute and permanent recession.” Whatever the reasoning, it was an unreasonable and vastly destructive idea when Banfield proposed it. It remains so today in ideas such as the “Central Library Plan,” a proposal to remove the books and gut the stacks at the main branch of the New York Public Library, which I mention in my essay.

At the heart of these ideas is both a contempt for the art-going, book-reading public and the elitist sense that they either don’t deserve or cannot appreciate the real thing. “It would not be unduly cynical,” Banfield wrote in 1982, “to say that many of the thousands who stood in line for a ten-second look at ‘Aristotle Contemplating the Bust of Homer,’ after the Metropolitan Museum paid $6 million to acquire it, would as willingly have stood to see the $6 million in cash.” Sorry, but to make such a statement is about as cynical as you can get.