Wired runs this AP piece about the latest estimate of the amount of digital data on Earth: we’re up to 161 exabytes! (161 billion gigabytes.) The last best estimate was 5 exabytes in 2003, so data has increased 32-fold in the last 4 years. Pretty impressive–and impossible to catalog.
Of course, that curve should level off somewhat as more fields complete their transition to digital. We can be fairly confident that some next-generation storage technology (e.g. holographic storage) will prevent us from running out of room in the physical world for all this data. But just as we have found that you can’t just throw computers at an knowledge problem, you also can’t just throw storage at a data problem.
Not every piece of data is suitable to be posted on the web where google can crawl and index it. Not every search needs an algorithmic best guess based on keywords, incoming links, tags, or user profiling. How we search and catalog our 161 exabytes of data is the next big question… and getting bigger.