The government began using the so-called “Hadoop” software for data collection, even after launching a $16 billion project to come up with its own way to manage 'big data.'