The End of Theory: The Data Deluge Makes the Scientific Method Obsolete

Wired:

The Petabyte Age is different because more is different. Kilobytes
were stored on floppy disks. Megabytes were stored on hard disks.
Terabytes were stored in disk arrays. Petabytes are stored in the
cloud. As we moved along that progression, we went from the folder
analogy to the file cabinet analogy to the library analogy to — well,
at petabytes we ran out of organizational analogies.

At the petabyte scale, information is not a matter of simple three-
and four-dimensional taxonomy and order but of dimensionally agnostic
statistics. It calls for an entirely different approach, one that
requires us to lose the tether of data as something that can be
visualized in its totality. It forces us to view data mathematically
first and establish a context for it later. For instance, Google
conquered the advertising world with nothing more than applied
mathematics. It didn’t pretend to know anything about the culture and
conventions of advertising — it just assumed that better data, with
better analytical tools, would win the day. And Google was right.

Google’s founding philosophy is that we don’t know why this page is
better than that one: If the statistics of incoming links say it is,
that’s good enough. No semantic or causal analysis is required.

[…]

This is a world where massive amounts of data and applied mathematics
replace every other tool that might be brought to bear. Out with every
theory of human behavior, from linguistics to sociology. Forget
taxonomy, ontology, and psychology. Who knows why people do what they
do? The point is they do it, and we can track and measure it with
unprecedented fidelity.

3 thoughts on “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete

  1. Yep – if your definition of “good” is “people link to this”, then all you care about is practical statistics. Now any logician who knows his Lewis Carroll will argue about whether this is an appropriate definition of “good”.
    If you could find a definition for any of the qualitative descriptors (good, pretty, elegant, beautiful, etc.) then we can quantify these descriptors and measure them. Many of the image evaluation programs try to create keyword lists and rankings to describe the images – this is the method that Google is trying to use to return image results.
    This is the best part about upper level mathematics – there are whole courses devoted to discussions of the underlying structures of mathematics (MA310 Abstract Algebra is running this fall)! What do we mean by addition, or multiplication, or commutative, or *smack* … sorry about that. :)

  2. “This is a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear.”
    I told you that the math geeks would rule the world :) He who controls the data controls the universe! We will assimilate all of the other fields eventually…

Leave a Reply to Joshua Sasmor Cancel reply

Your email address will not be published. Required fields are marked *