Yet with the advent of more and more (big) data, we have not fully used the power of math as a universal language; instead we’ve focused on more specific concepts like the query and pattern matching to help search information and find relationships in data faster. As technologists, we know all of this ends up as 1s and 0s, but the algorithms behind the data are less of a focus, leaving many problems deep within the lingo and existing practices specific to their domain.
Why is this important? Math is to data what abstraction layers are to software development. With it, we can translate physics and energy for the International Space Station, drug discovery, DNA search efficiency, finding the tomb of Genghis Khan, and nearly everything else into a common vocabulary for problem statements.
Researchers at U.S. Agency for International Development (US AID) even used this approach to create a way to predict where atrocities would occur around the world. They used a crowdsourced community of thousands to help identify new data sets, refine what should be predicted, and finally to create a better model for predicting atrocities. Most of the participants did not have deep domain knowledge of the political science of conflict or the existing sociological models for human behavior, but they could pursue the problem intelligently because they understood “how the math should work.”
You can Narinder Singh’s guest blog post on Big Data in it’s entirety on VentureBeat.