Today,a staggering proportion of the world's decision-making is done with the help of Big Data. When the world went from the "Website Age" to the "App Age", one big change happened. A change that most people never saw, amidst the other changes in their lives; that was the rise of data mining. There has never been a period in history when Humanity has known so much about it self. Large multinational tech companies hold petabytes upon petabytes of data on how we look, what we eat, and what our opinions and habits are. And they use that data to produce insights on what the next thing that we are going to buy is. Of course, they then sell this intelligence to others. Everything from jobs to social media to military decisions, are being decided using algorithms that use Big Data.But amidst all the craze, we may have neglected the dark side of letting Big Data steer the world.Private companies hold enormous power over our lives
Since most decisions today are made with the help of Big Data, the companies that design the algorithms hold an enormous amount of power. Most of these algorithms are proprietary. Many cases have emerged where governments used Big Data algorithms to make decisions. Then, the decisions were questioned. In the ensuing investigation, it was found out that no one in the entire government had access to the algorithm used to make that decision. Judicial systems in many countries have started to use data about people's lives to make judgements about whether they deserve bail or not. And again, in this case to, there is no way to dig into the algorithm that has such a strong influence on people's lives.
Another aspect of this is the data used to train these models. In a lot of cases, there are biases in how the data is collected,we don't whether the data was normalised for weeding out these biases or not.
Now you might say, normalising bias is Machine Learning 101. "Of course they normalise their training data, othe...