Big Data: 4 Things You Can Do With It, And 3 Things You Can't
Big Data – the term we give to drawing insight from the ever-growing amounts of digital information we are constantly generating by applying advanced computer analytics – can do some amazing things.
Sometimes, it seems tempting to think that Big Data can do anything. And it certainly can do a lot. In this post I want to take a look at the things that Big Data is very good at, as well as some of the things that it can’t do, yet.
1. Can do: Tell you what is likely to happen
A key Big Data skill is predictive modeling, which involves running simulations to “predict” the outcome of altering variables in a dataset – for example if you have weather data and visitor numbers for your theme park, you can predict what your sales are likely to be if you should have a week-long run of rainy days or a scorching heat wave. The more comprehensive your data, the more accurate your “model” of how weather affects visitor numbers is likely to be. Of course factors other than weather will influence the variable you are trying to predict – and by identifying them and layering them into your data analytics a more complete picture of the possible future outcome will emerge.
2. Can do: Find unexpected relationships
With huge amounts of data from many different data points, the relationships between different variables can be examined in detail. This has proven advantageous in many circumstances, where analysts have found that tweaking a certain variable had unexpected (good or bad) consequences or outcomes. One example comes from medicine, where huge databases of drug interactions and patient responses have found that medicines originally designed for one purpose have applications for others – such as the anti-depressant Desipramine which was found to have potential anti-cancer qualities after data on its effects on patients was studied.
3. Can do: Monitor a situation as it develops
Real-time analytics involves gathering and interpreting data “as it happens” in order that insights gained from it will be as timely as possible. In business, the motivation is competition – the more closely your data resembles what is actually happening, right now, in your market, the better the chance you can react and respond to it before your competition. Businesses engaged in real-time analytics are using the technology to do everything from predict when machine parts will fail, to analyzing social media to determine public sentiment towards products or topical issues.
4. Can do: Fix a problem before it becomes a crisis
Big Data can highlight dangers on the horizon by showing what the likely consequences of a path of action will be. This has extended into the physical world where machinery fitted with sensors which constantly feed data on their operations are becoming widespread across industry. These Big Data systems can build up extensive understanding of how machinery works and when it is likely to go wrong, meaning that repairs can be scheduled (reducing costly downtime to a minimum) and flaws in the design or manufacturing process identified at an early stage.
1. Can’t do: Read your mind
Computers cannot yet know what truly motivates us to make a decision, such as what to buy or where to go on vacation. They can come close to developing an understanding of this by processing data on external factors that correlate with us making a particular choice, but they can only ever infer a causal relationship between motivation and action. For example, Amazon may have systems in place that tell it that if we have bought books by a certain author, we are likely to also buy books by another – particularly if we browse categories of books which that author fits into every time we log into our accounts. It can’t know for certain that we would ever buy a particular book.
2. Can’t do: Come up with a new plan
Computers can only do what we tell them. To do. Cutting edge development in machine learning and cognitive computing, such as IBMs Watson project, might have shown us that computers are now able to “learn” but in reality they are restricted to learning in a highly pre-determined manner according to learning algorithms they have been programmed with, by humans.
This means that they aren’t very good at proposing solutions which haven’t already been suggested to them. If we give them a number of possible options for solving a particular problem, they can tell us which, statistically, is best, based on predictive modelling of all the known variables, and the unknown ones which can be extrapolated from them. But they can’t tell you “Actually all those ideas are stupid – this is what you should be doing …”
3. Can’t do: Predict the future (with certainty).
Big Data uses predictive modelling to come up with possible results of anything you could care to ask it about – what the weather will be like next week, who will win a baseball game or what sort of traits you should be looking for in the next employee you hire. But there are always just possibilities. It can give you a range of possible options or outcomes based on what the statistics show is most probable, but it won’t tell you with 100% certainty what will happen, or what course of action you should take. Any number of outside influences could throw a spanner in the works – and at the moment, unless the programmers designing the algorithms governing a system’s predictive abilities thought to take them into account, your Big Data will not have a clue about it.
As always, let me know your thoughts using the comment function below. Also, here at LinkedIn and at Forbes I regularly write about management, technology and the mega-trend that is Big Data. If you would like to read my regular posts then please click 'Follow' and feel free to also connect via Twitter, Facebook and The Advanced Performance Institute.
You might also be interested in my new big data case study collection, which you can download for free from here: Big Data Case Study Collection: 7 Amazing Companies That Really Get Big Data.
