Do you know what the term “big data” means? If you’re a data geek, chances are you’ll know exactly what I’m talking about. But I’ll admit, when I first came across the term I thought it simply meant a greater amount of data than usual.
But it’s not that obvious, the size of data is relative, and for the most part I was wrong. So despite the name, one would characterise big data not by its size, but what it is used for.
The best way to understand is to draw comparisons againt traditional data.
Big Data vs Traditional Data
Traditional data is defined and inflexible, with data fields aligned statically within each record (think relational databases), whereas big data comes in many forms and is unstructured (think everything else).
Structured data is easily searchable, whereas unstructured data is inherently not so, because this is data that comes from sources that don’t have a fixed format. This includes social media, documents, pictures, videos, audio – the list goes on.
Big data enables the use of data management and data analysis technologies previously unheard of.
When working with big data, the tools and techniques utilised are much more powerful than those employed in analysing traditional data.
Also, an organisation’s intended goals are usually different when examining big data as opposed to traditional data.
Big data platforms such as Hadoop and Spark enable scalability and can analyse huge amounts of data with virtually no impact on performance – unlike traditional investigative methods such as SQL.
Big Data and AI
Machine learning and artificial intelligence is, put simply, computers learning from data in a programmed manner. Of course it follows that the big data world is empowering machine learning and AI by providing an abundance of useful data that can be analysed through automation.
This is just one small example but imagine a shopper in a supermarket paying for their groceries and at the point of sale being generated coupons which are tailored towards them based on their buying history. This pretty much guarantees they will be back next week to spend more money. Sounds cool right?
This is the power of big data. Once you begin to leverage the three "V's" of big data (volume, variety and velocity) to discover valuable insights to benefit your business - the possibilities of what you could do with this power are endless.
Conclusion
As they say data is the new oil. It is the modern world’s most valuable resource and used in practically everything.
Almost every action we take leaves a digital trace which is more data that can be used to determine more trends and patterns to gain a deeper understanding upon which an organisation can base their decisions.
Big data is being used in more noble areas too, such as science and research and improving public healthcare and security.
As the tools become more widespread, so will the applications of big data and who knows what can be achieved in the future?
