Big data vs. small data

The focus is on the wealth of data that is generated and recorded every day — every minute. Very few of us are actually aware of the quantitative extent of this data.

Big data in figures


2.5 trillion bytes of data are generated worldwide every day.
In other words, this is 2,500,000,000,000,000 bytes. — This appears primarily overwhelming and only becomes apparent upon closer inspection.
A vast proportion of the world's population has Internet access. Considering that on average around 300 billion emails are sent every day or 55 billion WhatsApp messages leave the smartphone, it is possible to see how such a volume of data can come about.
Around 90 percent of this information is unstructured data — i.e. photos, voice data, etc. In order to utilize this data and to be able to draw insights from it, it must first be structured accordingly.

Big versus small 


Big data is usually generated with the help of the three V's Volume, Velocity & Varietycharacterized. 
But apart from that, big data differs from small data in key practical features: 


#1 location


When talking about small data, it is in most cases data that is bundled in a physical location (e.g. hard disk). Big data would simply be a problem of space here. Large amounts of data are spread across many different servers, often across different countries.

#2 Data structure

Classic Excel sheets and other tables are typical formats of smaller data sets. Due to the fixed form, these are usually already structured. Big data, on the other hand, comprises all generated data and therefore a wide variety of sources and formats.

#3 Measurement

While small data is collected in uniform quantities (e.g. a currency) within a certain period of time, large amounts of data from various sources must first be transferred in a complex manner.

#4 Repeatability

The chance of recovering small amounts of data in the event of a loss or technical error is (with a little effort) much greater than with big data. Depending on the extent of the problem, a complete reconstruction may be impossible.

#5 Analysis 


The analysis of small data is often carried out within a fixed period of time by a few people involved. With big data, an analysis works much more like a puzzle that is broken down into different pieces, analyzed and brought back together again.


#6 Introspection

Small amounts of data can describe themselves, which means that a type of core object is defined and enriched with appropriate information. With big data, this manageability can no longer be guaranteed; countless reciprocal relationships dominate here.

Even though everyone is talking about 'big data', the 'small' data sets are the less frequently cited heroes that can provide us with connections and trends in real time. This is data that - processed accordingly - helps us to analyse and gain insights in order to successfully steer our company and organization through everyday life.