Data analysis has become one of the most powerful tools available today thanks to the development of Big Data. This technology manages large volumes of data and analyses them to provide conclusions that will help in decision-making, in many different aspects and in many different sectors.
In order to approach this technology, and to understand how it works and its applications, it is possible to know what Big Data is through a definition articulated by industry analyst Doug Laney in the 2000s. Laney spoke of volume, speed and variety, three variables that determine its basis.
Volume, velocity and variety. What is Big Data?
The term Big Data refers to large volumes of data, which need to be collected and processed in a different way than in the past, as these are data sets with storage sizes of several Petabytes (1 Petabyte equals 1 million GB).
Speed is another of the foundations on which Big Data is built. Today, the development of infrastructure such as 5G has enabled the growth of digitalisation and connectivity. The next generation mobile network is faster, has a connectivity capacity a thousand times that of 2010 infrastructures and has virtually zero latency. What does this mean? That about 7 trillion devices are connected and exchange data.
All these connected devices generate data, millions of data of all kinds. Hence, the variety of data studied. This variety is one of the challenges because before analysing, all this variety of data has to be classified.
What types of data does Big Data analyse?
The variety of data studied is very wide. This is why we speak of so-called structured and unstructured data. Structured data are those that come from large databases. However, more and more, individuals and companies are generating data that comes from many different sources such as financial transactions, everything from the Internet of Things, IoT and connectivity between devices, emails or even videos, audios and information that appears on social media. These are the unstructured ones.
Before going on to analyse the different types of data, it is necessary to manage them. And within this management there is also an important aspect which is veracity, i.e. the quality of the data.
How does Big Data work?
After collecting and sorting all these millions of data from so many different sources, they are stored and analysed. In this regard, it is important to bear in mind that much of this data is generated in real time and stored in cloud solutions, and to obtain an immediate response, regardless of where the information is generated.
What this technology does is to fully automate these processes in order to keep only the data that is relevant to the application, because the information needs are not the same whether the aim is to improve decision-making in a pandemic or to improve traffic management in a large city.
What are the uses of the data?
Its applications are many and varied. Sometimes the data itself may not have much value, but it has the potential to be transformed into a source of great value through the use of Big Data tools. This technology is closely linked to the use of AI, which makes it a very powerful tool for identifying needs and problems, points of improvement and providing solutions to these issues or creating new products and services.
Many sectors use this technology: from tourism to industry, from natural resource management to city organisation, from logistics to customer service, and even public health care. Massive data analysis during the COVID-19 pandemic is not so far behind. In the midst of the health crisis, Big Data was used to monitor the evolution of mobility and to obtain “daily information on mobility at national, autonomous community, provincial and local level, to support the monitoring of the evolution of the disease, in order to evaluate the effectiveness of the mobility restriction measures adopted”, according to the authorities of the Ministry of Transport, Mobility and Urban Agenda.
Another health application relates to data from people’s medical records, which can help predict the onset of a disease based on patients’ lifestyle habits, for example, or recommend tailored treatment by cross-referencing data with other patients who may have similar symptoms.
And what are the benefits of mass data analysis?
A large number of companies and organisations of all kinds, such as public administrations, use this technology. In this way, they can learn about consumption habits and predict a trend before it materialises: such as rising energy demand, mobility flows or demand for certain healthcare products.
The combination of Big Data with other innovations such as cloud storage and AI can reduce the time it takes to make decisions and design strategic plans of all kinds in real time, helping to reduce costs. For example, industry can adapt production to demand, anticipate a potential crisis or improve the efficiency of services by analysing the needs of individuals and other businesses. Data analysis optimises many procedures, thanks to its ability to study the response (or feedback) of a market, also in real time.
