For several years now, the use of big data technology has been a thing. Analysts convince us how important are the conclusions from the processing of a huge amount of data, how much can be improved in the operation of business or society in general. Especially in global companies, statistical information and other big data types can improve the functioning, but also provide valuable knowledge about customers and their behavior. What’s more, they also make it possible to anticipate choices and preferences, while using graphical representation of data. Is this really the case? Or is it just fools gold that marketing specialists want to sell us. Let’s find out who big data is for!

Word of introduction: Big data technology is based on the analysis of huge databases, combining them into even larger collections and drawing conclusions from them. That’s it – in a nutshell. Of course, we are not talking about a database with 2000 records. We mean something bigger, way bigger! In the case of big data, the analysis usually concerns hundreds of millions of entries – in just one database. Only on such a sample there are reasons for using this technology. So if we run a small grocery, the implementation of the big data application for us will have little to non business value. However, if the scope of our activities extends to the entire province, and we collect a lot of data from various sources, then it is worth employing big data to learn about the relationships and connections taking place between different people or events.

Good customer relations

What can it give us? There are so many possibilities. If you run a medium-sized online shop, you can find out when you have the most customers and where they come from and at what time they buy. That allows to predict what sales we can count on in the near future, how to better organize logistics and shipping – this is the first example of application.

 A good implementation of Big Data in our business, provided of course that we have cross-sectional data, above all, gives us the opportunity to see the correlation between individual events and the opportunity to improve existing schemes or implement new ones. Of course, these data must come from legal sources, but this is a topic for a completely different article.

We’re going to the masses

We won’t learn too much from big data about individuals – this is not what this technology was created for, but the analysis of large communities, even of all our customers at once, tells us a lot more – after all, as individuals we act quite schematically. Especially when our business goes beyond the local market – big data turns out to be a useful tool, much better than traditional statistical research.

Exactly, what is big data technology better than standard statistical surveys? First of all, the scale and speed of calculations. While statistical tests start from several dozen people and at most they reach several thousand, in big data we start from tens of thousands! In the first model, however, we are able to conduct a deeper study of needs or behaviors
– in the second one we analyze events, records of activities performed: purchase of a specific product, leaving the basket or words entered in the search bar.

The treasure on the server

Without big data it would be difficult to study these dependencies because their number is too high and they are too often subject to fluctuations. Properly visualizing them, for example in Google Data Studio, which we have written about here, helps to compile and present data in such a way that they are as helpful as possible when building a company strategy or at the operational level.

– Companies themselves do not know what a treasure, in the form of data, is hidden on their servers – convinces Tomasz Andrzejewski from Greenlogic, who leads the implementation of big data technology in business. – You don’t need to be a huge corporation to use big data, although a certain volume of data is of course necessary to help build a competitive advantage – he adds.

Scalability of big data systems also allows to extend them with a new type of data, an additional database or new data nodes, analyzed in real time. If necessary, you can use language scripts, bridges to select data and cloud computing. It is thanks to the serverless computing we have written about here that big data technology can simultaneously compare billions of records in databases from all over the world.

Merrily and seriously

Comparing data is also fun. This is evidenced by the tylervigen.com website, where we find graphs of correlation between completely different aspects of human activity, such as cheese consumption and deaths from getting entangled in bedding or margarine consumption and the number of divorce in the state of Maine, USA. Of course, the use of such an analysis is insignificant, but it’s a fun.

However, if you want to use big data to develop your business, or in large social research, it is worth consulting the implementation with specialists, so as not to drown in the sea of data. As Tomasz Andrzejewski points out: – The analytical application of big data is different every time. At Greenlogic we did not meet with two identical databases and identical needs of two different clients. So before we run analytical scripts on servers or in AWS Lambda, it is worth to think thoroughly about what data we want to compare and what we want to use them for.