The Data Flood

We mostly think of data as something we put in into the systems using electronic forms. Human operators pour in data all around the world. There is also data about data, which is what mostly computers do with their invisible algorithms, and then there is data generated by machines equipped with sensors that measure all sorts of parameters. Referred to as the Internet of Things, this network of devices collects data incessantly streaming them into large databases. This data stream is about to explode, dwarfing the data collected by human operators.

Take the healthcare for example. Traditionally, the data collection occurs at healthcare facilities. You go there, a friendly nurse plugs into you a device that measures your blood, your heart, whatever is needed to help doctors produce a diagnostic. Once you are done, the data stream stops. On exceptional occasions you are given a device to carry with you for data sampling. As the sensors become cheaper and smaller, when it comes to data collection the healthcare industry starts to blur the boundaries between the inside and the outside of their facilities. Carrying monitoring devices will become normal creating a huge amount of data.

Everything that touches our lives will be equipped with sensors: the house, the office, the cars, the roads, you name it. With IPv6, the number of connectable devices is practically unlimited. Someone calculated that 1038 IP addresses will be available. Imagine how that world will look like.

WiTricity, a company that invented a system of charging a battery wirelessly, is considering the idea of powering the cars through mini generators planted into the road. This is an incredible idea. Cars flowing (driverlessly?) through the highway absorbing power from the road as they roll smoothly to their destination will be in constant dialogue with a huge network of small devices designed to identify them, and measure the energy consumption and other parameters. This is a data flood alright.

Who or what is going to handle all this data? Who/what is going to make sense out of all this? Forget about privacy – that will go away anyway – there is no place to hide, but handling this data will be a huge challenge.

On one hand we will use analytical tools to examine data and make decisions. This is a slow process that suits us, humans, to have time to figure out things. On the other hand you need faster decision systems that will respond to situations. Large financial institutions in US are already going through a huge redesign of their organisations by replacing human traders with ultra-fast machines that could execute optimised trades at lightning speed. We will have that adopted in healthcare, in transportation and other areas.

A good question is what are the system design principles we need to adopt in a world of fast computers and of an infinity of networked devices? Do we need to learn completely new skills that allow us to handle the increased cognitive load and to interact with computer systems in radically different ways, skills that are not taught in schools or elsewhere? At the moment, there is a growing disconnect between a schooling system obsessed with assessing numeracy and literacy skills and the transformed world in which we live in. Perhaps the technical system design needs to be merged into a social system design so that we don’t rely on highly skilled analysts and machines for making decisions, but integrate the computer network within higher order social networks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s