Data to knowledge in the Information Age

Our generation is chiefly characterized by the transition of traditional industry being goods and products driven to information driven. While the Industrial revolution, in the western world at least, brought about new manufacturing methodologies that created the boom of cities and plenty, our post industrial, post digital, information age brings us a new challenge:

To sift through the plenty to find what matters to us.

“Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?” — T.S. Eliot

This couplet is what is credited to the long standing data-information-knowledge-wisdom or DIKW model in information sciences according to Wallace in Knowledge Management. The model proposes that the relationships between Data, Information, Knowledge and Wisdom is linear, progressing on one another. For clarity, lets define the differences between these four layers of the hierarchical pyramid.

Data

Data, according to the Merriam Webster dictionary, is factual information (as measurements or statistics) used as a basis for reasoning, discussion, or calculation.

This means that data has inherent properties that differentiate it from information, knowledge and wisdom. It is factual, measurable, a key property of data is that is must contain a metric.

Its also the basis of something else. Meaning that while data might be interesting in on of itself, it the part of the calculation before the equals sign. It’s the stuff we need to solve what ‘x’ is.

Information

Information can be defined as the rational connection of data to form meaning. Traditionally this would answer the ‘who’, ‘what’, ‘why’, ‘when’ and ‘where’ questions.

This is the age that we are currently find ourselves and it’s evident in everyday activity. There are upwards of 70 millions photo’s shared on instagram daily, and together with the actual digital photo file users share other, measurable, data points such as date, time, geo-location, user tags and hashtags. These data points, when connected, creates the information that is instagram. The user who posted a picture of their childhood puppy because it’s throwback Thursday while they’re on lunch break.

Knowledge

Knowledge is handled by the model quite broadly but can be broken into 3 categories. Let’s approach them as modifiers to the root definition. Which is that knowledge is the product of a relationship between points of information. These typically but not exclusively answer the question of ‘how’.

This breaks into three groups of knowledge: tacit knowledge, explicit knowledge and embedded knowledge.

Tacit knowledge is knowledge that is difficult to share verbally, or through written means. It’s why HR wants individuals with no less than 3 years of experience to apply. It’s why you better at what you do after 3 years of doing it.

Explicit knowledge on the other hand is knowledge that can be codified. It can be written down, transmitted verbally or through visual presentation. It’s the stuff your school teacher helped you understand from the information in your prescribed textbook.

Lastly, embedded knowledge is the knowledge that is bundled together with a worldview, culture, locale or industry. Its implicit, to the group it comes from and while not difficult to transmit or codify, its perceived as unnecessary and given. It’s the river beside which we built our houses, because we just knew that the soil there was richer.

Wisdom

“Wisdom is the ability to make sound judgments and decisions apparently without thought.” It’s the evaluated understanding of all relevant knowledge points, so that informed decisions can be made. It’s what we strive for. Not so much as part of the human condition, but rather as a positive attribute through the embedded knowledge of our belief system (or the one we were raised in).

The DIKW Hierarchy

It’s important to note that there is a hard line between the existing and created in this model. Data and Information are existing. They are observed, collected and collated. Knowledge and Wisdom rely on human intellect to be created. There are no factual data points in a wise decision or teaching, but rather a firm understanding of the different pieces of knowledge that created it.

The information age and beyond

This specific era we find ourselves in is still being discovered. We are creating things, digital and physical based on any and all information we can collect and transmit. Resultantly we are seeing the discomfort of it in the public, and their sentiment toward data collection. We are in the uncanny valley where the value of what data we use and how we use it is poorly shadowed by what we give up. Raw data is not free, its cost is privacy, and that privacy is something many of us do not want to give up to a government or big corporation.

These corporations and governments don’t treat your information with the respect you do. Making public what should have been kept personal from a lack of respect for someone’s specific privacy settings in the case of Bobbi Duncan. Or jumping the gun and trying to be helpful in the worst way possible.

From what I see, these are teething pains that we will never be able to just skip ahead of. However we can and should be actively minimizing the time this process takes. Learn fast and don’t repeat the mistakes we’ve already made. Especially in a world where the Internet of Things or IoT is fast being the next thing, where there are even more data points to collect from.

There are two central focus areas where we need to do work. One is the corporation collecting the data, and two being moving beyond the drudge of information into a space of explicit knowledge.

The corporation, the government, the man

People are very concerned with who’s viewing their data. Rightly so, we have seen both business and governments take full advantage of the data they gather to oppress and force political will upon the public, regardless of the guise it went under.

We need to make people trust companies and governments again. Be that through reform, ousting, or the formation of watchdog committees I don’t know. But one thing is clear to me, that people have very little trust for the ‘powers’ that be.

This change will likely be the most difficult undertaking for us to make or invoke. The current system is embedded in its own work path and as people inside of the technology industry, we will need to learn the current system in order to improve it.

The bottom line is that people are willing to freely give their data to someone that they trust.

Information to Knowledge

I’d be overzealous trying to convince you that software will one day impart wisdom, but with the proper tools and guidance, we can start turning the information that we have into some sort of knowledge.

We do this in small steps, steps that return the highest possible value. Here’s an example based on the Google Now system.

Google collects location data about you. It asks where your home and place of work are. Those two personal data points, together with public traffic data lets them serve you a card telling you when you should leave for work. This improves over time as more data is collected on your location based on time to know when you likely should be at work. This is data, turned into information.

An example of a Google Now card

Now hypothetically, Google knows that Mondays I go to work, and I need to be in at 8:30am to beat the morning traffic. This is information, created through data points of location, time, and traffic. They also have my current location, which might be my local watering hole, as well as the time, which might be 11:00 pm on a Sunday night. This is another piece of information. The systems’ data created information indicates that getting home will take me at least 20 minutes, and in order for me to get a full nights of rest of 8 hours. Perhaps I have some health tracking device on my wrist and it has data about my sleep patterns. Another piece of information.

From these pieces of information, we could build some knowledge about when I should go home to be well rested and on time at work tomorrow. Using this knowledge, I could get a push notification telling me it’s probably a good time to head home right now. In fact the system has other information implicitly; where I am (at a pub) and it knows that it’s late, and how long I’ve been there because of my location data. The knowledge built from this information is that I’m most likely not fit to drive. So together with the push notification, it offers to call me a cab with pickup location and destination predefined.

The nuance about knowledge is that it’s dependent on context. The hypothetical proposed is generic, but in a real world context, the system will need to know that maybe I’ve become a easy target for criminals in my country, so together with getting me a cab, my girlfriend is notified that I’m on my way home. Perhaps Monday is a public holiday in my country, and the notification doesn’t fire at all.

My belief, is that when we get people to trust where their data goes, and they can see real value from it. We will accelerate the process of information based reactionary systems to knowledge based pre-emptive systems.