The 'internet of things' (IoT) and 'big data' are two of the most-talked-about technology topics in recent years, which is why they occupy places at or near the peak of analyst firm Gartner's most recent Hype Cycle for Emerging Technologies
Gartner's 2014 Hype Cycle for Emerging Technologies
If you have somehow missed the hype, the IoT is a fast-growing constellation of internet-connected sensors attached to a wide variety of 'things'. Sensors can take a multitude of possible measurements, internet connections can be wired or wireless, while 'things' can literally be any object (living or inanimate) to which you can attach or embed a sensor. If you carry a smartphone, for example, you become a multi-sensor IoT 'thing', and many of your day-to-day activities can be tracked, analysed and acted upon.
Big data, meanwhile, is characterised by 'four Vs
': volume, variety, velocity and veracity. That is, big data comes in large amounts (volume), is a mixture of structured and unstructured information (variety), arrives at (often real-time) speed (velocity) and can be of uncertain provenance (veracity). Such information is unsuitable for processing using traditional SQL-queried relational database management systems (RDBMSs), which is why a constellation of alternative tools -- notably Apache's open-source Hadoop
distributed data processing system, plus various NoSQL
databases and a range of business intelligence
platforms -- has evolved to service this market.
The IoT and big data are clearly intimately connected: billions of internet-connected 'things' will, by definition, generate massive amounts of data. However, that in itself won't usher in another industrial revolution, transform day-to-day digital living, or deliver a planet-saving early warning system. As EMC and IDC point out in their latest Digital Universe
report, organisations need to hone in on high-value, 'target-rich' data that is (1) easy to access; (2) available in real time; (3) has a large footprint (affecting major parts of the organisation or its customer base); and/or (4) can effect meaningful change, given the appropriate analysis and follow-up action.
As we shall see, there's a great deal less of this actionable data than you might think if you simply looked at the size of the 'digital universe' and the number of internet-connected 'things'.