In this blog, we’ll look at the transformational science of big data and analytics. Covering technological developments that are dragging big data from its subterranean roots to bask in the business-trend-setting sunlight, we’ll show how sense can be made of once-static blocks of corporate information.

Unprecedented big data tsunami

Around since 2005, the phrase ‘big data’ originally referred to any large data set stored in a data warehouse. Almost impossible to manage and process using the business intelligence tools then available, big data was a slightly disparaging term. It carried the sense of ‘whatever would do we do with it?’ Fast forward 15 years and the question’s done a quick 180 to ‘whatever would we do without it?’

The change occurred, of course, through the complementary effect of two colliding technologies: the development of cloud-based storage architectures; and the advent of advanced analytical tools designed to make sense of hidden wealths of information. We stand in the path of an unprecedented big data tsunami stirred up by IoT, 5G, social media and sharing apps, autonomics, cloud interconnectivity, and digital cities along with their digital twins, to name but a few. The exploration of orthogonal data sets, part-obscured at obtuse angles to conventional data, will lead to undreamed of analytical possibilities.

Changing the basis of competition

The barriers most face in extracting value from data and analytics are largely organisationally based. Many still struggle to incorporate data-driven insights into day-to-day business processes, obstructed in part by siloed data. Meanwhile, they’re challenged to attract and retain talented people like data scientists and business translators who can combine data savviness with industry and functional expertise.

For those determined to stay at the front of the pack, machine learning (ML) and artificial intelligence (AI) are data and analytics-related sciences with value potential everywhere. Research has shown 45% of work activities could potentially be automated by current techniques, with ML and AI the enabling technologies 80% of the time. Natural language processing breakthroughs could expand that proportion even further.

Lighting up dark data

Until recently, big data thinking had been focused on three Vs. Coined in 2001 by Gartner analyst, Doug Laney, they were volume, velocity, and variety. But as companies digitise more processes, we’ve seen the definition expand. In fact, dependent on who one listens to, big data can now run up to seven dimensions: variability, veracity, visualisation, and value have since been added.

The problem with including visualisation and value in the Vs is that most data is sitting there unseen and unused, which Gartner refers to as dark data. In a Forbes article from 2013, Svetlana Sicular of Gartner, explained the concept: “Like dark matter in physics, dark data cannot be seen directly, yet it is the bulk of the organisational universe.” She could have added much of it was only dark because no one was shining a torch on it. Or maybe a laser, as a more apt metaphor.

Which is where orthogonal data comes in. For example, as Svetlana indicated, who would have thought of examining elevator logs to optimise building occupancy, or viewing seemingly innocent

Tweets to pinpoint shoplifting activity, or analysing team members’ email messages to unlock the secrets of successful project management? It is such intellectual leaps that contribute to V for value.

In addressing velocity, the sheer volume and variety of big data has fuelled faster databases like Exasol and MemSQL, while the need for standardisation has seen widespread adoption of JavaScript Object Notation (JSON). Meanwhile query accelerators such as Apache Impala and Drill speed analysis across multi-format, structured and unstructured data. This is all good news for those looking to optimise the up-to-seven Vs for best-possible big data ROI.

It’s as well to remember, also, that a proportion of big data will increasingly be found at the edge of the infrastructure rather than the centre, especially true when IoT takes off.

Tools under development will afford massive data integration capabilities, breaking through organisational and technological barriers, reaching out to the edge rather than only concentrating on the core. They will enable new business models and deeper-than-ever-before analytical insights across previously inaccessible data living in stovepipes.

The possibilities are limited only by data scientists’ imaginations. Telehouse stands ready to help invent the enabling architectures. “The transition process to colocation might also be used to introduce and drive storage standards worldwide, offering a relatively painless way out of issues that lead to unsearchable siloed data.”

Inventing enabling architectures

Except for the most sensitive environments – usually central government, the military and core financial services systems – the days of on-prem big data are over. For the rest, achieving the scalability implied in the preceding paragraphs in traditional data centres would be unutterably expensive.

While public cloud domains might be okay for high concentrations of low criticality data, collocating with a company like Telehouse could arguably be the ideal big data solution. The transition process to colocation might also be used to introduce and drive storage standards worldwide, offering a relatively painless way out of issues that lead to unsearchable siloed data. It also offers seamless and secure connectivity choices to access third-party information repositories.

Telehouse: guiding big data and analytics journeys

Providing access to a highly connected digital ecosystem of service providers, Telehouse guarantees a highly secure IT operating environment. It also offers the ability to access seamless and secure low-latency and direct connections to multiple cloud services such as Amazon Web Services Direct Connect and Microsoft Azure ExpressRoute when traffic must not be exposed to the internet. Partnering with Telehouse puts at your disposal the imagination and know-how of Telehouse experts to assist you on your own unique and continuing big data and analytics journey.

Contact a Telehouse expert

At Telehouse we can help. We own the space where internet service providers congregate to interconnect and where enterprises come to introduce elegance into their storage and compute strategies. And it’s where we can bring the finest minds to bear on helping you solve the developing big data and analytics issues covered in this blog.

Contact Us

Telehouse International Corporation of Europe, Coriander Avenue, London, E14 2AA
T: +44 (0) 20 7512 0550
E: [email protected]

If you liked this blog, read our white paper which explains these opportunities further. Download it using the form below.