Data Analytics: Hindsight to Foresight
In an increasingly connected world, consumers, businesses, and governments are presented with unprecedented access to data. What they do with that data depends on the questions they wish to ask and the technological capabilities available to produce answers. As the ability to generate insights from data evolves at an increasingly rapid pace, both technology and humans will grow smarter and more empowered.
It Starts with Big Data
For any study, confidence in the outcomes and conclusions is directly tied to the size and quality of the dataset. Thanks to increasing worldwide digitization and Internet-connectedness, nearly every person on the planet creates mountains of data ripe for study. Entities on the receiving end of these large swaths of data find themselves in possession of a rich, albeit unwieldy, source of market insights.
The definition of this “Big Data” evolves as the digital landscape transforms. Think of it not as a thing, but as a concept. In that context, Big Data encompasses all of the information created and flung into the Digital Universe, which humans are free to access, consume, and organize in order to make sense of our world. Consumers, businesses, and governments are the primary beneficiaries of Big Data.
Big Data may seem amorphous and confounding, but Charles McLellan of ZDNet gives it shape with four basic traits:
- Large volume (at scale)
- A variety of forms (content types)
- Arrival at high velocity (streaming)
- Variable veracity (sometimes untrustworthy)
Big Data, by its very nature, is too vast and complex for traditional data-processing applications to handle, and it continues to be a driver of software and hardware innovation. Organizations receiving Big Data face challenges including capture, storage, transfer, security, and processing. While channels of Big Data include everything from social media websites to health care records to point of sale transactions, one of the fastest growing contributors is the Internet of Things (IoT).
Enter the Internet of Things
The term “Internet of Things” was coined by technology pioneer Kevin Ashton in 1999. One of the first applications of the IoT was tracking the locations of expensive equipment through radio-frequency identification (RFID) tags. Nowadays, the IoT has expanded to connect everyday objects such as automobiles, kitchen appliances, baby monitors, and even light bulbs to the Internet. Gartner, a worldwide research, and advisory company, predicts that 20.4 billion IoT devices will be in use by 2020.
According to Steve Ranger, UK editor-in-chief of ZDNet, “The IoT generates vast amounts of data: from sensors attached to machine parts or environmental sensors, or the words we shout at our smart speakers. That means the IoT is a significant driver of big data projects because it allows companies to create vast data sets and analyze them.” Access to such granular information about the lives of a company’s products has potential to shape future functionality and drive innovation. It also raises questions about privacy, security, and just what data is necessary to collect.
Data Reporting Progresses into Data Analytics
One way to make sense of collected data is via Data Reporting (DR). Well-designed reports include tables, visualization features, and customization capabilities. They can be pre-created by software developers, or assembled on demand by end-users. DR captures history, allowing humans a broad view into the past from the perspectives contained within an ever-growing ocean of Big Data.
As Big Data evolves, so do the means of deriving value from it. Beyond reporting, Data Analytics (DA) involves generating insights from sets of data. This concept is often referred to as Big Data Analytics (BDA), but can also be shortened to DA. Like the Big Data that feeds it, DA is a complex concept that is evolving as quickly as software and human capabilities can push it forward. While DR is a tool focused primarily on informing us about the past, DA is the art and science of using data to help organizations create a desired future.
George Anadiotis, an IT consultant writing for ZDNet, provides the below graphic from Gartner to summarize DA’s predicted evolution:
As DA’s application progresses from descriptive to prescriptive, humans will partner with Artificial Intelligence (AI) to more accurately and quickly derive actionable information from Big Data. Machine Learning (ML), a branch of AI that deals with the automation of analytical model building, can independently adapt to the variables inherent in Big Data while applying complex mathematical calculations that produce reliable, valid results at speed. Doug Henschen, an analyst at Constellation Research, tells us to “Expect a steady drumbeat of announcements throughout 2018 and beyond about ML applied to tasks including cleaning and combining data, discovering new data, and suggesting new combinations of data that could, in turn, uncover important insights.”
The graphic below illustrates how DA is becoming more accessible to the average business user, as AI technology evolves to assist.
Investment in Artificial Intelligence
Today, deriving meaning from DA is primarily the realm of Data Science and Analytics (DSA) professionals, but thanks to the emergence of AI-driven software tools, business users are becoming capable of generating actionable information through DA. Though an exciting advancement, investment in exploring the full potential of AI is slow to build.
In January of this year, Constellation Research, a technology advisory firm, published a study for which they surveyed C-level executives within their subscriber base, to examine AI adoption across 12 business sectors primarily located in the United States. For the purposes of this study, they define AI as “the culmination of technologies including deep learning, neural networks, natural language processing (NLP) and big data/predictive analytics to produce software that is self-improving and automatic and also emulates human intelligence”.
Constellation found that while 70% of respondents said their organizations currently use AI technology, 92% reported company-wide budgets of $5 million or less. This indicates that up until this year, AI projects have been modestly funded and limited in scope. The survey gives us a peek into their plans for growth by indicating 60% of companies expect to increase investment in AI development by more than 50% compared to last year.
The study also identified three ways in which responding businesses develop AI:
- 56% are creating applications with data science teams, utilizing open source frameworks
- 52% are creating applications using ML and deep learning services based in the cloud
- 42% are Integrating packaged applications with AI capabilities
These efforts are mostly for the benefit of consumer-facing departments and IT, as enterprises seek to prove the return on investment (ROI) of AI before applying it to back-office functions. This ties back to the DA trend of using AI technology to assist end-users. Software suites touting ease of generating actionable insights are emerging most noticeably in the areas of commerce, customer service, sales, and marketing.
Human Talent Remains in High Demand
As mentioned in the previous section, DSA professionals are still very much in demand despite projections about the capabilities of AI with regard to DA. According to research published by IBM and reported by Forbes contributor Louis Columbus, the DSA jobs market is expected to grow 15% by 2020. The Finance and Insurance industries provide the richest market for DSA jobs, accounting for 19% of all reported openings.
While the demand for experienced, educated individuals is growing out of proportion to supply. Future productivity and the ability of companies to realize go-to-market strategies are at risk for disruption if human talent cannot keep up with the pace of DA advancement. IBM reports the most challenging skills to recruit for are ML, Big Data, and data science.
Constellation Research warns, “Rising demand for talent with AI proficiency coupled with a preference to recruit new talent could culminate in a talent war as more AI projects come online.” To help close the DSA skills gap, IBM formed a research partnership with Burning Glass Technologies, a labor market data firm, and the Business-Higher Education Forum (BHEF), for the purpose of sharing data-driven insights between higher education and industry. Their work is ongoing.
The ability to leverage Big Data will be critical to surviving and thriving in the digital revolution. As DA continues to evolve and enhance our understanding of the past, present, and future, organizations will face challenges that push the limits of human and technological capability. In overcoming these challenges, the global marketplace and society at large will grow smarter.
To sum up, here are four areas of opportunity as we endeavor to create a desired future:
The Digital Universe, which is all digital data created, replicated and consumed in a single year, will swell to 44 trillion gigabytes in 2020, up from 4.4 trillion gigabytes in 2013. Entities wishing to mine Big Data for meaningful information will need to become better at targeting what data is useful, extracting and storing the mass in a secure manner, manipulating it with greater accuracy and speed, and providing actionable insights directly to business users.
Trade-offs in security and privacy will be part of life in a more connected, data-driven world. Implications range from individuals’ protected health information to business’ proprietary information, to cyberwarfare. Humans will be challenged to find the right balance between divulging personal information and their desire to take advantage of Big Data-powered applications and services. Organizations will bear the burden of protecting the private data they collect and using it ethically.
AI’s tremendous potential must overcome human distrust, scarcity of qualified workforce talent, and shaky support from budgetary and IT infrastructure perspectives. Building employee trust in smart systems will hinge on effective change management and restructuring of business practices. Further, the trust of AI by society at large will require transparency and communication of benefits relative to risks.
As the technologies supporting and advancing DA become more sophisticated, so will the need for higher skill levels in human workers. Aggressive recruitment efforts and a focus on developing AI in-house are indications that organizations are attempting to head off the looming talent war. Through ever-evolving training programs and reskilling of their workforces, firms can help employees keep pace with the rapid evolution of AI.