Beware of Creating a New Legacy of Artificial Intelligence Silos

This article was written by Adam Mayer and originally appeared on the Qlik Blog here:


Although the issue of silos in IT and data management are well known, companies appear to be falling back into this trap by not distributing their artificial intelligence (AI) and machine learning (ML) capabilities across their business. New research from Qlik and IDC revealed that just 20 percent of businesses widely distribute these capabilities across the organization.

However, with rise of analytics solutions that leverage AI and ML to augment users’ experience of and insights from data, many business leaders are recognizing that having these capabilities siloed in Business Intelligence teams will prevent them from generating the greatest value from their data. In fact, 42 percent consider expanding the use of AI and ML amongst workers as critical to improving the success of data analytics projects.

Three Sources of Silos

So, why are these silos arising once again? There are three key reasons, which many data leaders will be painfully familiar with.

The first is that many companies have gatekeepers to data across the organization. This, in and of itself, is not an issue as this approach often provides the simplest option for governance by keeping data secure. However, it does limit the opportunity of certain areas of the business to take advantage of all the data they need to run advanced analytics tools that incorporate AI and ML to augment users’ intelligence. As such, there needs to be a better balance between meeting the needs of IT and the business.

The second challenge is that there are some sources where it is difficult to get the data out – or where if you don’t do it in the right way, the data isn’t as useful as it should be. ERP systems, like SAP, are a prime example of this and limit the ability for business functions, like Sales, to incorporate its data into intelligent analytics solutions for predictive modelling.

Finally, many companies don’t have the skills widely dispersed across the organization to support a more democratized use of AI and ML. Research from Qlik and Accenture previously revealed just 18% of employees globally report that everyone in their organization has the skills they need to read, work, analyze and argue with data proficiently. Without these core data literacy skills, many knowledge workers will be unable to question and challenge the insights from intelligent solutions.

Democratizing the Benefits of AI and ML in Data Analysis

Understanding the issue is halfway to solving the problem. Those IT and data leaders that take affirmative steps now can break down these silos, so that their entire organization has the potential to drive Active Intelligence from its data.

But, how can businesses successfully overcome the aforementioned challenges and increase the use of intelligent insights across their whole organization?

  • Empower users to self-serve data – Given nearly two-thirds of business leaders (61%) cite that finding valuable data sources is one of their greatest challenges, the benefits of creating a searchable data catalog cannot be overstated. For example, a sales leader might search “customers” to be shown relevant data sets, from invoice to customer service data. Implementing it as a searchable SaaS platform rather than a static data store also supports in the management of governance and access privileges. This provides a single, self-serve data catalog for a consistent user experience, which ensures people can only access the right data for their role.
  • Unlock the potential of raw data sources – ERP and CRM systems hold masses of valuable data, but providing near real-time access to this data in a format that is optimized for the read processes of analytical systems is a massive hurdle that prevents CIOs and CDOs putting it in the hands of business users. The traditional process of extract, transform, load (ETL) used to move this transactional data to data warehouses where it can be governed, cleansed and queried often takes between six to nine months, by which point much of its value might be lost to the business. Switching to ELT and automating the process of streaming data with Change Data Capture (CDC) enables organizations to access real-time information from ERP and CRM systems, in turn fuelling advanced and predictive analytics engines for business users.
  • Choose intuitive platforms – With a small fraction of knowledge workers capable of AI and ML analysis, organizations must choose augmented analytics platforms that significantly reduce the barrier to actionable insights. Intelligent systems can support users on their journey to finding the right information: for instance, conversational analytics help users intuitively navigate data, while natural language processing removes the barrier of technical language and centers on user intent. Procuring platforms for AI and ML analysis that require more specialist expertise will significantly reduce the accessibility for knowledge workers and establishes a significant hurdle for a decentralized approach.
  • Invest in employee skillsets – Although our research with IDC revealed that currently just 16 percent of knowledge workers globally are equipped to do AI and ML analysis, it is encouraging to see that there are clear intentions to upskill more workers in this key area. Respondents predicted that this figure would rise to 25 percent of the workforce over the next two years, as well as increasing the proportion of those with data literacy skills from 45 percent to 63 percent. The role that employees’ skills play in removing barriers to data-informed decision making cannot be underestimated. These skills enable users to find, explore, analyze and question the key insights that AI and ML platforms generate, and which ultimately inform action and create positive business outcomes.

Beware of Silos – Again

Dismantling embedded, legacy silos continue to pose challenges to IT and data leaders the world over. As organizations embark towards a future of more intelligent analysis - with AI and ML enabling more proactive, personalized and collaborative experiences of data insights - these same leaders must ensure that they don’t fall into the trap of silos again. Democratizing the benefits of augmented analytics will not only improve the experience and outcomes of organizations’ analytical projects today, but will lay the foundations for more lucrative insights that will drive truly Active Intelligence in the future.