This article is written by Snowflake and originally appeared on the Snowflake Blog here: https://www.snowflake.com/blog/5-obstacles-to-successful-data-governance/
Organizational leaders worldwide agree that data governance is important. However, data governance programs in most companies are still being planned or in progress. In a 2020 Dataversity report¹, only 12 percent of companies had fully implemented programs, while 38 percent of programs were a work in progress, and 31 percent were just getting started.
That’s because companies often run into roadblocks while executing data governance. Below are five common obstacles organizations face. For details on how Snowflake’s cloud data platform can help overcome these challenges, download our ebook, 5 Critical Components of Successful Data Governance.
According to a recent Gartner survey², overcoming silo mentality in business areas is one of the most challenging aspects of data and analytics governance. Rigid legacy data architectures promote data isolation by hindering the sharing and dissemination of information throughout the entire organization. Legacy architectures also make it difficult for companies to organize information coherently. Siloed, disorganized information makes it impossible to apply data governance, whether that be tracing data lineage, cataloging data, or applying a granular security model.
Poor data quality
Data governance involves oversight of the quality of the data coming into a company as well as its use throughout the organization. Data stewards need to be able to identify when data is corrupt, inaccurate, old, or when it’s being analyzed out of context. They need to be able to set rules and processes easily. The ability to trust data is a cornerstone for data-driven organizations that make decisions based on information from many different sources. Fifty-eight percent of companies in the Dataversity report said understanding the quality of source data was one of the most serious bottlenecks in their organization’s data value chain. According to the report, “automating and matching business terms with data assets and documenting lineage down to the column level are critical steps to optimizing data quality.”
Data governance requires companies to achieve data transparency: What data do you have, and where does it reside? Who has access, and how do they use it? But legacy systems obscure the answers to these questions. Data management is key to performing this sort of data inventory: Having a strategy and methods for accessing, integrating, storing, transferring, and preparing data for analytics. According to Forrester Research³, “effective data governance grows out of data management maturity.” However, many organizations struggle with data management deficiencies.
Along with the proliferation of data sources both inside and outside enterprises, data breaches are on the rise. Data governance is vital to improving data security. Like successful data management, data security hinges on traceability: knowing where your data comes from, where it is, who has access to it, how it’s used, and how to delete it. Data governance sets rules and procedures, preventing potential leaks of sensitive business information or customer data so data doesn’t get into the wrong hands. However, legacy platforms create siloed information that is difficult to access and trace. Those silos are often exported, sometimes to spreadsheets, and duplicated to combine it with other siloed data, making it even harder to know where all the data went.
Lack of control over data
Businesses often begin thinking about data governance when they need to comply with regulatory policies such as GDPR, HIPAA, PCI-DSS, and the U.S. Sarbanes-Oxley (SOX) law. In the Dataversity report, 48 percent of companies ranked regulatory compliance as their primary driver for data governance. These regulations require organizations to be able to trace their data from source to retirement, identify who has access to it, and know how and where it is used. Data governance sets rules and procedures around ownership and accessibility of data. Without it, sensitive information can get into the wrong hands or be improperly expunged, leading to governmental or regulatory financial penalties, lawsuits, and even jail time.
Snowflake’s cloud data platform provides the right foundation in support of data governance programs. Snowflake helps companies break down data silos and has features that enable companies to achieve compliance as well as better decision making using secured, governed data. This includes availability on the three major data clouds, elastic storage and compute; data encryption, access controls, and tracking capabilities; and integration with external data management tools.