Welcome to the Velocity monthly newsletter for October 2020 where we share our latest use cases, upcoming events, product updates, and more...
Bring Your Enterprise Data to the Cloud with Snowflake and Qlik
The cloud has become the go-to platform for modern data analytics. But many solutions make it difficult to amass all your data, enable rapid analytics, and provide those insights across your organization. Are you looking for a solution to deliver continuously analytics-ready data at scale?
Join us at 2-3PM on October 22 and learn how Qlik and Snowflake keep your business running at the same pulse as your data. The topics would cover:
- How to move your data to the cloud in real-time?
- How to accelerate your analytics projects in cloud data platform?
- How to automate your entire data warehouse lifecycle?
INDUSTRY VIEW POINTS
Governing Cloud Data Stores
As you move data from legacy systems to a cloud data platform, you need to ensure the quality and overall governance of that data. In this digital age, data and it’s governance are the responsibility of the entire organization. Read the article.
Why Valuable Data Needs To Be Identifiable to The Entire Business
One of the greatest challenges that organizations are grappling with starts right at the beginning of the data pipeline. Research with IDC revealed that a staggering 96 percent of global business leaders reported that it is challenging for their company to identify potentially valuable data sources – with 56 percent stating that it is either very or extremely challenging. Read the article.
Customer Use Case
Qlik points way ahead for DCH
The challenges that Dah Chong Hong Holdings were facing was that the DCH consumer products business extends across the supply chain and includes the distribution of thousands of household favorites with a network of services spanning the entire supply chain and ensuring the highest levels of traceability, quality control, and product excellence. It is the largest agency distributor in Hong Kong with more than 500 brands of food and FMCG products under the Sims, IMSA and DCH banners. With such a widespread operation, DCH realized that it needed to sustain a deep understanding of both its customers and its business.
Gaining such insights would help it work more efficiently today and plan more comprehensively for the future. Data was the key to obtaining this business intelligence and DCH realized that to create a firm basis for the business, it needed to update its Enterprise Resource Planning solution.
As a result of using Qlik, a daily sales and inventory analysis that used to take half a day to compile is now instantly generated automatically in Qlik. While DCH is currently using historical data to predict future sales and revenue, looking to the future it is investigating the use of Qlik Sense Machine Learning (ML) and Augmented Intelligence (AI) functionality for forecasting. Further benefits of using Qlik includes; Qlik’s swift analysis enables the FMCG and Pharmaceutical divisions to increase productivity and save manpower costs, comprehensive business insights gives management the business intelligence it needs to make informed decisions, and the FMCG and Pharmaceutical divisions benefits from a better understanding of the DCH operation and its customers.
“Using Qlik Sense enables everyone to work more efficiently and productively, saving both manpower and costs. They can analyze valuable data to produce well-focused reports that deliver valuable business intelligence for the Dah Chong Hong Holdings (FMCG and Pharmaceutical divisions).” Amy Tso Yin Mui, Assistant General Manager, Group Information Technology, Dah Chong Hong Holdings. Read use case.
Solution Updates
Alteryx Data Science Portal
Alteryx has launched their Data Science Portal in the Alteryx Community (under the Blogs & Podcasts menu) where data scientists and machine learning enthusiasts can find all data science content offered across the Alteryx Academy, podcasts, blogs, discussions and more.
Alteryx Everything Podcast
A podcast about data science and analytics culture. Subscribe to the Alteryx Everything Podcast in the Alteryx Community to hear from data enthusiasts like yourself.
What’s New in DataRobot Release 6.2?
The DataRobot Release 6.2 includes enhanced automated feature discovery (feature engineering), a new comprehensive Autopilot mode for maximum accuracy when you need it, anomaly assessment insights to help you uncover root cause, governed approval workflows in MLOps, and much, much more.
Here are some of the main highlights of the release:
- Next-Level Feature Discovery: An enhanced dataset relationship workflow, makes it much easier to select multiple datasets, and define, edit, and visualize all your relationships at the same time. You can now access logs to get details on which features were explored, discarded, and generated. You can also download the full training dataset, including all the derived features.
- Comprehensive Autopilot Mode (PUBLIC BETA): This new mode runs every single model in the repository for your project, taking as long as necessary to maximize accuracy when you need it most. We also have a new Get More Accuracy feature, so you can kick off Autopilot in Quick mode, then start Comprehensive mode only after you’ve seen your initial results.
- Anomaly Assessment Insights: New in Automated Time Series 6.2, this interactive visualization allows you to quickly investigate anomalies and anomalous regions in your data, and access SHAP scores for the underlying features causing the anomaly. This allows you to understand the root cause, as well as be able to explore all of your anomalies, panning through different time segments and zooming in to see the detail.
- Model Comparison Reimagined: The ability to compare models has taken a huge leap forward in Release 6.2. We worked with our most experienced data scientists to give you the best possible user experience where you can compare models and choose the best for deployment. We have enhanced the Lift Charts, ROC Curve, and Profit Curve, added support for more bins, and have added new tooltips to enhance overall ease of use.
- Governed Approval Workflows: Customizable governance policies and review and approval workflows come to MLOps in 6.2. This introduces accountability in your production AI, and enables you to continue to deploy and manage production models while at the same time increasing your overall level of AI governance for your entire organization.
- Connect to Remote Repositories: We recognize that your data science teams often govern and manage their models in popular open source code repositories. In Release 6.2, MLOps allows you to connect directly to your GitHub and S3 repositories and dynamically pull model code and model artifacts into DataRobot, making it simple to package, test, deploy, and monitor them in your production environment of choice.
What’s New in Qlik Sense September 2020?
Augmented intelligence
New search-based visual analysis chart types When generating insights in search-based visual analysis, Insight Advisor will now auto-generate several new types of visualizations. These include:
- Cluster Chart – Shows clusters using the new k-means clustering function. Correlation Chart – Shows correlations using the correlation function. Control Chart – Shows how a process changes over time.
Search-based visual analysis on mobile devices
Search-based visual insight generation is now available on handheld mobile devices. This provides a more intuitive means of creating new analytics and insights on small form factors, as opposed to traditional authoring. It is available on handheld devices through the browser-based user interface.
Advanced analytics calculation - K-means clustering
With this release, advanced analytics calculations on-board the Qlik Associative Engine are being introduced. The first function available is k-means clustering - allowing data points to be grouped into clusters based on similarity. This is a highly useful function for customer segmentation, fraud detection, and many other use cases.
Visual analytics improvements
Improved reference lines
Bar and line charts now include dimension based (vertical) reference lines with formatted labels, useful for enhancing charts with added detail along a time axis (such as start and end dates of important events)
Animator control
The Animator control gives users the ability to animate changing data over time by auto-selecting values in any dimension and playing them in a loop, with or without aggregation.
Other chart improvements
- Number formatting of master measures
- Turn on and off borders in containers
- Custom sorting in Sankey Chart
- Frequency counts in filter pane
- WMS (web map service) layer opacity
- Hover icons toggle
Administration improvements
Table view in the hub presents a scalable and easy to use app manager. The following features are included:
- Columns for name, description, owner, published (when applicable), Data last reloaded (when applicable), details
- Sort by clicking on the column headers
- Keyboard shortcut just like grid (Ctrl + g) and list view (Ctrl + Shift + l)
- Navigate with keyboard
- Support for Qlik-supported screen readers
Other improvements
Data literacy built-in
Qlik has focused on our vision for a data-literate world. As part of this mission, and keeping first-time users in mind, new capabilities have been added within Qlik Sense to assist and educate users about their data. For first-time users, these capabilities help explain data concepts unique to Qlik to expedite their onboarding and shorten the time to insights, including more context for chart creation and additional explanation for dimensions/measures.
New Snowflake Features
Multi-Cloud, Cross-Cloud, and Pattern-Matching Support in Snowpipe
Snowflake previewed the following features to make Snowpipe more robust:
- Automated continuous data loading for Google Cloud Storage stages: Aligning with the spirit of near-zero maintenance as a core tenet of Snowflake, this feature enables automated continuous data loads for Google Cloud Storage (GCS) without requiring administrative intervention. Using event notifications, Snowpipe automatically loads new data from cloud storage into Snowflake, making it more readily available to analytics teams for faster time to insight.
- Continuous data loading to Snowflake on AWS: Many Snowflake customers require a multi-cloud approach, whether it’s to avoid vendor lock-in or to accommodate different platforms resulting from mergers and acquisitions. Often organizations look for ways to use Snowflake’s cloud data platform across the leading cloud providers Snowflake supports, namely Amazon, Google, and Microsoft.
- Support for pattern matching: Event-driven architectures are commonplace today. As certain types of events occur (such as fraud detection or real-time customer service requests), using data to take the right action is critical.
Snowsight Usability Updates
Snowsight, the Snowflake user interface, is where analysts can build queries and interact with Snowflake’s cloud data platform. Snowflake continually works to extend Snowsight functionality to include the capabilities required for all Snowflake-supported workloads.
- User refresh of schema metadata: Previously in Snowsight, the schema metadata was refreshed automatically every 24 hours. When a new table or view was created in a database, users with the right roles and permissions could query the object; however, the schema browser would not list the new object immediately, and the new object could not be searched, meaning important updates or new data could be missed.
- Execution of multiple SQL statements sequentially in a single worksheet: Analysts use worksheets within Snowsight to build and run SQL statements on Snowflake. Previously, analysts had to either build complex scheduling routines or execute SQL statements manually.