This blog is a HIGH-level viewpoint/discussion around ‘What is a Data Fabric’ and high-level positioning around Data Virtualisation/Data Mesh’s or other ‘marketing’ speak that is used.
The first key thing to say is that a DATA FABRIC is NOT a PRODUCT…. it’s an ARCHITECTURE based upon a set of services, to deliver managed cross organisation data integration. Indeed, many organisations may already have many elements of a Data Fabric from tools or software solutions that have been acquired to address business data needs. When these various components are integrated they become a DATA FABRIC.
So, what’s new and why is it relevant today and for the future?
Well, traditionally many businesses had solutions that were based ‘on premise’ ie in a data centre or their own facilities. There are many good reasons for some applications for to that continue, ie sovereignty of the data. As businesses modernise their technical architecture many are looking to exploit both Private and Public cloud platforms and the proliferation of Software as a service offering, now being offered by most vendors. The flexibility, agility and accessibility of software as a service offering are very attractive as are subscription payment models. With cloud and SaaS came a different approach to accessing and sharing data often via REST interfaces utilising JSON and XML. To make decisions and run their businesses organisations still need access to all of the data across the modern data landscape, which spans on-premise, cloud and SaaS. Quenching the thirst to exploit data across all these new and traditional data silos (often called HYBRID solution) is the new data challenge for businesses, which a Data Fabric helps to address.
One of the key drivers is that many enterprises are now data-driven (growth of Data Scientists/Analysts) but research shows that up to 65% plus of data isn’t analysed in most organizations and up to 80% of enterprises are inhibited by data silos ie the data sits in multiple locations and multiple owners.
So, in simple terms, a data fabric is an architecture and set of data services that provide consistent and secure access to data across a choice of DATA endpoints spanning hybrid multi-cloud environments. It is architected to drive and support standardised data management practices and practicalities across cloud, on-premises, and edge devices.
Additionally, a Data Fabric architecture facilitates unifying data across an organisation. Agnostic to the data environments, processes, use and geography, this architecture helps automate data discovery, data governance and data consumption to deliver business-ready data for reporting, analytics and Artificial Intelligence (AI) all of which are key ‘use cases’.
However, please note it’s not to be confused with say a Data mesh or Data Virtualisation, these are a new (some legacy as well) agile approaches based on a modern, distributed architecture for agile data integration. Enabling end-users to easily access and query data where it resides without first transporting it to a data lake or data warehouse. This is a worthy subject in its own right and forms part of a Data Fabric if applied correctly, but alone they are not.
There are several key benefits of a data fabric architecture:
- Safeguarding the enterprise data: Data fabric embeds automated governance within the data platform. Enabled by active metadata, data fabric ensures automatic policy enforcement for all data access, providing a high level of data protection.
- Enabling self-service consumption of trusted data: Business users have a single point of access to find, understand, shape and utilize data throughout the organization. The data fabric, therefore, extends the personas and roles within the organization that may access and use data. Data fabric liberates and democratises data consumption.
- Automating the integration of data and providing a single view: Many organisations are having to cobble together at a data platform from different point solutions and manually or programmatically integrate the data. A Data fabric facilitates the automated integration, transformation and orchestration of data to support analytics, business and AI applications.
- Reducing the effort and cost of data acquisition for reporting and analytics.
- Enables Data Governance to be actively applied in real-time, taking Data Governance from being a passive reference point to the managed point of entry for all data.
So, some of the challenges we are seeing that play strongly to a DATA FABRIC are companies that are on a digital transformation journey are looking to harness their data across their existing and new applications. They are faced with a sprawl of data across disparate data stores, locations, clouds and, in most cases many vendors and are struggling to provide the data to their various internal customers on-demand with their existing tools.
Additionally, we are seeing those organisations looking to connect and deliver data from hybrid, multi-cloud and multi-vendor sources, while having to adhere to regulatory compliance and policy enforcement pertaining to their data, in some cases across multiple jurisdictions.
All of these challenges can be addressed by a modern DATA FABRIC platform. While the name Data Fabric may be new to some, the concepts and approaches are not new to us at SmallNet Consulting we have been addressing DATA solutions of this nature for over 21 years.
To learn more about HOW and WHY a DATA FABRIC solution can help your organisation please simply contact us for a free of charge introductory call, with no obligations either way. We have various supporting materials and white papers and details on how you can get started.
