Data as a Product – the foundational pillar for the new-gen Data Mesh Architecture

In our previous article, Beyond the monolithic era- Data as a Product (https://www.finextra.com/blogposting/23986/beyond-the-monolithic-era--data-as-a-product)  we had touched upon the new concept, Data as a product which is the way forward for organisations to manage data and move towards a data mesh architecture.

This article elaborates this new concept and provides a high-level view of the next-gen concepts that will rule in the data management world.

So, let’s start by understanding this new concept, Data as a Product that seems to have caused much confusion and garnered a lot of attention in the recent times.

Data as a Product – the context amongst the confusion

Today, data is a crucial enabler for business to sustain and grow and must be used for quicker business decisions and business growth. However, the current architectures do not support the right ownership and accountability that is required to manage data as an asset and maintain it to the highest level of quality.

Organisations that have progressed in their thought process have embarked on the journey to bring in the accountability that is the need of the hour, by incorporating a product mindset towards managing data and moving towards the next-gen architecture, Data Mesh, which necessitates managing Data as a Product and empowering the creators of Data with full ownership of the Data produced and managed by them.

Evolving Architectures lead to this new concept

The transition from Monolithic Data Frameworks, which were not scalable and did not really accord data the importance it deserved. It was almost like a by-product and was not used for business decisions and growth. This forced organisations to rethink their operating models. The next move towards a Centralised Governance structure also witnessed several hurdles and bottlenecks like ownership issues, knowledge gaps that could not be easily bridged, communication problems and many more challenges. The importance of data to was thus undermined and not leveraged to the full.

The disadvantages eventually forced a change in thought process. New-age architectures like Data Mesh started gaining traction and is currently the buzzword in the data management world. A Data Mesh architecture mandates a product thinking mindset, which gave rise to Data as a Product, the futuristic way to manage data. Data as products enabled organisations to empower the producers of Data with full ownership, aligning data owners with the needs of the data consumers. Data thus became the king and moved to be a critical asset for business decisions and growth.

This Data as a Product gained attention and organisations aspired to build and implement the same. However, there is a lot of confusion that exists on this term, which is used interchangeably with a similar term, Data Products.

Hence, let’s clear the confusion between these terminologies before we move ahead.

Data products vs Data as a Product

A Data Product is any technological product /platform / element that uses and analyses data to provide results.

On the other hand, Data as a Product is a consumption-ready data sets that are of high quality, trusted, comprehensive, curated, and ready for consumption to solve business problems and can be used for business decisions. Data as a product can be developed to be used within a single unit, across the organization or outside the organization. They are built on top of they are built on large purpose engineered Data sources like EDWs, ODS or Data lakes.

Data as a Product is organized by business entities and governed by the respective domains under a federated governance structure.

While there is a clear distinction between the two concepts, let’s understand why organisations aspire to move towards productizing data.

Why Data as a Product is the way forward?

Data as a Product is the way to go because it:

• Drives Customer Value and Business Growth
• Opens revenue streams
• Empowers Data Citizens (reduces complexity through smaller data sets that are easier to regulate & comprehend)
• Faster speed to insights
• Ensures end-to-end responsibility within each domain (right from data production, value creation, documentation, quality, training & support
• Promotes Reusability & Eases accessibility
• Lowers risk of failure
• Enhances Data Quality
• Creates Cost savings
• Promotes better contextualization to create value from Data

According to Harvard Business Review, companies with a data-led product vision reduce the time it takes to adopt existing data heritage in new business cases by 90%.

Additionally, Data as a Product is a critical foundational pillar for the new-age Data Mesh architecture, which is what organisations are moving towards to build Data in their DNA

Data Mesh Architecture

A Data mesh architecture address various challenges of current architectures that exist. Data Silos, Data Ownership issues, Data Timeliness leveraging data to derive insights and democratizing data are some of them.

Moving towards a Data Mesh Architecture will enable organisations to implement:

1)      A domain oriented decentralized data ownership

2)      Data as a Product concept

3)      Self-service and Data Democratization

4)      Federated Governance Model

As is evident, evolving to a more matured Data Management Model, which is the Data Mesh architecture, mandates four core principles as the foundational blocks: Domain oriented decentralized data ownership and architecture, Data as a Product, self-serve Data Platform and a Federated Governance structure.

While organisations have put on their thinking cap and are strategizing on creating Data as a Product, there are a few requirements that need to be in place for the concept to bring in the required benefits.

• A robust Data Governance structure and operating model
• Deep Domain knowledge
• Cross functional team to build Data as a Product
• Data Literacy
• Understanding and a map of Data Flows
• Detailed Data Models
• Definition of key metrics (Business KPIs & SLAs)
• Knowing the consumers of data and building as per their skill-sets and needs
• Self-service capabilities in Data Platform
• Data Mesh architecture – strategy to move towards the same and foundational blocks being put in place

Data as a Product can be successfully implemented in an organisation by ensuring the following principles are adhered to:

• Stakeholder involvement and push to bring in a product management mindset towards handling of data
• Fixing Data Quality issues and maintain high data quality standards. Ensuring Data Observability is in place to ensure high quality of Data
• Investments need to be made in establishing a Data Platform and creating a best fit structure for Data Governance
• Clearly defined and operationalized Data Ownership (Data as a Product Owner for each Data as a Product)
• Team of experts supporting the Data as a Product Owner who help build, support, and enhance Data as a Product
• Data Literacy across the organisation
• Understand and use of generative AI to bring data together and help build holistic and market relevant Data products
• Prioritize Data Security, Privacy and Transparency – understand the regulations in the spirit and create guidelines around Data Ethics, Data Security, Privacy and Transparency and stay ahead of potential risk red flags, especially about customer’s PII data

Data as a Product addresses the challenge of ownership and control. If an organisation creates Customer 360 as a Data Product, and assigns a Product Owner to the same, the person will be accountable for responsible to manage the data content, quality, security, protection, accuracy, timeliness, and access to the data across various consumers. This ensures a crisp and convenient flow of communication between the owner and consumers of data, instead of intra-communication across various units/ stakeholders.

Several benefits accrue due to adoption of this concept: the key one being the improvement in the quality of the data with a higher degree of democratization. This leads to value creation and operational efficiencies, which augment achievement of an organisation’s goals and business growth strategy.

Before starting out, always remember to keep your customer at the forefront. Align on your organisation’s vision and objectives.  Build the base. Create a foundation of safety and transparency. Put in place a Data Ethics Team. Develop a robust Risk mitigation strategy.  And move towards the new-age Data Management architecture by building Data as a Product and democratizing data to your business users. Your customer delight index and competitive edge depends on it!