IoT News – Use cases and applications of data mesh in IoT, AI and machine learning

0

An article by Yash Mehta, expert in IoT, M2M and Big Data technology.

In Web 3.0, the dynamics not only of the Internet, but also of data flows, undergo a decentralized transformation. First, with distributed data governance, each domain can now manage and govern its data products, but at the same time it also relies on central control over security policies, data modeling, and compliance.

Data mesh distributes data across physical and virtual networks in a decentralized manner. Unlike conventional data integration tools that require a highly centralized infrastructure, a data mesh instead operates in on-premises, multi-cloud, and single-cloud edge environments.

In this article, we discuss the actual applications of the mesh in different configurations.

Data Mesh: Fixing Several Common Problems

According to MIT findings, only 13% of organizations surveyed could successfully deliver their data strategy. Data Mesh tackles many responsible root causes.

Using a data mesh can solve several problems found in smaller scale data pipelines. If left unresolved, they can quickly become problematic and brittle over time, as messy point-to-point systems can build up their webs over time.

At the same time, a data mesh also resolves larger issues in an organization, such as key business facts that different departments in a company may disagree on.

By implementing a data mesh, the system is less likely to have copies of facts.

Using a data mesh not only brings order to a system, but also gives you a more manageable, mature, and evolved data architecture.

As we witness the rise of cloud-based applications, application architectures are moving away and evolving from conventional centralized computing and evolving towards a distributed service mesh or microservices. A real-time data platform called K2view has taken a step ahead and successfully implemented the use of micro-DBs in their fabric and mesh architectures. Each micro-DB stores data of a particular trading partner (customer) only while their mesh platform stores millions of these micro-DBs.

Data Mesh: Use Cases

A data mesh can support multiple analytical and operational use cases across multiple domains. Some examples include:-

1. Understand the customer lifecycle

It provides 360° support for customer service and significantly reduces average customer handling time. It also improves customer satisfaction and increases first contact resolution.

Marketing can also deploy a single customer view for next-best decision making or predictive churn modeling.

2. Usefulness in the Internet of Things (IoT)

Through IoT device monitoring, product teams can gain insight into usage patterns of edge devices. They can use this model information to iterate and improve their profitability and product adoption.

By adopting a mesh network for IoT devices, businesses can gain several benefits that make it a popular technology when it comes to choosing networks.

Enterprises can collectively store all of their IoT, enterprise, streaming, and third-party data in an S3 data lake at very low cost.

3. Self-Healing Algorithm

As mentioned earlier for Shortest Path Bridging, the self-healing algorithm automatically selects the best path to send data even in the event that some nodes lose their connection.

This algorithm allows the system to use only available and working connections. So even if some devices stop working, the network is still able to send and receive the information needed to maintain or complete a given task.

4. Distributed and more efficient security

Now, when it comes to security, companies are well prepared and continue to update their protocols. However, SMEs do not have the necessary advice. According to Accenture’s study on cybercrime, 43% of attacks target small businesses while only 14% are able to prevent themselves.
With contemporary data management solutions like Mesh, SMBs have the opportunity to stay on top of the trends.

Security is essential in a situation where data is highly decentralized and distributed.

These systems should delegate authorization and authentication activities to different users, providing them with different levels of access as needed.

The following key security capabilities of Data Mesh have been identified in the first market report 2022:

  • Data privacy management in all its forms
  • Data encryption, whether at rest or in motion
  • Data masking, to effectively manage PII obfuscation
  • CCPA and GDPR compliance with other legislations
  • Identity management that covers all IAM/LDAP type services

5. Auto-configuration

IoT devices can now self-configure with automatic detection of mesh networks. It automatically calibrates new nodes and connects them to the desired network without any prior configuration.

With this feature, the network can be extended and governed easily.

6. Marketing and sales

The marketing and sales team can easily organize a 360 degree view of consumer profiles and behaviors from different platforms and systems using distributed data.

This allows them to create better targeted campaigns, CLVs (customer lifetime values), better lead scoring accuracy, and perform several other critical performance metrics.

Marketing teams use hyper-segmentation to deliver a campaign to the right customer through the right channel at the right time.

7. AI and machine learning

Intelligence and development teams can easily create data catalogs and virtual warehouses from multiple sources to power AI and machine learning models.

This gives them more information without having to collect all the data in one central location.

Teams can also use federated data prep, which enables domains to deliver reliable, quality data for data analysis workloads.

8. Loss Prevention

By implementing a data mesh in the financial industry, companies can develop faster time-to-information with reduced risk and operational costs.

This feature allows international financial institutions and organizations to analyze their data locally. This can be done in any region or country and helps identify any fraud threats without creating copies of datasets that can be transported to the central database.

Data privacy management allows companies to protect their customers’ data as they must comply with evolving regional data and privacy laws, such as the VCDPA.

Some Real Data Mesh Implementations

Financial services institution

In one of their blogs, Thoughtworks discussed the impact of data meshing on a financial institution‘s data process.

Since such an application processes large volumes of real-time transactional data, it is important to deliver accurate and timely feeds to analytics systems.

In this case, executives had the flexibility to quickly operationalize data and they could access domain-oriented data products.

This allowed them to ask more relevant questions and ultimately got them more reliable answers and valuable information to act on in less time.

Not only that, but the domain team was also able to use analytics data and integrate it directly into their users’ digital experience.

AWSS3

There was a sea change when AWS commoditized its storage layer and replaced it with the AWS S3 object store about 15 years ago.

Due to the affordability and ubiquity of S3 and other cloud storages, companies are now transferring their data to cloud object stores. This allows them to create data lakes where the data can optionally be analyzed in different ways.

A fashion retailer brand

Zalando, Europe’s largest online fashion retailer, has learned there’s an easy way to ensure access and availability at scale. This can be done by shifting more responsibility to the teams that initially collect this data and also have the required knowledge in the area. And also keeping all metadata information and central data governance.

Trust me, the space just isn’t enough to cover all use cases. It is a buoyant market and companies want to make the most of it.

And after? Adopt a data product mindset

There are several innovative practices for data products that merge different concepts, such as design thinking, task theory, and breaking down organizational silos that prevent cross-functional innovation. Companies, in 2022, should seize the opportunity and revamp their data management strategy with Web 3.0 in mind.

Authors biography: Yash Mehta is an internationally recognized expert in IoT, M2M and Big Data technologies. He has written a number of widely recognized articles on data science, IoT, business innovation and cognitive intelligence. His articles have been featured in the most authoritative publications and awarded as one of the most innovative and influential work in the connected technology industry by IBM and Cisco IoT departments.

Share.

About Author

Comments are closed.