Data Mesh learnings and challenges: a retrospective with 50 people from different organisations

17 Nov 2022 | Agile Culture


What are the Data Mesh common challenges? What are the people involved with Data Mesh saying about it?

Many people are talking about Data Mesh. But there are even more people asking for references on the topic. I have been involved in many Data Mesh engagements and I will be sharing more about it in future articles. In this short article, I share the result of a Data Mesh retrospective with 50 people from different organisations.

In October 2022 I had the pleasure to be with Zhamak Dehghani in a 20 hour Data Mesh training. It was not the first time I helped her with the Data Mesh training, and, most likely, not the last one. I will try adjusting my agenda so I can be with her on her future trainings. She has been doing it once every other month as she is super busy with many engagements. More info on the next trainings here.

There are quite a few amazing benefits for me to be part of the training: 

  • listen again and again about Data Mesh from its creator,
  • share a little on my experience with Data Mesh on-boarding and inceptions (very little as her training is already full of amazing contents; at least, I share a few hints and links to help the participants)
  • listen to the avid participants as they share conversations, questions, challenges, learnings and valuable tips about Data Mesh in their organisations.

At the last training, I did a quick retrospective for the participants. The result of the retrospective reveals important information for all of the training participants. But I realised these revelations are useful for many people getting started in Data Mesh. So I decided to share the results in this post.

I used two activities to gather the information I am sharing with you. They are:

  • Slider on the Data Mesh principles , and
  • Anchors and engine for data mesh in your organisation

Slider on the Data Mesh principles

I started by asking the group to have a quick recap about all the teachings Zhamak shared on the four principles of Data Mesh: Domain Ownership, Data as a Product, Self-serve Data Platform and Federated Computational Governance

Then I asked each participant to vote on the mostly challenging principle for the Data Mesh transformation in their organisation.

The image below is the result for the voting session.

Domain ownership and Federated computational governance are considered to be the most challenging principles. Then, Data as a product and Self serve data platform are considered to be challenging, but a little less challenging.

The result from this group (about 50 people from different organisations) is very representative. I have seen many groups going through the same activity. In general, people are having more challenges in Domain ownership and Federated computational governance than Data as a product and Self serve data platform.

One very important aspect: this activity is only for raising awareness and starting important conversations. These principles are not to be worked on in isolation.

Anchors and engine for Data Mesh in your organisation

I asked two very straight-forward questions: In the context of Data Mesh, What are the things pushing you forward? What are the things holding you back?

Below is the image, result of the Data Mesh Anchor and Engine activity.

Please note the colour coding for each of the Data Mesh principles (same color used in the previous image activity).

Below is the text from all the notes.

Engine, the things moving you forward:

  • Clear separation of the concerns, ownership, responsibilities
  • Perception of the data from different perspective, multiple ways of use cases
  • There is a lot of technical challenges that engineers will enjoy 
  • Creation of pizza teams
  • Seeing data as value and product
  • Transformation to product teams
  • Flexibility, scalability
  • Setting standards and SLO’s on data will be very happily accepted
  • Having teams organised in Value Streams, or something similar and already have DevOps maturity to enable automation.
  • Shared responsibility over common goal against centralised committees and processes

Anchor, the things holding you back:

  • In our landscape the distance from operational domain to data domain is very big
  • Organisational Mindset and culture
  • Getting a common understanding of domains in the organisation
  • No ready-made tools to choose yet, building complete infrastructure will take huge efforts.
  • Convince org to change when not organised around domain today
  • Cross company initiative against Data team activity
  • Required mind shift in business regarding ownership
  • Defining  the products
  • No familiarity, nowhere to show a working example
  • Create Data Products that are valuable on their own
  • I have to sell it without complicated words..too many new words in this workshop
  • Centralised thinking of existing data teams
  • No existing products yet
  • DevOps maturity
  • How to justify required setup / resources 
  • Maturity of data platform, mastering of the technologies
  • Organisational politics & mindset
  • Expectation of centralised governance and approaches which is not happening
  • Data office not in line with decentralisation of governance

This short article has no intention to tell you how to handle each of these things holding you back, or to share how to take advantage of the forces pulling you forward. My main intention is to share with you how other people pursuing Data mesh are feeling.

As Zhamak says so clearly:

“Data M;esh is in its infancy. Please stay open to and aware of many advances in this area. More important than any specific practice or tool is to adhere to the Data Mesh principles.”

Paulo Caroli

Paulo Caroli is the author of the best-selling book “Lean Inception: How to Align People and Build the Right Product” (the first on a series of books about Lean Strategy and Delivery). He's also the creator of FunRetrospectives.com , a site and book about retrospectives, futurospectives and team building activities. Caroli writes on this blog frequently. Receive the next post in your email. Sign up here .
Assumption Mapping Activity

Assumption Mapping Activity

Assumption mapping is a powerful activity that equips teams to evaluate their assumptions based on their significance and supporting evidence. By following a structured process, teams can clarify their assumptions, enabling well-informed decisions, mitigating risks, and bolstering the likelihood of success in their endeavors.

read more
Design Ahead: Harmonizing Design and Development Timelines for Optimal Results

Design Ahead: Harmonizing Design and Development Timelines for Optimal Results

This article explores the strategic integration of the Design Ahead approach and dual track development methodologies in the realm of software development. By delving into the nuances of design, concurrent workflows, and seamless collaboration, it offers insights into orchestrating a harmonious symphony between design and implementation. From refining user experiences to developing new features, this exploration illuminates the path for organizations seeking to excel in the ever-evolving landscape of software delivery.

read more
Working Backwards – Press Release Template

Working Backwards – Press Release Template

This comprehensive Press Release template provides a structured framework for implementing the innovative Working Backwards approach, a strategy favored by industry leaders like Amazon. By starting with the end customer experience and working backward, this method offers a clear pathway for product development and planning. The template’s sections guide teams in articulating the product/initiative/solution’s significance, key features, and benefits.  Whether for remote collaboration or in-person workshops, this tool equips teams with a collaborative framework to craft impactful press releases.

read more

Pin It on Pinterest