#212 Reflections on Building a Data Mesh Platform from Scratch - Interview w/ Jyotshna Karki

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/Please Rate and Review us on your podcast app of choice!If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts (most interviews from #32 on) hereProvided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.Jyotshna's LinkedIn: https://www.linkedin.com/in/jyotshna-karki-81a24038/In this episode, Scott interviewed Jyotshna Karki, Data Engineer at Novo Nordisk. To be clear, she was only representing her own views on the episode.Some key takeaways/thoughts from Jyotshna's point of view:In data, especially in data engineering, people need to be curious. There are so many new innovations that may really be majorly beneficial. Look to try out more approaches and technologies.You can have happy data producers and consumers with a centralized data lake setup and still have data mesh be the right evolution. In the long-run, at scale, it isn't efficient to have a centralized data team coordinating all data use cases.For many domain teams, the centralized data processing and storage can be a black box. Data goes in, it gets transformed and stored by the central team and then served out. This can create a high dependency on experts and technology.?Controversial?: If your domain team consists of their own data engineers and data scientist with domain knowledge experts to manage their own data products, it's okay to work with multiple teams at the start of a mesh journey. Scott note: if you don't need to drive buy-in and your org can do this, I don't see it as a major risk. But probably at most a few hundred (tens maybe even) organizations are like this worldwide.Don't try to enable every tool as part of your platform. You should focus and create a good experience on the most widely used tools rather than trying to support every tool available out there.?Controversial?: Probably don't try to automate processes at the proof of...