"Digitale Wissensbissen": The future of data analysis – A conversation with Christian Schömmer
Introduction
Companies that are able to use their data effectively have a clear competitive advantage. But how can the huge amounts of structured and unstructured data that are generated every day be used sensibly? Especially in connection with generative AI? In our new episode, we invited Christian Schömmer, Senior Partner Manager Data Analytics at AWS, to talk about the challenges and opportunities of modern data architectures. Christian shares his experiences and provides valuable insights into the world of data lakes, data warehouses and the emerging technology of the lakehouse.
Key Points from the episode
- On the evolution of data analysis:
- “The hope that you can now do something with data that is beyond your own ability to analyze it is what drives the whole development.”
- “In the past, the first use case for enterprise search was simply ‘I'll find stuff’.” - On the challenges and opportunities of Gen AI:
- “Gen AI has generated a lot of utopian expectations, but there is still a glass ceiling when it comes to reliability.”
- “The combination of Gen AI and data storage offers a gigantic promise, but also many potential risks.” - About the differences between data warehouse, data lake and lakehouse:
- “A data warehouse is optimized for structured data and specific queries, while a data lake is more flexible and can accommodate both structured and unstructured data.”
- “The lakehouse combines the advantages of both worlds and makes it possible to use both structured and unstructured data efficiently.” - About practical implementation and barriers to entry:
- “The barrier to entry is currently lower than it has been in a long time. There is no reason not to do anything in this area.”
- “A data scientist would certainly not be a bad start, but the team size has been significantly minimized compared to ten years ago.”
Summary
In our conversation with Christian Schömmer, we delved deep into the world of modern data architectures. A central topic was the evolution of data analysis tools and the role of Gen AI. While Gen AI has raised many utopian expectations, there are still challenges in terms of reliability and integration into existing data structures.
Another focus was on the differences between data warehouses, data lakes and lakehouses. While data warehouses are optimized for structured data and specific queries, data lakes offer a more flexible solution that can accommodate both structured and unstructured data. The lakehouse combines the advantages of both worlds and enables companies to use their data more efficiently.
Christian also emphasized that the barriers to entry for modern data analysis tools are lower than ever. Companies, even small and medium-sized ones, can start using their data in a meaningful way with relatively little effort and without large teams. The availability of out-of-the-box tools and the ability to iterate make it easier to take the first steps towards becoming a data-driven organization.
Overall, the combination of Gen AI and modern data architectures offers a promising future for companies that want to use their data effectively. The risks are present, but the potential benefits far outweigh them. It is an exciting time for anyone involved in data analysis, and the possibilities are almost endless.
(Episode in German only)
