Partnership with Databricks

"Working with data just got easier”

Norderstedt, October 9, 2023 - LHIND partner Databricks combines the best of data warehouses and data lakes. Using an open and unified platform, it enables innovative ways of analyzing data and using AI. In this interview, Dael Williamson, EMEA CTO for Databricks, and Michael Koch, Director Artificial Intelligence & Data Analytics at Lufthansa Industry Solutions (LHIND), explain how Databricks and LHIND work together to provide customers with useful tools and systems for data and information analysis.

What are the challenges that companies are currently facing in their digital transformation?

Dael: Many companies lack vision. They are too static and neglect unpredictable factors. My advice is to experiment more and make better use of data! Data and AI offer enormous potential for optimizing operational processes. It's time to unlock this treasure trove of data and turn it into a competitive advantage. At Databricks, for example, we have a clear shared purpose that gives us an innovation advantage. Every year, we come together to determine the direction we want to go. And if that direction occasionally changes due to market influences or new technologies, we can always navigate flexibly. This is also true for many other data-driven companies.

Michael: Overall, I see companies in Germany and the entire DACH region as well positioned to successfully implement digital transformation due to their thorough approach to technology and their growing interest in AI. However, we often get in our own way in Germany. In my experience, German engineers have one remarkable trait: they value thorough preparation. Before implementing new technologies or solutions, they want to identify and address potential problems and challenges in advance. This is undoubtedly a valuable trait, but it can sometimes be challenging in an area like digital transformation, which is characterized by a high degree of uncertainty. Progress often requires a willingness to take risks and break new ground. That's why it's important to build trust in new technologies, and AI in particular.

My advice is to experiment more and make better use of data! Data and AI offer enormous potential for optimizing operational processes.

Dael Williamson, EMEA CTO for Databricks

How can this be achieved?

Michael: The right balance between thorough preparation, a spirit of innovation and appropriate regulation helps to drive digital transformation forward safely and sustainably. For example, the new EU AI Act on the responsible use of AI is helping to increase acceptance of the new technology. The regulation offers Germany and the entire EU the opportunity to promote innovation in the field of AI. At the same time, it ensures that AI systems are used ethically and responsibly. In addition, innovations from the business community are helping to raise public awareness of the technology. ChatGPT, in particular, has recently been instrumental in breaking down prejudices against AI.

Dael: The more people understand how these technologies work, the more open-minded they will be. At the same time, they will realize that these technologies are not yet autonomous beings, but can only simulate human-like behavior and communication. The EU AI law emphasizes important principles such as truth, transparency, accountability and traceability. These principles build trust in AI systems. Europe is well positioned for open platforms and models that can significantly lower barriers to entry. This allows us to foster innovation and progress without stifling the development of the technology.

Progress often requires a willingness to take risks and break new ground. That's why it's important to build trust in new technologies, and AI in particular.

Michael Koch, Director Artificial Intelligence & Data Analytics at Lufthansa Industry Solutions

What is special about Databricks' solutions?

Dael: We provide our partners, such as Lufthansa Industry Solutions, with a platform on which they can develop solutions to customer problems. We do this by providing users with a set of intuitive tools to simplify processes and leverage new technologies. The solutions developed can then be sold directly to customers or distributed as value-added offerings. In this way, we enable innovative partnerships that deliver results quickly. Compared to traditional approaches that often take months, we create new solutions quickly and significantly shorten the time frame. Our approach is like building with Lego bricks: with Databricks, you don't just buy a set of tools, you buy a complete methodology for building a village. This moves us toward an ecosystem model where partnerships go beyond traditional customer-supplier or vendor relationships. Instead of getting lost in transactional relationships, we become part of the team. We embrace this shift because it represents a fundamental change in mindset.

Michael: The development of platforms like Databricks has made working with data even easier. What's fascinating is that these advances are not just accessible to experts who have been in the industry for a long time, but to more and more people who want to use data. This is an exciting and right development. After all, it's important to give more people access to data.

Our approach is like building with Lego bricks: with Databricks, you don't just buy a set of tools, you buy a complete methodology for building a village.

Dael Williamson, EMEA CTO for Databricks

Databricks has also developed industry-specific offerings. Why is this important?

Dael: It allows us to easily tailor our platform to industry-specific needs. We do this by categorizing companies based on common characteristics and developing accelerators to address industry-specific challenges. Our offerings are like starter kits that help companies get started. They don't have to start from scratch but can draw on best practices from different industries. This accelerates innovation and makes it easier to integrate the technology into the business.

Michael: So, we don't have to start from scratch all the time, we can take well-prepared platforms and adapt them to our needs. That really speeds up development. We also follow a "eat your own dog food" approach. This means that we use these tools in our own company for our own processes and experiment with them to show our customers new possibilities.

What is the roadmap for the future?

Dael: Our goal is to drive the integration of technology and business and increase the speed of innovation. To achieve this, we want to bridge the gap between users and developers by leveraging technologies such as AI and deep learning. Along the way, we have also acquired external companies to bring in talent and innovative solutions. These strategic moves, along with expanded partnerships with companies like SAP, position us as a leading provider of tools and support for the data and AI world of the future. We are actively leveraging our own language models, including technologies like Copilot, to accelerate our teams. With over 1,500 people in product development and engineering, we are focused on reducing the cognitive load of distractions to drive innovation.

Michael: Breaking boundaries is a big theme! We will continue to pursue future themes, and the partnership with Databricks is one of them. I always emphasize that Databricks is a platform for generative AI, not a fad. Generative AI is a revolution, and we are just getting started. We are already investing in areas that will really take off in the next year or two. That's what we're putting our whole team behind, and that's what we want to drive.

To me, Databricks is synonymous with innovation. When I started looking for solutions for our customers ten years ago, Databricks already had a new technology for in-memory computing and big data. It has stayed with us over the years, and exciting new topics have been added all the time. Now we're looking at the next step, and again I feel that Databricks knows exactly where the business is going.

Dael, you were a biochemist before you entered the IT world. How did that happen?

Dael: My scientific background shapes my approach to problem solving. I worked extensively in proteomics, not only in the lab but also as a data scientist. My goal was to purify proteins and understand their structure. This required the use of AI techniques and the processing of large amounts of data. In my current work, we apply essentially the same principles. For example, we use AI to make logistics networks more dynamic and supply chains more resilient. To do this, we look for weak signals in big data, similar to protein research. This combination of determinism and probability actually gives companies a kind of superpower to look into the future. We detect signals and link them to internal information. That way, we are better prepared for the unexpected.

About Lufthansa Industry Solutions

Lufthansa Industry Solutions is a service provider for IT consulting and system integration. This Lufthansa subsidiary helps its clients with the digital transformation of their companies. Its customer base includes companies both within and outside the Lufthansa Group, as well as more than 300 companies in various lines of business. The company is based in Norderstedt and employs more than 2,400 members of staff at several branch offices in Germany, Albania, Switzerland and the USA.