Elevating problem-solving from an art to a science
By A M Howcroft, assisted by ChatGPT
Effective problem-solving is not just a useful skill; it’s a critical survival tactic. In a fast-paced world, successful problem-solving is essential for both people and organizations. The number of challenges and disruptions we face on a regular basis can feel overwhelming, from pandemics through to global supply shortages, the impact of climate change, labor shortages and so much more. Given the absolute necessity of problem-solving you would imagine there might be a well-documented process to guide us, and yet problem definition is still an art-form rather than a science. Let’s explore the world of problem-solving together and see whether a more disciplined approach would help.
The Art and Science of Problem Solving
For centuries, humans have honed their problem-solving abilities through trial and error, intuition, and creativity. These skills have led to ground-breaking innovations, scientific discoveries, and solutions to some of the most intricate puzzles known to humanity. “The formulation of the problem is often more essential than its solution”, as Einstein said. While this artistic and free-flowing approach to solving problems has undoubtedly yielded remarkable results, it's far from being a systematic and repeatable process – especially for those of us not in the ‘genius’ category.
We live in an era of data abundance, Artificial Intelligence, and networked systems, which provides us with tools and resources to approach problem-solving in a more scientific manner. But what exactly does it mean to turn problem-solving into a science? It involves a shift from relying solely on intuition and creativity to incorporating structured frameworks, data-driven analysis, and systematic approaches. It's about adopting a method that can be tested, improved, and applied across various domains, making problem-solving a predictable and scalable process. This transformation is not meant to stifle creativity or the human element of problem-solving. Instead, it seeks to enhance our capabilities by providing a solid foundation upon which creativity can flourish. By embracing a more scientific approach, we can reduce the element of luck and guesswork and increase our chances of consistently finding effective solutions – especially if we begin to learn how to better utilize the new AI tools at our disposal.
Hollywood and Creative Rigor
Let’s look at a domain where structure has already been applied to a creative process. Whether you love or hate them, Hollywood movies have historically dominated the silver screen. The film script is the blueprint on which movies are made, a guide for actors, directors, and producers. It is a meticulously structured document with constraints, a specific layout, jargon and instructions, broken into very precise acts and scenes. There are high expectations on the formal content for a character arc, dialogue, theme, pacing, and consistency.
In every respect it is a formal document with exacting expectations - and yet it has consistently been a vehicle for creativity and imagination. We may argue about whether some movie franchises have become formulaic and predictable, but this is more to do with the commissioning and selection process, rather than the screenplay structure. Art and science have successfully combined in the film industry to provide a place where the two approaches can support each other and prosper. Similar combinations exist in music, architecture, and other domains, but what about business?
If Software Engineers Built Bridges
Modern organizations often rise and fall based on the software applications they have bought or built. There is a possibly apocryphal quote from James Martin the gist of which is that “If software programmers built bridges, I would never drive over one”. James was a British-American author and futurist, famous for his book Information Engineering which pioneered a systematic, rigorous approach to writing software. It is no accident that we now have software engineers, rather than programmers, although as anyone who writes code will tell you, there is still a huge place for creativity in the process of building an application. However, the application of frameworks, structure, and methodology, resulted in huge leaps forward in both productivity and quality. Anyone who doubts that might look back at some old software programs littered with the ‘GOTO’ statement, resulting in what was frequently called spaghetti code. It was hard to read, even harder to maintain. Modern high-level languages rarely permit the GOTO command, and for those that still do its use is heavily discouraged. The amazing applications we use today are built on the foundational work of Martin and others, in putting engineering principles into practice for software development.
A Timeline of Engineering
The shift of software coding from an art to a science took place in the 1970’s and 1980’s, but we have seen several other functional areas move in this direction. Michael Hammer and James Champy caused a huge transformation in the way people viewed their business processes in the 1990’s with the release of the best-selling book Reengineering the Corporation. Note the use of the word engineering, again. I was part of the team that brought Hammer to the UK for his first ever conference, and I can tell you the event was bigger than a rock concert – with a host of CEOs present from all the UK’s top organizations. There were some huge improvements built on the back of their ideas, at companies such as Ford, GE, Citibank, Dell, American Express, IBM, Procter & Gamble, and McDonalds to name just a few. The techniques evolved over time, and you can still see the impact of the core concepts at organizations such as Amazon, and how they review their supply chain.
The last decade or so has seen the rise of cloud and Software-as-a-Service (SaaS). One way to look at these technologies is to see the outsourcing of infrastructure layers to dedicated engineering specialists, who provide a structured way to manage complex aspects of IT that used to be managed by internal specialists with a more ‘art’ based approach. Those internal IT experts relied on their knowledge, intuition, experience, and frequently a healthy dose of luck to keep things working, and it paid to have a good relationship with them. Getting a fix for your computer could take a long time (and still does in many organizations who have not yet moved to the cloud!). We can see a recurring pattern emerging here.
Summary
Let’s recap the key points:
Problem solving has historically been an art-form.
Industries such as film-making have demonstrated that a rigorous, more scientific-based approach can help creativity flourish, while streamlining processes.
Business has been slower to adopt this philosophy, but there is a long-term trend of adding more structure & science to various layers of the organization.
Software Engineering, Business Process Engineering, and more recently cloud/SaaS have deployed engineering principles to deliver significant benefits to adopters.
Given the critical nature of problem-solving, it seems remarkable that so few organizations have taken a close look at adopting a more science-based methodology. Of course, it helps if there are some well-known approaches to choose from. Over the years, there have been a handful of approaches tested, such as fishbone diagrams and the six whys, with varying levels of success.
In the next blog post, we will investigate these methods, and see why they only met with moderate success at best. We will also discover why AI may be the key to unlocking a transformational approach to problem-solving, which could be the next big wave of organizational engineering.
Before then, let’s try a little experiment – I dare you to post a comment on LinkedIn that sums up the approach your organization most frequently takes to solve challenges: post-it notes on a whiteboard? Excel? Google Sheets? Call in the consultants? Or perhaps the well-known ostrich approach… ;)