I would not give a fig for the simplicity this side of complexity, but I would give my life for the simplicity on the other side of complexity.
Oliver Wendall Holmes
You probably found this article because either 1) you are considering a job offer from Spiff, 2) you recently started at Spiff and are learning about our culture, or 3) you are interested in creating your own culture. This continues a series of articles that we have published about Spiff’s culture.
Building software is hard. It’s hard because it’s complex. We live in a VUCA world. We believe managing complexity has become one of the biggest sources of competitive advantage.
One of the most important concepts I’ve learned in my career is the importance of “simplicity on the far side of complexity.” We call this “elegant thinking” at Spiff. You will likely hear these phrases frequently at Spiff.
A related concept is the importance of creative thinking. Breakthrough ideas can occur at any level of an organization and at any level of a problem. Cultivating a culture of elegant and creative thinking can do more to create a culture of success and growth than almost anything else.
This article talks about how we foster elegant and creative thinking at Spiff. In this post, we’ll talk about:
- Elegant Thinking
- Types of Solutions to Problems
- Creative Thinking
- Defining Creativity
- Brainstorming vs “Noticing”
- Permutational Thinking
- Blind Variation Selective Retention (BVSR)
- Conceptual Blending
- Effective Surprise
- The Novelty-Utility Matrix
- Innovative Thinking
Elegant Thinking
If you’re like us, you may think of a stuffy aristocratic gentleman when you hear the phrase “elegant thinking.”
But we mean something different. We’re talking about a kind of creative thinking that can solve hard problems.
Hard problems occur beyond physics and math. They exist in interpersonal relationships, science, storytelling, and in building software.
We believe that finding solutions to these hard problems requires elegant thinking.
Types of Solutions to Problems
In the 1970s and 1980s, management consultants used a puzzle to foster creative thinking with clients.
The idea is to think of a way to connect all of the dots using four straight lines or fewer, without lifting the pen and without tracing the same line more than once.
Before, scrolling down, see if you can do it.
Here is the answer. There are several others but they are all rotations of this shape.
Once you get the “aha” experience of realizing that you can extend a line beyond the frame of the dots, the problem becomes fairly “obvious.” This puzzle became the basis for the phrase “out of the box” thinking.
It’s this kind of creative problem solving that allows us to find simple solutions to complex problems.
You could divide solutions to all problems into 4 types:
- Complex Solution, Shallow Understanding. These solutions are like Rube Goldberg machines. They try to solve a problem in an overly complicated way stemming from a shallow understanding of the problem. They feel overengineered.
- Complex Solution, Deep Understanding. These solutions are hard to understand but they work. Understanding these solutions is like trying to understand how Google’s AlphaGo AI beat the world Go champion. Humans often don’t have the RAM to fully comprehend them.
- Simple Solution, Shallow Understanding. Be especially careful to avoid these solutions because they mimic the next kind of solution. They are easy to understand. But they miss hidden complexities. We call these “naive” solutions.
- Simple Solution, Deep Understanding. These are the “best” solutions. They comprehend the full complexity of a problem but they are easy enough for humans to fully understand. We call these “elegant” solutions.
At Spiff, we seek elegant solutions and in some cases AI-style solutions. Most of all though, we avoid “naive” and “overcomplicated” solutions. Most of these solutions aren’t really solutions at all.
So how do you find these elegant solutions? This leads us to our next section on “creative thinking.”
Creative Thinking
As far as I can conjecture the art consists in habitually searching for the causes and meaning of everything which occurs. This implies sharp observation and requires as much knowledge as possible of the subjects investigated.
Charles Darwin on how he came up with the theory of evolution.
When someone asked Darwin how he came up with the Theory of Evolution the quote above was his reply.
We don’t have a formula for solving every problem in an elegant way. But we believe it has a lot to do with creative and disciplined thinking. And we have some general thoughts and guidelines about creative thinking that we will discuss below.
Defining Creativity
What is creativity? Most experts define creativity as producing something new and useful. So, at its core, creativity is a mixture of novelty and utility.
As you think about some creative breakthroughs you’ve had in your life you might recognize these 5 Stages of Creativity based on pioneering work by Graham Wallas:
- Preparation. Preparatory work on a problem that focuses the individual’s mind on the problem and explores the problem’s dimensions.
- Incubation. Where the problem is internalized into the unconscious mind and nothing appears externally to be happening.
- Intimation. The creative person gets a “feeling” that a solution is on its way.
- Illumination or Insight. Where the creative idea bursts forth from its preconscious processing into conscious awareness.
- Verification. Where the idea is consciously verified, elaborated, and then applied.
You’ve probably had those moments where you’re relaxed and thinking about a problem in a non-rushed way and had a sudden flood of ideas pop into your mind. We want to facilitate that kind of creativity as much as possible.
We believe preparation and incubation are particularly important. Preparation often means looking for great ideas, insights, and inspiration in areas that aren’t directly related to a particular problem.
Steve Jobs famously brought a lot of what he learned from a college calligraphy course to his experience building computers. I wonder if that experience became the foundation for the nearly $1T success that Apple has built by focusing on beauty and computation.
In 1939, James Webb Young published his amazing book on creativity: A Technique for Producing Ideas. Years later, he published a postscript in the reprint of his book where he also underscored his most important learning:
From my own further experience in advertising, government, and public affairs I find no essential points which I would modify in the idea-producing process. There is one, however, on which I would put greater emphasis. This is as to the store of general materials in the idea-producer’s reservoir.
I am convinced, however, that you gather this vicarious experience best, not when you are boning up on it for an immediate purpose, but when you are pursuing it as an end in itself.
Brainstorming versus “Noticing”
Any sufficiently advanced technology is indistinguishable from magic.
Arthur C. Clark
People often think of creative thinking as “brainstorming.” In our experience, most brainstorming sessions are ineffective. Worse, they often produce bad ideas.
I think it’s because of a phenomenon startup “philosopher” Paul Graham calls “sitcom” ideas.
Why do so many founders build things no one wants? Because they begin by trying to think of startup ideas. That m.o. is doubly dangerous: it doesn’t merely yield a few good ideas; it yields bad ideas that sound plausible enough to fool you into working on them.
At YC we call these “made-up” or “sitcom” startup ideas. Imagine one of the characters on a TV show was starting a startup. The writers would have to invent something for it to do. But coming up with good startup ideas is hard. It’s not something you can do for the asking. So (unless they got amazingly lucky) the writers would come up with an idea that sounded plausible, but was actually bad.–Paul Graham
In contrast to bad brainstorming sessions, we’ve all had “shower moments” where we are thinking about a problem in a non-hurried, indirect way and felt sudden bursts of inspiration.
Why does inspiration flow so much better in one setting versus the other? I’d guess there are subtle subconscious processes at work during the incubation period of an idea. This is why I started this section with the Arthur C. Clarke quote about magic. Ideation is only magic to us because we don’t understand it fully yet. As we understand what our subconscious mind is doing more and more, we will be able to better facilitate inspiration. That’s one of our goals here at Spiff for every team member.
Scholars are starting to understand the creative process better. From research on the topic, I have found three basic frameworks to make the creative process more explicit: BVSR, Conceptual Blending, and Effective Surprise. We add one more at Spiff, we call Permutational Thinking.
Permutational Thinking
Permutational thinking involves grouping all of the dynamic elements of a problem in an organized way. Once you have identified these, then you start to sense “the shape of the problem.”
When you understand “the shape of complexity,” you can start to think about how to simplify without over-simplifying. You can search for elegant simplicity (on the far side of complexity) not naïve simplicity (on the near side of complexity). You will know how to group certain levers, eliminate or minimize others, while still preserving and even emphasizing the solution to the problem.
Blind Variation and Selective Retention (BVSR)
We often think that brainstorming is the most basic form of ideation, but it’s actually part of a much larger concept called blind variation selective retention, or BVSR for short.
BVSR is a way to test and filter ideas to find the strongest one. This happens in nature, too, when living things go through the process of evolution. In evolution, nature selects the gene combinations that are most likely to reproduce. The ability to reproduce is competitive, leading to an increase of the fittest genes in surviving populations.
Because gene replication involves randomness, new possibilities are constantly being tested, and animals with new, successful gene combinations are more likely to reproduce.
NOTE: The concept of randomness followed by a filtering process is a key concept in any creative act (including brainstorming and natural evolution).
Blind variation is just another word for novelty and selective retention is just another word for usefulness. So BVSR processes lead to useful novelty aka creativity.
Conceptual Blending
Conceptual blending is the idea of mixing two (or more) frameworks. Analogical reasoning, structure mapping (in AI), and bisociation are all examples.
Let’s look at Arthur Koestler’s theory of bisociation as an example of conceptual blending.
The basic idea of bisociation is that creativity comes from the sudden and unexpected merger of two (or more) frames of reference previously considered incompatible. Koestler describes it like this:
The pattern underlying [the creative act] is the perceiving of a situation or idea, L, in two self-consistent but habitually incompatible frames of reference, M1 and M2. The event L, in which the two intersect, is made to vibrate simultaneously on two different wavelengths, as it were. While this unusual situation lasts, L is not merely linked to one associative context, but bisociated with two.
I have coined the term ‘bisociation’ in order to make a distinction between the routine skills of thinking on a single ‘plane,’ as it were, and the creative act, which … always operates on more than one plane. The former can be called single-minded, the latter double-minded, transitory state of unstable equilibrium where the balance of both emotion and thought is disturbed.
The basic idea of bisociation is mixing conceptual frameworks and finding important similarities and differences between them.
One of my favorite classes at Harvard B-School was one where I learned about the Bass Diffusion Model. It’s a mathematical model for the virality of a product and network effects. It was based on transferring ideas from the study of epidemics to business products.
For example, who knew that modeling Facebook adoption is very similar to modeling the spread of infectious disease in a population? The epidemic model that Bass relies on in turn borrowed heavily from a chemical model called the Law of Mass Action.
Here’s another example. NPR had a piece last night about Aretha Franklin’s famous song “Respect.” It turns out she stole huge elements of it from Otis Redding’s song “Respect.” Aretha’s big innovation was that she re-interpreted Otis’ song from a woman’s perspective–giving it it’s timeless feminist appeal.
So, not surprisingly, mixing previously unrelated frameworks can lead to useful novelty as well.
Effective Surprise
How extremely stupid not to have thought of that.
Thomas Huxley on being shown Darwin’s theory of evolution
With regards to creativity, Jerome Bruner states, “An act that produces effective surprise [is] the hallmark of the creative enterprise.”
We have fallen in love with the idea that at its core, all creativity is just “effective surprise.” When you look for it, you can find effective surprise in nearly every creative activity–jokes, art, science, and business–especially startups. If you can find a way to surprise people with a groundbreaking product or service, you should have a good chance at success.
In many ways, this is probably why young people create so many of the biggest startup successes. Young people are experiencing the world in a very different way than the old guard who is running it. So their business ideas are often surprising to the status quo. In fact, any minority group that frequently gets ignored by those currently running the world can often produce effective surprise.
At first glance, the previous two major frameworks we discussed–brainstorming and conceptual blending–may seem to have nothing in common, but if you look closer you can see that a common link is effective surprise.
Instead of thinking about discovering novelty, I’ve found it useful to instead think about creating effective surprise.
All of this has emphasized two major aspects of creativity: novelty and usefulness. Let’s look at how we can come up with more novel and useful ideas.
The Novelty-Utility Matrix
It’s probably an oversimplification, but we can divide ideas into four basic categories:
In reality, nothing is as cut-and-dried as this chart. Some ideas hover near the boundaries and some ideas move from one category to another; they seem strange at first but become paradigm-shifting. Others seem incremental at first but become revolutionary.
We can also use the idea matrix to distinguish between two different types of startup ideation: those that focus more on utility and those that focus on novelty.
Sarah Kaplan’s new research on patent innovation shows that highly novel patents have the highest economic value. This view would seem to confirm the paramount importance of novelty.
But the research shows that these kinds of innovations are exceedingly rare. Fewer than 1% of all patents fall into this category.
This again seems to validate our hypothesis that focusing on novelty is like swinging for the fences. If you connect with the ball, you’ll probably hit a homer. But trying to swing with that much force decreases your chances of connecting with the ball at all.
Kaplan also highlights two kinds of brainstorming:
Basically, in research on innovation, there has been an idea that breakthrough innovations, those with the highest impact, are ones that are produced through … what we call “re-combination” or combination processes of distant and diverse knowledge…
For practitioners, the important thing is to remember that not all creativity happens through combination processes. We have this idea out in the world that … bringing distant and diverse knowledge together is the way to get creative insights, and that’s certainly true. However, what we discovered is that there’s an equally important process of the deep dive, of deep knowledge in one domain.
She says this about deep-dive, expertise-driven brainstorming:
Brainstorming is not enough without deep knowledge development that you would need in a particular domain to understand what the issues are so that you can break away from existing ways of thinking.
Kaplan’s research invites us to consider two major modes of brainstorming. Brainstorming that focuses more on novelty (re-combination) and brainstorming that focuses more on utility (deep domain knowledge).
It turns out there are some big benefits to utility-driven brainstorming and some big risks associated with novelty-driven brainstorming. We can see this in our idea matrix.
If we limited our brainstorming only to “highly useful” ideas, we would only come up with two kinds of ideas:
- Paradigm-shifting / revolutionary ideas
- Incremental / evolutionary ideas
We might luck out and find a paradigm-shifting idea, but chances aren’t in our favor. Remember, these kinds of innovations are very rare. But we’d still have a great shot at finding some incremental / evolutionary ideas, and those can be very valuable!
If we limited our brainstorming to only ideas that are “highly novel,” we would come up with these two kinds of ideas:
- Paradigm-shifting / revolutionary ideas
- Strange ideas
Again, we might luck out and find a paradigm-shifting idea. It’s no coincidence that as Seneca said “no great genius has ever existed without some touch of madness.” But chances are, we’ll end up with a strange idea.
Some fields of human endeavor–like art–often seem to value novelty for novelty’s sake. We even have a phrase for this: “novelty act.” But strange ideas that aren’t particularly useful typically fail.
In fact, strange ideas are basically the exact ones the PG Warning refers to–they are often bad ideas masquerading as good ones.
I think most fast-paced, forced ideation is novelty-driven ideation. This is what we typically think of when we think about “brainstorming.”
On the other hand, utility-driven ideation is quite different. Remember, BVSR teaches us that determining the usefulness of an idea is just “selective retention.” Knowing what to throw out and what to retain is often much harder than it seems.
It turns out, the key difference between traditional “brainstorming” and utility-driven ideation is expertise.
Innovative Thinking and Expertise
Let’s distinguish traditional brainstorming–which often overemphasizes novelty–from an approach I’ll call “innovative thinking,” which focuses more on usefulness.
Many people believe the best startup ideas spring naturally from expertise.
Arthur Koestler put it this way: “Minor, subjective [creative] processes do occur on all levels, and are the main vehicle of untutored learning. But objective novelty comes into being only when subjective originality operates on the highest level of the hierarchies of existing knowledge.”
I have often felt that “recognizing” great ideas is one of the central challenges of creativity. For example, I recently googled the term “best painting ever.” Here’s what I saw.
With apologies to my art historian wife, I was surprised to see a painting called Las Meninas by Velazquez at the top of the list.
At first blush, I couldn’t understand why it was in the company of other more obvious paintings like the Mona Lisa, Starry Night, The Scream, etc.
Reading on, I found a critic who called Las Meninas “one of the most overpowering paintings in western art history” while noting “It doesn’t seem so mysterious at first glance.”
At first, it just seemed like a very nice, traditional, albeit somewhat crowded, painting. On closer review, the painting reveals layers and layers of recursive questions about reality, illusion, art, and the artist.
It served as a nice reminder that sometimes a great idea is staring you in the face but you don’t have the expertise to recognize it.
Experts are far more likely to recognize great ideas in their industry. They are also more likely to know what has been tried before, what new ideas have the most potential, and how best to execute an idea.
But there are drawbacks to expertise as well. Sometimes it can limit ideation. In certain circumstances, experts disregard novel ideas that they see as too unconventional or that disagree with broadly-accepted truths of the day. (Just think about expert predictions for the 2016 U.S. presidential race.) Experts sometimes limit themselves by not thinking outside of the box of their own expertise.
Conclusion
Software allows humans to handle increasing complexity. You might even define technology as any tool that allows humans to handle additional complexity. So when we hire and promote here at Spiff we are looking for who can help us find elegant and creative solutions to complex problems
