How Do You Judge an Idea?

If you are building an innovation lab or an innovation program for your organization, you will need to build an idea funnel. One of the first challenges in building an idea funnel is setting up a process to find ideas and then a way to vet the ideas. It is the latter problem, the vetting of ideas that really is a wicked hard problem.

There really is no shortage of good ideas or even great ideas. If you want to find ideas just go to a pitch fest or entrepreneurial meetup group or your local Innovation center. The supply of good ideas usually far exceeds an organization’s capacity to implement them.

The bigger problem for most organizations is trying to figure out which ideas to work on. In theory, this sounds like a simple challenge, however, in practice, the “best” ideas are often overlooked. I’ll go one step further and this is especially true if the idea is revolutionary or well ahead of its time.  The biggest challenge in building a truly innovative idea funnel is finding a way to recognize the very best ideas.

Problem #1
The best ideas are often overlooked! 

Mini-Case Study 1: Jeff Bezos & Amazon

Mini-Case Study 2: J.K. Rowling & Harry Potter

Photography Debra Hurford Brown © J.K. Rowling 2018

J.K. ROWLING’s path to publishing Harry Potter was full of roadblocks and rejections. J.K. Rowling was turned down by many publishers and it was a bit of a quirk of fate that landed her first publisher Bloomsbury. The chairman of Bloomsbury gave the first chapter of Harry Potter to his eight-year-old daughter. She loved the book and immediately asked for the next chapter. The enthusiasm of the publishers eight-year-old is what won the deal for J.K. Rowling.

Mini-Case Study Summary

The key to finding the best idea is having the skill and ability to recognize the idea in the first place. Neither Warren Buffet nor J.K. Rowling’s publisher Bloomsbury were able to recognize a revolutionary idea.  However, Bloomsbury was clever enough to rely on the good judgement of his eight-year old daughter.

In both case study some incredibly brilliant ideas were overlooked by a lot of very smart and knowledgeable people. Both Buffet/Bloomsbury had proven methodologies for picking winners in established markets, but Amazon/Harry Potter were undetectable outliers. If the people who judge your ideas are using proven risk management techniques and trying to calculate ROI, they will miss ideas with a high degree of uncertainty.

Both Amazon and the book Harry Potter were new ideas at the time they were being judged. People with highly developed skills in risk management and calculating ROI are not well suited to judging new and revolutionary ideas. Therefore, great ideas are often overlooked because they are being judged based on the wrong criteria.

Problem #2
What skills do you need to recognize the best ideas?

Paul Graham and Reid Hoffman are investors with a proven track record of spotting revolutionary ideas very early. Let’s look at the skills these have enabled these investors to consistently pick the best ideas at the very earliest stages.

Paul Graham – Founder of Y-Combinator

Photo by Dave Thomas, released under the Attribution Creative Commons license.

Paul Graham, the founder of Y-Combinator which is arguably the best incubator in the world, has been consistently picking some of the most revolutionary ideas for over a decade. Paul Graham’s early investment thesis is summed up in one of my favourite blog posts of all time; “What Microsoft is this the Altair Basic of?” Let’s take a minute to deconstruct this question. Everyone has heard of Microsoft the software company. However, most people have not heard of Altair Basic. The very first product Microsoft produced was a Basic interpreter for the Altair.    “Bill Gates wrote a version of the Beginner’s All Purpose Symbolic Instruction Code, or BASIC , programming language for the MITS Altair microcomputer. Gates, who was an undergraduate student at Harvard University, developed this language with Paul Allen, and it was the first product Microsoft sold.” Source: https://www.quora.com/What-programming-language-did-Bill-Gates-use-to-build-Windows

If you were put in a time machine and sent back to the Microsoft office when the team was little more than Bill Gates and Paul Allen, you would see a start-up whose product was a BASIC Interpreter for Altair. It would take real vision to recognize that a start-up whose only product is Altair Basic could turn into Microsoft. Gates and Allen could see the Microsoft of the future and had the vision even in those very early days.

Even Paul Graham can’t judge the idea itself. However, his investor superpower is letting the founders convince him that they have a clear vision for the idea and can see the future state. If Paul Graham had been transported back in time to interview Gates and Allen, the question he would have been trying to answer is; Is there a corporation waiting to be born from this start-up? (What Microsoft is this the Altair Basic of?). Said another way does this idea have tremendous potential?

Reid Hoffman – Founder of PayPal and LinkedIn

The second investor is Reid Hoffman. Reid has a solid track record of not only picking winners very early, but also building enduring organizations. In the Startup 10 commandments Tim Ferris interviews Reid Hoffman. When asked about “how do you judge an idea”, this is Reid’s response; HOFFMAN: So how can you tell a truly bad idea from a bad-sounding idea? How can you be sure your ugly duckling could become a swan? This is the key: You have to pay attention to the quality, not the quantity of rejections. You want to see at least a teeny minority of investors squirm. You don’t have to get them to a “yes,” but you should detect some friction, as they reason their way to a “no.”

Reid Hoffman’s response to start-up founders seeking funding for their idea implies that the secret to knowing if you have a GREAT idea is based on the quality of the rejections you receive. You want to have at least a minority of investors giving you the “squirmy no”. A Squirmy no is rejection of the idea because the investors don’t understand it. A squirmy no can be a positive indicator you are on the right path.

The other important bit in this answer is that the majority of investors will reject the idea, but a minority will give the squirmy no. The people you want on your idea funnel governance committee are those with the skill and ability to reason a squirmy no.

Some dos and don’t for building your idea funnel 

Don’t apply risk management techniques to the individual ideas in your innovation pipeline

Applying risk management techniques to your idea funnel is not recommended. When Warren Buffet declined to invest in Amazon is the perfect example of Problem #1, the best ideas are often overlooked. Warren Buffet was trying to calculate the ROI on his potential investment in Amazon. However, investing in Amazon in 1997 before the Internet had taken off would have been “outside my circle of competence” according to Buffet. If you are using a value-based formula or some form of risk management to calculate the ROI on an idea with a lot of uncertainty you will likely overlook the very best ideas. Therefore, great ideas are often overlooked because they are being judged based on the wrong criteria.

Do remove uncertainty from the ideas in your innovation pipeline

In general, you should try to remove uncertainty from your idea funnel and mitigate market risk in your product funnel. For a quick overview on the difference between risk and uncertainty check out our previous post here; Why start-ups are better innovators than large enterprise.

Don’t stack your idea governance committee with corporate executives

Calculating ROI and business case building are skills that are more suited to the product funnel and minimized on your idea funnel team. The fastest path to innovation irrelevance is paved with the good intentions of corporate executives. Having too many corporate executives (skilled at-risk management, ROI, business case building) on your idea governance committee can be the kiss of death for an innovation program.

Do build a wise crowd to judge your idea

Bloomsbury, Paul Graham and Reid Hoffman all have something in common with how they find the best ideas. They harness the wisdom of crowds and other people. J.K. Rowlings first publisher Bloomsbury had the wisdom to recognize he couldn’t see the value in Harry Potter for himself but used the judgement of his eight-year-old daughter instead. Paul Graham can’t judge the idea himself, however, he has developed the skill and ability to recognize if the founders (people with the idea) have the vision to see the future state.  Reid Hoffman advises founders to gauge the value of their idea by the quality of the rejection they receive from investors.

The lesson in all of this is that to build a good idea governance committee you need to harness the wisdom of crowds. Getting the right skills for your innovation governance committee are very qualitative in nature and much harder to judge, but here are a few examples of what skills you should be looking for in the people judging ideas in your idea funnel.

• Opportunity identification is an essential skill,
• Ability to deal with uncertainty is a valuable skill,
• ability to detect “true” vision in the idea inventor. (Perhaps rarest skill)

NOTE: There is an important distinction between vision and delusion. The ability to detect the difference … priceless!

How can you be sure your ugly duckling (idea) could become a swan? 

Here is the hard truth about judging ideas that lead to radical innovation. Being able to predict the future of the best ideas is a wicked hard problem. Even the smartest investors in the world have trouble recognizing truly great and revolutionary ideas. However, you can improve your probability of success by building a wise crowd to judge the idea and a robust process to vet ideas.

Blog post was written by Stratford Alumni Ian Graham

This article was published more than 1 year ago. Some information may no longer be current.