This site is part of the Informa Connect Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 3099067.

Enlighten, inspire, and learn to build a culture of innovation that disrupts your industry.

The Single Most Important KPI for Building Innovation Muscle

Share this article

By: Tim Woods, VP of Marketing & Product Strategy, Hype Innovation

This post was originally published on Hype Innovation's Blog
The best literature on innovation
all points to the same thing: innovation is highly uncertain, and therefore the
best approach is to experiment and prototype, iterating until you find the right
product/market fit, and conduct this iteration with the diligence of the
scientific method. The advice is so consistent, yet when we look at innovation
metrics, there is rarely any kind of KPI measurements around the details of
experimenting and prototyping. Let's look at why it's so important, and what a
KPI might look like.

Fire bullets, then cannonballs (Jim Collins)

In Great by Choice, Jim Collins describes the idea
of firing bullets, then cannonballs. Very rarely is an innovation a big single
shot, but rather it's the outcome of many smaller steps that helped to
calibrate towards that right outcome. When Apple moved into retail stores, they
started with one shop, prototyping, firing bullets to see what hits; when they
got it right they expanded, and became the most
square feet retailer in the world.
What makes a bullet?
'  A bullet is low cost. The size of the
enterprise determines what low cost means, a cannonball for a $1 million
enterprise might be a bullet for a $1 billion enterprise.
'  A bullet is low risk. That doesn't mean low
probability of success, but it means minimal consequences if the bullet hits
'  A bullet is low distraction. It might be
high distraction for a few individuals working with it, but for the enterprise
as a whole, it's very low distraction.
The behaviors you need to develop
to be successful:
'  Ensure you are firing enough bullets. They
should be low enough cost and risk to allow for many.
'  Resisting the temptation to fire uncalibrated
cannonballs. Innovation is not about plowing money into big bets without
the data to prove the investment.
'  Committing fully when ready: by converting
bullets into cannonballs once you have the empirical validation.

Disciplined Experimentation (Govindarajan and Trimble)

In The Other Side of Innovation, authors
Govindarajan and Trimble go to great lengths to describe how to organize
yourself for innovation, and how generating ideas is the least difficult part.
Execution on ideas is where you need to focus, and the fundamental ingredient
is the ability to experiment in a disciplined way.
'In managing their ongoing
operations, companies strive for performance discipline. For the innovation
initiatives, however, they ought to strive for discipline of a different form:
disciplined experimentation. Indeed, all innovation initiatives, regardless of
size, duration, or purpose, are projects with uncertain outcomes. They are
For an idea to mature, it must
start with an experiment. Create a plan, with a scorecard, which explains the
assumptions, and the data points you will measure. Formalize a clear
hypothesis, in very simple terms, which states what you think will happen. Then
find ways to spend a little, but learn a lot. Keep assessing the plan as you
go, and allow formal revisions to the predictions you made.
'...the scientific method is the
innovator's indispensable friend ' when running innovation initiatives,
businesspeople need to behave more like scientists.'

Source: The Other Side of Innovation

The 5x5 Methodology (Michael Schrage)

In Michael Schrage's excellent
book, The Innovator's Hypothesis, he
believes experimentation is the difference between those that innovate, and
those that don't.
'Look carefully at the history of
technology, entrepreneurship, or business innovation. A persistent pattern
emerges. Successful innovators talk about ideas, but they invest their time,
money, and ingenuity in expressive experimentation. Their competitive success
comes from getting more value faster from expressive experimentation.'
Ultimately, what you're seeking
is insight, so that you can get closer and closer to an innovation; you want to
buy a dollar's worth of innovation insight for 50 cents, or 20 cents, or less.
Fast and cheap, but an extremely high return on validated learning. His 5x5
methodology is designed to achieve this:
'Give a diverse team of 5 people
no more than 5 days to come up with a portfolio of 5 business experiments that
cost no more than $5,000 (each) and take no longer than 5 weeks to run.'
The results are then presented to
a senior management board, and the best ideas get more funding to continue. The
goal is to build a portfolio of experiments, but the problem is that most
organizations just don't know how to design or manage a portfolio of business
experiments. The 5x5 methodology quickly gets you up and running.
Schrage provides extensive detail
on how to run and measure the 5x5 method, along with compelling examples - it
makes for tantalizing reading. It's very easily attainable, and has enormous
potential to build innovation muscle. But as he notes, big organizations find
it incredibly hard to instill as a discipline:
'Creating simple, compelling, and
readily testable business hypotheses is managerially unfamiliar, uncomfortable,
and unrewarding. So managers avoid them.'

The DNA of an Innovator (Dyer, Gregersen, Christensen)

In The Innovator's DNA, business gurus Jeff Dyer, Hal
Gregersen and Clayton Christensen, analyze what makes an innovator. They define
5 skills that you need to master to become a disruptive innovator: Associating,
Questioning, Observing, Networking, and .... Experimenting.
In their analysis they found that
experimenting skills were significantly higher in innovators of all types, not
just the start-up entrepreneurs, but also process innovators at large

The Innovator's DNA
One of the many example
innovators in the study is Jess Bezos, who puts the ability to experiment at
scale at the heart of Amazon's innovation strategy:
'Bezos's experience has taught
him that experimenting is so critical to innovation that he has tried to
institutionalize it at Amazon. 'Experiments are key to innovation because they
rarely turn out as you expect, and you learn so much ' I encourage our employees
to go down blind alleys and experiment. We've tried to reduce the cost of doing
experiments so that we can do more of them. If you can increase the number of
experiments you try from a hundred to a thousand, you dramatically increase the
number of innovations you produce.'
To learn more about the five
skills, and how to develop them, check this out.

Build, Measure, Learn (Eric Ries)

When The Lean Startup was published, it hit a home run. It nailed
exactly the ethos and method for innovating in the 21st Century. Large
corporates scrambled to figure out how to adapt it to their environment, and
lean startup consultants appeared everywhere. The basic ideas are so simple and
so effective though.
Generate a hypothesis, define a
way to test it (build), define how you can strictly monitor it (measure), and
define how to validate the results (learn). Then after each short cycle through
that process, decide whether to persevere (do another loop with an adjusted
hypothesis and new build), or pivot (move on to another hypothesis). It works
for small incremental changes, and it works for whole product launches.
'The essential lesson is not that
everyone should be shipping fifty times per day but that by reducing batch
size, we can get through the Build-Measure-Learn feedback loop more quickly
than our competitors can. The ability to learn faster from customers is the
essential competitive advantage that startups must possess.'
'The Lean Startup works only if
we are able to build an organization as adaptable and fast as the challenges it
faces. This requires tackling the human challenges inherent in this new way of
To see the 10 methods of the Lean
Startup, check this out.

This is just a quick selection of
some of the excellent literature out there, but many more exist which similarly
stress the importance of experimentation, including The Innovator's MethodLittle Bets, and anything by Tom Peters (ready,
fire, aim!
 And of
course his 'bias for action' from In Search of Excellence).

Measuring Experimentation

So if experimentation is so
widely regarded as the basis for innovation, why do we rarely see KPIs in place
to track it? After all, what gets measured, gets done, right?
In the HYPE process, this is how
it can be done. Let's assume you're using Idea Campaigns to generate high
quality ideas, which target a known problem, with a sponsor who backs (and
funds) selected ideas. Ideas are rarely ready for implementation right away,
they need to be developed, or combined with other ideas, and then tested. This
is where Concepts come in:

Enterprise process for full-lifecycle innovation management
With Concepts you can create
customized processes and forms for managing the iteration and the evaluation
steps. In the example below you can see a form included to track the iterations
of a build, measure, learn cycle (Lean Startup). You capture the information
for each iteration, then flag whether to pivot or persevere.

Concept feature allows for configurable process steps and data capture
The two critical KPIs are then:
Number of experiments in my innovation
portfolio (number of Concepts created)
Number of iterations through those experiments
(cycles through the build, measure, learn cycle)
Seeing these numbers go up over
time, gives you a direct pulse on whether you are building innovation muscle.
Providing you are following the scientific method, and seeking out validated
learning, not just experimenting with no rigor.
Supporting these metrics, you'll
also want to know more granular details about the portfolio, such as:
Number of experiments being run per
"strategic innovation area,"
 to show you the health of each major area you're
Number of experiments by their status (pivot, persevere, or whatever other status
you will determine for experiments).
Number of experiments per organizational unit
/ business division / department
, to help you understand what areas of the business are
"getting it," or struggling. Experimentation is not the exclusive
domain of the innovation department, it should be part of the culture across
the entire organization.
'  You may also want to measure the time
to cycle through an iteration
. As Eric Ries notes, the question
is how fast can you get through each loop, so that you're learning faster and
moving closer to the right outcomes?
We're still spending a lot of
time thinking about number of ideas, number of participants, and other
rudimentary KPIs. These are nice to know, but they don't tell you anything
about your progress towards a more innovation-capable organization. The
back-end is where the true health of your program is measured, and for that, we
must measure and promote the experiments. As Michael Schrage nicely puts it in
The Innovator's Hypothesis:
'Good ideas have nothing to do
with good implementations ' Implementation - not the idea - is the superior
unit of analysis for assessing value creation. How organizations enact ideas -
not the ideas themselves - is the soul and substance of innovation. More often
than not, implementation ends up redefining both the boundaries and the essence
of the original idea.'
Are you tracking how you
experiment? I'd love to know.
About the Author: Tim is VP of Marketing and Product Strategy for HYPE Innovation. With a background in software development, Tim has worked in the Innovation Management space for over a decade, working on software and services solutions that support the innovation initiatives of some of the worlds largest organisations. Tim is passionate about how software can drive innovation results faster, better and more reliably.

Share this article

Upcoming event

FEI USA: Front End of Innovation

20 - 21 Sep 2021, Boston
FEI: Front End of Innovation
Go to site