Innovation: reduce cycle-time of the built-measure-learn loop

My learnings/book review from:
Eric Ries: “The Lean Startup” Penguin, London 2011.

For me, one of the most fascinating books about innovation under conditions of extreme uncertainty. Every day you delay to read this book you risk to waste money.

Most of the innovation books are not concrete enough but this book is full of concrete recommendations – a must-read book for IT-based innovators, and not only for startups but also for large enterprises.

Key Messages:

  • The worst thing regarding productivity you can do is to build the wrong product – a product which nobody wants. The Problem: most of the innovators build the wrong product and fail. Therefore:
  • The key success criteria of innovation (under extreme uncertainty):
    make the cycle-time of the build-measure-learn loop as short as possible.
  • The effort that is not absolutely necessary for learning what the customers want, can be eliminated.
  • Validated Learning: based on real data with real customers. Today we need not to aks “can the products be built?” but “should the product be built?”.
  • Investors (and you) want to know if the value hypothesis and the growth hypothesis are working:
    • Value Hypothesis: does the product really deliver value to the customer? Examples for metrics: retention rate.
    • Growth Hypothesis: how fast will new customers discover the product? Examples for metrics: referral rate.
  • Minimum viable product (MVP): enables a full turn of the Build-Measure-Learn loop with a minimum of effort.
    • Concierge MVP: the CEO himself
    • manually instead of automatically
    • limited feature set (and limited quality)
  • “Vanity metrics” like number of users do not help much. It is easy to look successfully even if you don’t. Use actionable metrics.
  • Split-Test: the impact on a new feature on the strategic metrics needs to be tested before it is completely rolled out. This can be done with a split test (make the new feature available only to a part of the users and measure the difference). The effort for additional development pays off by doing the right things (because of learning more quickly).
  • Cohort analysis: e.g. measure the behavior of the new customers of the last month (and not the average of all users since the go-live as early customers have a different behavior).
  • Kanban (capacity constraint). Four boxes (one for each phase), each of them can have maximal 3 entries at a time: Backlog, In Progress, Built, Validated. As a consequence, the whole process will stop as long as the validation tasks are not done. Without validation – no learning. If validation fails the feature is removed.
  • Problem of Business Plans: they are based on a set of assumptions. If the assumptions are wrong then the business plan is wrong. Therefore short cycle time for the Build-Measure-Learn loop in order to learn quickly if the assumptions are correct.
  • Pivot or Preserve. Pivoting (significantly changing the direction) requires that one foot is kept rooted, e.g. zoom-in pivot, zoom-out pivot customer segment pivot, customer need pivot, platform pivot, business architecture pivot (e.g. from high margin, low volume to low margin, high volume), value capture pivots, engine of growth pivot, channel pivot, technology pivot. Signs that the time is ready for piloting: decreasing effectiveness of the product experiments. Regular pivot or preserve meeting approx. every two month or quarter. If you don’t, you risk to getting stuck in the land of the living dead.
  • Small batch sizes. E.g. continuous deployment (special setup required, good product’s immune system required).
  • Growth: “New customers come from the actions of past customers”.
    Engine of Growth:
    • The sticky engine of growth:
      be locked in to the vendor. Important metrics: attrition rate and churn rate. Rate of compounding: natural growth rate minus churn rate.
    • The viral engine of growth:
      Important metric: viral coefficient: how many new customers will use a product as a consequence of each new customer who signs up. Should be larger than 1!
    • The paid engine of growth:
      buy clicks at Google and Co or hire sales team. Important metric: cost per acquisition (CPA) and lifetime value (LTV). CPA needs to be less than LTV.

Startups should focus on just one of the three.

Attention: at some point the number of early adopters is exhausted. The transition to mainstream customers requires tremendous additional work. You need a different team and a different product to serve either early adopters or mainstream.

  • Innovation Accounting: explicit hypotheses, actionable metrics instead of vanity metrics, show from meeting to meeting the progress on the validation of the value and growth hypotheses based on those metrics (e.g. registration, activation, retention, referral, purchasing rates). Kanban Diagram.
  • Example: buy clicks for 5$ a day, then measure with cohort analysis or split tests the rate of customer registration, app download, trial, repeat usage, purchase.
  • Metrics need to be
    • Actionable: a measure can be derived if the metric behaves different from what is expected.
    • Accessible: understandable also for the investors and the management.
    • Auditable: as the metric decides about dead or life of a feature it is important that everybody has the transparency how the metric is computed and trusts in the correctness of the metric.
  • Toyota: genchi gembutsu “go and see yourself”, or Steve Blank: “get out of the building”.
  • Adaptive organization: The 5 Why culture: when a problem pops up ask 5 times iteratively “why?” until you end up at the root cause, and then assign appropriately resources to each problem level. This acts as a natural speed regulator. You implement just as many processes as you really need. Professionally facilitate 5-Why-meetings, the facilitator needs to have enough seniority, all stakeholders need to be present.
  • Creating the Innovation Sandbox in large companies: any team can create a true split-test experiment that affects only the sandboxed parts. Limited time, limited number of customers.

It is impossible to communicate the content of this 300 pages book in 2 pages. You need to read the book to really understand those messages. There are many examples which make clear how much resources are wasted when not following such a lean concept. The book gives a good insight about the current atmosphere in Silicon Valley and what a professional investor is looking at.

The only problem: you need to read it from the start until the end and cannot skip pages. For me the book was such exiting that it was more difficult to stop reading than to continue.

Link collection:


About heisss
This entry was posted in Book Review and tagged , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s