Eric Ries' book The Lean Startup has been out since 2011, but I've disregarded reading it because I assumed by 'lean startup' he meant: being cheap and building small minimal viable products. This seemed like an over discussed topic.
It turns out, my assumption of this was entirely inaccurate. The term 'lean startup' actually draws its name from lean manufacturing, a work process developed in Japan and exemplified by the Toyota Production System (TPS). A method of working designed to reduce waste and continuously deliver the right product.
[As you know, I’m a big fan of lean manufacturing, and the post on Stocking Hospital Supply Rooms (Two-Bin Kanban) has been widely read. My summary series of The Handbook of Healthcare System Design also discusses lean manufacturing principles].
In applying lean manufacturing to startups, Eric sees every assumption about a startup - its product, services, prospective customers, marketing campaigns, etc - as a hypothesis that must be tested. For a startup, the “question isn’t can it be built? But, should it be built? And for whom? And can we make a sustainable business around it?”
The lean startup approach of customer tested hypotheses and data-driven decisions avoids two equally problematic approaches to startups. It gets over the ‘just-do-it school of entrepreneurship,’ where things happen, but a startup doesn’t actually learn or understand what works and what doesn’t. The other problematic approach is the ‘analysis-paralysis startup’ where every decision is over-thought and over-planned without every getting real-world data.
This book is a must-read for anyone trying to do something new. This blogpost tries to summarize the key themes from the book. Direct quotes from the book are in quotation marks.
The five main sections of this post, come from Eric’s Lean Startup Principles
- Entrepreneurs are everywhere
- Entrepreneurship is management
- Validated Learning
- Build – Measure - Learn
- Innovation Accounting
(The header image “Stop wasting people’s time” is from Eric Ries’ January 12, 2012 London School Economics Department of Management Lecture available here, with addition of his name)
1. Entrepreneurship is everywhere
When the term ‘startup’ is used, many associate it with a new software company. However, Eric insists the term applies far more broadly.
A startup is “a human institution designed to create a new product or service under conditions of extreme uncertainty.” This means a startup is any new, or old, company that is trying to find a new source of value: such as a new product, service, customer, or market. If there were no uncertainty about what to do, it wouldn’t be a startup; one could execute a well-built business plan to achieve massive success.
Eric puts forth the lean startup, as “a set of practices for helping entrepreneurs increase their odds of building a successful startup.” Because, by definition, a startup operates in settings of extreme uncertainty, it must view almost everything it does as a hypothesis. The hypothesis requires testing via a build-measure-learn feedback loop to results in ‘validated learning’ (true insights about the hypothesis).
The goal of a lean startup isn’t to be cheap, but to accelerate the build-measure-learn feedback loop in order to enable continuous innovation through a process of decision making that is data-driven and reflective of the customer’s actual behaviour. Critical decisions should not be based on hypothetical strategic planning, unreliable surveys, or focus groups.
Using this hypothesis-driven approach Eric hopes to “improve the success rate of new innovative products worldwide.”
2. Entrepreneurship is management
The traditional ‘startup’ views management as a bad word. Something that is stifling and too controlling for the unstructured nature of their venture. “If management is the problem, chaos is the answer.”
However, Eric disagrees and proposes a new type of management, one that is designed for startups.
Entrepreneurial management “requires creating conditions that enable employees to do the kinds of experimentation that entrepreneurship requires.”
In traditional management, success is executing a business plan on time and on budget. Failure to deliver a product is considered either poor planning or poor execution.
However, Eric continuously emphasizes, that in a startup the only thing worse than building a bad product, is “building something that nobody wants.”
Traditionally management engages in this development process via strategic planning - a long, slow process based on untested hypothesis. Eric strongly suggests that management move away from strategic planning and instead towards rapid experimentation as its way forward. “Experiments can start immediately.” And unlike strategic planning, experiments provide real results, and if successful results in a usable product.
Many organizations have shifted to agile development and believe they have solved their startup managerial issues. However, Eric makes a fundamental observation that agile development does not provide insights about what to build for customers.
Agile is a way to “make work more flexible.” It is a way of working in smaller batches, in small integrated teams, to “develop as you go,” with the potential to focus on “the most important things at that time.” But what is the most important thing?
The process, of understanding what creates value for customers is called validated learning.
3. Validated Learning
Eric suggests the whole purpose of a startup is to obtain “validated learning” in order to “demonstrate valuable truths about a business prospect.” Such as what to build, for whom, to create a sustainable business. This needs to be done quickly before the startup venture runs out of funds.
Startups think they know what to build, or who their typical ‘customer archetype is.’ But the core truths a startup takes for granted are often unvalidated ideas, based on leap of faith assumptions, or arguments by analogy. Without actual paying customers buying the product, everything remains a hypothesis.
The process of validated learning breaks down the large vision a startup has about what it thinks it should do, into clear testable hypotheses.
Two critical hypotheses that must be tested in a startup are the value hypothesis and the growth hypothesis. The value hypothesis aims to understand if the product or service actually creates value for the customers (i.e. is it something they want). The growth hypothesis looks at how new customers discover the product or service. For a startup to succeed, both of these hypotheses require reliable answers.
Eric’s methodology of hypothesis-driven development is clear: “We must learn what customers really want, not what they say they want or what we think they should want.”
[Eric (and others before him like Steven Jobs, or Henry Ford have observed) customers often don’t know what they want. They want a faster horse or a better battery on their tape player.]
Eric instead suggests that ‘customers reveal their preferences by their behaviour.’ The purpose of validated learning is to test every assumption about the business strategy or product as a hypothesis and experiment with actionable metrics. And “unlike focus groups, measure what customers actually did.”
This process of experimentation is outlined in the build-measure-learn feedback loop.
4. Build-Measure-Learn
The build-measure-learn feedback look is the engine of generating validated learning. It may sound counterintuitive, but the process actually starts at the end with learn.
Learn: “What do we want to learn” is the question that initiates this process. This is the hypothesis that needs to be tested, such as a business assumption, technical issue, or marketing plan.
Build: The second step is to understand what type of experiment needs to be built to test that hypothesis. This is where the concept of a minimal viable product (MVP) comes in. Ideally, the hypothesis can be tested even without making something (eg. a video MVP), or without having to develop too much (eg. a smoke-test MVP, or wizard of oz MVP).
The MVP, in the lean startup lexicon, is a way to answer the specific hypothesis by building a product or service. Only those features that contribute to testing the hypothesis should be built. Any other features not required to test the hypothesis are considered, in lean manufacturing terms, “waste.” (note, the exception is If an idea is small enough, it may not require an MVP, and the actual idea can be built for the experiment). Either way, the goal is to avoid waste in this step and efficiently test the hypothesis.
Measure: The experiment must produce metrics that are clear to interpret and can validate or refute the hypothesis. The next section, ’ innovation accounting’ goes into this in more detail.
Learn: the experiment has been run, the results are in, now a decision must be made based on the data. If the hypothesis was rejected, an alternative approach needs to be tried. If the experiment is successful, the experiment itself can become the product.
Eric gives an example of how Zappos early on had to test the hypothesis that people will purchase shoes online. Instead of building an entire online store, all they did was take photos of shoes from a physical store and post them online. Customers could purchase a shoe from the pictures, and then Zappos would go and purchase that shoe from the physical store the photo came from on behalf of the customer.
This slide from a talk by Eric Ries shows specific techniques that may be helpful at each stage of the process.
Startups can’t live in stealth mode because they are unable to go through the build-measure-learn loop. Groups often like to remain in steal mode because the absence of real numbers “invites possibility and imagination” about the future, “whereas small numbers raise questions”. But this doesn’t help validate any of the hypothesis about what is being built, and so there is a high chance the hypothesis may be incorrect.
5. Innovation Accounting
As we’ve discussed many times, the point of a startup is to determine what route to take; to quickly “kill things that don’t work, and double down on those things that do.” This is only possible if one avoids vanity metrics and tracks meaningful metrics.
Vanity metrics present numbers such as the total number of downloads, total customers, or total revenue. But these metrics do not actually reveal much about how a customer values the product. If someone downloads and deletes an application without using it, the metric of ‘total downloads’ does not demonstrate this.
Vanity metrics can also easily be manipulated by success theatre. A large add spend may raise these metrics, and give the outward appearance of success. But the purpose of a startup isn’t to simply add new customers, but determine if a sustainable business is being developed.
Metrics must be actionable, accessible, and auditable. Actionable means something that can be influenced by an experiment Accessible means the metrics’ make sense’ to people in the organization interpreting them. Auditable means employees must be confident the data populating these metrics is right, and they can check it.
One way to make metrics more useful is to connect them to human behaviour. For instance, 'downloaded product,' 'used product,' and 'upgraded to paid version.' This is more tangible than a ‘website hit.'
Eric tracked four key metrics for some time:
- The total number of downloads
- Repeat usage: 1x
- Repeat usage: 5x
- Number of customers who paid money for the product
By focusing on these metrics, it was clear if their experiments were helping them build a sustainable business. Retention rate metrics often are a valuable metric to track. It helps provide crucial insight into the value hypothesis. The total number of downloads (when analyzed daily), helps provide insight into the growth hypothesis (but is important to realize doesn't help with the value hypothesis).
It is often valuable to track these metrics using cohort analysis. Especially user retention. This enables an organization to see if the changes they make on a rolling basis retain users better over time.
Before running a new experiment, it is essential first to determine what the baseline metrics are. These are the metrics the experimental intervention aims to move closer to the ideal. Second, deploy the intervention. Consider using a method such as a split testing where features deployed to a small group of users are compared against the rest.
Third, after many experiments, an organization needs to determine if the metrics are not moving closer to the ideal. This decision is one to “pivot or persevere.” If the metrics are not improving adequately, perhaps a significantly new approach (a pivot) is required. The pivot uses the validated learning to date, to adopt a new approach.
“3 Engines of Growth”
As an organization hones in on a value hypothesis that is working, it may begin to test its growth hypothesis in more detail. Eric outlines three engines of growth.
1. Sticky engine of growth:
The sticky engine of growth is best suited for products that customers use over a long time. If someone leaves using the product (the churn rate) it likely mean they are very dissatisfied or they have gone to a competitor.
The way to improve product growth using this strategy is to retain existing customers by making the product better. This goes against the traditional idea of growth to invest in marketing and sales.
2. Viral engine of growth.
The viral engine of growth works because the customers do the marketing. Although this often is thought of as word of mouth, another way this can happen is as a side effect of product usage that results in person-to-person transmission. For instance, when one receives an email from Hotmail, the message receiver may see it was sent from Hotmail.
A ‘viral coefficient’ can be tracked. If it is 0.1, for every 10 new users to the platform, they will bring 1 additional user. This is not sustainable. However, if the viral coefficient is greater than 1, for every 1 new user, they will bring an additional user (fraction of a user). Growth now can happen.
3: Paid engine for growth.
The paid engine of growth uses funded advertising. Growth can be found by either reducing the cost to acquire a customer (CPA, cost per acquisition), or increasing the revenue from each customer (CLV, customer lifetime value).
Start Today
Your organization can follow the Lean Startup process by building a cross-functional team that can take on a hypothesis, build it, ship it, and measure the results. This cross-functional team reduces the inefficiencies of handing work off to other groups and ensures the development team has ownership in the outcome of their work.
An innovation team cannot be slowed down by bureaucracy. They need to have the autonomy to validate hypothesis as required and build a culture of split testing, continuous deployment, and customer testing.
Moving to the Lean Startup approach moves your organization away from leader driven development - where product managers, business analysts, or leadership plays Caesar in making decisions - to one where the organization strives to make frequent evidence-based decisions.
The lean startup approach also moves away from decision making via compromised agreements. A compromised agreement is when a decision is made by taking the middle ground between two parties. There are two problems with this. First, the decision is not rooted in data but a happy middle ground. Second, each party over time will progressively 'anchor' their position in a more extreme way to create a final outcome closer to the middle ground result they initially hoped for.
To learn more details on one way to build a lean startup approach into your existing company, see Eric’s 7 Rules for an Innovation Sandbox.
Conclusion
Eric Ries strongly believes that organizations waste tremendous personnel (passion, energy, vision) and economic resources working on the wrong thing. His hope is the lean startup approach will help organizations that are operating in settings of extreme uncertainty (aka startups) learn what they should be working on, and help bring new innovative ideas reliably to market.
Why does this matter?
“Startups don’t starve; they drown.” - Shawn Carolan
“There are always a zillion new ideas about how to make the product better floating around, but the hard truth is that most of those ideas make a difference only at the margins.”
The lean startup approach aims to help see through this noise quickly and determine which is the correct path forward.
“There is surely nothing quite so useless as doing with great efficiency what should not be done at all.” - Peter Drucker 1963
I highly recommend reading (or listening to the full book). All quotes from The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. If any quotations are inaccurate or missing, please let me know. There is certainly a chance of error, as I took notes while listening to the Audible version of the book at the gym.
An interesting collection of aggregate statistics for reasons why Startups Fail (Updated 2020).
Related Posts
Pair Together: Design Thinking, Lean Startup, & Agile