Validated Learning

Overview

Validated Learning is a cornerstone of the Lean Startup methodology, a process that emphasizes speed, learning, and a customer-centric approach to building businesses. Coined by Eric Ries, the pattern is a systematic and rigorous approach to demonstrating progress in the highly uncertain environment of a startup. It is the practice of using scientific experimentation to test business hypotheses and learn what customers actually want, rather than what founders assume they want. The core idea is to reduce waste by avoiding the development of products or features that nobody will use. Instead of focusing on traditional metrics of progress like lines of code written or milestones met, the unit of progress in a lean startup is validated learning: a clear and demonstrable understanding of what creates value for customers.

This learning is achieved through a continuous feedback loop of building, measuring, and learning. The process begins with identifying the riskiest assumptions in a business model—the leap-of-faith assumptions upon which the entire venture rests. These assumptions are then tested through a series of experiments, each designed to generate empirical data from real customers. The goal is not to build a perfect product from the outset, but to build a Minimum Viable Product (MVP) – the simplest version of the product that can be used to gather feedback and validate or invalidate the core assumptions. This iterative process allows startups to make small, incremental adjustments to their strategy based on real-world evidence, rather than relying on intuition or extensive market research that may not reflect actual customer behavior.

Validated Learning is not just about collecting data; it is about turning that data into actionable insights. It requires a shift in mindset from a “build it and they will come” mentality to a more scientific and evidence-based approach to entrepreneurship. By embracing uncertainty and treating every decision as an experiment, startups can navigate the complexities of the market more effectively, pivot when necessary, and ultimately increase their chances of building a sustainable and successful business. It is a disciplined and methodical way to discover the right product for the right market, ensuring that every effort and every dollar spent contributes to a deeper understanding of the customer and the business.

Core Principles

1. Entrepreneurs are Everywhere: The principles of the Lean Startup and Validated Learning are not limited to startups in a garage. They can be applied by anyone working in a “human institution designed to create a new product or service under conditions of extreme uncertainty,” including established companies.
2. Entrepreneurship is Management: A startup is not just about the product; it is an institution that requires a new kind of management. This management is geared towards navigating extreme uncertainty and is centered around the principles of validated learning.
3. Build-Measure-Learn: This is the fundamental feedback loop of the Lean Startup. The process involves turning ideas into products (Build), measuring how customers respond (Measure), and then learning whether to pivot or persevere (Learn). The goal is to accelerate this feedback loop to learn as quickly as possible.
4. Innovation Accounting: To hold entrepreneurs accountable and measure progress, a new kind of accounting is needed. Innovation accounting focuses on actionable metrics that demonstrate cause and effect, rather than vanity metrics that can be misleading. It helps to track the progress of learning and the validation of the business model.

Key Practices

1. Identify Leap-of-Faith Assumptions: The first step is to identify the two most critical assumptions: the value hypothesis and the growth hypothesis. The value hypothesis tests whether a product or service really delivers value to customers once they are using it. The growth hypothesis tests how new customers will discover a product or service.
2. Build a Minimum Viable Product (MVP): The MVP is the smallest possible experiment to test the leap-of-faith assumptions. It is not a smaller version of the final product; it is the fastest way to get through the Build-Measure-Learn feedback loop with the minimum amount of effort.
3. Run Experiments and A/B Tests: Each iteration of the product is an experiment. A/B testing is a powerful technique to compare different versions of a product or feature and see which one performs better against a specific metric. This allows for data-driven decision-making.
4. Use Actionable Metrics: It is crucial to use metrics that are actionable, accessible, and auditable. Actionable metrics demonstrate a clear cause and effect, helping to understand what actions lead to what results. Vanity metrics, such as the total number of sign-ups, can be misleading and should be avoided.
5. Pivot or Persevere: Based on the data and learning from the experiments, a startup must make one of the most difficult decisions: to pivot or to persevere. A pivot is a structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth. Perseverance means staying the course.

Implementation

Implementing Validated Learning requires a disciplined and systematic approach. Here is a step-by-step guide:

1. Define Your Assumptions: Start by clearly articulating your business model and identifying the riskiest assumptions. Use a tool like the Lean Canvas to map out your assumptions about the problem, solution, key metrics, customer segments, and more.
2. Prioritize Your Assumptions: Not all assumptions are equally risky. Prioritize them based on their potential impact on your business and the level of uncertainty. Focus on testing the riskiest assumptions first.
3. Design Your Experiments: For each assumption, design an experiment to test it. Define the hypothesis, the metric you will use to measure the outcome, and the success criteria. For example, if you assume that customers are willing to pay for a certain feature, you could run a “concierge MVP” where you manually provide the service to a small group of customers to see if they are willing to pay.
4. Build and Run Your MVP: Build the simplest possible version of your product or experiment that allows you to collect data. This could be a landing page, a video, a prototype, or even a simple conversation with a potential customer. The goal is to get feedback as quickly as possible.
5. Measure and Analyze the Data: Collect the data from your experiment and analyze it. Did you validate or invalidate your hypothesis? What did you learn about your customers and your business model?
6. Learn and Decide: Based on your analysis, decide what to do next. Do you need to run another experiment to gather more data? Do you need to pivot and change your strategy? Or have you validated your assumptions and are ready to move on to the next stage of development?
7. Iterate: Validated Learning is a continuous cycle. Repeat the process of building, measuring, and learning, constantly refining your product and business model based on real-world feedback.

The 7 Pillars Assessment

• Purpose (Score: 5): Validated Learning is deeply aligned with the purpose of creating value. Its entire focus is on discovering what is truly valuable to customers and eliminating waste. By focusing on learning and evidence, it ensures that the organization is building something that matters.
• Governance (Score: 4): The pattern promotes a decentralized and evidence-based approach to decision-making. It empowers teams to run experiments and make decisions based on data, rather than relying on top-down directives. This fosters a culture of ownership and accountability.
• Culture (Score: 5): Validated Learning fosters a culture of curiosity, experimentation, and continuous improvement. It encourages teams to embrace failure as a learning opportunity and to challenge their own assumptions. This creates a dynamic and adaptive organization.
• Incentives (Score: 3): While the pattern promotes a focus on learning, traditional incentive structures in many organizations are still tied to short-term financial results or the successful delivery of features, which can conflict with the principles of Validated Learning. Aligning incentives with learning and experimentation is a key challenge.
• Knowledge (Score: 5): The pattern is a powerful engine for knowledge creation and dissemination. Every experiment generates new knowledge about the customer, the market, and the business model. This knowledge is then used to inform future decisions and is shared across the organization.
• Technology (Score: 4): Technology plays a crucial role in enabling Validated Learning, especially in the digital realm. Analytics tools, A/B testing platforms, and rapid prototyping tools are essential for running experiments and collecting data. The pattern encourages the use of technology to accelerate the learning cycle.
• Resilience (Score: 4): By promoting a culture of adaptation and continuous learning, Validated Learning builds organizational resilience. It allows companies to navigate uncertainty and change more effectively, and to pivot when necessary to survive and thrive.

When to Use

• Early-stage startups: When you are in the early stages of a new venture and are facing a high degree of uncertainty.
• New product development: When you are developing a new product or service, even within an established company.
• Entering a new market: When you are entering a new market and need to understand the needs and behaviors of a new customer segment.
• When resources are limited: When you have limited time and money and need to make sure that you are investing in the right things.

Anti-Patterns

• Vanity Metrics: Focusing on metrics that look good on the surface but do not provide real insights into the health of the business. For example, focusing on the total number of sign-ups instead of the number of active users.
• Analysis Paralysis: Getting stuck in the “measure” and “learn” phases of the loop without taking action. It is important to make decisions and move forward, even with incomplete information.
• The “Everything is an MVP” Fallacy: Using the term MVP as an excuse to release a low-quality product. An MVP should be a high-quality experiment, not a crappy product.
• Ignoring Qualitative Feedback: Relying solely on quantitative data and ignoring the rich insights that can be gained from talking to customers and observing their behavior.

References

• [1] Ries, E. (2011). The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Crown Business.
• [2] Wikipedia. (2023). Validated learning. https://en.wikipedia.org/wiki/Validated_learning
• [3] The Lean Startup Co. (n.d.). The Lean Startup Methodology. https://theleanstartup.com/principles