We all have exciting ideas to enhance a product. During my time at Jooble, one of the largest job search aggregators, I often received suggestions from colleagues like, “What if we add a CV review feature?” or “What if we introduce interview practice tools?” These ideas sound great—they could add value for job seekers and improve the platform. But as we often say in Product, "it depends." Not every idea will work, and guessing can only take us so far. Instead, it is worth validating hypotheses with data. One effective method is the Fake Door test: This is a way to assess user interest before investing significant time and resources.
It involves presenting users with a CTA button or banner inviting them to try a feature that doesn’t yet exist.
It helps teams determine if there is potential interest in the idea. This is a way to assess user interest before investing significant time and resources. user interest It involves presenting users with a CTA button or banner inviting them to try a feature that doesn’t yet exist. CTA button or banner It helps teams determine if there is potential interest in the idea. potential interest However, what if you have 20+ beautiful ideas to validate but need an entire sprint of development effort just to launch one Fake Door A/B test, followed by three days of data collection? That’s 17 days per idea, which means nearly a year to validate 20 ideas. Clearly, this isn't scalable. Jooble has a strong data-informed culture with an ownership mindset. Everyone is encouraged to bring their ideas to improve the product and make it more valuable to job seekers. To support this, we envisioned a solution that allows anyone to validate an idea in just three days instead of 17, with zero development effort. Our goal was simple: empower everyone to test their ideas quickly and determine their potential value without wasting time or resources. we envisioned a solution that allows anyone to validate an idea in just three days instead of 17, with zero development effort. Discovery of the Solution Before we go to the solution, let me walk you through the process we had in place for validating ideas and launching Fake Door tests. At the time, Fake Door tests were treated as Product Backlog items. This meant they had to be prioritized, refined, developed, and tested by the development team. If you had an idea to validate, you first needed to convince the product team to prioritize it for development, and only then could engineers and QA invest their time. The product team was often hesitant to take on new ideas because they inherently carried costs without clear evidence of their value. That’s why they asked for market research and competitor analysis to back ideas with data. This complexity discouraged people from proposing new ideas and going through the process altogether. We asked ourselves: is there a way to launch a Fake Door test as easily as making a cup of coffee? We started a product discovery journey and uncovered some key insights that shaped the solution. Target Persona: The primary users who needed to validate ideas were business people. These individuals were non-technical, extremely busy, and lacked the time and skills to validate ideas themselves.
Fake Door Tests: 90% of Fake Door tests used banners on two main pages of our platform, which proved to be an effective way to fit into the existing design. Target Persona: The primary users who needed to validate ideas were business people. These individuals were non-technical, extremely busy, and lacked the time and skills to validate ideas themselves. Target Persona: business people. non-technical, extremely busy, Fake Door Tests: 90% of Fake Door tests used banners on two main pages of our platform, which proved to be an effective way to fit into the existing design. Fake Door Tests: banners Assessing risks was another critical step in our discovery process. Risk assessment is essential because it helps identify potential challenges early, that can impact achieving our goal. Are there any limitations to enable Fake Door Test launch for everyone? Fortunately, we already had tools in place to track data for newly launched A/B tests automatically and display Test Results. With this clarity, using the “How Might We” framework, we formulated our hypothesis: How might we enable anyone to validate their hypothesis themselves in less than 5 minutes by streamlining the launch process and leveraging automated tools? This challenge became the foundation for building a simpler, faster way to validate ideas. The Solution: Building the Component With a clear hypothesis in mind, we started researching tools that would help us build the solution efficiently. After evaluating various options, we chose Strapi and TeleportHQ as our no-code tools. Strapi TeleportHQ TeleportHQ: Allowed us to design and deploy visually appealing UI elements without coding.


Strapi: Enabled seamless content management, with any changes automatically reflected on the product website. TeleportHQ: Allowed us to design and deploy visually appealing UI elements without coding. TeleportHQ: Allowed us to design and deploy visually appealing UI elements without coding. TeleportHQ: Strapi: Enabled seamless content management, with any changes automatically reflected on the product website. Strapi: Enabled seamless content management, with any changes automatically reflected on the product website. Strapi: This combination formed the first iteration of Easy Widget. Easy Widget. However, we soon encountered an unexpected challenge: The user guide we created to help people launch Fake Door tests with Easy Widget was lengthy and complex. Users were reluctant to engage with it, fearing it was too complicated and that they might break something on the product website. While engineers could now launch widgets in days instead of weeks, we still hadn’t achieved our goal of enabling everyone to launch tests independently. We went back to the drawing board, reflecting on how we could make the process even simpler. Key Insight: Most banner designs for Fake Door tests were almost identical, with only minor changes in text, styles, or images.


Master Easy Widget: We created a preconfigured template for Fake Door tests, enabling users to quickly modify text, styles, and images without starting from scratch. Key Insight: Most banner designs for Fake Door tests were almost identical, with only minor changes in text, styles, or images. Key Insight: Most banner designs for Fake Door tests were almost identical, with only minor changes in text, styles, or images. Key Insight: Master Easy Widget: We created a preconfigured template for Fake Door tests, enabling users to quickly modify text, styles, and images without starting from scratch. Master Easy Widget: We created a preconfigured template for Fake Door tests, enabling users to quickly modify text, styles, and images without starting from scratch. Master Easy Widget: This led to Easy Widget 2.0, where we replaced the lengthy documentation with a short Loom video. The simplicity of this approach resonated with our business users—they could now launch Fake Door tests independently in just a few minutes without technical assistance. Easy Widget 2.0 The Results With Easy Widget, the results were remarkable: Easy Widget, the results were remarkable: Old Process: Validating one idea cost $6,000 and took 17 days—limiting us to just 20 ideas tested annually, costing $120,000.


Optimized Process: The cost per idea dropped to just $5, with validation completed in 3 days and a 5-minute launch time. This allowed us to test up to 121 ideas annually for just $605. Old Process: Validating one idea cost $6,000 and took 17 days—limiting us to just 20 ideas tested annually, costing $120,000. Old Process: Validating one idea cost $6,000 and took 17 days—limiting us to just 20 ideas tested annually, costing $120,000. Old Process cost $6,000 17 days costing $120,000. Optimized Process: The cost per idea dropped to just $5, with validation completed in 3 days and a 5-minute launch time. This allowed us to test up to 121 ideas annually for just $605. Optimized Process: The cost per idea dropped to just $5, with validation completed in 3 days and a 5-minute launch time. This allowed us to test up to 121 ideas annually for just $605. Optimized Process: just $5, in 3 days and a 5-minute launch time just $605. Sounds great, right? But is it really everything? There’s always a hidden cost. One critical concept in Product Management is Cost of Delay—the economic impact of postponing a product feature. This includes lost revenue, missed market opportunities, and increased competition. Time spent not validating ideas matters because it slows down innovation and delays potential revenue streams. By enabling Easy Widget, we were able to validate over 10 hypotheses in just two months. Interestingly, we backlogged 80% of them because there was no interest from users in adopting those ideas. However, this process also uncovered a new market opportunity that allowed us to create new revenue streams and increase product revenue significantly. Additionally, Easy Widget empowered marketing teams to launch promotional banners independently—without engineers—to drive traffic to other Jooble offerings. This allowed us to leverage the Jooble ecosystem effectively by tailoring solutions based on country-specific user behavior. By customizing promotions for different regions, we were able to boost engagement and further increase revenue across multiple markets. How Other Teams Can Implement the Same Solution For anyone interested in building a similar solution, here are some best practices based on our lessons learned. While you might choose different tools than Strapi and TeleportHQ—and that’s perfectly fine—remember that tools are just “means” to achieve the desired outcome. What truly matters are the core principles that remain unchanged and are essential for successful implementation. Do the Homework Once the problem is defined, dive deep into data and gather feedback from people to understand how the solution should look. Explore the challenges users face today and how these impact business results. Doing this homework will give you clarity on pain points and potential impact. Look for patterns—analyze the types of tests users are launching and identify similarities. This will help you design the simplest solution right from day one. Define Success Criteria and Stick to Them Clearly define what success looks like and stay committed to it. For us, the goal was to enable everyone to launch a Fake Door Test in just 5 minutes, and we iterated as much as needed until we achieved it. Having clear success criteria keeps your team focused and ensures you deliver meaningful outcomes. Make It “Grandma Simple” This might seem obvious, but it’s worth emphasizing: simplicity is subjective. Based on our backgrounds and skills, what feels “simple” can vary greatly. For a technical person familiar with coding, configuring widgets using Strapi and TeleportHQ might feel straightforward when following documentation. But what if we asked my grandma to do it? The goal should be to make the process so simple that even she could run a Fake Door Test without hesitation. Always aim for maximum simplicity—this is key for adoption. Engage Target Users From Day One Involve your target users in testing as early as possible—even if the solution is still running on a development machine. Early feedback is invaluable for identifying challenges users might face and making improvements before launch. This iterative approach ensures your solution addresses needs effectively. Conclusion In conclusion, I wish I could give you a silver bullet for optimizing A/B testing processes, but the truth is—it depends. Based on my experience, one approach that has consistently helped me while working on multiple initiatives to optimize A/B testing strategies is going back to the question: “What do we want to achieve?” and working backwards from there. This is a powerful approach because it starts with a clear vision of the end goal and maps out the steps needed to achieve it. Start by mapping the end-to-end flow. Conduct a user journey mapping exercise to identify every step in the current process and how long each step takes. For example, in my case, I imagined myself as a business user who wanted to launch a Fake Door Test to validate an idea. I walked through the entire process step by step, as if I were the user. This exercise helped me pinpoint the main pain points, inefficiencies, and areas of waste in the process. When I spoke with actual users, their feedback validated my assumptions: the main issue was that it took too long and required development resources. With this clarity, we were able to formulate a hypothesis and start exploring potential solutions. By clearly defining the goal and performing user journey mapping, we identified inefficiencies and optimized the process effectively. This method not only helps uncover hidden bottlenecks but also ensures that any solution you implement directly addresses real user pain points and aligns with your ultimate objectives.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

How a No-Code Tool Changed Our Testing Process

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

4 Lessons We Learned in 2019 (and How Marketers Can Apply Them in 2020)

A/B Testing For Email Marketing — What Should You Test For?

A/B Testing Our Way to a Better Payment Gateway

A/B Testing - You’re doing it wrong

AB Testing For Digital Products

A/B Testing In Flutter With Statsig

4 Lessons We Learned in 2019 (and How Marketers Can Apply Them in 2020)

A/B Testing For Email Marketing — What Should You Test For?

A/B Testing Our Way to a Better Payment Gateway

A/B Testing - You’re doing it wrong

AB Testing For Digital Products

A/B Testing In Flutter With Statsig

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps