date: 2013-07-23

categories: thoughts, business

author: Glenn

Behind the curtain - Pretotyping CLI ideas

https://medium.com/rocket-startup/what-is-pretotyping-and-how-is-it-different-from-prototyping-and-building-mvp-b44f21611aa https://www.pretotyping.org

https://www.exponentially.com/what-is-pretotyping

One of the things people working with me probably get sick of hearing me say is "how do you know you've built the right thing?". For those that think they've built it, the follow-up is inevitably "When did you know?". The answer to these questions can be more difficult and confronting than they appear on the surface, even for a successful product. I can point to amount of revenue generated, is that how we know? The feedback loop on that for a B2B SaaS product, given an enterprise sales cycle could average a few quarters, could be measured in a year or more. Does it even really tell us we've built the right thing, or is it just that our sales team are really great at selling? I guess we'll have to wait another year or two and look at the renewal rates. Oof.

At best that's given a lossy and lagged approximation of success at a very macro product-wide level. What's impact on a specific feature on revenue? 🤷‍♂️ You could start measuring adoption, usage, and depth of engagement and correlate that to revenue and retention. Now we're getting some earlier signals, hopefully with higher conviction too. There's still a considerable delay in us knowing we've built the right thing. How many people need to use it for us to be confident? What if it's a fairly niche need for a particularly high value segment? Is the lack of adoption because we've not done a good job marketing it or some other awareness issue rather than the feature itself? All that time. All that uncertainty. All that cost. If this turns out to have been a miss, what other opportunities have we lost because of the extended distractions?

Confidence over time

Imagine for a moment you've got great answers to "How do you know?" and "When did you know?". The next question is "How could you have known sooner?". This is, to me, the core of a high functioning Product Management group. Learning a year or two after a feature has been released that it wasn't quite what customers needed is incredibly expensive. Conversely, the cheapest time abandon a bad idea is before you've let it take down roots. The sunk cost fallacy & plan continuation bias are difficult things to overcome. We're hard wired to want to make our things work. The more time we spend on them the more emotionally invested we are. The more real something is the more we try to justify how or why it will work, if only it has a couple more tweaks. The more code we write the harder it technically becomes to unwind the decision.

If we work backwards from the ends state of the problem we're trying to solve we can usually think about different ways we could objectively measure whether we've been successul. Expectations that a certain percentage of people will do X, will change their behaviour, will provide a qualititative response. From there we can keep working backwards and question the assumptions we have. Would people change their behaviour? Is such a change actually desirable to them? Does it solve the problem? All of these changes creates forks in the road. If any of our assumptions are wrong then we'll probably have to rethink a large part of our approach. Keep pulling on that thread and you'll usually get to a set of questions you could answer today. With minimal effort.

At that point the game changes. It's not about being absolutely certain you've built the right thing. It's about finding support for your general approach, and discovering where the dead ends are. Early. Really early. Before you've committed to any specific plan. You test an assumption or hypothesis, you get an answer, you adjust your course and continue on. Each day brings you closer to the answer. Each adjustment increases your conviction. Knowing you've got the right thing can be hard, but you can quickly learn what the wrong things are. And that can be just as valuable.