Great Beta Programs — A Contrarian View

If I told you that you could directly impact the success of your product by rethinking how you run your beta programs, would you think I was nuts? We suggest that you beta differently. Let me explain how.

Today, we officially kicked off the Aha! beta. It is always a great milestone for any software company, but what happens before the emails go out and after customers start using the product is what really matters. If you had to summarize what matters most about beta programs, what would you put at the top of your list?

For us, betas are solely about feedback — quality feedback that leads to pattern recognition. By the end of the beta program, you should be able to predict what a customer will tell you. And great beta programs start long before you even send out the first beta invites.

Many folks have recently asked us about how we would run our beta program. We clearly have a certain point of view and repeatedly found people surprised that we were not going to follow the conventional wisdom which pushes for early release and public availability. Our perspective is hardly radical, but some might argue that it is contrarian. And that is likely because we believe the two prevailing recommendations that most entrepreneurs receive are all wrong.

And that is likely because we believe the two prevailing recommendations that most entrepreneurs receive are all wrong:

1. Just get it out there At our last startup, Paglo, we went big. When we opened the doors to our beta, within 24 hours we had over 800 signups (mind you, this was an enterprise IT SaaS product). But most users never really got going and we only spoke with a handful of active customers. We failed to segment beta customers, learned nothing about them before they started the program, and gained lousy insights based on their limited use and contact with our team. The approach led to lots of initial use but little engagement. While it appeared to be a terrific success, its long-term value was dubious.

2. If you release a product and are not embarrassed by it, you waited too long How many times have you heard or read this? We think the reverse is actually true. If you release a product and are embarrassed by it, you did not wait long enough. In today’s market, being tolerated is not good enough. You should strive for love. Check out this previous post on creating the Minimum Lovable Product. If your goal is to receive feedback during beta, people actually need to use it, and that means they need to sacrifice to make the time. And people only sacrifice if what is new excites them. A lousy beta product excites no one.

With these lessons in mind, we rethought everything. We approached the goal of the Aha! beta with a single purpose to get great feedback. And to receive great feedback we knew we needed to focus our efforts to identify forward-thinking product management experts, deliver a Minimum Lovable Product, and help them succeed using it. (Note that we believe this approach is preferred no matter whether you are developing business or consumer software.)

By no means was this the easiest path as there was lots of upfront work to connect with the right folks. But think of it this way. We had something to offer (access to the beta) and in return, we got to speak with and demo Aha! for 50-plus companies who were interested enough to talk before we allowed a single customer into the beta. Early-stage companies really do not have much to offer, but access to an interesting demo and the chance of getting into a beta has real mystery and value.

So, before the beta even began today, we received nearly 50 hours of priceless feedback which led to important changes and unique enhancements. It is paying off as folks are visibly excited when they see a demo and consider starting to use Aha! We have also created meaningful customer relationships which humanize what we are doing and add personality to the customer’s beta experience. We are fortunate enough to have loyal customers and advocates before we have sold anything.

And by using Aha! to build Aha! we now have a completely customer-driven roadmap that we are working against. It is amazing to consider that all of that happened without having a single customer using the application.

Now, it is hard work and not every meeting goes swimmingly. If you follow this approach, one of the hardest aspects is saying “No.” There are lots of terrific folks who work at growing companies that will not be a great fit for you. If the goal is to capture insightful feedback, you need active beta users working with the teams that help them get work done.

I listened closely to potential beta customers and if either there was no excitement or I heard phrases like “Sure, I would like to play with it,” I humbly suggested that the timing might not be right and we should connect at a later time. The potential beta customer is not always right, and it is actually good for you to say no. It is important to say no because great products have a driving vision, and who and what are left out is just as important as what goes in.

It will be interesting to learn how the feedback changes with actual beta customer use, but we have already changed the trajectory of the company. I think you can too if you think differently about your own beta programs moving forward.

Building great software is hard, but it should not be excruciating. We built Aha! with this goal in mind — we wanted to create a new way for product managers and engineers to create brilliant roadmaps and be happy doing it. Sign up for the second phase of our beta now to see if we do what we say.