Continuous interviewing with Kristian Collin Berge (CEO & Co-founder at UX Signals)
Tips & insights to foster a habit of continuous interviewing and why it's so important
👋 Hey there! I’m Afonso, founder of Scandinavian Product Group.
Every week or so, I explore a new product organization, share an insightful story from brilliant product minds, or rant about a topic I think is relevant.
My goal is to share insights to help people build better products with happier teams - and create stronger businesses.
Consider becoming a free subscriber so you don’t miss out.
My guest today is Kristian Collin Berge.
Kristian is the Co-Founder/CEO of UX Signals and an absolute expert in Product Discovery.
At UX Signals they make software that helps automate the user interviewing process, and he coaches product teams and organizations on how to build user interviewing habits.
He’s also the founder of Produktleder.no, the Slack community with 1500+ product managers in Norway, which has been essential in building the Norwegian product community.
Previously, he has held various positions as CPO and product manager, specializing in continuous product discovery.
I recently sat down with Kristian to discuss all things continuous interviewing - and share his wisdom with the community. Let’s dive in.
1. Continuous discovery is a hot topic in our industry. Why should company leaders and product teams care?
How you do product discovery can influence the life or death of your product and company.
For example: 10 years ago, I founded a tech company. We had good feedback from our users, a growing customer base around Europe and US, and top-tier investors. There were a lot of indications that we were going to succeed.
But a few years later, we started to realize that we were not going to make it. Somehow we had not been looking closely enough at our underlying assumptions, and our product could not make the jump from early adopters to the mass market. We ended up shutting the company down.
This is a quite common story in product development. 87% of product people say they have built products or features nobody wanted. Only 1 in 5 features in a typical software product are in regular use. And even top companies like Google and Microsoft, only succeed with 10-30% of the new solutions they AB-test.
Having experienced this first hand, it has made me obsessed with talking to users regularly, and frequently testing assumptions, to ensure that we always stay in tune with what customers need.
It has made a real difference in my work and in the companies I’ve worked with since then. Contributing directly to cost savings, revenue growth, and startup acquisitions.
And maybe most importantly, it’s just more fun and rewarding to build something customers want and need. Life is too short not to.
2. I still hear some people talking about Discovery as a “phase”. How is “continuous discovery” different?
When I first started working in Product, I tried many times to separate discovery and delivery into different phases. But my perspectives have changed over time.
Yes, there are times when almost everything you do is discovery, like before you have a line of code, when you are just talking to potential customers.
But as soon as you want to track and learn from actual user behavior, it starts to include some form of delivery. I.e. building a live prototype, or launching a fake door experiment inside your current product.
Similarly, there are times when almost everything you do is delivery, when you are rolling out big changes at scale. In those situations, you will focus a lot more on scalability and performance. But even then you will likely roll out to a small segment of users first to measure their response, like Facebook does when they first launch a feature in a few countries, before gradually rolling it out to the entire world.
To me, continuous discovery is a mindset and practice of always testing your assumptions, no matter what phase you are in. It’s just the type of assumptions and the level of risk they incur that changes depending on which phase or context you are in.
3. Why is it so important to develop a habit of continuous interviewing?
Most teams have analytics enabled in their product to see where their users drop off. But it does not tell you why.
Many also use in-product surveys or feedback widgets to learn what customers feel. But this only gives you the tip of the iceberg, and it’s very risky to base your decisions on this alone.
To really understand why users sign up, engage, buy, retain or churn, there is nothing like actually talking to them, seeing their facial expressions, and being able to ask why.
It gives you instant empathy for their situation.
Talking to users also allows you to test new ideas, content and designs, before you put them into production. This lets you iterate much faster, and get better results.
Once in a while, you’ll discover small insights that let you make a huge leap forward. This happened to Airbnb, when they found that people were afraid to click a button in their checkout flow, because they didn’t understand what would happen next. Changing 7 characters of text on the button, increased their conversion rate and added millions of dollars in additional revenue overnight.
There is even evidence that suggests that companies who talk regularly with their customers, grow 15-20% faster than their competitors.
Marty Cagan goes so far as to say that “without direct unencumbered access to actual users and customers, the product team has little hope of any kind of success”. (Transformed, Cagan 2024)
With all the teams we have helped get started with continuous interviewing, the usual response is: “Holy shit, we should do this more often!”, because it brings such clarity into the user situation, and aligns the team on what they should prioritize next.
4. One challenge I often hear from folks is that doing continuous interviewing takes too much time and effort. What are some of the things teams can do to help with this?
If interviewing takes too much time, this is usually how it happens:
When you need insights, you think you don’t have time for interviews, so you do it very rarely.
When you actually do interviews, since you don’t do it very often, you feel that you need to prepare a perfect interview guide or prototype for testing, which can take days or weeks.
Then you need to recruit the right users, but if you don’t do it very often, you need to start from scratch every time, and it can take weeks.
When recruiting users, you also have to get their consent for data processing. If there is not a clear policy on how to do this at your company, you are afraid of making mistakes. Or if there is a policy, it might be a time consuming manual process.
When you finally get into your interviews, you feel the need to answer everything, so interviews become very long, and you produce a lot of unstructured data.
So you may feel the need to record, transcribe and analyse all the data, to make sure you don’t miss anything.
And after analysis, you may feel the need to make a report or presentation, to start convincing other teammates or stakeholders of what you have learned.
In total, this can take weeks or months.
But you can do it a lot faster:
Start by automating the recruiting process. We recommend recruiting users from your website or app automatically into your calendar. Recruit from locations in your product that are most relevant for what insights you need. For example, if you are working to improve a particular feature, recruit users who are using this feature right now. This radically improves the relevance of your insights.
Automatically collect consent for data processing when you recruit participants. Build this into your automatic recruiting process, so that you’re sure that everything is handled correctly, without needing any manual work.
Start bringing teammates or stakeholders into your interviews as co-moderators or observers. Summarize your findings together immediately after each interview. This allows you to shorten the analysis, and directly apply your learnings in your development.
Now you can start doing interviews more often. As you get more experience, you don’t need to prepare as much for each interview. And you get more comfortable testing your work-in-progress designs, instead of needing to make perfect prototypes every time.
You can really build a habit in your team and product organization to talk regularly with users.
The more you do it, the easier it gets. And it will greatly improve your impact.
5. You now work full-time with this at UX Signals. How did it all start and where are you now with this new exciting venture?
My first experience with continuous interviewing was in a previous role as product manager at Porterbuddy. I had read an article by Teresa Torres and was inspired to do 5 interviews every week.
It was really difficult to set up, mix and match many tools, and having problems with no-shows, data privacy etc. But it brought us closer to our customers, and we succeeded way past expectations to increase our main success metric by 250%. It started spreading across teams, and stakeholders started to request to join the interviews.
Coming out of this experience, my heart and mind were set to make it possible for every product team in the world to do the same. So I started UX Signals to make that happen.
Now we build software that helps teams and companies automate as much of their interview operations as possible: Interview recruiting, consent management and more. And we coach teams and organizations on how to build interviewing habits into their development process, working closely with their research-, insights- or product ops people.
We want to empower every product team in the world to talk to their users regularly.
And we’re thankful to be working with some very knowledgeable customers early on, like NAV, Vipps MobilePay, Norsk tipping, Oslo municipality, Aidn and more. We believe building together with our customers will bring the best outcome for everyone involved.
Thanks for reading!
Consider subscribing for free so you don’t miss out on the next stories that are coming soon😉