Not creating things you don’t need: testing at Bulb

Hi everyone! I’m Caroline, a User Researcher here at Bulb. I’m part of Jenna’s team and have been here since the start of the year (in fact, on my second week at Bulb, I had the pleasure of meeting @mowcius testing out a prototype of an automated chatbot!). Currently, I’m working in the Growth pod to understand how we improve the sign-up and referrals experience. Before that, I was working the internal tooling pod.

One project we were doing recently was investigating whether we should build an “Email me my quote” feature on the website where people join Bulb. Rather than plunging a lot of time and money into developing a feature we’re not sure we need, we always test things first quickly and cheaply. Our designer Juan put together a quick clickable prototype that looked like a website, and we tested it with some non-Bulb members, who switched between a mobile phone and desktop.

Juan, our designer in the Growth pod at Bulb, sitting with the testing set-up with a computer and mobile phone

Our hypothesis was that by allowing users to email themselves their quote, it would be smoother for users to switch between browsing on a mobile phone, and concentrating on signing up on a desktop.

But… we were wrong.

It was clear that users don’t want to email themselves a quote from our website, for three reasons:

  1. Personal data and spam - users don’t want to enter their email address, handover personal information, and risk being included a spammy email list that they’ll never be able to turn off.
  2. Their own ways of saving - users already had their own ways to save quotes and other information, such as taking a screenshot, which was quicker than typing in their email.
  3. Too simple - sometimes users told us that they had (reluctantly) emailed themselves their quote in the past, but this was because the quote would otherwise fluctuate (such as with insurance) or because it was a very complicated list of options. Bulb’s quote’s are neither of these things, so there was no need to email it to themselves.

Through testing with a simple prototype, we knew this wasn’t something that we should invest time and money into - so we’re not going to!

In the future, we might revisit the idea of saving quotes, but in a way that ensures we don’t hold onto user’s personal information, and that is quicker than typing in an email address.

If you have any thoughts or stories related to saving quotes on website, I’d love to hear them below :smile:

:blush: Well now I do feel like a celebrity!

Interesting findings on this one but it makes sense!

I’d be interested to know more about your process for finding/picking people for user testing. I presume you have a process to try and ensure a varied selection of testers representative of your members?

@“Caroline W at Bulb” has just gone on a quick holiday. Maybe @“Spyri at Bulb” might be able to help with this?

@mowcius That’s an interesting question! There’s a quite a few factors that go into recruitment and checking that we have a representative sample. The big starting question would be whether participants need to be Bulb members or not. From there, we look to recruit a sample that isn’t too skewed towards one demographic (for example, we wouldn’t want to only talk to 18-30 year olds, as Bulb members are all ages and this wouldn’t be representative). However, we don’t have strict quotas - it’s up to the researcher to keep things balanced for their project.

Another example - when we recruit community members, we do try to spread them across different projects instead of having projects consisting of only community members, as we know the community tends to be a lot more energy conscious and informed than the average person in the UK! The exception to this would be if we have projects where we’re looking specifically for a certain group of people to talk to, such as Jenna’s EV project where she researched with only EV drivers.

Hope this answers your question!