Incentives, their goal and bias
First up, let's get specific. What do we mean when we talk about incentives and how are they different from rewards? The goal of incentives are to do just that, incentivize people to take time out of their day to spend on your study. A monetary reward can be an incentive, but it not necessarily is.
Monetary incentives are where UX research can face criticism. "Were people paid?" is not an uncommon question. With any (monetary or other) incentive you set, you introduce participation bias (e.g. self-selection bias, experimenter bias). Being well-informed about the impact of incentives, you can interpret results accordingly, and even use biases to your advantage. The amount and type of incentive people respond to are another data point to inform you about audiences' interest in your product.
Setting incentive amounts
At Web3UX we believe that in most cases a monetary incentive is a prerequisite, and recommend ~$1 per minute; Whether people are highly motivated to contribute to your project or not, they are giving you their time, and time holds value. However, the exact amount and other types of incentives (swag anyone?) will depend on the participant profile, their motivation, and your ask.
Profile
The specificity of the people you want to participate in your study, dramatically impacts how much you're going to have to pay to get people interested. It's a question of supply. If the people you want to talk to are scarce, you'll need a higher response rate to your invitation and you want to reduce the number of people dropping off. In market research, the likelihood of finding your participant profile is referred to as incidence rate. Low incidence rate means you pay a high incentive.
Motivation
To understand what makes an effective incentive we first need to understand what might motivate people to participate. Here's a few reasons that we've encountered.
- Earn an income
- Earn something extra
- Contribute to the project
- Learn about the project
- Favour the researcher
If an incentive is to learn about a project, it might be appealing to get exclusive access or information about this project. If people simply want to earn something extra a monetary incentive will carry more pull.
Understanding your target audience and participants' motivation can also help you pre-empt any eligibility deception and unintended coercion. In other words, if incentives are substantial, people might try and game your screener to appear eligible and feel like they have to answer questions to not lose out on the incentive. In these cases, it's is all the more important to thoroughly screen and get explicit informed consent; Pointing out that participants are free to stop or pause at any time without risk of losing the incentive.
When considering your target audiences motivation, also take localization into account. $500 might be a moderately reasonable incentive for some, but a substantial amount for others!
Ask
There are so many ways to conduct user research! Depending on your research question, you might need a high volume of people completing a single task or deeply understand an individual's use case. Your study method translates to both time and effort asked from participants. Some examples:
- First click study: ~2 min, low effort
- A usability test: 30 min, medium effort
- An in-depth interview: 60 min, high effort
- A diary study: 1 min a day for 2 weeks, high effort
If you're asking a lot from participants, but their motivation is low, expect to have to pay more than ~1 per minute. If you're asking people to do a small, fun task, lower incentives can be sufficient. (A good reason to think about how to make your research fun!)
Guidance
While incentives will vary for your study, here's some guidance to get started.
Format and type
In tandem with the amount, there are various formats and incentive types for you to consider. With format, in this context, we mean the way in which a participant will receive the incentive. A few examples:
- A lottery
- Flat, one-off
- Staggered, based on how long or how people participate
Another consideration, especially in Web3 research is whether to provide your incentive (partially), before, during or after participation. For example, you can provide people with crypto before or during a session so that they can use the funds with your product. As long as you are transparent about your approach when inviting participants!
Even monetary incentives can take different forms. For example you can transfer crypto to a participant's address, but you might also offer a gift card or airtime. Check out Bitrefill for some great options! Also, don't forget the power of personal notes and swag in case of in-person testing. To decide on the type of incentive we'd recommend checking with your marketing, finance and legal team. We also recommend setting up a policy for your project to introduce some consistency (and fairness) over time for similar study types. Here's a template to help you get started with that.
Basics
Regardless of the incentive amount, format and type you choose for your study, two things are critical: Set clear expectations and pay on time.
Here's how we describe paying participants in our Code of Conduct
You must set clear expectations for Participants before they agree to take part in your studies about how much they will be paid, what method of payment will be used, and when they will receive the payment. You agree to pay Participants on time.
In conclusion
There is no silver bullet when it comes to incentives. As a guideline we recommend ~$1 per minute and encourage you to consider your participant profile, motivation and your ask. Get creative with the format and type, keeping in mind the goal of incentives being to contribute to an appealing rewarding, and fun participant experience. If you'd like to bounce ideas about a appropriate incentive for your studies, please drop by on Discord.
The Web3UX panel is a community-owned marketplace for user research in Web3.
Web3UX is a design partner of Casama and supported by Web3 Creatives Foundation.