Concept testing questions: 10 examples for your next survey

What should concept testing questions measure?

Useful concept testing questions separate understanding, relevance, appeal, alternatives, tradeoffs, and conditional intent. Start with what people think the idea does, then ask how it fits their situation. Every retained question should inform a named decision: revise the description, change the offer, investigate a barrier, or plan a more realistic test.

Concept testing evaluates a proposed product, service, or offer. Message testing evaluates how an offer is communicated. If two versions change both the service and its headline, a favorable response cannot tell you which change helped. Use comparable descriptions when the decision concerns the underlying concept.

Set up one concrete decision and a neutral stimulus

This guide uses a fictional household-repair organizer. Its description is: “A service that stores household repair receipts and repair history and lets you schedule reminders. You enter or upload the information. It does not book contractors or inspect your home.” This is an invented teaching example, not an available product or a study MTB has conducted.

An initial decision could be whether to prototype this organizer or first revise its explanation. Record the intended audience, what counts as accurate comprehension, the primary outcome, and the unacceptable misunderstanding before recruitment. For example, belief that the service inspects a home is a guardrail concern. A team must justify its own decision threshold; this worksheet supplies no validated pass mark.

Consent and essential eligibility checks come before exposure. Avoid detailed questions about repair frustrations before your primary concept measures unless that context is deliberately part of the design. Use the same description, image detail, device presentation, and exposure rules across comparable groups.

Ten concept testing questions with decisions attached

These are proposed questions to adapt and pretest, not a validated scale or a requirement to ask all ten. The downloadable matrix contains the complete response options, decision and interpretation risk for each row. The conditional price question belongs in a later block; the first measures assess the unpriced concept.

• Q01 — In your own words, what would this service do? Decision: revise the concept description if the central function is misunderstood.
• Q02 — What, if anything, is unclear about how this service would work? Decision: identify missing information before adding features.
• Q03 — How relevant, if at all, would this service be to the way you currently manage household repairs? Decision: decide whether the proposed use case fits this audience.
• Q04 — Overall, how appealing or unappealing do you find this service? Decision: assess appeal after checking comprehension.
• Q05 — What most influenced your answer about appeal? Decision: identify the reason behind a favorable or unfavorable rating.
• Q06 — How do you currently keep track of household repairs? Decision: identify the real alternative the concept would replace.
• Q07 — Which, if any, of the described features would you be least willing to lose? Decision: choose a feature to preserve in a simplified prototype.
• Q08 — What would you need to know before deciding whether to try this service? Decision: identify the next information or prototype requirement.
• Q09 — If this service were available for $4 USD per month with monthly cancellation, how likely or unlikely would you be to try it in the next three months? Decision: decide whether a realistic next-stage offer test is worth planning.
• Q10 — What, if anything, would make you decide against trying this service? Decision: identify an unresolved barrier or guardrail for the next test.

Keep the answer choices honest

Separate a neutral opinion from “cannot judge.” An undecided respondent should not be silently coded as a negative response or removed to inflate favorable ratings. The example uses balanced positive and negative categories with a labeled midpoint. Ordered rating scales should remain in a readable order.

For the feature question, vary the position of the three feature names while keeping “none,” “no preference,” and uncertainty choices available. Feature prompts reveal specific benefits, so place them after the unaided comprehension measures. Ask why people chose their appeal rating in both directions; probing only positive answers would leave the explanation incomplete.

Choose monadic or sequential monadic deliberately

In a monadic design, each participant evaluates one concept. Random assignment to separate groups helps compare concepts without showing participants the competing alternatives. More concepts usually mean more separate cells to recruit. The MTB planner can show this operational requirement, but its cell totals are not a statistical power calculation.

In a sequential monadic design, each participant evaluates multiple concepts in turn. This can reduce the number of distinct people needed for a fixed number of evaluations, but it lengthens exposure and lets earlier concepts influence later judgments. Randomizing concept order distributes position effects; it does not erase memory or fatigue.

Repeated ratings from the same person are correlated. Do not put those totals into MTB’s independent-groups message-comparison calculator. A paired or repeated-measures analysis appropriate to the outcome and assignment is needed. With separate, independent groups and a pre-specified binary outcome, that calculator may be relevant; a five-category appeal scale must not be treated as a binary count without a justified coding rule.

Interpret a conflict instead of averaging it away

Imagine, without assigning invented respondent counts, that a future test finds high appeal alongside frequent belief that the organizer books contractors. The defensible next step would be to investigate or revise that misunderstanding. Averaging appeal and comprehension into one score would hide the reason the favorable response may not transfer to the real service.

Likewise, a respondent can understand the concept and dislike it. That is useful evidence about the offer, not automatically a wording failure. Keep separate records of comprehension, appeal, barriers, and the reasons offered. Preserve uncertain and skipped answers and disclose the denominator for each measure.

The $4 example asks about an explicit hypothetical offer. It does not establish willingness to pay across prices, predict conversion, or prove market demand. Availability, competing offers, trust, actual effort, and payment behavior differ from a survey scenario. Use intent to plan the next learning step, then evaluate real behavior only through a lawful, truthful test.

Turn the matrix into a reviewed instrument

Download the CSV and replace the fictional stimulus context before use. For each row, record whether the question stays, changes, or is removed, and why its answer could change your decision. Keep the original question ID so revisions remain traceable. Ask a small pretest group to explain how they understood the wording; document resulting edits without claiming that a small pretest validates the instrument.

Before fielding, write recruitment and exclusion rules, coding instructions, comparison plans, and limits on subgroup exploration. After fielding, report exact wording, response options, exposure sequence, achieved sample, dates, exclusions, uncertainty, and limitations. This page and its CSV contain no collected responses. MTB currently provides planning resources, not a recruiting service.

Sources and scope

1. Writing Survey QuestionsPew Research Center

   Primary methods guidance on wording, response categories, pretesting and question-order effects. The question bank here is MTB’s own proposed application, not a Pew instrument.

2. Concept Testing ProgramQualtrics

   Product documentation distinguishes single-concept and multiple-concept exposure. It is a commercial platform source, not validation of this worksheet.

3. Monadic versus sequential monadic survey designSurveyMonkey

   Commercial research-platform explanation of separate and repeated exposure, questionnaire burden and recruitment tradeoffs.

Sources support the specific statements described above; they do not validate this publication’s rubric, guarantee a compliant execution, or replace context-specific professional advice.