QUICK SUMMARY
The session explores how traditional survey research creates a “say-do gap” where consumers’ stated intentions differ from their actual behaviors. Speakers Rashed Chowdhury from Orchard and Sarah Snudden from JDE Peet’s present a solution using social media as a behavioral research lab to test concepts in real-world contexts. Their approach helped the coffee brand Lore successfully leverage its French heritage in marketing by testing content in authentic social media environments rather than relying solely on traditional survey methods.
KEY QUOTES
- “The say-do gap where we as humans say one thing with our intent and then do something very different in reality.”
- “What if you could move beyond hypotheticals, beyond this rational environment and actually measure what consumers do by going where they naturally live?”
- “How often do you get to test your claims and get media impressions at the same time? You’re getting how they pause, how they click, what they like… levels of engagement that are so much more of a true signal.”
FULL SESSION SUMMARY
Understanding the Say-Do Gap
The session began with Rashed Chowdhury, CEO of Orchard, introducing the concept of the “say-do gap” – the disconnect between what consumers say they will do and what they actually do. He illustrated this with a personal example of consistently failing to wake up early for the gym despite his stated intentions. Sarah Snudden, his partner from JDE Peet’s, shared a similar example about intending to stretch more but only watching stretching videos on Instagram instead. These examples highlighted how human behavior often contradicts rational intentions.
Chowdhury explained that while the say-do gap isn’t new, it’s growing wider in today’s chaotic consumer environment. Consumers are overwhelmed with choices, attention is scarce, and decisions are increasingly impulsive and emotional. Traditional surveys place respondents in a rational, test-taking mindset that differs significantly from their actual decision-making processes in daily life. This creates substantial business risk when companies make multi-million dollar decisions based on what consumers say rather than what they do.
The Social Media Solution
Orchard’s approach, described as “testing in the Wild,” leverages social media as a behavioral research laboratory. With 5 billion global users across platforms, social media offers an “always-on global focus group” where companies can conduct scalable experimentation in real-world contexts. The key advantage is that consumers don’t realize they’re part of a test, providing authentic feedback through their natural behaviors.
Instead of asking hypothetical questions, this approach measures actual consumer actions – clicks, likes, scrolls, add-to-carts – throughout a complex consumer journey with all its real-world friction. It reaches consumers when they’re in their natural state (e.g., scrolling on their phones late at night) rather than in an artificial survey environment.
Case Study: Lore Coffee
Sarah Snudden shared her experience applying this approach to Lore, a premium French coffee brand. She first contextualized the challenge by recounting a previous experience at Keurig where traditional research had led them astray. In that case, survey results strongly favored an antioxidant coffee blend, but when both products were launched, a focus-enhancing blend significantly outperformed the antioxidant option in actual sales.
For Lore, the goal was to differentiate the brand by emphasizing its French heritage in a market dominated by Italian espresso brands. Traditional black-and-white survey testing provided useful but limited insights. By contrast, Orchard’s approach allowed them to test how consumers responded to actual social media content featuring visually appealing coffee imagery alongside different messaging approaches.
The team tested three campaign concepts, focusing on quality, variety, and French excellence. While they knew quality and variety were important attributes, they particularly wanted to determine if they could successfully leverage the brand’s French roots. Traditional rational testing had struggled to evaluate the emotional appeal of “French excellence,” but by presenting it through engaging social media content, they could measure genuine consumer response.
Results and Implementation
The testing revealed that the French excellence messaging resonated strongly with consumers when presented in this authentic context. This gave the team confidence to proceed with a French-focused campaign. Snudden highlighted that this approach provided multiple benefits: they could test claims while simultaneously gaining media impressions, and they could measure deeper engagement signals like pauses, clicks, and likes.
The insights helped Lore successfully compete against larger brands in the Nespresso original capsule space by creating meaningful differentiation through its French heritage. For the testing, they created approximately 30 different content variants and deployed them across Facebook and Instagram (with Orchard also mentioning capabilities for testing on Google and TikTok).
KEY TAKEAWAYS
- The say-do gap is widening as consumer environments become more chaotic, making traditional survey research increasingly unreliable for predicting actual behavior.
- Social media platforms offer a powerful alternative research environment where companies can test concepts in real-world contexts with consumers in their natural state.
- Emotional and experiential brand attributes that are difficult to evaluate in rational survey environments can be effectively tested through engaging social media content.
- Measuring actual behavior (clicks, engagement, etc.) provides more reliable insights than asking consumers to predict their future actions.
- Testing in authentic environments allows brands to evaluate how messaging resonates when competing for attention in consumers’ actual media consumption contexts.
- The approach combines research with actual marketing exposure, providing dual benefits of insights and initial market impressions.
DELIVERY ON EVENT FOCUS:
Aligning Innovation with Business Strategy
This session directly addressed the event’s focus on aligning innovation with business strategy by demonstrating how more accurate consumer insights lead to better strategic decisions. The speakers showed how traditional research methods can misalign innovation efforts with actual consumer preferences, while their social media testing approach helps ensure that product and marketing innovations truly resonate with target audiences. The case study demonstrated how this alignment helped Lore successfully differentiate itself in a competitive market.
DELIVERY ON EVENT THEME:
Harvesting Innovation and Sowing the Seeds of Future Growth
The session connected to the event theme of “harvesting innovation and sowing seeds of future growth” by showing how better research methods can cultivate more successful innovations. By testing concepts in authentic environments, companies can identify which innovations will truly flourish in the market rather than wasting resources on concepts that only perform well in artificial testing environments. The approach helps companies plant the right seeds (innovations that genuinely resonate with consumers) and harvest greater returns on their innovation investments.
ACTION PLAN FOR INNOVATION EXPERTS
- Audit current research methods for potential say-do gaps by comparing past research predictions with actual market performance.
- Pilot social media testing alongside traditional research to compare results and identify discrepancies.
- Develop more engaging, realistic stimuli for concept testing that better represents how consumers will encounter products and messaging in real life.
- Incorporate behavioral metrics into innovation evaluation processes rather than relying solely on stated preferences or purchase intent.
- Create cross-functional teams between research, marketing, and innovation departments to design tests that simultaneously gather insights and build market awareness.
- Establish a framework for evaluating emotional and experiential brand attributes that traditional rational testing struggles to measure accurately.
- Implement continuous testing throughout the innovation process rather than relying on single go/no-go research moments.