Turning a low-adoption beta into a multi-year strategy
Email Insights launched at Inbound 2024 and reached only 7.5% of customer portals in beta. I diagnosed why and led the work to increase the impact. Then I turned the research and beta feedback into the foundations of HubSpot’s QA agent, and a multi-year vision for messaging reporting.
My role
Sole product designer, from first research to the QA agent’s foundations. Wrote the beta impact review that set the team’s strategy.
Team
Product triad: PM, engineering, design
Timeframe
Research Feb 2024 · Inbound launch Sep 2024 · beta review Jan 2025 · QA agent foundations 2025
Scope
Email performance insights for Marketing Hub, ungated to 11,527 customer portals, then pre-send checks
The call I made
Treat low adoption as two problems, reach and impact, and own impact
Result
Happy CSAT 38% → 48% · Unhappy 33% → 21% · Pre-send work shipped as QA agent
The stakes
The business problem was churn risk: marketers who can’t tell what’s working in their email compare HubSpot with other tools, and leave. Email Insights was the answer, launched as Messaging Insights in HubSpot’s Fall 2024 Spotlight at Inbound, with a bar of 60%+ of email customers using it.
Five months in:
7.5%
of customer portals reached (861 of 11,527)
10%
of surveyed users came back to it consistently
30%
of surveyed users wouldn’t recommend it
60%
pre-send insights topped what users asked for in the survey
Users asked for more. The CSAT said something else:
Customers without insights
Complained there were none.
Customers with insights
Said they were too generic to act on: “they don’t pinpoint specific issues and instead focus on pushing generic resources.”
My decisions
01
Set the comparison criteria that define “good” for each email
5 of 5 marketers judge results against their own history: “I am my benchmark.”
Insights compare an email with its campaign and its own past months, shipped as comparison modes.
What I didn’t do
Rank every email only against the account average
Trade-off I accepted
More comparison modes for marketers to choose from, in exchange for numbers they trusted
02
Challenged (with data) the idea that the email list wasn’t for reporting
Marketers in the sessions reported from the list: “I would definitely show it to my boss.”
Insights on both pages: a quick signal on the list, full depth on the post-send page. The list’s insights button became the most clicked action.
Before: the email list as a management table: numbers, but no sign of what’s good or bad.After: 1. anomaly detection on the list, 2. root cause and 3. recommended actions in the panel.
What I didn’t do
Keep insights to the post-send page only
Trade-off I accepted
Two surfaces to keep consistent, and a list that later had to scale to hundreds of insights
03
Split low adoption into reach and impact, and owned how to improve impact
Two problems held the beta back. Reach: most portals never got an insight, and engineering took that on. Impact: those that did found insights too generic to act on.
Exact, prioritized insights, one summary per email. Happy CSAT 38% → 48%.
Type of insights: from the whole portal down to a program, all the insights the system could generate.Fallback order: the most valuable insight available always shows first, from the portal’s own data down to best practices.
What I didn’t do
Wait for reach to be solved, or write generic tips
Trade-off I accepted
Fewer, actionable insights per email, built in parallel with engineering’s reach work
04
Challenged (with data), again, the push to remove insights from the list
After launch, the team’s old assumption came back: the list is for managing emails, not reporting. I answered it with new data, not opinion: 42% of insight clicks came from returning users.
Kept the most used action in the product, and solved scale without removing it.
What I didn’t do
Remove a feature people relied on to solve a scaling problem
Trade-off I accepted
A capped counter and a longer panel, instead of a cleaner list page
05
Pushed insights to pre-send, and surfaced the engineering risk early
Pre-send was the most requested feature (60% in the survey).
Tech debt and AI uncertainty surfaced in an early workshop with the engineering team, not after a full design. The project paused, and later became the QA agent’s starting point.
What I didn’t do
Push a design the team wasn’t ready to build
Trade-off I accepted
The most requested feature waited, and the cost of learning that was a workshop, not a build
06
Owned the foundations of QA agent, launched at Unbound 2026
160 marketers ranked links, spam and spelling top of 37 needs.
An AI agent that reviews every email before it goes out. Deterministic checks catch what has one right answer, like broken links. Non-deterministic LLM checks judge what needs reading, like tone and readability. Marketers write their own custom checks and set a severity for each. Launched in 2026 as QA agent.
What I didn’t do
Send every check to an LLM
Trade-off I accepted
Safety checks first; AI recommendations came in later phases
07
Set a multi-year vision: moving from email-specific insights to insights across all messaging platforms, tied to business goals
Connecting email to revenue was marketers’ #2 pain point.
A three-stage north star, led with my PM. Its first two pieces, post-send insights and pre-send checks, have shipped.
Efficient messaging (marketers): insights before and after every send.
Maximise impact (marketers and analysts): a “My programs” view against monthly goals.
Maximise results (managers and executives): one view across channels, with revenue attribution.
[Vision video]
[Caption for the vision video]
What I didn’t do
Present it as the final roadmap
Trade-off I accepted
A north star that overlaps campaigns, reporting and lead scoring, with ownership questions left open
Beyond the triad
Leadership
Leadership adopted my multi-year vision, and it set the order of what came next: pre-send insights first, then the QA agent.
The editor
My triad owned post-send reporting. The vision moved insights into the email editor, a surface outside our scope, and shaped a product beyond my team’s remit.
QA agent task force
My pre-send research brought the task force to an agreed POC direction in two weeks. That work became the foundation of QA agent, launched at Unbound 2026.
My triad
My beta impact review answered what impact we’d driven, what customers told us and what we’d do next. Its answer, reach and impact in parallel, became the plan. I also ran the North Star workshop: do marketers who use insights see their open and click-through rates rise?
What happened
38% → 48%
Happy CSAT, beta to GA
33% → 21%
Unhappy CSAT, beta to GA: down by more than a third
Shipped. Comparisons by campaign, HubSpot benchmark and same email month over month, plus “To improve” / “Doing well” recommendations, are documented here.
What I’d do differently
Test the data threshold before launch. Insights need 30 emails of the same type, each to 250+ recipients, in 90 days. Most portals don’t send that. Checking reach against real portal data before Inbound would have caught it before customers did.
Measure the North Star from day one. CSAT shows customers liked it more. Tracking open and click-through rates from launch would show whether they became better email marketers, which is the result that matters.
Public sources
Fall 2024 Spotlight (HubSpot): the Inbound 2024 launch, as Messaging Insights