All Case StudiesCommunity & Product Feedback Programs

Ring Customer Feedback and Escalation Program

Built a repeatable system that turned scattered community signals into prioritized product work — delivering $1M in annual savings from one app update and ~$400K from SEO-focused content improvements.

$1M

Annual savings from app update driven by community feedback

~$400K

Estimated savings from SEO-focused community content

Average monthly page views on the community forum

Product teams in the feedback loop

1 quarter

To realize SEO content savings

At a Glance

Role: Community Manager, Ring (an Amazon company)
What I built: A repeatable system that turned community signals into prioritized product work, tied each issue to contact volume and cost, and tracked whether fixes reduced customer contacts
Impact: $1M in annual savings from an app update, ~$400K from SEO content improvements, and a lasting feedback loop between customers and product decision-making
Partners: 5 product teams, Analytics, Operations, and cross-functional leaders
Timeline: Ongoing program with key results confirmed within one quarter to six months of each initiative

Tools Used

SQL
Discourse
Excel
Jira
Confluence

Methodologies

Hybrid operating model
Kanban
Agile
RACI-style governance
Lean (voice of the customer)

Where We Started

Ring had just launched a customer community forum, and it was already drawing high traffic and interaction, at roughly 1M monthly page views by the end of the first quarter. The conversations in the forum mirrored what we were seeing on Facebook and Reddit, but the forum gave us an advantage those channels didn't: we could steer the conversation with targeted content, and we had access to richer data, such as views and search behavior, that helped us understand customer issues and needs on a deeper level. What was missing was a connection to the business. There was no defined way to measure how community activity affected the company, and no established pathways for internal teams to act on the feedback or use it to influence product and content.

The Core Problem

High-volume community feedback wasn't consistently becoming prioritized product work. There was no defined way to measure its impact on the business and no clear pathway for internal teams to act on it. Three challenges stood out: separating signal from noise, creating shared visibility across Product, Technology, and community teams, and proving measurable value, since sentiment alone wasn't enough and the impact needed a dollar figure.

Approach

I used a hybrid operating model: continuous feedback intake combined with structured, cross-functional prioritization. This fit because the work was ongoing and needed to adapt, product changes required clear owners and handoffs, and the program needed both qualitative customer context and quantitative business impact. I centralized signals from community discussions, identified recurring themes, built a feedback loop with five product teams, and linked community engagement metrics to contact-reduction metrics to guide roadmap priorities.

What I Did

1
Launch and intake. I launched Ring's customer community forum as a source of direct customer feedback and product insight. Moderators logged and tracked key issues in a weekly-updated spreadsheet, giving us one consistent record.
2
Sizing and escalation. Issues were sized by community activity — a Large or XL issue had more than 1,000 views on a single post, or an average of six or more new posts per day on the same concern. When an issue crossed that threshold, I used SQL to pull call center volume and sentiment, then built models estimating full-year contact reduction if the issue were fixed.
3
Weekly targeted email updates covering the most important metric changes, sent to product teams and cross-functional leaders.
4
Bi-weekly stand-ups with the five product teams to review metric and sentiment shifts, resolve issues faster, and give teams quick sentiment reads after feature updates of every size.
5
Weekly community health reporting to operations leads and cross-functional leaders in the WBR, covering contact changes, trending issues, launch metrics, and feature performance tied to suggestion boards.
6
Case Example — The Missing Device Generation: Ring was releasing new device generations with different features, but the app had no way to show which generation a customer owned. I surfaced the pattern through community volume, matched it to call center data, and modeled the contact reduction. The app team added device generation to the device health section, delivering an estimated $1M in annual savings confirmed after six months.
7
Case Example — Targeting the Top 50 Issues: Customers searching for answers to common problems weren't finding them, so they contacted support instead. We identified the 50 issues customers looked for most and updated the community's SEO and backend structure. I built a model comparing forum views and engagement against call center contact reductions, validated by Analytics and Operations. Result: approximately $400K in estimated customer service savings within one quarter.

Results

$1M in annual savings from the device generation update, estimated at prioritization and confirmed accurate after six months in market
Approximately $400K in estimated savings from SEO-focused content on the top 50 customer issues, validated by Analytics and Operations and realized within one quarter
A roadmap change for a core customer pain point, driven by the data I surfaced
A lasting feedback loop between customers, community signals, and product decision-making, with goals and roadmap set using measurable metrics

The Hard Moment

Product teams didn't initially see the value in community feedback, and they didn't fully grasp the impact that high-visibility threads had on the company. Rather than trying to convince them, I started by answering the questions they already had, using the data and insights they cared about, and I did it on a regular cadence. That earned their trust within the first month. From there, I dug into other pieces of feedback with them and we moved toward real solutions. The model I built to put a dollar amount on each issue helped a great deal, because it made the impact something they could weigh against their other priorities.

What I'd Do Differently

I would have connected with product teams before launch. We were already seeing posts about the company in non-company-owned forums, and I could have shared those themes up front and asked product teams what they wanted to learn from our users. Instead, I approached them after launch and had to explain the value of the community while also learning what they needed. Starting earlier would have built that trust and alignment from day one.

All Case StudiesAI & Quality Programs

AI Language Quality Evaluation Program

Designed a scalable CX-focused scoring model for AI customer service transcripts, trained a cross-functional review team, and turned transcript-level defects into patterns senior leaders could act on, adopted across an entire product area to inform company-wide AI and LLM training.

500+

Transcripts evaluated in three weeks

Trained volunteer reviewers across disciplines

~40%

Of all defects traced to the top three quality categories

Defined quality categories in the scoring rubric

Product area adopted the model company-wide

At a Glance

Role: Senior Program Manager, Amazon
What I built: A scalable, CX-focused scoring model for AI transcript quality, a trained cross-functional review team, and a leadership readout tying findings to customer sentiment scores and KPIs
Impact: 500+ transcripts evaluated, ~40% of defects concentrated in top 3 categories, findings adopted across a product area and used to inform company-wide AI and LLM training
Partners: Science, Content and UX Design, Operations, Product, and a business analytics professional
Timeline: Three-week initial review; ongoing adoption across product area

Tools Used

Excel
QuickTime Screen Recording
PowerPoint
Word

Methodologies

Evaluation and governance model
Agile (iteration)
Lean continuous improvement (Plan-Do-Check-Act)
Stage-gate governance
Lean/Six Sigma-style measurement

Where We Started

AI-enabled customer service products were generating large volumes of transcripts, but teams lacked a consistent, CX-focused way to evaluate quality at scale. Quality findings were hard to translate into clear signals for product leaders, and there was no shared standard for what 'good' looked like.

The Core Problem

Customer-facing AI quality could not improve consistently without a shared standard. Four challenges stood out: no common criteria across reviewers, patterns hidden inside individual transcripts, hidden critical customer trust risks, and the need for an ongoing process rather than a one-time audit.

Approach

I used a hybrid, iterative evaluation and governance model. AI quality has to be re-evaluated as models evolve, a standardized score makes it possible to compare results and track trends, and leaders needed a concise link between transcript-level findings and customer outcomes. I designed an 11-category CX-focused scoring rubric, reviewed 500+ transcripts over three weeks, validated and totaled scores, aligned them with customer sentiment data, and shared results with senior leadership.

What I Did

1
Designed the scoring rubric. I created 11 defined categories of quality issues, covering everything from grammar to meeting customers' emotional needs — plus an 'other' category for anything outside them.
2
Built the review team. I recruited 12 volunteers from different backgrounds across the company so assessments would be fair and not shaped by a single team's viewpoint. I trained every reviewer the same way and gave them the same documents.
3
Ran the scoring process. Each transcript was reviewed by a volunteer who logged highlighted defects in a spreadsheet, matching each to a rubric category. The head of UX content design evaluated the spreadsheets, and I totaled the scores after validation.
4
Connected findings to business outcomes. A business analytics professional aligned the results with customer sentiment scores, showing how defects overlapped with sentiment and connecting them to leadership KPIs.
5
Cross-functional partnership. I worked with Science, Content and UX Design, Operations, and Product to understand how transcripts were currently feeding the models, so the evaluation could be as useful as possible and adjustments could be made based on what it showed.
6
Leadership readout. I reviewed the business analytics comparison against my findings and supported the leadership presentation. The readout focused on quality patterns and customer impact rather than confidential defect specifics.
7
Scaled the model. The evaluation was adopted across an entire product area and used to inform company-wide customer service AI and LLM training.

Results

500+ customer-service transcripts evaluated in three weeks by a team of 12 trained volunteer reviewers
Roughly 40% of all defects logged traced to the top three quality categories, giving teams a clear place to focus
Clear evidence of AI language-quality impact delivered to senior customer service leadership, with findings tied to customer sentiment scores and leadership KPIs
Adoption across an entire product area, with the evaluation used to inform company-wide customer service AI and LLM training
An ongoing input for improving AI training and LLM models

The Hard Moment

Everyone was already at maximum capacity, so few people had time to support the transcript audit. It was important to me that the audit not reflect a single point of view, so I wanted reviewers from a variety of disciplines and backgrounds across the company. To make the ask easy to say yes to, I built a presentation deck that laid out exactly how much time was expected and what the work involved, so it was never an open-ended volunteer request. I also offered something in return: an inside look at how AI and LLM work was being done in our organization, which was very new at the time. That combination got people on board.

What I'd Do Differently

If I were doing this work today, I would train an AI agent on the transcript data from this case study and use it to run larger reviews across transcripts from different locations. That would deepen our understanding of how to score transcripts, and I could hand it over to teams so they could run their own audits more easily.

Ring Customer Feedback and Escalation Program

Turning VP-level CX feedback into a repeatable prioritization system

AI Language Quality Evaluation Program