Agora.io · 2024 · 9 months
Designing for reducing time to first agent from hours to under 3 minutes.
Role
Principal Product Designer
Team
1 PM, 5 FE, 3 BE, 3 QA, +1 Designer
Timeline
9 months to launch
North star
Time to first agent: 30-60 min → 3 min
Context
Agora had the infrastructure. It didn’t have a product around it.
A Conversational AI SDK was live on top of Agora’s real-time audio network. Developers still had to stitch speech-to-text, an LLM, and text-to-speech together on their own before hearing a single reply.
Customers coming from call-center work wanted a faster path to production. Most never reached it.
Baseline TTFD
30-60 min
Sign-up to a first working agent before the Agent Studio redesign.
Objective
Shorten the path from evaluation to a first working agent.
Design a studio that improved confidence in moving beyond a demo. The product and business goals stayed narrow: help teams reach a reliable first test, then move more of them toward live deployment.
Product Goals
- Deliver a reliable first meaningful test with strong perceived responsiveness.
- Reduce integration effort so teams can embed the agent into existing systems quickly.
- Support one coherent workflow across build, test, deploy, and monitor.
- Strengthen observability with logs, alerts, campaign visibility, and issue tracking.
Business Goals
- Accelerate time-to-POC and move more evaluators into live deployment.
- Grow agentic voice minute consumption on the Convo AI engine.
- Establish a production-ready AI call-center offer for enterprise use.
- Support enterprise expansion through compliance and global readiness.
Research
Finding out where people actually gave up.
One week of discovery during product definition. Five interviews, one designer, a PM sitting in on half of them. This wasn’t a full study. It was lean research to understand where trust broke, where momentum dropped, and what had to change to get more teams to live use.
Research stack used to separate setup friction from production blockers
| Method | Source / sample | Why it was used | What it revealed | Limitation |
|---|---|---|---|---|
| Competitive analysis | LiveKit, Vapi, Retell: docs + trial sign-ups | Map what developers expected from setup, testing, and production workflows. | Comparable products reduced setup anxiety by making the first outcome and the production path clearer earlier. | Static product review without customer validation on each competitor. |
| Sales + customer asks | Active deals, solutions conversations, and customer requests | Surface the production questions showing up before teams committed. | Readiness concerns centered on cost, scale, permissions, and failure handling, not just setup speed. | Weighted toward enterprise and high-intent accounts. |
| Interviews | Five exploratory conversations with integration engineers, solutions architects, and product managers | Understand where confidence dropped between first interest and real deployment. | People wanted to hear the agent fast, know the next step, and understand what blocked them when something failed. | Small qualitative sample and not a long-term usage study. |
| Funnel + secondary research | ConvAI SDK analytics plus enterprise voice-AI demand reports | Check whether the interview friction also showed up in behavior and market expectations. | The biggest leak sat between sign-up, first test, and production-readiness questions, confirming trust had to be earned earlier. | Explains where drop-off happened better than why each team left. |
Insights and opportunities
01
Users trusted the agent only after hearing it.
When the first audible test took too long, most teams stalled before they could judge the product. Trust was earned through the first real conversation, not the setup screen. The experience had to get people to a working spoken interaction within minutes.
02
Production questions arrived before commitment.
Cost, scale, and failure handling came up before users were ready to commit. That made production readiness part of the buying decision, not a later documentation step. Those answers had to appear earlier in the product.
03
Deploy was not the end of the journey.
Getting an agent live did not tell users what to do next. Once people had to guess between more testing, web launch, telephony, or monitoring, momentum dropped and docs took over. The product had to guide the next move after launch.
Goals
Goals that shaped the studio.
Research clarified what users needed to accomplish first. Those user goals anchored the redesign, while the product and business priorities were defined earlier in the objective. We then measured the work through four behaviors that mattered commercially: adoption, activation, retention, and confidence.
- Reach a working first conversation within minutes of sign-up.
- Understand the next step across setup, testing, deployment, and monitoring.
- Diagnose blockers across channels, permissions, integrations, and infrastructure.
- Validate safely and trust the system enough for live customer-facing use.
We treated card added as the activation threshold because it showed a team had moved past free testing and was preparing the agent for real production use.
How these goals were measured
We tracked whether the redesign improved four behaviors that mapped directly to commercial progress: adoption, activation, retention, and confidence.
Adoption
A developer can configure a voice agent without writing integration code.
Time to a first configured agent in about 3 minutes.
Activation
A user reaches paid intent by adding a card after the 10,000-minute test stage.
Card-added activation rate at ~3× baseline.
Retention
People came back to monitor, test and configure agent or campaign performance.
Repeat build rate above 25 percent.
Confidence
The product answers production questions before a developer has to ask.
60%+ of deployers opened observability on their first deploy.
Growth Loop Hypothesis
Each goal was designed to feed the next. If the studio got teams to a first audible success quickly and kept guiding them into deployment, more of them would reach real production use. Production usage would then show which templates, prompts, and monitoring surfaces actually reduced friction, so the platform could improve the next team’s starting point. Held as a bet going in, not a proven mechanic.
A developer hears their agent respond in the same session.
Low friction means more builds get past the blank canvas.
We learn where developers drop off, what they need, and what is missing.
Those signals shape which models, prompts and flows we surface first.
The next developer inherits a smarter starting point.
- 01
First call
A developer hears their agent respond in the same session.
- 02
More agents
Low friction means more builds get past the blank canvas.
- 03
Usage signals
We learn where developers drop off, what they need, and what is missing.
- 04
Better defaults
Those signals shape which models, prompts and flows we surface first.
- 05
Shorter path
The next developer inherits a smarter starting point.
Assumptions
Explicit bets going in. Each shaped the studio’s design and became measurable.
| Assumption | Rationale | KPI | Success Metric |
|---|---|---|---|
Users need to talk to a working agent within the first three minutes. | Research showed trust in voice AI arrived through the first audible interaction, not through setup alone. | TTFD | ~3 minutes to first working agent |
If the studio builds confidence in scale and production readiness, more teams will move from testing to deployment. | Research showed cost, scale, permissions, and failure handling were commitment questions, not post-setup details. | Test-to-deploy conversion | 3x baseline rate |
Surfacing observability at first deploy will increase production confidence. | Enterprise buyers flagged cost, performance, and failure visibility as a gating factor for moving beyond a demo. | Observability adoption | 60%+ of deployers opened observability on their first deploy |
Clear next steps after the first live test will increase activation and repeat use. | Interviews showed teams lost momentum after deploy when the product stopped guiding whether to test more, launch, or monitor. | Repeat build rate | Repeat build rate above 25 percent |
Users would prefer a node-based graph editor for configuring agents. | Early exploration suggested advanced automation users might want visual orchestration. | Automation setup speed | Faster multi-step automation setup |
Solution
What Agent Studio shipped with.
Six surfaces covering the full agent lifecycle. Each one maps to a goal from the table above and the metric badge shows which one it drives.
01 · Build
AdoptionAgent configurable in a few quick steps.
- What it is: A guided form for wiring STT, LLM, and TTS with sensible defaults.
- Why it exists: Removed the stitch-three-vendors-by-hand step that blocked every first-time user in interviews.
Results
What moved, before and after.
Measured over the first 90 days post-GA, against the baseline funnel from the problem section.
Adoption
Time to a first configured agent
Before30 to 60 minutes
After3 minutes
Activation
Card-added activation rate
Before~1 in 10 (pre-studio funnel)
After~3× baseline
Retention
Repeat build rate (same dev, 2nd agent)
BeforeNot tracked pre-studio
After25%+
Confidence
Observability adoption
BeforeSurface didn’t exist
After60%+ of deployers opened observability on first deploy
Where assumptions failed
One of the clearest assumptions behind the original builder was that users would prefer a node-based graph editor for configuring agents. That assumption failed: the graph added setup overhead, buried the next action, and pulled the product away from the adoption outcome in the table above.
To move time to a first configured agent from roughly 30 to 60 minutes down to 3 minutes, the agent builder had to be redesigned around a simpler setup flow with clearer defaults and a faster path to live testing.
Before: node-based graph editor.
After: simpler setup flow with clearer defaults.
What stuck
TTFD dropped from roughly 30 to 60 minutes down to 3 minutes. Observability went from nonexistent to a default surface more than 60 percent of deployers opened on their first deploy. The growth loop from the goals section started compounding: more shipped agents fed better templates, and better templates shortened the next developer’s path.
What changed my thinking
It was making the agent audible within the first three minutes of signup. Once developers could hear their agent respond, trust followed. In voice AI, confidence is an audio problem before it is a screen problem.