Multilingual Voice AI Claims: How to Actually Test Them

Every few weeks another Indian voice AI company announces a model with better numbers than the last one. On November 2, it was Bengaluru-based Arrowhead AI launching Fletch, its own text-to-speech model for customer calls, claiming speech generation starting in under 100 milliseconds and support for 12 "emotion styles" like calm, apologetic and firm across 14 languages including English, Hindi, Tamil, Telugu, Bengali, Malay and Indonesian. The company says it's already handling collections, sales, renewals and inbound support calls.

Those are good numbers, if they hold up on your calls, in your language mix, under your network conditions. That "if" is the part most buyers skip. Here's how to actually check it, whichever vendor you're evaluating.

Why vendor numbers alone don't tell you enough

Latency and language-count numbers are almost always measured in the vendor's own lab, on clean audio, in the vendor's choice of language and sentence. That's a reasonable way to report a benchmark. It's not a reasonable way to decide whether a model will work on a call center agent's patchy 4G connection in a tier-2 city, switching between Hindi and English mid-sentence the way your customers actually talk. Fletch's architecture — built on a diffusion-transformer approach called DiTAR that generates continuous speech rather than stitching together fixed units — is a genuine technical approach to lower latency, not marketing language. But the only latency number that matters to you is the one you measure on your own calls.

The same caution applies to every Indic voice AI vendor you'll evaluate this year, not just this one. Sarvam, ElevenLabs, Gnani.ai and Arrowhead are all racing on similar claims. Testing them the same way lets you compare honestly instead of comparing marketing pages.

This matters more in voice than in almost any other AI use case, because the cost of a wrong answer is immediate and public. A chatbot that fumbles a query gets a follow-up question. A voice agent that mishears a loan amount or mispronounces a customer's name on a live call loses the customer's trust in the three seconds it takes to notice. You can't A/B test your way out of that after the fact the way you might with a draft email a human reviews before sending.

Five things to test yourself before you pilot

1. Code-switching, not just language count. A model supporting 14 languages separately is a different claim from a model handling a customer who starts in Hindi, drops into English for a product name, and switches back. Feed it real transcripts from your own calls, including the mid-sentence switches, and listen for where it stumbles.
2. Latency under your actual network, not a lab connection. Run the test from the device and connection your agents or IVR will actually use, not a developer's office Wi-Fi. A 300ms round-trip over a weak mobile connection changes a "sub-100ms" model into a noticeably laggy one by the time it reaches the caller.
3. Emotion appropriateness, not just emotion variety. Twelve emotion styles sound impressive until the model picks "cheerful" for a collections call about an overdue EMI. Test the model on your worst-case scripts — a complaint, a payment reminder, a cancellation — and judge whether the tone fits the moment, not whether the feature list is long.
4. Accent and dialect coverage within a language. "Supports Tamil" can mean Chennai Tamil tested well and Madurai or Coimbatore accents didn't. If your customer base is regionally concentrated, test with speakers from that specific region, not just the language in general.
5. What happens when it fails. Every model mishears or misreads something eventually. Ask the vendor to show you the failure path: does the call route to a human cleanly, does the system flag low-confidence responses, or does it plough ahead and answer wrong with total confidence? This matters more for a collections or sales call than almost any other number on the spec sheet.

How to structure a one-week test

Don't start with a full pilot. Pull 20 to 30 real call transcripts or recordings from the workflow you're considering — collections reminders, renewal calls, support queries, whatever's relevant — strip anything sensitive, and run them through the model with your own team listening, not the vendor's. Score each one on the five points above using a simple pass/borderline/fail, and only move to a live pilot once you see more passes than borderlines on your own material. This takes a sales engineer a week, and it will tell you more than any comparison chart a vendor hands you.

Bring the people who actually talk to your customers into that scoring exercise, not just the team evaluating the technology. A collections lead will catch a tone problem on a sensitive call that an engineer scoring for latency and word accuracy will miss entirely. A frontline support supervisor will know immediately whether a regional accent sounds natural or sounds like a text-to-speech demo. Treat this week as due diligence you'd apply to any vendor handling customer-facing work, not as a technical formality before procurement signs off.

It's also worth asking what changes when you scale from 30 test calls to your actual daily volume. A model that handles occasional code-switching well in a curated test can behave differently against the full variety of how your customers actually speak once thousands of calls a day go through it. Ask the vendor for their own production numbers, not just lab benchmarks, and ask specifically how those numbers were measured — on what call volume, in what languages, over what time period. A vendor that can answer that clearly is a different proposition from one that only has a demo.

Voice is genuinely one of India's best-suited AI use cases — most customer conversation here still happens over the phone, in Hindi, Hinglish and regional languages, not in a chat window. That's exactly why it's worth testing claims this carefully instead of picking on a demo call: a model that sounds right in English and breaks down in Bhojpuri or code-switched Hinglish is a model that will under-serve the calls that matter most to your business. Our sister company, IndusLabs, builds voice AI and workflow automation specifically for Indian languages, accents and code-switching, so this is a test we run constantly, not a one-off checklist.

If you're building toward a voice agent pilot and want help setting up this kind of evaluation before you commit to a vendor, our AI Automation & Agents Bootcamp walks through exactly this process. And if you've already piloted voice AI and are trying to figure out whether you're getting full value from it, our India Voice AI Readiness framework is a useful next read.