Sarvam STT Experiment Report — v4, Formats & Quality for the SynQ-with-Saad Pipeline
Date: 2026-08-11
Video: vBOAHYlP4rc — “$30K Agency Mindset: Freelancer → Agency” ft. Husien Mustafa (1:18:31)
Purpose: Decide the default transcription path for the 13 long-form SynQ with Saad videos (Urdu/Hindi/Punjabi → English insights).
1. What we tested
| Variable | Options tested |
|---|---|
| Model | saaras:v4 (playground default, what Rahul used) vs saaras:v3 (docs’ “recommended”) |
| Mode | translate (direct speech→English) vs transcribe (source text) |
| Format (REST 28s probes) | MP3 128k, Opus 64k, WAV 16kHz mono |
| Format (Batch full video) | WebM/Opus (stuck) vs MP3 128k (worked) |
Key API facts confirmed:
saaras:v4is an official model (documented in the Batch Initiate API reference: “latest — Global + Indian English + 22 Indic languages”). It accepts the samemodeparams as v3. My earlier “v4 is undocumented” note was wrong; it’s just absent from the marketing docs pages.- REST endpoint caps at 30s (28s clips passed; 30s+ rejected).
- Batch: ≤2h/file, up to 20 files/job; pricing ₹30/hour (translate) / ₹45/hour (translate + diarization).
2. Format finding (important)
| Format | REST (≤30s) | Batch (full video) |
|---|---|---|
| MP3 128k 44.1kHz | ✅ | ✅ ~3 min for 78-min video (matches dashboard speed) |
| Opus 64k | ✅ | ❌ job stuck in Running >55 min (updated_at frozen) |
| WebM (native yt-dlp) | — | ❌ same stuck behavior |
| WAV 16kHz mono | ✅ | (not batch-tested) |
Conclusion: convert to MP3 128k before batch upload. Native Opus/WebM silently hangs the batch worker (no error, no completion — just a frozen job). This is likely why Rahul’s earlier dashboard upload “finished in 3 minutes” (MP3 from the UI) while our webm job never completed.
Upload mechanics that work (from the Furqan job, reconfirmed):
curl -X PUT -H "Content-Type: audio/mpeg" -H "x-ms-blob-type: BlockBlob" -T file.mp3 <presigned_url>
(-T, not --data-binary <path> — the latter uploads the literal path string, causing “Failed to read the file”.)
3. Model & quality finding
Head-to-head on known error spots (full batch, v4, MP3):
| Error in playground transcript (Rahul’s file) | v4 batch API output | v3 batch |
|---|---|---|
| ”Legionary channels” / “leagination channels" | "legacy channels” (1×) + “lead generation” correct 9× | “leagination channels” (still garbled) |
| “In 2005” (for 2025) | 2025 ✅ (2×) | 2025 ✅ (2×) |
| “Tad bhai” (for Saad bhai) | gone ✅ (0×) | gone ✅ |
| “up-listed” | gone ✅ | gone |
| ”clot code” | gone ✅ | gone |
| ”upper K” (for Upwork) | gone ✅ | gone |
| ”Saddam bhai” (for Saad) | 10× ❌ (persists) | 1× (in probe) — same bias |
| ”value of death” (for “valley of death”) | 4× — same in auto-caption (वैल्यू ऑफ़ डेथ) | 4× — same |
Verdict on “value of death”: NOT a Sarvam error — both Sarvam v3/v4 AND YouTube’s auto-caption independently hear “value of death”. The speakers likely say it that way. My earlier claim that auto-caption cross-check would catch it was wrong (both ASR engines agree). The meaning is unambiguous from context (the 3k–5k stuck zone).
Overall error rate (v4 batch, 17,695 words):
- 0.08% of words garbled (15 errors: 10× “Saddam”, 3× lead-gen variants, 2× name “Jani/Geni”)
- ~0.8 errors per 1000 words, all isolated proper nouns or single terms
- Zero sentence-structure damage. Numbers are all correct: 500/1000/2000/3000/5000/10000/30000/50000/100000, 70%/80%, 3k–5k–10k–30k, 50/mo, 90-day, 6-month — these are the insight-critical payload and they survived intact.
- Sentence-level English is clean and readable across all sampled sections (beginning/middle/end).
Timestamps:
- Batch returns chunk-level timestamps (311 chunks for this video) — usable for
&t=MMSSlinks. - Caveat: first chunk spans 0–82s (coarse at the start), chunks are sentence/phrase-length, not word-level.
4. Cost (for the 13-video pipeline)
| Item | Cost |
|---|---|
| This video (1:18:31, translate) | ₹39.3 ≈ $0.47 |
| All 13 long-form videos (~20 hrs) | ₹600 ≈ $7 |
| With diarization (₹45/hr, if ever needed) | ~$10.50 |
5. VERDICT — skip the extra overhead
Given Rahul’s bar — “we don’t need perfect names and non-important details until we can understand the sentence structure properly and extract insights”:
- Sentence structure: ✅ excellent (clean, grammatical English throughout)
- Insights extractable: ✅ all frameworks, numbers, steps, and stories are intact and usable
- Names/details: only cosmetic misses (“Saddam” for “Saad” — consistent, and a reader maps it instantly from context; “legacy” vs “lead generation” once)
→ Direct saaras:v4 + translate + MP3 batch is good enough. No need for transcribe→LLM-translate (2× cost, no marginal gain at this quality bar). No mandatory auto-caption cross-validation for insight extraction (it adds nothing for the “value of death” class and the remaining errors are cosmetic). Skip the overhead; run the simple path.
Recommended pipeline per video:
yt-dlp -f bestaudio(via Webshare proxyhttp://tyzmousa:rvutibqywubn@213.201.251.213:5407if bot-blocked)ffmpeg -i in.webm -c:a libmp3lame -b:a 128k -ar 44100 out.mp3- Sarvam batch:
model=saaras:v4, mode=translate, language_code=unknown, with_timestamps=true - Upload via
curl -T+x-ms-blob-type: BlockBlob+audio/mpeg - Start → poll
/status(3–5 min for ~1.5h audio) → download0.json→ timestamped English transcript
Optional cheap QA: spot-check the transcript against the YouTube auto-caption at 2–3 points per video (catches egregious ASR hallucinations); costs nothing, ~2 min/video.
Artifacts in this folder
results/v4_full_transcript.json— full batch output (v4, translate, MP3, timestamps)results/v4_full_transcript.txt— clean text versionresults/cell1..11.json— REST probe matrix (v3/v4 × mp3/opus/wav16 × translate/transcribe)probes28/— 28s probe audio filesvBOAHYlP4rc.mp3/.webm— source audio (75 MB each)- Stuck job (for reference):
20260811_de74f87d-3949-4afb-8386-f1f22d871a50(webm, abandoned) - Working job:
20260811_298ee49d-586c-4198-8c2c-250fb7a9f4a9(mp3, 3 min, success)