How Far Can the Gemma 4 Speech Encoder Go Without Changing Its Weights? Saudi and Gulf Arabic Speech Recognition with a Frozen and an Adapter-Augmented Encoder
Paper · https://arxiv.org/abs/2610.28673
arXiv:2610.28673v1 Announce Type: new Abstract: We study the 305M-parameter audio encoder shared by Gemma 4 and EmbeddingGemma 2 (identical except for its final projection) for Saudi and Gulf Arabic speech recognition, without changing any of its weights. A frozen system trains only a 22.6M-parameter CTC reader over a mix of encoder layers; an adapted system adds 3.1M low-rank adapter parameters, a 41.5M-parameter hybrid CTC/attention reader and joint decoding with a sub-word language model. Against Cohere Transcribe Arabic (2.07B parameters), re-run on identical audio, the frozen system trails in word error rate (WER) but is statistically indistinguishable in character error rate (CER) on five of six human-labelled conditions, while running 3.4 times faster on CPU with greedy decoding. The adapted system is statistically indistinguishable in WER on a public Saudi customer-service call (19.20% against 19.27%) and on clean Emirati speech, is significantly better on telephone-coded Emirati speech, has a significantly lower CER on all six conditions of these sets, and beats a 0.6B recogniser fine-tuned on SADA in WER on all six; it still trails Cohere in WER on long-form Saudi speech. On a blind, human-corrected public call-centre sample it ranks 4 of 15 systems, behind three commercial APIs (significantly only one) and ahead of the other self-hostable systems by point estimate. An optimised joint decoder runs it at 139 times real time on one GPU. We also show that references made by correcting machine drafts favour the drafting system. Code and public-set results are released.
First seen by Oriel on .
AI summary
The study evaluates Gemma 4’s unchanged 305M-parameter audio encoder for Saudi and Gulf Arabic speech recognition, finding that a small adapter-augmented system approaches or beats larger recognizers on several benchmarks.
- The frozen setup trains a 22.6M-parameter CTC reader; the adapted setup adds 3.1M low-rank adapter parameters, a 41.5M hybrid CTC/attention reader, and joint decoding.
- The frozen system runs 3.4 times faster than Cohere Transcribe Arabic on CPU and has statistically indistinguishable character error rate on five of six human-labelled conditions.
- The adapted system matches Cohere in word error rate on a Saudi customer-service call and clean Emirati speech, and performs significantly better on telephone-coded Emirati speech.
- It beats a 0.6B recognizer fine-tuned on SADA in word error rate across six conditions but trails Cohere on long-form Saudi speech; an optimized decoder reaches 139 times real time on one GPU.
- The adapted system ranks fourth among 15 systems on a blind public call-centre sample, and the study finds that machine-draft-corrected references favor the drafting system.
#speech recognition #arabic #gemma #parameter-efficient fine-tuning
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2610.28673v1 | https://arxiv.org/abs/2610.28673 Read discussion |