Code-switching — the practice of alternating between two or more languages within a single conversation or even a single sentence — is one of the most common features of multilingual speech, and one of the most poorly handled scenarios in automatic speech recognition.
If you’ve ever tried to transcribe a bilingual conversation with a standard cloud transcription tool, you’ve likely seen the result: the tool picks one language at the start and forces everything into that language, producing nonsense at every point where the speaker switches.
This guide explains what code-switching is, why it’s hard for ASR systems, and how Kajo Voice handles it.
What is code-switching?
Code-switching is the alternation between two languages or language varieties by a multilingual speaker. It’s not a sign of language deficiency — research consistently shows that code-switching is a sophisticated linguistic behavior that follows its own grammatical rules.
Common code-switching contexts:
- Spanglish (Spanish-English): Dominant in US Latino communities. “¿Me entiendes what I’m saying?”
- Francarabe (French-Arabic): Standard in North Africa and among the French diaspora. “Je pensais qu’il était مجنون but that’s fine.”
- Hinglish (Hindi-English): Pervasive in Indian professional and urban speech. “Meeting cancel हो गई because the client didn’t show up.”
- Taglish (Tagalog-English): Common across the Philippines. “Nagpunta na ako doon pero it was closed.”
- Singlish and Singaporean Mandarin/English: Complex trilingual mixing common in Singapore.
- Cantonese-English: Standard in Hong Kong professional contexts.
- Lebanese Arabic-French-English: Triple code-switching in Beirut professional circles.
Why code-switching breaks standard ASR
Most ASR systems are designed around a single-language assumption: the model is configured for a language at the start, and all audio is transcribed as that language.
When a speaker switches to another language, several failure modes occur:
Forced same-language transcription: The model tries to interpret the foreign-language words as the configured language. “¿Me entiendes what I’m saying?” becomes “Me intensive what I’m saying?” (if English mode) or “¿Me entiendes gua Amiens?” (if Spanish mode).
Language detection confusion: Some models attempt to detect the language dynamically, but do so at the file or paragraph level — not sentence by sentence or word by word. A short code-switch gets absorbed into the wrong language context.
Silence or deletion: Some models simply drop or silence segments in an unrecognized language, creating gaps in the transcript.
Phonological interference: Sounds in the switched language that don’t exist in the first language may be phonologically “corrected” into the closest equivalent in the wrong language, producing unintelligible output.
How Kajo Voice handles code-switching
Kajo’s engine decides the recording’s language once, from its opening, then decodes with a vocabulary that spans all 98 languages it knows. In practice short switches — a borrowed phrase, a name, a technical term — usually come through as spoken; long runs in the second language can be pulled toward the detected one.
In practice that means:
- One language decision per recording — automatic, or the language you set in Settings → Transcription
- A multilingual vocabulary during decoding, so switched words are not forced through a single-language dictionary
- A mixed transcript that keeps each language as spoken — e.g. “We discussed the خطة التسويق and they seemed interested” rather than forcing everything into Arabic or English — with the switch points reviewed by you
This is the model’s own behaviour inside a desktop pipeline — not a separate dedicated code-switching product layer. Pairs where both languages are well represented in public speech data (English with Spanish, French, Arabic, Mandarin, or Hindi; French with Arabic) tend to hold up; pairs involving lower-resource languages, and three-way mixing, need a closer review. The full list of transcribable languages is on /transcribe.
Practical implications for researchers
For sociolinguists and anthropologists studying code-switching patterns, Kajo’s output is not just convenient — it’s a meaningful methodological improvement over tools that erase the code-switching signal entirely.
A researcher studying code-switching in Mexican-American families gets a transcript that preserves the language alternation patterns rather than forcing homogenization. This is analytically significant: the choice of when to switch and into what language is data.
The JSON export carries each segment’s start, end, speaker, and text, so a researcher can tag switch points in a script or spreadsheet and compute switch frequency and triggers (before technical terms? after address terms? at discourse boundaries?) from the timestamps.
Bilingual subtitle export
For documentary makers and journalists working with multilingual tape, Kajo’s bilingual subtitle export is directly relevant to code-switching material. A documentary featuring code-switching interview subjects can be exported with:
- an original-language track, preserving the code-switching as the speaker produced it
- a translation track: a full on-device translation of both languages into a target language
This respects the linguistic authenticity of the source while making it accessible to monolingual audiences.
Limitations
No current ASR system handles all code-switching scenarios correctly. The most persistent challenges:
Very short insertions: A single word from another language inserted into a sentence may be phonetically approximated in the dominant language.
Rare language pairs: Code-switching between two low-resource languages (e.g., Somali-Amharic) is not well supported by any current model.
Heavy accent interaction: When code-switching occurs with heavy regional accent, both the accent and the language transition affect accuracy independently.
Phonological blending: Bilingual speakers sometimes blend the phonology of two languages in ways that are different from either language alone — early-age heritage speakers, in particular, may produce language-blended forms that aren’t clearly one language or the other.
For research where code-switching transcription accuracy is critical, human review of Kajo’s output by a fluent bilingual reviewer is still recommended. Kajo reduces the review burden compared to transcribing by hand, but doesn’t eliminate it for high-stakes research.
Getting started with multilingual content
- Import your multilingual audio file into Kajo
- If one language dominates, set it in Settings → Transcription so a short opening in the other language cannot mislabel the whole file; leave detection automatic only when the languages are roughly balanced
- After the first file, skim domain-specific terms in either language and fix misses with inline edit
- Review the transcript, focusing on code-switch points for accuracy
The transcript will show the mixed-language speech as spoken — review switch points carefully.
Related: For the broader multilingual workflow (98 languages, bilingual subtitles, accented speech), see Multilingual Transcription.