rr.Ronit Reddy
← All research
Multilingual speech

LinguaSense

Multilingual speech processing under code-switching and background noise.

PublishedCo-author · system development
Paper ↗

The problem

How can a speech system follow language changes when speakers switch languages in noisy environments?

A hybrid RNN–Transformer system combining phonetic language-boundary detection, contextual verification and DeepFilterNet noise suppression. The work began during my exchange at UCSI University and continued at Woxsen.

My role & contributions

I developed the hybrid speech-processing pipeline and integrated noise suppression and 8-bit quantization.

Supervision: Professor Shayla Islam, then Dr. Syed Javeed Pasha

Method & system

LinguaSense architecture

  1. Speech preparationDeepFilterNet enhancement, audio preprocessing and acoustic features
  2. Phonetic analysisRNN-based language-boundary detection
  3. Contextual verificationTransformer-based interpretation of the surrounding language
  4. Deployment8-bit processing; local initial analysis with cloud verification for complex cases
Overview adapted from manuscript Figure 1. Speech preparation includes enhancement, preprocessing and acoustic feature extraction.
  • Phonetic language boundaries and surrounding linguistic context serve different roles: the RNN tracks short-term changes, while the Transformer checks whether a switch fits its context.
  • Noise suppression precedes later acoustic feature extraction so that background sound is reduced before language-boundary analysis.
RNN–TransformerDeepFilterNet8-bit quantization
Implementation details
  • DeepFilterNet enhancement and audio preprocessing prepare MFCC acoustic features and temporal segments for the classifier.
  • The paper describes 8-bit processing with preprocessing and initial RNN analysis at the edge, and cloud Transformer verification for more complex cases.

Evaluation & results

The paper compares system accuracy on clean mixed-language data and at 15 dB SNR. Table III separately compares word-error rate with and without DeepFilterNet under light (20 dB) and heavy (10 dB) noise.

At 20 dB SNR, WER is 12.10% without DeepFilterNet and 4.90% with it. At 10 dB, WER is 31.45% without and 9.20% with DeepFilterNet. Lower is better.Enlarge figure ↗
Noise-suppression comparison · LinguaSense, Table III. Bars start at zero; lower word-error rate is better.
96.45%

Accuracy on clean mixed-language data

94.12%

Accuracy at 15 dB SNR

Accuracy follows the paper’s terminology; it is distinct from word-error rate. Source: LinguaSense, Table II.

Accuracy and word-error rate measure different properties and describe the paper’s stated test conditions.

Future work

  • Extend evaluation across additional acoustic conditions, keeping accuracy and word-error rate separate.
← Back to researchDiscuss this work ↗