Communication Voice Translation and Replay Function

1 Core Function Implementation

Deploy a dedicated local voice translation large model at the client's site without relying on external APIs, enabling real-time conversion of voice streams into text plus real-time Chinese-English bilingual interpretation during cross-border voice calls. Translated text is displayed in real time on the agent screen side with latency ≤3s. Implement automatic full-call recording and centralized storage for all calls, with recording files automatically associated with call records. Support synchronized highlighted playback of translated text and audio, full-text keyword search, online recording query period ≥6 months, and long-term archiving for future reference.

2 Technical Implementation Solution (Core Highlights, Quantified Implementation Across All Stages)

(1) Local Large Model Real-time Voice Translation

Adopt an offline-deployable local voice translation technology stack: ASR uses the FunASR (Paraformer/Zipformer) training and fine-tuning toolchain, with inference side using sherpa-onnx (ONNX Runtime) for offline inference; NMT uses the CTranslate2 + Marian/OPUS-MT (or NLLB-200 Distilled) offline translation engine. All model files are stored locally, no external APIs are invoked, and one-click offline installation and upgrade can be performed within the client's intranet.

Real-time processing pipeline: audio stream is segmented into 500ms chunks entering VAD (Voice Activity Detection) → ASR streaming decoding → NMT streaming translation → sentence segmentation/punctuation restoration (CT-Transformer/Paraformer PUNC) pipeline; end-to-end latency ≤3s (meeting tender requirements), with support for hotwords/custom vocabulary taking effect immediately to improve recognition and translation accuracy in the civil aviation domain.

Translated text supports real-time sentence segmentation, automatic punctuation insertion, text editing and saving. The translated text layout is clear, synchronized in real time with the voice stream, and supports re-editing of translated text in the playback interface.

(2) High-reliability Recording and Tiered Storage Solution

① Recording Service Implementation

Deploy a domestic high-availability recording server cluster, implementing automatic full-call stereo recording of all internal/external line calls, VHF communications, conference calls, and video presentation audio/video based on the SIP REC protocol. Support standard audio formats such as WAV (lossless) and MP3 (universal). Recording is complete without omissions or stuttering, with recording success rate ≥99.99%.

② Tiered Storage Architecture and Capacity Planning

Adopt a tiered storage and archiving strategy to ensure the online recording query period is no less than 6 months, with support for longer-term archiving for future reference. Provide a storage architecture and capacity planning solution to meet tender requirements.

③ Quantified Capacity Planning

Based on 70 agent seats at the client's site, with an average of 4 hours of calls per seat per day, estimated at 16kbps in MP3 format, the annual new recording storage requirement is approximately 1.2TB. This solution initially configures ≥50TB of usable storage space, supporting elastic expansion by adding hard drives to meet storage needs for more than 10 years.

(3) Voice Translation Model Iterative Upgrade Solution (Including Technical Implementation + Data Security)

Offline training and customization capability: Support incremental training/fine-tuning using FunASR on GPU servers within the client's intranet (supporting hotwords, proper noun vocabulary, custom dictionaries, and domain corpora). After training is completed, export the ONNX model and deploy it via sherpa-onnx hot update; the translation side supports glossary and domain corpus fine-tuning (Marian/Fairseq), also exported offline and deployed locally.

① Technical Implementation Solution

Establish a closed-loop iterative system of "user feedback - data collection - model training - version upgrade" to continuously improve model recognition and translation accuracy for various foreign accents and civil aviation terminology:

Feedback entry: Provide a "translation feedback" entry in the system management console and playback interface, allowing users to mark low-accuracy translation segments and perform manual text correction;

Data collection: After authorization from the client, the system automatically performs data desensitization on marked low-accuracy cases (removing personal information, flight-sensitive information, and agent seat information), storing the "original audio - incorrect translation - manually corrected text" triplet into the local training sample database;

Model training: Periodically (quarterly/on demand), use FunASR in the client's intranet training environment for incremental training/fine-tuning of the ASR model; after training output is exported to ONNX, it is loaded by the sherpa-onnx inference service and released via canary deployment; the entire process stays within the intranet, with no raw audio or text data uploaded.

Version upgrade: After model optimization is completed, push updates through the system's canary upgrade package, supporting one-click upgrade and rollback. The upgrade process does not affect normal system operation. A test report is provided after the upgrade, and the system is officially launched after client acceptance.

② Data Security Solution

Data collection: Only collect user-marked low-accuracy cases, not full voice data. The collection process is authorized by the client, with support for enabling/disabling the collection function;

Data desensitization: All collected sample data undergoes multiple layers of desensitization to remove all identifiable sensitive information, ensuring data cannot be traced back to specific individuals or flights;

Data storage: The training sample database is stored on the client's local servers with encrypted storage. Only authorized technical personnel can access it, and all access processes are logged;

Model training: All model training processes are completed within the client's intranet, using the client's server resources without relying on external computing power, ensuring data and model security;

Audit logs: The entire process of sample collection, model training, and version upgrade generates detailed audit logs recording the operator, operation time, and operation content, supporting traceability and auditing.

(4) Recording Playback and Global Full-text Search

Translation text and recording association: Translated text is automatically associated and stored with the corresponding recording files. Translated text is indexed in the domestic Elasticsearch global full-text search engine, supporting rapid keyword search;

Synchronized highlighted playback: The playback interface provides a "one-click to text" button. Clicking it automatically converts the selected recording into a text transcript, achieving synchronized highlighted playback of text and audio. Clicking any text segment immediately jumps the audio to the corresponding time point, with support for searching recording segments by text;

Multi-condition search and export: Support combined filtering of recordings by date, agent seat, flight number, phone number, call duration, keywords, and other conditions. Support batch export of recording files and translated text in ZIP format (including recording files and TXT/Excel text transcripts).

3 Highlight Features

Multilingual translation extension: In addition to Chinese-English bilingual support, additionally support real-time interpretation for Chinese-Japanese and Chinese-Korean minority languages to meet the client's cross-border international flight communication needs. Minority language translation latency ≤3s, accuracy ≥90%;

Intelligent recording classification: The system automatically classifies and archives recording files based on flight number, call type, and call counterpart, generating categorized directories for quick user retrieval;

Recording playback speed adjustment: Support playback speed adjustment at 0.5/1.0/1.25/1.5/2.0x, as well as pause, fast forward, rewind, and loop playback to improve playback efficiency;

Translated text keyword highlighting: In search results, the system automatically highlights search keywords in red for quick user location of key content.