K100_AI单卡全离线部署智能会议记录系统

一、引言

会议记录是企业日常沟通与决策的重要载体，然而传统人工记录耗时费力，且容易遗漏关键信息。随着人工智能技术的发展，语音识别（ASR）与声纹识别技术逐步成熟，使得自动生成带说话人标注的会议纪要成为可能。但当前多数方案依赖云端API，存在网络延迟、数据隐私与长期使用成本三大痛点。对于政务、金融、医疗等对数据安全要求高的场景，全离线部署是刚性需求。

K100_AI加速卡的推出，使得在单张加速卡上同时承载大语言模型与大参数量语音模型成为现实。本文设计并实现了一套基于Qwen3-ASR-1.7B语音识别模型与ERes2Net V2声纹识别模型的智能会议记录系统，全部推理工作均在一张K100_AI单卡上完成，无需任何互联网连接。系统采用Flask+Socket.IO框架实现前端网页与后端服务的实时双向通信，前端通过浏览器麦克风采集音频，后端进行低延迟的语音活动检测（VAD）、流式语音分割、ASR转写与说话人识别，最终在前端界面实时呈现，并支持一键保存全文记录。

本文在保证转写准确率的同时，充分考虑了单卡资源的合理调度：Qwen3-ASR模型使用vLLM部署为高性能推理API服务，声纹模型通过ModelScope本地加载并常驻显存。整个系统安装部署简单，所有模型与服务均封装在单节点内，真正做到了开箱即用的全离线会议记录体验。

二、方案设计

系统整体采用“浏览器采集—服务端处理—浏览器展示”的B/S架构，核心服务均部署于配备单张K100_AI加速卡的服务器上。功能模块如下：

1、前端采集与交互：基于HTML5-WebAudio-API实时捕获麦克风音频，以16kHz单声道格式分帧发送至后端；同时提供会议控制、噪音校准、声纹注册/测试、阈值调节和记录保存等交互功能。

2、后端核心服务：使用Python Flask与Flask-Socket.IO构建，负责接收音频流、执行两级VAD（基于能量与静音时长）、语音段切割、调用ASR服务转写文本，并通过声纹模型标注说话人。识别结果通过WebSocket实时推送至前端。

3、ASR推理服务：Qwen3-ASR-1.7B模型以兼容OpenAIAPI的HTTP服务形式部署在K100_AI上（使用vLLM或FastAPI封装），后端通过REST调用完成语音片段转录。配置文件中的API地址可指向localhost的本机服务，实现全离线。

4、声纹识别模块：加载iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common模型，通过提取说话人嵌入向量并与注册声纹库比对完成身份确认。对输入音频进行去静音、过滤过短片段和动态阈值匹配，有效提升识别鲁棒性。

5、并发与缓冲设计：采用线程池异步处理语音段，避免阻塞主线程；设置语音活动缓冲区和分段逻辑，确保长句与停顿的正确切分。

下图展示了整体数据流：

单卡资源调度策略上，ASR服务占据大部分计算与显存，声纹模型轻量常驻，两者分时复用GPU，避免了资源冲突。

三、实施方法及代码

3.1硬件环境

本方案的硬件平台为一台H3C服务器，配置如下：

只需占用一张K100_AI显卡

3.2软件栈

本方案的软件栈基于Docker容器化技术构建，使用经过海光DCU适配的vLLM推理镜像：

镜像（Qwen3-ASR-1.7B）：

harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm0.15.1-ubuntu22.04-dtk26.04-0130-py3.10-20260220

该镜像基于vLLM0.15.1推理框架、DTK26.04，Python3.10环境，单卡部署Qwen3-ASR-1.7B，200ms即可完成语音识别。

软件项目：

1、Qwen3-ASR-1.7B环境部署参照https://developer.sourcefind.cn/codes/modelzoo/qwen3-asr_pytorch里面操作步骤，项目下载链接为：https://developer.sourcefind.cn/codes/modelzoo/qwen3-asr_pytorch/-/archive/main/qwen3-asr_pytorch-main.zip。

2、智能会议记录系统自编程序代码如下：

#!/usr/bin/env python3"""智能会议记录系统 — Flask + SocketIO 实时版（Qwen3-ASR API）修复：正确处理英文语言标识，避免吞字优化：声纹识别增加去静音、短片段过滤、动态阈值新增：声纹测试功能、会议记录保存到本地文件"""import eventletimport sysif 'gunicorn' not in sys.modules:eventlet.monkey_patch()import osimport ioimport waveimport jsonimport timeimport base64import structimport threadingimport tempfileimport refrom collections import defaultdictfrom concurrent.futures import ThreadPoolExecutorimport numpy as npimport torchimport torchaudioimport soundfile as sffrom scipy import signalimport requestsfrom flask import Flask, render_template_string, requestfrom flask_socketio import SocketIO, emitfrom modelscope.models import Model# ================== 配置 ==================DEFAULT_CONFIG = {"asr_api_url": "http://192.168.222.65:8084/v1/audio/transcriptions","asr_model": "Qwen3-ASR-1.7B","asr_api_key": "PassWord@123456","asr_sample_rate": 16000,"asr_frame_duration": 30,"asr_vad_mode": 1,"asr_silence_thresh": 15,"vad_energy_threshold": 0.02,# 修改为 0.02"vad_silence_duration": 0.9, # 修改为 0.9"vad_max_speech_sec": 15,"noise_words": ["ok.", "Yeah.", "hmm", "uh", "um", "ah", "oh", "mhm","嗯。", "啊", "哦"]}SPEAKER_MODEL_ID = "iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common"SPEAKER_DB_PATH = "speaker_db.json"SAMPLE_RATE = 16000DEFAULT_ENERGY_THRESHOLD = DEFAULT_CONFIG["vad_energy_threshold"]DEFAULT_SILENCE_DURATION = DEFAULT_CONFIG["vad_silence_duration"]MAX_SPEECH_SEC = 5BUFFER_TIMEOUT = 0.1# 声纹识别动态阈值参数SPEAKER_ABS_THRESH = 0.55SPEAKER_MARGIN = 0.12MIN_SPEECH_DURATION = 1.0NOISE_WORDS_SET = set(word.lower().strip() for word in DEFAULT_CONFIG["noise_words"])executor = ThreadPoolExecutor(max_workers=4)# ================== 声纹管理 ==================class SpeakerManager:def __init__(self, db_path=SPEAKER_DB_PATH):self.db_path = db_pathself.db = self.load_db()self.lock = threading.Lock()self.device = "cuda:6" if torch.cuda.is_available() else "cpu"self.speaker_model = Model.from_pretrained(SPEAKER_MODEL_ID,revision=None,device=self.device,)self.speaker_model.eval()print("✅ 声纹模型加载成功", flush=True)def load_db(self):if os.path.exists(self.db_path):with open(self.db_path, 'r', encoding='utf-8') as f:return json.load(f)return {}def save_db(self):with open(self.db_path, 'w', encoding='utf-8') as f:json.dump(self.db, f, ensure_ascii=False, indent=2)def extract_embedding_from_array(self, audio_array, sr=16000):if audio_array is None or len(audio_array) == 0:return Nonetry:data = audio_array.astype(np.float32)if data.ndim > 1:data = np.mean(data, axis=1)waveform = torch.from_numpy(data).unsqueeze(0)if sr != 16000:resampler = torchaudio.transforms.Resample(orig_freq=sr, new_freq=16000)waveform = resampler(waveform)waveform = waveform.to(self.device)with torch.no_grad():embedding = self.speaker_model(waveform)if isinstance(embedding, torch.Tensor):embedding = embedding.squeeze().cpu().tolist()return embeddingexcept Exception as e:import tracebacktraceback.print_exc()return Nonedef register_speaker(self, name, audio_path):if not os.path.isfile(audio_path):return False, "音频文件无效"data, sr = sf.read(audio_path, dtype='float32')if data.ndim > 1:data = np.mean(data, axis=1)embedding = self.extract_embedding_from_array(data, sr)if embedding is None:return False, "无法提取声纹特征"with self.lock:self.db[name] = embeddingself.save_db()return True, f"说话人 '{name}' 注册成功！"def delete_speaker(self, name):if not name or not name.strip():return False, "说话人姓名不能为空"with self.lock:if name not in self.db:return False, f"说话人 '{name}' 不存在"del self.db[name]self.save_db()return True, f"说话人 '{name}' 已删除"def identify_speaker(self, audio_array, sr=16000):if not self.db:return None, 0.0, "声纹数据库为空"embedding = self.extract_embedding_from_array(audio_array, sr)if embedding is None:return None, 0.0, "无法提取声纹特征"emb_tensor = torch.tensor(embedding)with self.lock:db_copy = self.db.copy()scores = []for name, db_emb in db_copy.items():db_tensor = torch.tensor(db_emb)similarity = torch.nn.functional.cosine_similarity(emb_tensor.unsqueeze(0), db_tensor.unsqueeze(0)).item()scores.append((similarity, name))scores.sort(reverse=True, key=lambda x: x[0])if not scores:return None, 0.0, "数据库为空"best_score, best_name = scores[0]second_score = scores[1][0] if len(scores) > 1 else -1.0if best_score >= SPEAKER_ABS_THRESH and (best_score - second_score) >= SPEAKER_MARGIN:return best_name, best_score, f"识别成功: {best_name} ({best_score:.3f})"else:return None, best_score, f"未匹配 (best={best_score:.3f}, 2nd={second_score:.3f})"def get_speaker_list(self):with self.lock:return list(self.db.keys())speaker_mgr = SpeakerManager()# ================== 工具函数 ==================def trim_silence(audio_np, sr=SAMPLE_RATE, top_db=30):if len(audio_np) < sr * 0.5:return audio_npframe_len = int(sr * 0.025)hop = int(sr * 0.010)energy = np.array([np.sum(np.abs(audio_np[i:i+frame_len]**2))for i in range(0, len(audio_np)-frame_len, hop)])if len(energy) == 0 or energy.max() < 1e-10:return audio_npthreshold = 0.01 * energy.max()mask = energy > thresholdif not mask.any():return audio_npstart_idx = np.argmax(mask) * hopend_idx = (len(mask) - np.argmax(mask[::-1]) - 1) * hop + frame_lenreturn audio_np[start_idx:max(end_idx, start_idx + int(sr*0.2))]def pcm_to_wav_bytes(pcm_data, sample_rate, num_channels=1, sampwidth=2):wav_io = io.BytesIO()with wave.open(wav_io, 'wb') as wf:wf.setnchannels(num_channels)wf.setsampwidth(sampwidth)wf.setframerate(sample_rate)wf.writeframes(pcm_data)return wav_io.getvalue()def is_noise_text(text):cleaned = text.strip().lower().rstrip('.!?,;:。！？，；：')return cleaned in NOISE_WORDS_SETdef is_punctuation_only(text):cleaned = re.sub(r'[^u4e00-u9fffu3400-u4dbfa-zA-Z0-9]', '', text)return len(cleaned) == 0# ================== ASR 调用 ==================class ASRGenerator:def __init__(self, config):self.api_url = config["asr_api_url"]self.api_key = config["asr_api_key"]self.model = config["asr_model"]self.sample_rate = config["asr_sample_rate"]self.frame_duration = config["asr_frame_duration"]self.vad_mode = config["asr_vad_mode"]self.silence_thresh = config["asr_silence_thresh"]self.session = requests.Session()if self.api_key:self.session.headers.update({'Authorization': f'Bearer {self.api_key}'})def transcribe_pcm(self, pcm_bytes):if not pcm_bytes:return ""wav_bytes = pcm_to_wav_bytes(pcm_bytes, self.sample_rate)try:files = {'file': ('audio.wav', io.BytesIO(wav_bytes), 'audio/wav')}data = {'model': self.model,'language': 'zh','response_format': 'json','sample_rate': self.sample_rate,'frame_duration': self.frame_duration,'vad_mode': self.vad_mode,'silence_thresh': self.silence_thresh}response = self.session.post(self.api_url, files=files, data=data, timeout=30)response.raise_for_status()result = response.json()if 'text' in result:raw_text = result['text'].strip()elif 'choices' in result and len(result['choices']) > 0:raw_text = result['choices'][0].get('text', '').strip()else:raw_text = ""clean_text = re.sub(r'<[^>]*>', ' ', raw_text)clean_text = re.sub(r'blanguages+[A-Za-z]+s*', '', clean_text, flags=re.IGNORECASE)clean_text = re.sub(r's+', ' ', clean_text).strip()return clean_textexcept Exception as e:print(f"[ASR] 请求错误: {e}", flush=True)return ""asr_gen = ASRGenerator(DEFAULT_CONFIG)print("✅ ASR API 就绪", flush=True)# ================== Flask 应用 ==================app = Flask(__name__)app.secret_key = os.urandom(24).hex()socketio = SocketIO(app, async_mode='eventlet', cors_allowed_origins="*",ping_timeout=60, ping_interval=25,max_http_buffer_size=10 * 1024 * 1024)sessions = defaultdict(lambda: {'vad': {'voiced_frames': b'','silence_duration': 0.0,'speaking': False,'speech_duration': 0.0,'energy_threshold': DEFAULT_ENERGY_THRESHOLD,'silence_thresh': DEFAULT_SILENCE_DURATION},'recording': False,'stop_event': threading.Event(),'lock': threading.Lock(),'transcription': [],'noise_threshold': None,'noise_filter_enabled': False,'debug_counter': 0})def process_segment(sid, pcm_bytes):text = asr_gen.transcribe_pcm(pcm_bytes)if not text or is_noise_text(text) or is_punctuation_only(text):returnaudio_np = np.frombuffer(pcm_bytes, dtype=np.int16).astype(np.float32) / 32768.0duration = len(audio_np) / SAMPLE_RATEspeaker = "未知说话人"if duration >= MIN_SPEECH_DURATION:trimmed_audio = trim_silence(audio_np, sr=SAMPLE_RATE)name, score, msg = speaker_mgr.identify_speaker(trimmed_audio, sr=SAMPLE_RATE)if name:speaker = nameelse:speaker = f"未知说话人({score:.3f})"else:print(f"[SEGMENT] 语音片段过短 ({duration:.2f}s)，跳过声纹识别", flush=True)record = {"time": time.strftime("%H:%M:%S"),"speaker": speaker,"text": text.strip()}sessions[sid]['transcription'].append(record)socketio.emit('transcription', record, room=sid)# ---------- SocketIO 事件 ----------@socketio.on('connect')def on_connect():emit('connected', {'sid': request.sid})@socketio.on('disconnect')def on_disconnect():sessions.pop(request.sid, None)@socketio.on('clear_audio_state')def on_clear_audio_state():sid = request.sidsessions[sid]['vad'] = {'voiced_frames': b'','silence_duration': 0.0,'speaking': False,'speech_duration': 0.0,'energy_threshold': DEFAULT_ENERGY_THRESHOLD,'silence_thresh': DEFAULT_SILENCE_DURATION}@socketio.on('start_recording')def on_start_recording():sid = request.sidsessions[sid]['recording'] = Trueon_clear_audio_state()sessions[sid]['transcription'] = []@socketio.on('stop_recording')def on_stop_recording():sid = request.sidsessions[sid]['recording'] = Falsevad = sessions[sid]['vad']if vad['voiced_frames']:executor.submit(process_segment, sid, vad['voiced_frames'])vad['voiced_frames'] = b''@socketio.on('set_noise_threshold')def on_set_noise_threshold(data):sid = request.sidrms = data.get('rms', 0.0)if rms > 0:sessions[sid]['noise_threshold'] = rms * 2.0sessions[sid]['noise_filter_enabled'] = Truenew_energy = max(rms * 8.0, 0.02)sessions[sid]['vad']['energy_threshold'] = new_energysocketio.emit('update_energy_threshold', {'value': round(new_energy, 4)}, room=sid)emit('log', {'msg': f'✅ 噪音阈值已校准: {rms*2:.6f}，能量阈值自动调整为: {new_energy:.4f}'}, room=sid)@socketio.on('set_silence_duration')def on_set_silence_duration(data):sid = request.sidduration = float(data.get('duration', DEFAULT_SILENCE_DURATION))if duration < 0.1:duration = 0.1elif duration > 5.0:duration = 5.0sessions[sid]['vad']['silence_thresh'] = durationemit('log', {'msg': f'✅ 静音检测时间已设置为 {duration:.1f} 秒'}, room=sid)@socketio.on('set_energy_threshold')def on_set_energy_threshold(data):sid = request.sidthreshold = float(data.get('threshold', DEFAULT_ENERGY_THRESHOLD))if threshold < 0.001:threshold = 0.001elif threshold > 0.5:threshold = 0.5sessions[sid]['vad']['energy_threshold'] = thresholdemit('log', {'msg': f'✅ 能量阈值已设置为 {threshold:.4f}'}, room=sid)@socketio.on('audio_chunk')def on_audio_chunk(data):sid = request.sidsession = sessions[sid]if not session['recording']:returnif isinstance(data, bytearray):raw = bytes(data)else:raw = dataif len(raw) < 4:returnoriginal_rate = struct.unpack_from('= threshold:if not vad['speaking']:vad['speaking'] = Truevad['voiced_frames'] += pcm_framevad['silence_duration'] = 0.0vad['speech_duration'] += frame_durationelse:if vad['speaking']:vad['voiced_frames'] += pcm_framevad['silence_duration'] += frame_durationif (vad['silence_duration'] >= silence_thresh orvad['speech_duration'] >= MAX_SPEECH_SEC):segment = vad['voiced_frames']vad['voiced_frames'] = b''vad['silence_duration'] = 0.0vad['speaking'] = Falsevad['speech_duration'] = 0.0if segment:executor.submit(process_segment, sid, segment)# 声纹管理事件@socketio.on('register_speaker')def on_register_speaker(data):name = data.get('name', '').strip()audio_b64 = data.get('audio', '')if not name or not audio_b64:emit('register_result', {'success': False, 'msg': '姓名或音频为空'})returntry:audio_bytes = base64.b64decode(audio_b64)except Exception as e:emit('register_result', {'success': False, 'msg': f'音频解码失败: {str(e)}'})returntmp = tempfile.NamedTemporaryFile(suffix='.wav', delete=False)tmp.write(audio_bytes)tmp.close()success, msg = speaker_mgr.register_speaker(name, tmp.name)os.unlink(tmp.name)emit('register_result', {'success': success, 'msg': msg})@socketio.on('delete_speaker')def on_delete_speaker(data):name = data.get('name', '').strip()success, msg = speaker_mgr.delete_speaker(name)emit('delete_result', {'success': success, 'msg': msg})@socketio.on('get_speaker_list')def on_get_speaker_list():emit('speaker_list', {'speakers': speaker_mgr.get_speaker_list()})@socketio.on('test_speaker')def on_test_speaker(data):audio_b64 = data.get('audio', '')if not audio_b64:emit('test_result', {'success': False, 'msg': '音频数据为空'})returntry:audio_bytes = base64.b64decode(audio_b64)except Exception as e:emit('test_result', {'success': False, 'msg': f'音频解码失败: {str(e)}'})returntmp = tempfile.NamedTemporaryFile(suffix='.wav', delete=False)tmp.write(audio_bytes)tmp.close()try:data_np, sr = sf.read(tmp.name, dtype='float32')if data_np.ndim > 1:data_np = np.mean(data_np, axis=1)name, score, msg = speaker_mgr.identify_speaker(data_np, sr)emit('test_result', {'success': True,'name': name if name else '未知','score': round(score, 4),'msg': msg})except Exception as e:emit('test_result', {'success': False, 'msg': f'识别处理失败: {str(e)}'})finally:os.unlink(tmp.name)# ================== HTML 模板（含保存功能） ==================HTML_TEMPLATE = '''
🎤 智能会议记录⚪ 未连接
⚙️ 语音检测
能量阈值 0.0200
实时能量: 0.000000
静音阈值 0.9 秒
🔇 噪音校准
开始校准（安静3秒）
👤 声纹管理
注册说话人删除🔍 声纹测试刷新列表
▶️ 开始会议清屏💾 保存记录🎤 麦克风未激活
'''@app.route('/')def index():return render_template_string(HTML_TEMPLATE)if __name__ == '__main__':print("🚀 智能会议记录系统（Qwen3-ASR，声纹优化版，含声纹测试与记录保存）启动", flush=True)print(" 访问地址: http://0.0.0.0:7898", flush=True)socketio.run(app, host='0.0.0.0', port=7898, debug=False)

3.3模型下载与准备

1、Qwen3-ASR-1.7B（语音识别模型）：

下载链接：https://modelscope.cn/models/Qwen/Qwen3-ASR-1.7B/files

2、iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common（说话人确认模型）：

下载链接：https://modelscope.cn/models/iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common/files

3.4模型启动参数

1、Qwen3-ASR-1.7B启动参数：

cat Qwen3-ASR-1.7B.shHIP_VISIBLE_DEVICES=6 vllm serve '/home/models/Qwen3-ASR-1.7B' --trust-remote-code --gpu-memory-utilization 0.3 --limit-mm-per-prompt '{"audio": 1}' --port 8084 --served-model-name Qwen3-ASR-1.7B --api-key PassWord@123456

2、iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common启动参数：

SPEAKER_MODEL_ID = "iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common"SPEAKER_DB_PATH = "speaker_db.json"# ================== 声纹管理 ==================class SpeakerManager:def __init__(self, db_path=SPEAKER_DB_PATH):self.db_path = db_pathself.db = self.load_db()self.lock = threading.Lock()self.device = "cuda:6" if torch.cuda.is_available() else "cpu"self.speaker_model = Model.from_pretrained(SPEAKER_MODEL_ID,revision=None,device=self.device,)self.speaker_model.eval()print("✅ 声纹模型加载成功", flush=True)

四、运行测试

1、启动Qwen3-ASR-1.7B

docker exec -it qwen3-asr bashcd /home/models/nohup ./Qwen3-ASR-1.7B.sh &

2、启动智能会议记录系统自编程序

python 会议记录系统.py /home/models/会议记录系统.py:8: EventletDeprecationWarning: Eventlet is deprecated. It is currently being maintained in bugfix mode, andwe strongly recommend against using it for new projects.If you are already using Eventlet, we recommend migrating to a differentframework.For more detail seehttps://eventlet.readthedocs.io/en/latest/asyncio/migration.htmlimport eventletDownloading Model from https://www.modelscope.cn to directory: /root/.cache/modelscope/hub/models/iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common2026-07-11 16:09:15,218 - modelscope - INFO - initialize model from /root/.cache/modelscope/hub/models/iic/speech_eres2netv2w24s4ep4_sv_zh-cn_16k-common✅ 声纹模型加载成功✅ ASR API 就绪🚀 智能会议记录系统（Qwen3-ASR，声纹优化版，含声纹测试与记录保存）启动 访问地址: http://0.0.0.0:7898

3、访问智能会议记录系统自编程序

用谷歌浏览器访问，访问之前需要做安全授权设置：

地址栏执行：chrome://flags/

搜索：Insecureoriginstreatedassecure

填入：http://192.168.222.65:7898

然后访问：http://192.168.222.65:7898

先执行噪音校准，获取背景噪声。根据说话人语速调整VAD静音持续时间，语速越快，该时间就越小。注册说话人、声纹测试，最后开始会议。