突发技术信号：AI 安全网关开始升温 提示词防护 权限校验 内容审计成为大模型应用标配

{"type":"doc","content":[{"type":"heading","attrs":{"id":"0c7a624c-5b3d-4578-b9d3-88942989edfb","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"概述"}]},{"type":"paragraph","attrs":{"id":"fda7ef2b-1a69-48d8-bb5c-9c0f20eda094","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"2026 年，大模型应用正在从“能生成内容”进入“需要安全治理”的阶段。"}]},{"type":"paragraph","attrs":{"id":"cc34578d-765e-4e57-b4f9-49ae213e0bc2","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"过去，很多企业接入大模型时，重点关注的是调用效果。模型回答是否准确、响应是否足够快、是否能接入知识库，是早期最重要的问题。"}]},{"type":"paragraph","attrs":{"id":"91ddc19f-f377-4e33-9f35-0cc8f1710cf7","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"但随着大模型应用进入客服、办公、研发、数据分析、企业知识库和 Agent 工具调用场景后，安全问题开始变得明显。"}]},{"type":"paragraph","attrs":{"id":"81fd0b5e-0c13-47f5-8a3c-5299a41ff32d","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"用户可能输入恶意提示词，诱导模型绕过规则；模型可能输出不合适内容；Agent 可能调用不该调用的工具；RAG 系统可能把敏感资料召回给无权限用户；内部接口也可能被大模型应用间接暴露。"}]},{"type":"paragraph","attrs":{"id":"4ec8ef97-95a7-41f7-b9c8-1eba7a0f6baa","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"因此，AI 安全网关开始成为新的工程能力。"}]},{"type":"paragraph","attrs":{"id":"3df8ec3a-5ada-4db4-aa1c-830cdb9353ec","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"它位于用户请求和模型服务之间，负责完成提示词检查、用户权限校验、敏感词过滤、工具调用控制、输出审计和日志记录。"}]},{"type":"paragraph","attrs":{"id":"015b2091-1e1e-4013-9835-5584ab86fe7b","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"这意味着，大模型应用不能只关注“生成结果”，还需要关注“生成过程是否安全”。"}]},{"type":"horizontalRule","attrs":{"id":"f0dacf94-e44e-4323-978a-b5dcae709738","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"2cf78541-b0a8-467c-a148-e39912a304da","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"一、为什么 AI 应用需要安全网关？"}]},{"type":"paragraph","attrs":{"id":"89e29511-3659-46e3-ad13-0f891d57c6e5","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"传统 Web 应用通常会有 API 网关、防火墙、权限系统和日志审计。"}]},{"type":"paragraph","attrs":{"id":"3de1bafe-8e3f-447c-9051-831808e85b86","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"大模型应用也需要类似能力。"}]},{"type":"paragraph","attrs":{"id":"25f9de3b-aa45-4a50-94af-064fb5ee8e18","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"因为一次 AI 请求不只是普通接口调用，它可能包含用户输入、知识库检索、工具调用、模型生成和最终输出多个环节。"}]},{"type":"paragraph","attrs":{"id":"a89b4dc4-34c7-41a8-a2c0-ccac380275a2","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"任何一个环节缺少控制，都可能带来风险。"}]},{"type":"paragraph","attrs":{"id":"d7987e10-4afb-4eec-a863-4af73ba75409","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"例如："}]},{"type":"orderedList","attrs":{"id":"7f216c3c-c6cb-4611-ac21-60f636036f4e","start":1,"isHoverDragHandle":false},"content":[{"type":"listItem","attrs":{"id":"ebc4ccf8-3a9b-439e-8bb0-ee7101c6d3d3"},"content":[{"type":"paragraph","attrs":{"id":"f0fa938f-b672-410d-931e-9b8e6fb4ba4c","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"用户输入恶意提示词；"}]}]},{"type":"listItem","attrs":{"id":"74c32d09-4997-4900-a7ee-2868f536dcca"},"content":[{"type":"paragraph","attrs":{"id":"17353b26-488c-4f54-a9e3-f05e7f7a99ae","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"模型输出敏感内容；"}]}]},{"type":"listItem","attrs":{"id":"346354b6-1156-4d6d-8a39-f715cf8519be"},"content":[{"type":"paragraph","attrs":{"id":"2c560416-6ec9-44a8-aa3f-614d51746f85","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"Agent 调用高风险工具；"}]}]},{"type":"listItem","attrs":{"id":"f7d79afc-1e04-4635-b262-0321dfb5c8d6"},"content":[{"type":"paragraph","attrs":{"id":"f49312f2-944e-4b94-b447-fee72729b542","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"RAG 召回了无权限文档；"}]}]},{"type":"listItem","attrs":{"id":"5d9bc509-dbe5-4747-9367-fe7e86e1b5cc"},"content":[{"type":"paragraph","attrs":{"id":"269d7d6c-3ef3-4944-ab35-191218104335","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"系统没有记录请求链路。"}]}]}]},{"type":"paragraph","attrs":{"id":"c02ec42b-916d-4ffb-98ce-bb236fe629b5","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"下面用 Python 写一个简化版 AI 安全网关。"}]},{"type":"horizontalRule","attrs":{"id":"5b3b5a75-cfbf-4839-9220-018891d429e8","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"2700c750-b90d-411a-8eed-29ce9e61f7d7","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"二、基础配置：定义安全规则"}]},{"type":"paragraph","attrs":{"id":"e52bd857-076f-4215-86eb-442c00cff566","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"第一步是定义安全规则。"}]},{"type":"paragraph","attrs":{"id":"6f75418e-3389-4483-898e-abee1c10d24d","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"这里包括敏感词、危险动作、高风险工具和角色权限。真实系统中，这些规则可以存储在数据库或权限中心中。"}]},{"type":"codeBlock","attrs":{"id":"11436c47-f66a-427b-9e7d-ed729b3aa281","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"import jsonnimport timenimport hashlibnfrom datetime import datetimennnSENSITIVE_WORDS = [n"敏感数据",n"内部密钥",n"用户隐私",n"数据库密码",n"访问令牌"n]nnnDANGEROUS_ACTIONS = [n"删除数据",n"绕过权限",n"导出全部用户",n"关闭审计",n"修改权限"n]nnnHIGH_RISK_TOOLS = [n"database_write",n"user_export",n"permission_update",n"shell_execute",n"file_delete"n]nnnROLE_PERMISSIONS = {n"guest": ["search", "summary"],n"developer": ["search", "summary", "code_review"],n"admin": ["search", "summary", "code_review", "database_read", "audit_query"]n}nnnAUDIT_LOG = []nnndef build_request_id(user_id, prompt):nraw = f"{user_id}-{prompt}-{time.time()}"nnreturn hashlib.md5(nraw.encode("utf-8")n).hexdigest()n"}]},{"type":"paragraph","attrs":{"id":"9cc71bef-54cd-4853-ac33-99e87a0db682","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"这部分规则相当于安全网关的基础策略。"}]},{"type":"paragraph","attrs":{"id":"d7941138-cab5-4743-8c2c-3274fe03c29b","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"系统会基于这些规则判断请求是否应该放行。"}]},{"type":"horizontalRule","attrs":{"id":"fe650b36-ae8b-49b8-afe5-982a6c401789","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"5d6e8a20-c83e-47dd-a135-5440412e1ccb","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"三、输入检查：识别提示词风险"}]},{"type":"paragraph","attrs":{"id":"5ab79789-d38a-4b51-9c85-d70981e78bc1","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"第二步是检查用户输入。"}]},{"type":"paragraph","attrs":{"id":"1e19b93b-fbf6-4b79-9ac6-a98b5cbb87b5","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"如果用户输入中包含危险动作或明显敏感内容，系统可以直接拦截，也可以标记为高风险请求。"}]},{"type":"codeBlock","attrs":{"id":"e7b8175e-3935-4090-8ade-98bc6d6364dc","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"def check_prompt_risk(prompt):nrisk_items = []nnfor word in SENSITIVE_WORDS:nif word in prompt:nrisk_items.append({n"type": "sensitive_word",n"value": wordn})nnfor action in DANGEROUS_ACTIONS:nif action in prompt:nrisk_items.append({n"type": "dangerous_action",n"value": actionn})nnrisk_level = "low"nnif len(risk_items) >= 2:nrisk_level = "high"nelif len(risk_items) == 1:nrisk_level = "medium"nnreturn {otterly.cnn"risk_level": risk_level,n"risk_items": risk_itemsn}n"}]},{"type":"paragraph","attrs":{"id":"704f21c0-e20e-4ac9-ac1c-8e4519352a4e","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"输入检查是 AI 安全治理的第一道防线。"}]},{"type":"paragraph","attrs":{"id":"116e1991-8ee6-451a-9c63-04ca51128e3e","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"它不能解决所有问题，但可以过滤明显异常请求。"}]},{"type":"horizontalRule","attrs":{"id":"6a3d525c-8f0f-4a93-be4f-932ad9d8a464","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"11432d3b-4f8d-471f-8947-9484491cb8d2","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"四、权限校验：判断用户能否执行任务"}]},{"type":"paragraph","attrs":{"id":"48f8f15f-1022-4393-aa51-0fcaedb150aa","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"第三步是权限校验。"}]},{"type":"paragraph","attrs":{"id":"d2bbeeb9-9680-42d5-939a-ca2c027865e6","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"不同用户角色可以执行不同任务。比如访客只能搜索和摘要，开发者可以进行代码审查，管理员可以查询审计数据。"}]},{"type":"codeBlock","attrs":{"id":"13dce4c2-febb-480b-89ee-966383d0d9c8","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"def check_user_permission(role, action):nallowed_actions = ROLE_PERMISSIONS.get(role, [])nnreturn action in allowed_actionsnnndef build_permission_result(role, action):nallowed = check_user_permission(role, action)nnreturn {n"role": role,n"action": action,n"allowed": allowed,n"checked_at": datetime.now().isoformat()n}n"}]},{"type":"paragraph","attrs":{"id":"cfbae84f-1462-4a3c-aeed-a6c1593eab65","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"权限校验可以防止 AI 应用变成绕过系统权限的入口。"}]},{"type":"paragraph","attrs":{"id":"b618b637-c828-4ed3-99aa-7fc5c37757fa","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"尤其是 Agent 场景中，模型会调用工具，更需要控制用户能调用哪些能力。"}]},{"type":"horizontalRule","attrs":{"id":"f951dd65-002a-4b7b-b10f-1753759bf977","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"ef1c2d0c-2d64-4a6f-a090-e7305bb82393","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"五、工具调用检查：拦截高风险工具"}]},{"type":"paragraph","attrs":{"id":"f69c8e2c-79e2-4f75-a8e8-80eabcadb9f0","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"第四步是检查工具调用。"}]},{"type":"paragraph","attrs":{"id":"5e169650-c567-4e2b-b39f-236a3d78a1ca","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"如果 Agent 想调用高风险工具，系统必须先判断用户角色是否允许。"}]},{"type":"codeBlock","attrs":{"id":"0373b26c-d9f4-4ddb-ab4c-f507b2259c39","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"def check_tool_call(role, tool_name):nif tool_name not in HIGH_RISK_TOOLS:nreturn {n"tool_name": tool_name,n"allowed": True,n"reason": "normal tool"n}nnif role == "admin":nreturn {n"tool_name": tool_name,n"allowed": True,n"reason": "admin allowed"n}nnreturn {n"tool_name": tool_name,n"allowed": False,n"reason": "high risk tool requires admin role"n}nnndef simulate_tool_call(tool_name, payload):nreturn {n"tool_name": tool_name,n"payload": payload,n"result": "工具执行结果",n"run_time": datetime.now().isoformat()n}n"}]},{"type":"paragraph","attrs":{"id":"14300fe6-b503-4930-b908-8478ce4ceec5","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"工具调用检查是 Agent 安全的关键。"}]},{"type":"paragraph","attrs":{"id":"70f940dd-9446-4fbf-836c-7d02668a41d3","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"如果没有这一层，模型可能在用户诱导下调用危险工具。"}]},{"type":"horizontalRule","attrs":{"id":"9897d163-44a7-4914-8b2b-8c308a624a29","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"380aba02-66a6-480e-92ea-776e4b288dcd","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"六、输出检查：过滤敏感结果"}]},{"type":"paragraph","attrs":{"id":"43c2d086-3fad-4380-9584-da91b70e95ad","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"第五步是输出检查。"}]},{"type":"paragraph","attrs":{"id":"7e65d771-6b19-48ea-a19d-7bfac43f519b","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"模型生成内容后，系统需要再次检查是否包含敏感词或不应该返回的信息。"}]},{"type":"codeBlock","attrs":{"id":"7e949472-1f87-4112-a194-e007f769bf9b","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"def check_output_safety(output_text):nissues = []nnfor word in SENSITIVE_WORDS:nif word in output_text:nissues.append({n"type": "sensitive_output",n"value": wordn})nnsafe = len(issues) == 0nnreturn {n"safe": safe,n"issues": issuesn}nnndef mask_sensitive_output(output_text):nmasked = output_textnnfor word in SENSITIVE_WORDS:nmasked = masked.replace(word, "***")nnreturn maskedn"}]},{"type":"paragraph","attrs":{"id":"57518ec3-99c3-41de-b069-92165142ba0a","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"输出检查可以降低模型泄露敏感内容的风险。"}]},{"type":"paragraph","attrs":{"id":"aa43d3e3-eab6-489f-b029-d296f35269be","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"尤其在企业知识库和内部问答场景中，这一步非常重要。"}]},{"type":"horizontalRule","attrs":{"id":"12893449-daa3-4070-9750-2bdd34f7437c","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"295920b8-a803-4194-9e7d-05c4a5d4d1cb","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"七、审计日志：记录完整请求链路"}]},{"type":"paragraph","attrs":{"id":"8ad3733e-6eb1-4a50-bd23-09e40d1e56bf","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"第六步是审计。"}]},{"type":"paragraph","attrs":{"id":"18881640-10d0-4f32-a2a0-49251954f958","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"每一次请求都应该记录用户、角色、动作、风险等级、工具调用、输出检查结果和最终状态。"}]},{"type":"codeBlock","attrs":{"id":"64655d60-c376-46cf-b3fe-2ec660e73982","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"def write_audit_log(record):nAUDIT_LOG.append(record)nnndef build_audit_record(nrequest_id,nuser_id,nrole,naction,nprompt_risk,npermission_result,ntool_result,noutput_check,nstatusn):nreturn {n"request_id": request_id,n"user_id": user_id,n"role": role,n"action": action,n"prompt_risk": prompt_risk,n"permission_result": permission_result,n"tool_result": tool_result,n"output_check": output_check,n"status": status,n"audit_time": datetime.now().isoformat()n}n"}]},{"type":"paragraph","attrs":{"id":"613af4c3-57a5-491c-8677-495c1c679193","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"审计日志的意义，是让 AI 应用可以被追踪、被复盘、被治理。"}]},{"type":"paragraph","attrs":{"id":"b44d4482-5897-4898-b224-771c6514af96","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"没有审计能力的 AI 系统，很难进入严肃业务场景。"}]},{"type":"horizontalRule","attrs":{"id":"47a14fc0-e612-4f91-9495-f4e3eec52635","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"0a2136b3-0d27-4b7c-bc60-20bb5ac05208","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"八、统一安全网关：串联完整流程"}]},{"type":"paragraph","attrs":{"id":"a6a3e70b-bab6-42a1-8c49-42fe95195abf","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"第七步是构建统一安全网关。"}]},{"type":"paragraph","attrs":{"id":"eff75c83-32f8-44b7-ae70-80d2e713de7d","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"所有请求先经过网关，再决定是否调用模型或工具。"}]},{"type":"codeBlock","attrs":{"id":"b1c0e18a-a999-49d8-b3f3-a8045a30b150","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"def fake_model_generate(prompt):nif "敏感数据" in prompt:nreturn "模型回答中可能包含敏感数据，需要处理。"nnreturn "这是模型生成的安全回答。"nnndef ai_security_gateway(user_id, role, action, prompt, tool_name=None):nrequest_id = build_request_id(user_id, prompt)nstatus = "success"ntool_result = Nonennprompt_risk = check_prompt_risk(prompt)nnpermission_result = build_permission_result(nrole=role,naction=actionn)nnif not permission_result["allowed"]:nstatus = "blocked_permission"nnoutput_text = "请求被拦截：当前角色没有执行该操作的权限。"nnoutput_check = check_output_safety(output_text)nnaudit_record = build_audit_record(nrequest_id=request_id,nuser_id=user_id,nrole=role,naction=action,nprompt_risk=prompt_risk,npermission_result=permission_result,ntool_result=tool_result,noutput_check=output_check,nstatus=statusn)nnwrite_audit_log(audit_record)nnreturn {n"request_id": request_id,n"status": status,n"answer": output_textn}nnif prompt_risk["risk_level"] == "high":nstatus = "blocked_prompt"nnoutput_text = "请求被拦截：输入内容存在较高安全风险。"nnoutput_check = check_output_safety(output_text)nnaudit_record = build_audit_record(nrequest_id=request_id,nuser_id=user_id,nrole=role,naction=action,nprompt_risk=prompt_risk,npermission_result=permission_result,ntool_result=tool_result,noutput_check=output_check,nstatus=statusn)nnwrite_audit_log(audit_record)nnreturn {n"request_id": request_id,n"status": status,n"answer": output_textn}nnif tool_name:ntool_check = check_tool_call(role, tool_name)nnif not tool_check["allowed"]:nstatus = "blocked_tool"nnoutput_text = f"请求被拦截：{tool_check['reason']}"nnoutput_check = check_output_safety(output_text)nnaudit_record = build_audit_record(nrequest_id=request_id,nuser_id=user_id,nrole=role,naction=action,nprompt_risk=prompt_risk,npermission_result=permission_result,ntool_result=tool_check,noutput_check=output_check,nstatus=statusn)nnwrite_audit_log(audit_record)nnreturn {n"request_id": request_id,n"status": status,n"answer": output_textn}nntool_result = simulate_tool_call(ntool_name=tool_name,npayload={"prompt": prompt}n)nnoutput_text = fake_model_generate(prompt)noutput_check = check_output_safety(output_text)nnif not output_check["safe"]:noutput_text = mask_sensitive_output(output_text)nstatus = "masked_output"nnaudit_record = build_audit_record(nrequest_id=request_id,nuser_id=user_id,nrole=role,naction=action,nprompt_risk=prompt_risk,npermission_result=permission_result,ntool_result=tool_result,noutput_check=output_check,nstatus=statusn)nnwrite_audit_log(audit_record)nnreturn {n"request_id": request_id,n"status": status,n"answer": output_textn}n"}]},{"type":"paragraph","attrs":{"id":"7352f308-5fce-4094-8582-6315cd8e0207","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"安全网关的价值，是把 AI 请求的风险控制前置。"}]},{"type":"paragraph","attrs":{"id":"326b9f72-6044-418e-adc3-d6436fb9bd07","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"它让模型调用不再是裸奔，而是经过输入、权限、工具和输出多层检查。"}]},{"type":"horizontalRule","attrs":{"id":"a7f5f0ac-f11c-4af9-be65-182352387fb3","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"80c3a771-48c5-4bcb-aa7f-5ea054f71675","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"九、生成安全审计报告"}]},{"type":"paragraph","attrs":{"id":"840f45ac-8517-48bf-9a6e-6308b1320a97","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"最后生成审计报告。"}]},{"type":"paragraph","attrs":{"id":"d398c1c7-345a-4a3a-9597-85598231b73e","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"报告可以统计请求数、拦截数和风险类型。"}]},{"type":"codeBlock","attrs":{"id":"cc9ddd8f-381d-4534-9f0c-628e02558178","language":"javascript","theme":"atom-one-dark","runtimes":0,"isHoverDragHandle":false,"key":"","languageByAi":"javascript"},"content":[{"type":"text","text":"def generate_security_report():ntotal = len(AUDIT_LOG)nblocked = len([nitem for item in AUDIT_LOGnif item["status"].startswith("blocked")n])nnmasked = len([nitem for item in AUDIT_LOGnif item["status"] == "masked_output"n])nnrisk_count = {}nnfor item in AUDIT_LOG:nrisk_level = item["prompt_risk"]["risk_level"]nrisk_count[risk_level] = risk_count.get(risk_level, 0) 1nnreturn {n"report_name": "AI 安全网关审计报告",n"total_requests": total,n"blocked_requests": blocked,n"masked_outputs": masked,n"risk_count": risk_count,n"generate_time": datetime.now().isoformat()n}nnnif __name__ == "__main__":nsamples = [n{n"user_id": "u001",n"role": "guest",n"action": "summary",n"prompt": "请总结这篇技术文章",n"tool_name": Nonen},n{n"user_id": "u002",n"role": "developer",n"action": "code_review",n"prompt": "请帮我检查代码问题",n"tool_name": "search"n},n{n"user_id": "u003",n"role": "guest",n"action": "database_read",n"prompt": "请导出全部用户数据",n"tool_name": "user_export"n},n{n"user_id": "u004",n"role": "admin",n"action": "audit_query",n"prompt": "查询审计记录，但不要暴露内部密钥",n"tool_name": "audit_query"n}n]nnfor sample in samples:nresult = ai_security_gateway(**sample)nnprint(json.dumps(nresult,nensure_ascii=False,nindent=2n))nnreport = generate_security_report()nnprint(json.dumps(nreport,nensure_ascii=False,nindent=2n))n"}]},{"type":"horizontalRule","attrs":{"id":"0f6b5efc-0a55-4f72-85e6-c099ac9ead5e","isHoverDragHandle":false}},{"type":"heading","attrs":{"id":"3d7a6b95-fe20-4284-96dd-e3c4af825b3e","textAlign":"inherit","indent":0,"level":2,"isHoverDragHandle":false},"content":[{"type":"text","text":"十、趋势判断"}]},{"type":"paragraph","attrs":{"id":"5c72c5cf-4245-4afd-a5c5-8539a08d68d1","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"从这套流程可以看到，AI 安全网关正在成为大模型应用的重要基础设施。"}]},{"type":"paragraph","attrs":{"id":"d207aa1b-3ae4-4d1a-b69e-443846255047","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"过去，企业更关注模型能力；现在，企业还需要控制模型使用边界。"}]},{"type":"paragraph","attrs":{"id":"5ef85780-ecf3-4426-a26c-5a535997953b","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"输入检查、权限校验、工具控制、输出过滤和审计日志，将成为 AI 应用进入生产环境的标配能力。"}]},{"type":"paragraph","attrs":{"id":"6e44572d-31b0-493f-91f0-f850250b3699","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"未来，企业不会只问“模型能不能回答”，还会问“模型能不能安全地回答”。"}]},{"type":"paragraph","attrs":{"id":"86f961d2-f254-402f-b948-7bdb5b1a2c8c","textAlign":"inherit","indent":0,"color":null,"background":null,"isHoverDragHandle":false},"content":[{"type":"text","text":"谁能更早建立 AI 安全治理体系，谁就更容易把大模型应用接入真实业务流程。"}]}]}","createTime":1782897639,"ext":{"closeTextLink":0,"comment_ban":0,"description":"","focusRead":0},"favNum":0,"html":"","isOriginal":0,"likeNum":0,