联犀开发者文档

WebRTC全双工AI与音视频关联

约 2960 字大约 10 分钟

2026-10-16

本页包含已接线核心流程及可选扩展；具体子能力必须按设备、平台与用户授权求交集。参见支持范围、场景索引和命令手册。图中S编号只用于本文定位，不放入实际JSON。

前置条件

先在本次连接上报以下真实能力并收到acceptedRevision=1；这是创建请求的前置，不从S01示例的16kHz能力推断支持本页48kHz音轨。设备没有实际AEC时不能复制此全双工声明。

S20-M28 · capabilityReport

{"msgToken":"s20-cap","method":"capabilityReport","params":{"revision":1,"purposes":["aiConversation"],"transports":["webrtc"],"supportedInteractionModes":["fullDuplex"],"aecMode":"device","features":["audio"],"audio":{"format":"opus","sampleRate":48000,"channels":1,"frameDuration":20}}}

S20-M29 · requestAck

{"msgToken":"s20-cap","method":"requestAck","code":200,"data":{"requestMethod":"capabilityReport","result":{"acceptedRevision":1}}}

音频回复跨信令/内容/媒体无全序。WebRTC轨用mid/trackId/mediaRevision绑定；多个回复顺序使用同一音轨，RTP包不强塞respId。平台使用reply音频队列隔离、cancel屏障和供给侧取消；设备清旧回复队列。供应商无逐片reply标识时仍需真机证明无串音，不能用设计保证替代。视觉帧需独立consent和采集时窗，音频能力不自动授予视觉。

D/D2是不同设备参与端，A是应用端，P是平台和媒体节点。各端分别使用鉴权Topic/登录态，不能冒用另一端身份。基础会话与媒体协商见S01/S06，复用时保留本场景sessionId并独立分配密钥和token。

完整时序

报文索引

编号按图阅读；编号保留历史引用，不要求数值连续。字段定义以命令链接为准。

编号发送→接收Topic/接口方法及契约应答/等待状态或作用
S20-M28D→PSIG↑capabilityReportrequestAck；10秒当前连接声明真实48kHz能力
S20-M29P→DSIG↓requestAck—核验acceptedRevision后创建
S20-M01D→PSIG↑sessionCreate对应应答见命令；10秒受理，媒体协商15秒
S20-M02P→DSIG↓sessionCreated应答/状态事件；不等待ACK
S20-M03D→PSIG↑mediaOffer对应应答见命令；10秒受理，媒体协商15秒
S20-M04P→DSIG↓mediaAnswer应答/状态事件；不等待ACK
S20-M05D→PSIG↑mediaState对应应答见命令；10秒受理，媒体协商15秒
S20-M06P→DSIG↓requestAck应答/状态事件；不等待ACK
S20-M07P→DSIG↓sessionEstablished应答/状态事件；不等待ACK
S20-M08D→PSIG↑audioStart对应应答见命令；10秒受理，媒体协商15秒
S20-M09P→DSIG↓requestAck应答/状态事件；不等待ACK
S20-M10P→DMSG↓respCreated应答/状态事件；不等待ACK
S20-M11P→DMSG↓respTextDone应答/状态事件；不等待ACK
S20-M12P→DSIG↓respAudioStart应答/状态事件；不等待ACK
S20-M13D→PSIG↑respCancel对应应答见命令；10秒受理，媒体协商15秒
S20-M14P→DSIG↓requestAck应答/状态事件；不等待ACK
S20-M15P→DSIG↓audioSpeechInterrupted应答/状态事件；不等待ACK
S20-M25P→DMSG↓respDone应答或事件，无ACK核对补齐，不以受理替代完成
S20-M16P→DMSG↓respCreated应答/状态事件；不等待ACK
S20-M17P→DSIG↓respAudioStart应答/状态事件；不等待ACK
S20-M26P→DMSG↓respDone应答或事件，无ACK核对补齐，不以受理替代完成
S20-M27P→DMSG↓respDone应答或事件，无ACK核对补齐，不以受理替代完成
S20-M18P→DSIG↓sessionError应答/状态事件；不等待ACK
S20-M19D→PSIG↑audioStop对应应答见命令；10秒受理，媒体协商15秒
S20-M20P→DSIG↓requestAck应答/状态事件；不等待ACK
S20-M21D→PSIG↑audioStart对应应答见命令；10秒受理，媒体协商15秒
S20-M22P→DSIG↓requestAck应答/状态事件；不等待ACK
S20-M23D→PSIG↑sessionEnd对应应答见命令；10秒受理，媒体协商15秒
S20-M24P→DSIG↓sessionEnded应答/状态事件；不等待ACK

逐条JSON

S20-M01 · sessionCreate

{
  "msgToken": "s20-77",
  "method": "sessionCreate",
  "params": {
    "purpose": "aiConversation",
    "transport": "webrtc",
    "audioParams": {
      "format": "opus",
      "sampleRate": 48000,
      "channels": 1,
      "frameDuration": 20
    },
    "tracks": [
      {
        "trackId": "mic",
        "kind": "audio",
        "sourceId": "mic",
        "direction": "sendonly",
        "codec": "opus",
        "state": "requested"
      },
      {
        "trackId": "speaker",
        "kind": "audio",
        "sourceId": "speaker",
        "direction": "recvonly",
        "codec": "opus",
        "state": "requested"
      }
    ],
    "supportedInteractionModes": [
      "fullDuplex"
    ]
  }
}

S20-M02 · sessionCreated

{
  "msgToken": "s20-77",
  "method": "sessionCreated",
  "sessionId": "s20-session",
  "data": {
    "state": "negotiating",
    "selected": {
      "transport": "webrtc",
      "interactionMode": "fullDuplex",
      "audioParams": {
        "format": "opus",
        "sampleRate": 48000,
        "channels": 1,
        "frameDuration": 20
      }
    },
    "tracks": [
      {
        "trackId": "mic",
        "kind": "audio",
        "sourceId": "mic",
        "direction": "sendonly",
        "codec": "opus",
        "state": "requested"
      },
      {
        "trackId": "speaker",
        "kind": "audio",
        "sourceId": "speaker",
        "direction": "recvonly",
        "codec": "opus",
        "state": "requested"
      }
    ]
  },
  "code": 200
}

S20-M03 · mediaOffer

{
  "msgToken": "s20-78",
  "method": "mediaOffer",
  "sessionId": "s20-session",
  "params": {
    "mediaRevision": 1,
    "webrtc": {
      "iceMode": "complete",
      "sdp": "<SDP offer>"
    }
  }
}

S20-M04 · mediaAnswer

{
  "msgToken": "s20-78",
  "method": "mediaAnswer",
  "sessionId": "s20-session",
  "data": {
    "mediaRevision": 1,
    "webrtc": {
      "iceMode": "complete",
      "sdp": "<SDP answer>"
    }
  },
  "code": 200
}

S20-M05 · mediaState

{
  "msgToken": "s20-79",
  "method": "mediaState",
  "sessionId": "s20-session",
  "params": {
    "mediaRevision": 1,
    "state": "connected"
  }
}

S20-M06 · requestAck

{
  "msgToken": "s20-79",
  "method": "requestAck",
  "sessionId": "s20-session",
  "data": {
    "requestMethod": "mediaState",
    "result": {
      "mediaRevision": 1,
      "state": "connected"
    }
  },
  "code": 200
}

S20-M07 · sessionEstablished

{
  "msgToken": "s20-80",
  "method": "sessionEstablished",
  "sessionId": "s20-session",
  "data": {
    "state": "established",
    "mediaRevision": 1
  },
  "eventSeq": 1
}

S20-M08 · audioStart

{
  "msgToken": "s20-81",
  "method": "audioStart",
  "sessionId": "s20-session",
  "params": {
    "inputId": "continuous-wrtc"
  }
}

S20-M09 · requestAck

{
  "msgToken": "s20-81",
  "method": "requestAck",
  "sessionId": "s20-session",
  "data": {
    "requestMethod": "audioStart",
    "result": {
      "inputId": "continuous-wrtc"
    }
  },
  "code": 200
}

S20-M10 · respCreated

{
  "msgToken": "s20-82",
  "method": "respCreated",
  "sessionId": "s20-session",
  "data": {
    "respId": "r-old",
    "inputId": "continuous-wrtc"
  },
  "messageSeq": 1
}

S20-M11 · respTextDone

{
  "msgToken": "s20-83",
  "method": "respTextDone",
  "sessionId": "s20-session",
  "data": {
    "respId": "r-old",
    "contentIndex": 0,
    "text": "请继续说。"
  },
  "messageSeq": 2
}

S20-M12 · respAudioStart

{
  "msgToken": "s20-84",
  "method": "respAudioStart",
  "sessionId": "s20-session",
  "data": {
    "respId": "r-old",
    "trackId": "speaker",
    "mediaRevision": 1
  },
  "eventSeq": 2
}

S20-M13 · respCancel

{
  "msgToken": "s20-85",
  "method": "respCancel",
  "sessionId": "s20-session",
  "params": {
    "respId": "r-old"
  }
}

S20-M14 · requestAck

{
  "msgToken": "s20-85",
  "method": "requestAck",
  "sessionId": "s20-session",
  "data": {
    "requestMethod": "respCancel",
    "result": {
      "respId": "r-old",
      "state": "cancelled"
    }
  },
  "code": 200
}

S20-M15 · audioSpeechInterrupted

{
  "msgToken": "s20-86",
  "method": "audioSpeechInterrupted",
  "sessionId": "s20-session",
  "data": {
    "respId": "r-old",
    "trackId": "speaker",
    "mediaRevision": 1
  },
  "eventSeq": 3
}

S20-M25 · respDone

{
  "method": "respDone",
  "msgToken": "s20-audit-5",
  "sessionId": "s20-session",
  "messageSeq": 3,
  "data": {
    "respId": "r-old",
    "inputId": "continuous-wrtc",
    "status": "cancelled",
    "contentCount": 1,
    "audioState": "cancelled",
    "reason": "bargeIn"
  }
}

S20-M16 · respCreated

{
  "msgToken": "s20-87",
  "method": "respCreated",
  "sessionId": "s20-session",
  "data": {
    "respId": "r-new",
    "inputId": "continuous-wrtc"
  },
  "messageSeq": 4
}

S20-M17 · respAudioStart

{
  "msgToken": "s20-88",
  "method": "respAudioStart",
  "sessionId": "s20-session",
  "data": {
    "respId": "r-new",
    "trackId": "speaker",
    "mediaRevision": 1
  },
  "eventSeq": 4
}

S20-M26 · respDone

{
  "method": "respDone",
  "msgToken": "s20-audit-6",
  "sessionId": "s20-session",
  "messageSeq": 5,
  "data": {
    "respId": "r-new",
    "inputId": "continuous-wrtc",
    "status": "completed",
    "contentCount": 0,
    "audioState": "sent"
  }
}

S20-M27 · respDone

{
  "msgToken": "s20-final-60",
  "method": "respDone",
  "sessionId": "s20-session",
  "messageSeq": 6,
  "data": {
    "respId": "r-new",
    "inputId": "continuous-wrtc",
    "status": "failed",
    "reason": "upstreamUnavailable",
    "contentCount": 0,
    "audioState": "failed"
  }
}

S20-M18 · sessionError

{
  "msgToken": "s20-89",
  "method": "sessionError",
  "sessionId": "s20-session",
  "data": {
    "reason": "upstreamUnavailable",
    "retryable": true,
    "scope": "response",
    "respId": "r-new"
  },
  "eventSeq": 5,
  "code": 502
}

S20-M19 · audioStop

{
  "msgToken": "s20-90",
  "method": "audioStop",
  "sessionId": "s20-session",
  "params": {
    "inputId": "continuous-wrtc"
  }
}

S20-M20 · requestAck

{
  "msgToken": "s20-90",
  "method": "requestAck",
  "sessionId": "s20-session",
  "data": {
    "requestMethod": "audioStop",
    "result": {
      "inputId": "continuous-wrtc"
    }
  },
  "code": 200
}

S20-M21 · audioStart

{
  "msgToken": "s20-91",
  "method": "audioStart",
  "sessionId": "s20-session",
  "params": {
    "inputId": "continuous-wrtc-2"
  }
}

S20-M22 · requestAck

{
  "msgToken": "s20-91",
  "method": "requestAck",
  "sessionId": "s20-session",
  "data": {
    "requestMethod": "audioStart",
    "result": {
      "inputId": "continuous-wrtc-2"
    }
  },
  "code": 200
}

S20-M23 · sessionEnd

{
  "msgToken": "s20-92",
  "method": "sessionEnd",
  "sessionId": "s20-session",
  "params": {
    "reason": "userHangup"
  }
}

S20-M24 · sessionEnded

{
  "msgToken": "s20-92",
  "method": "sessionEnded",
  "sessionId": "s20-session",
  "data": {
    "state": "ended",
    "reason": "userHangup"
  },
  "code": 200
}

媒体与本地动作

编号路径格式与约束
S20-R01D→PICE与DTLS握手，非MQTT；失败走S11
S20-R02D→PWebRTC Opus麦克风持续轨，平台适配为供应商要求的采样率和帧长
S20-R03P→DWebRTC Opus回复轨；网络到达不等于扬声器播放
S20-R04D→P播放中插话的持续上行
S20-R05P→D新回复音轨内容；旧终态仅更新旧respId

异常、状态与资源释放

路由与信封见公共约定；阶段性状态不代表最终完成。操作终态时限见命令手册；编号、超时等元信息不发送。

能力不足422、无权限403、资源/修订号冲突409、超时504按S11处理，不静默开启设备能力。同token只重放原结果；事件按eventSeq去重，旧会话/端代次不可恢复。验证本页正常路径、alt分支及支持范围断言。文档通过不代表代码或硬件已实现。

验证入口

检查本场景对应的支持范围，以及公共流程。示例可解析不等于网络、权限或硬件运行已验证。

更新日志

2026/10/16 16:04
查看所有更新日志
  • fe3a7-docs(AI与音视频): 按场景整理完整接入流程与命令契约于