ESP32S3+IDF環境下websocket連接阿里百煉實時多模態大模型

首先貼一個阿里百煉實時多模態大模型的文檔（找對這個文檔花了我不少時間）：
大模型服務平臺百煉控制臺

文檔給出了一個 websocket python 示例，我們要做的是參考并移植到C環境下：

# pip install websocket-client
import json
import websocket
import os

API_KEY=os.getenv("DASHSCOPE_API_KEY")
API_URL = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen-omni-turbo-realtime"

headers = [
    "Authorization: Bearer " + API_KEY
]

def on_open(ws):
    print(f"Connected to server: {API_URL}")
def on_message(ws, message):
    data = json.loads(message)
    print("Received event:", json.dumps(data, indent=2))
def on_error(ws, error):
    print("Error:", error)

ws = websocket.WebSocketApp(
    API_URL,
    header=headers,
    on_open=on_open,
    on_message=on_message,
    on_error=on_error
)

ws.run_forever()

根據這個示例，我們得出需要以下兩個參數：

API_URL = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen-omni-turbo-realtime"

headers = [ "Authorization: Bearer " + API_KEY]

所以改寫進ESP32-idf環境：

#include "esp_websocket_client.h"

void websocket_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data);

void websocket_init(void)
{
    esp_websocket_client_config_t ws_cfg = {
        .uri = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen-omni-turbo-realtime",                  // 使用帶鑒權參數的URL
        .transport = WEBSOCKET_TRANSPORT_OVER_SSL,   // 明確使用SSL傳輸
        .reconnect_timeout_ms = 10000,               // 重連超時設置
    };

    client = esp_websocket_client_init(&ws_cfg);

    esp_websocket_client_append_header(client, "Authorization", "Bearer sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX");

    esp_websocket_register_events(client, WEBSOCKET_EVENT_ANY, websocket_event_handler, NULL);
    esp_websocket_client_start(client);
}

void websocket_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data)
{
    esp_websocket_event_data_t *data = (esp_websocket_event_data_t *)event_data;
    switch (event_id) {
        case WEBSOCKET_EVENT_CONNECTED:
            ESP_LOGI("WS", "連接成功");
            vTaskDelay(4000 / portTICK_PERIOD_MS); // 延遲2秒再啟動任務
            break;
        case WEBSOCKET_EVENT_DATA:
            printf("收到數據: %s\n", (char*)data->data_ptr);
            break;
        case WEBSOCKET_EVENT_DISCONNECTED:
            ESP_LOGI("WS", "連接斷開，嘗試重連...");
            break;
    }
}

連接成功后得到以下打印：

I (8740) WS: 連接成功
收到數據: {"event_id":"event_R6ALkF8t1bCgkkNymZvvT","type":"session.created","session":{"object":"realtime.session","model":"qwen-omni-turbo-realtime","modalities":["text","audio"],"instructions":"...model instructions here...","voice":"Chelsie","input_audio_format":"pcm16","output_audio_format":"pcm16","input_audio_transcription":{"model":"gummy-realtime-v1"},"turn_detection":{"type":"server_vad","threshold":0.5,"prefix_padding_ms":300,"silence_duration_ms":800,"create_response":true,"interrupt_response":true},"tools":[],"tool_choice":"auto","temperature":0.8,"max_response_output_tokens":"inf","id":"sess_CFP8GFog5dp8RGOrhoJY3"}}

連接成功！