應(yīng)用引入LLM實(shí)踐-一次性輸出和流式輸出(思維鏈)
在大模型應(yīng)用時(shí),有的場(chǎng)景希望根據(jù)prompt要求一次性輸出結(jié)果,有的場(chǎng)景則希望輸出整個(gè)思維過程以及最后的結(jié)果。
這部分在網(wǎng)上看了一些文章說的都不一樣,自己嘗試了一下,正確的寫法是這樣的,記錄一下。
一次性輸出:
from openai import OpenAI def generate_huoshan(prompt): client = OpenAI( # 從環(huán)境變量中讀取您的方舟API Key api_key="**", base_url="https://ark.cn-beijing.volces.com/api/v3", # 深度推理模型耗費(fèi)時(shí)間會(huì)較長(zhǎng),建議您設(shè)置一個(gè)較長(zhǎng)的超時(shí)時(shí)間,推薦為30分鐘 timeout=1800, ) response = client.chat.completions.create( model="deepseek-r1-250120", messages=[ {"role": "system", "content": "You are a professional market research assistant who needs to accurately obtain retail price information for specified electronic products in a specific market"}, {"role": "user", "content": prompt}, ], max_tokens=1024, temperature=0.6, stream=False ) answer = response.choices[0].message.content return answer.strip()
流式輸出:
from openai import OpenAI def generate_huoshan(prompt): client = OpenAI( api_key="*", base_url="https://ark.cn-beijing.volces.com/api/v3", # 深度推理模型耗費(fèi)時(shí)間會(huì)較長(zhǎng),建議您設(shè)置一個(gè)較長(zhǎng)的超時(shí)時(shí)間,推薦為30分鐘 timeout=1800, ) response = client.chat.completions.create( model="deepseek-r1-250120", messages=[ {"role": "system", "content": "You are a professional market research assistant who needs to accurately obtain retail price information for specified electronic products in a specific market"}, {"role": "user", "content": prompt}, ], max_tokens=1024, temperature=0.6, stream=True ) for chunk in response: delta = chunk.choices[0].delta # 優(yōu)先提取思維鏈內(nèi)容 if hasattr(delta, 'reasoning_content') and delta.reasoning_content: yield delta.reasoning_content #print(f"[推理過程] {delta.reasoning_content}", end="\n", flush=True) # 處理最終回答內(nèi)容 elif delta.content: yield delta.content #print(f"[最終回答] {delta.content}", end="", flush=True) else: continue
外層通過這樣返回:
def generate_stream(): try: for chunk in generate_text(model_name, prompt): #yield chunk yield json.dumps({"msg": "Success", "code": 200, "data": chunk})+ '\n' except Exception as e: yield json.dumps({"code": 500, "message": str(e)})+ '\n' # Yield a JSON string headers = { 'Content-Type':'text/event-stream', 'Cache-Control': 'no-cache', 'X-Accel-Buffering':'no', } return Response(generate_stream(), mimetype='text/event-stream',headers=headers)
然后,前端相應(yīng)做解析即可。


浙公網(wǎng)安備 33010602011771號(hào)