共50道题 | 难度分布:★★★★☆平均 | 20K+ AI应用岗位流式输出必考
★★★★★
Content-Type: text/event-stream,Cache-Control: no-cache,Connection: keep-alive。data:开头,以两个换行符结束。支持event:(事件类型)、id:(事件ID)、retry:(重连间隔)字段。
data: {"content": "你好"}
data: {"content": ",我"}
data: {"content": "是AI"}
data: [DONE]
★★★★★
★★★★
data:字段:消息内容,可以有多个data行(用\n分隔)。event:字段:事件类型,客户端可按类型监听。默认是message事件。id:字段:事件唯一标识,用于断线重连时通过Last-Event-ID恢复。retry:字段:建议客户端的重连间隔(毫秒)。
id: 1
event: token
data: {"text": "你"}
id: 2
event: token
data: {"text": "好"}
★★★
fetch + ReadableStream替代EventSource,支持POST和自定义Header。★★★★
Last-Event-ID:重连时自动携带最后收到的事件ID,服务器可以从断点继续发送。retry:字段:服务器可以动态调整客户端的重连间隔。★★★★
text/event-stream。data: {"type":"heartbeat"})防止连接被代理/防火墙关闭。data: [DONE]或客户端调用eventSource.close()。★★★★★
@GetMapping("/chat/stream")
public SseEmitter streamChat(@RequestParam String message) {
SseEmitter emitter = new SseEmitter(300_000L); // 5分钟超时
executor.submit(() -> {
try {
chatService.streamChat(message, token -> {
emitter.send(SseEmitter.event()
.data("{\"token\":\"" + token + "\"}")
.id(String.valueOf(tokenCounter++)));
});
emitter.send(SseEmitter.event().data("[DONE]"));
emitter.complete();
} catch (Exception e) {
emitter.completeWithError(e);
}
});
return emitter;
}
onCompletion、onTimeout、onError。★★★★★
@GetMapping(value = "/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ServerSentEvent<String>> streamChat(@RequestParam String message) {
return chatService.streamChat(message)
.map(token -> ServerSentEvent.<String>builder()
.data(token)
.build())
.concatWith(Flux.just(
ServerSentEvent.<String>builder().data("[DONE]").build()
));
}
★★★★
★★★★
server.tomcat.threads.max=500)。@Async + 自定义线程池)。★★★
Content-Type指定UTF-8:text/event-stream;charset=UTF-8。emitter.send(event, MediaType.APPLICATION_JSON_UTF8)。produces = "text/event-stream;charset=UTF-8"。★★★★
@CrossOrigin(origins = "*") 或全局配置。Access-Control-Allow-Origin、Access-Control-Allow-Headers。
http.authorizeHttpRequests(auth -> auth
.requestMatchers("/chat/stream").permitAll()
);
★★★★★
location /chat/stream {
proxy_pass http://backend;
proxy_http_version 1.1;
proxy_set_header Connection '';
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 300s;
}
proxy_buffering off:禁用响应缓冲,SSE事件立即转发。proxy_read_timeout:SSE长连接超时设置,默认60s太短。proxy_buffering。★★★★
sessionAffinity: ClientIP)。★★★★★
★★★★★
stream: true启用流式响应。ChatCompletionChunk对象。
data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"你"},"index":0}]}
data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"好"},"index":0}]}
data: [DONE]
delta字段:增量内容(只包含本次新生成的Token)。[DONE]:流结束标记。stream_options: {"include_usage": true}可以在最后一条消息中返回Token用量。delta是增量而非全量——客户端需要自行拼接完整响应。★★★★
data字段→解析JSON→提取delta.content→拼接。data: [DONE]或choices[0].finish_reason非null。data可能包含错误JSON(如data: {"error":...}),需要容错。StringBuilder,每次收到delta追加。
let fullText = '';
eventSource.onmessage = (event) => {
if (event.data === '[DONE]') { eventSource.close(); return; }
const chunk = JSON.parse(event.data);
const token = chunk.choices[0]?.delta?.content || '';
fullText += token;
updateUI(fullText);
};
★★★★★
stream()方法返回Flux<ChatResponse>。
Flux<ChatResponse> stream = chatClient.prompt()
.user("解释Java虚拟线程")
.stream()
.chatResponse();
stream.subscribe(response -> {
String token = response.getResult().getOutput().getText();
// 发送SSE事件
emitter.send(SseEmitter.event().data(token));
});
emitter.send()。★★★★
@GetMapping(value = "/chat", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ServerSentEvent<String>> chat(@RequestParam String message) {
return chatClient.prompt()
.user(message)
.stream()
.chatResponse()
.map(resp -> {
String content = resp.getResult().getOutput().getText();
return ServerSentEvent.<String>builder().data(content).build();
})
.concatWithValues(
ServerSentEvent.<String>builder().data("[DONE]").build()
);
}
★★★★
[DONE]后获取usage。stream_options: {"include_usage": true}在最后一条消息返回Token用量。ChatResponse.getMetadata().getUsage()获取Token用量(流式最后一条包含)。max_tokens限制单次生成的最大Token数。★★★★
event: error通知客户端。eventSource.close()或AbortController.abort()。★★★★
delta.content,Claude用content_block_delta,国产模型各有差异。ChatModel接口屏蔽不同模型的流式格式差异。[DONE],Claude用message_stop),delta字段路径不同。★★★★
const source = new EventSource('/chat/stream?message=hello');
source.onmessage = (event) => { console.log(event.data); };
source.onerror = () => { source.close(); };
★★★★★
async function streamChat(message) {
const response = await fetch('/chat/stream', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': 'Bearer ' + token
},
body: JSON.stringify({ message })
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n\n');
buffer = lines.pop();
for (const line of lines) {
const data = line.replace(/^data: /, '');
if (data === '[DONE]') return;
const chunk = JSON.parse(data);
appendToken(chunk.choices[0].delta.content);
}
}
}
★★★★
**粗体只收到一半)。`后进入代码块模式,收集到闭合`后再渲染高亮。
let fullText = '';
let renderTimer = null;
function appendToken(token) {
fullText += token;
clearTimeout(renderTimer);
renderTimer = setTimeout(() => {
document.getElementById('output').innerHTML = marked.parse(fullText);
}, 50);
}
★★★
`计数),如果是则延迟高亮直到代码块闭合。★★★★
const tokenQueue = [];
let isTyping = false;
function enqueueToken(token) {
tokenQueue.push(...token.split(''));
if (!isTyping) typeNext();
}
function typeNext() {
if (tokenQueue.length === 0) { isTyping = false; return; }
isTyping = true;
appendChar(tokenQueue.shift());
setTimeout(typeNext, 30);
}
★★★★
const controller = new AbortController();
fetch('/chat/stream', { signal: controller.signal, ... });
// 用户点击停止按钮
controller.abort();
eventSource.close()。onCompletion回调,可清理资源。★★★★
onBackpressureBuffer(缓冲)、onBackpressureDrop(丢弃)、onBackpressureLatest(只保留最新)。★★★
buffer(n):缓冲最多n个元素,超出抛异常。drop():消费者处理不过来时丢弃新元素。latest():只保留最新元素,丢弃中间的。latest()——如果网络慢,跳过中间Token只显示最新的,比丢帧好。
chatService.streamChat(message)
.onBackpressureLatest()
.subscribe(token -> emitter.send(token));
latest()策略最合理——宁可跳过中间Token也不能让内存溢出。★★★★
max_tokens限制、服务端节流)。★★★
max_tokens限制单次响应长度,防止内存溢出。★★★★
server.tomcat.max-connections=8192控制最大连接数。★★★★★
@RestController
@RequestMapping("/api/chat")
public class ChatController {
private final ChatClient chatClient;
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ServerSentEvent<String>> streamChat(
@RequestParam String message,
@RequestParam(defaultValue = "default") String model) {
return chatClient.prompt()
.user(message)
.stream()
.chatResponse()
.map(resp -> {
String content = resp.getResult().getOutput().getText();
return ServerSentEvent.<String>builder()
.data(Map.of("token", content).toString())
.build();
})
.concatWithValues(
ServerSentEvent.<String>builder()
.event("done").data("").build()
)
.onErrorResume(e -> Flux.just(
ServerSentEvent.<String>builder()
.event("error").data(e.getMessage()).build()
));
}
}
★★★★
ChatMemory接口在流式完成后自动记录对话。★★★★
stream_ttft_seconds、stream_token_throughput、stream_active_connections。★★★★
Retry-After Header,让客户端延迟重试。★★★★
netstat -an | grep ESTABLISHED | wc -l查看活跃连接数。onCompletion回调清理资源。★★★★
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"。sessionAffinity: ClientIP保持SSE连接到同一Pod。terminationGracePeriodSeconds: 120)。★★★★
★★★★
★★★★
event: thinking\ndata: {"step":1,"content":"需要查询数据库..."}
event: tool_call\ndata: {"tool":"sql_query","args":"SELECT ..."}
event: tool_result\ndata: {"result":"[...]"}
event: answer\ndata: {"token":"根据查询结果..."}
★★★★
tool_calls delta时,暂停文本流,发送工具调用事件。★★★
★★★★
★★★★
data:前缀+换行,约10字节额外开销。★★★★★
★★★
★★★★★
★★★★★