Claude cli >= 2.1.154 breaks local use with vLLM by introducing "ctx", "msg" and "system" roles for API messages. This 1-line patch to vLLM fixes it.
r/LocalLLaMA
•
Generative AI
AI Tools
Diff --git a/vllm/entrypoints/anthropic/protocol.py b/vllm/entrypoints/anthropic/protocol.py index 3ebc17117.2d5726d73 100644 --- a/vllm/entrypoints/anthropic/protocol.py +++ b/vllm/entrypoints/anthropic/protocol.py @@ -65,7 +65,7 @@ class AnthropicContentBlock(BaseModel): class AnthropicMessage(BaseModel): """Message structure""" - role: Literal["user", "assistant"] + role: Literal["user", "assistant", "ctx", "msg", "system"] content: str | list[AnthropicContentBlock] The changes are (I suspect) related to the new "workflows" feature.