docs: add vllm semantic router blog #76
Closed
+113
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Xunzhuo
force-pushed
the
add-vsr-blog
branch
2 times, most recently
from
6792af7 to
a6c1fe5
Compare
Xunzhuo
force-pushed
the
add-vsr-blog
branch
2 times, most recently
from
52f372e to
5a632bd
Compare
Signed-off-by: bitliu <[email protected]>
Xunzhuo
force-pushed
the
add-vsr-blog
branch
from
5a632bd to
5fd1503
Compare
hmellor
reviewed
Sep 1, 2025
Signed-off-by: bitliu <[email protected]>
rootfs
reviewed
Sep 1, 2025
|
|
||
| Take **GPT-5** as an example. Its real breakthrough isn't in the number of parameters, but in the **"automatic routing + thinking quota"**: | ||
|
|
||
| * **Light queries → Light models**: For example, "Why is the sky blue?" does not require expensive inference models. |
There was a problem hiding this comment.
nit: "light" -> "simple"/"trivial"/or "casual"
rootfs
reviewed
Sep 1, 2025
|
|
||
| * **Complex/High-value queries → Strong inference models**: Legal analysis, financial simulations, etc., are routed to models with Chain-of-Thought capabilities. | ||
|
|
||
| The logic behind this mechanism is called **"Per-token Unit Economics"**. |
There was a problem hiding this comment.
"Per-token Unit Economics"
Maybe we can borrow a page from here and call it "AI Token Economics"
|
please continue at #77 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this PR adds vllm semantic router blog