We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 7b09318 commit 847e270Copy full SHA for 847e270
microsoft/Phi-3-medium-4k-instruct/performance/server.yml
@@ -0,0 +1,6 @@
1
+enable-chunked-prefill: true
2
+max-model-len: 4096
3
+tensor-parallel-size: 1
4
+trust-remote-code: true
5
+uvicorn-log-level: debug
6
+no-enable-prefix-caching: true
0 commit comments