Skip to content

Commit 847e270

Browse files
authored
Add performance config for microsoft/Phi-3-medium-4k-instruct (#51)
add phi3 perf config
1 parent 7b09318 commit 847e270

File tree

1 file changed

+6
-0
lines changed
  • microsoft/Phi-3-medium-4k-instruct/performance

1 file changed

+6
-0
lines changed
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
enable-chunked-prefill: true
2+
max-model-len: 4096
3+
tensor-parallel-size: 1
4+
trust-remote-code: true
5+
uvicorn-log-level: debug
6+
no-enable-prefix-caching: true

0 commit comments

Comments
 (0)