feat: support for api_key param

[nicor88]

Feature request

Based on what I see in the README:

The API key: The API key used for client authentication. This is optional.

Anyhow, there is no documentation on how to pass an api_key, and if not possible will be a needed feature to secure access to the LLM models.

This question also address the same issue.

Motivation

Hosting LLM in our AWS infra without a protection mechanism is dangerous, and expose security concerns.

Other

No response

[aarnphm]

not sure what you mean. but wouldn't api_key into the client would work?

[nicor88]

@aarnphm the question that I have is how I set the api_key in the LLM serving endpoint - then once set the client can pass the api_key, but I don't know how to set the api_key, any example?

[aarnphm]

Probably you want do setup custom repo with bentovllm, then uses vLLM’s support for api key there.

[nicor88]

Probably you want do setup custom repo with bentovllm, then uses vLLM’s support for api key there.

Why that vs having a feature to support custom API keys in this repo? Maybe I'm missing something. But pretty much I want to use OpenLLM with the possibility to set a custom API key, in order to have the client requests to the LLM (hosted in my infra) authenticated.
Let me know if it's clear - anyhow I don't have any issue on forking this repo, add the feature and propose a PR to implement this.