Skip to content

Commit 9a19f6d

Browse files
jongallowayCopilotMackinnonBuck
authored
Add note on extending timeout for Ollama client (#6258)
* Add note on extending timeout for Ollama client Standard HttpClient timeout of 10 seconds applied by Service Defaults under .NET Aspire is likely too short for Ollama calls. Add note with code to extend this timeout. * Update src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/README.Aspire.md Co-authored-by: Copilot <[email protected]> * Update src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/README.Aspire.md Co-authored-by: Mackinnon Buck <[email protected]> --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Mackinnon Buck <[email protected]>
1 parent 32b3cbf commit 9a19f6d

File tree

1 file changed

+17
-0
lines changed
  • src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData

1 file changed

+17
-0
lines changed

src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/ChatWithCustomData/README.Aspire.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,23 @@ This project is configured to run Ollama in a Docker container.
7070

7171
To get started, download, install, and run Docker Desktop from the [official website](https://www.docker.com/). Follow the installation instructions specific to your operating system.
7272

73+
The response time from Ollama depends on your GPU. If you find that default resilience timeout of 10 seconds used by .NET Aspire is timing out for the HTTP call from the web app to the Ollama service, you can update the web app's `Program.cs` to extend the timeout by adding the following below the line which reads `builder.AddServiceDefaults();`:
74+
```csharp
75+
// Extend the HTTP Client timeout for Ollama
76+
builder.Services.ConfigureHttpClientDefaults(http =>
77+
{
78+
#pragma warning disable EXTEXP0001
79+
http.RemoveAllResilienceHandlers();
80+
http.AddStandardResilienceHandler(config =>
81+
{
82+
config.AttemptTimeout.Timeout = TimeSpan.FromMinutes(3);
83+
config.CircuitBreaker.SamplingDuration = TimeSpan.FromMinutes(10); // must be at least double the AttemptTimeout to pass options validation
84+
config.TotalRequestTimeout.Timeout = TimeSpan.FromMinutes(10);
85+
});
86+
#pragma warning restore EXTEXP0001
87+
});
88+
```
89+
7390
Note: Ollama and Docker are excellent open source products, but are not maintained by Microsoft.
7491
#### ---#endif
7592
#### ---#if (IsAzureOpenAI)

0 commit comments

Comments
 (0)