Failed to load model

[jesseadams]

LocalAI version:

localai/localai:v3.5.0-gpu-hipblas

Environment, CPU architecture, OS, and Version:

When I try to use the chat feature with any model.

To Reproduce

1. Click on Chat
2. Type in "Hi" and press enter.

Expected behavior

The model to generate a response and display it on the screen.

Logs

10:31AM ERR Failed to load model llama-3.3-70b-instruct with backend llama-cpp error="failed to load model with internal loader: could not load model: rpc error: code = Canceled desc = " modelID=llama-3.3-70b-instruct

Additional context

Graphics:
  Device-1: Advanced Micro Devices [AMD/ATI] Strix [Radeon 880M / 890M]
    driver: amdgpu v: 6.12.12
  Device-2: Framework Laptop Webcam Module (2nd Gen) driver: uvcvideo
    type: USB
  Display: wayland server: X.Org v: 24.1.6 with: Xwayland v: 24.1.6
    compositor: gnome-shell v: 48.0 driver: X: loaded: amdgpu dri: radeonsi
    gpu: amdgpu resolution: 2880x1920~120Hz
  API: EGL v: 1.5 drivers: kms_swrast,radeonsi,swrast
    platforms: gbm,wayland,x11,surfaceless,device
  API: OpenGL v: 4.6 compat-v: 3.3 vendor: amd mesa v: 25.0.0-devel
    renderer: AMD Radeon Graphics (radeonsi gfx1150 LLVM 19.1.5 DRM 3.63
    6.14.0-29-generic)
  Info: Tools: api: clinfo, eglinfo, glxinfo gpu: amd-smi,radeontop
    x11: xdriinfo, xdpyinfo, xprop, xrandr

[mapson01]

Hi have the same behavior. It seems LocalAi does not work with my internal GPU, in my case got the same error and checking the logs does not identify the graphic card correctly and report cero VRAM that is,I believe, the reason of why can not load the model, but does detect the iGPU . IGPUs does not have dedicated VRAM so never will be enough VRAM.. it seem related to #6057 but is not clear what is the solution or how workaround it.

any clue ?
tnks

by the way my iGPU is a

00:02.0 VGA compatible controller: Intel Corporation Alder Lake-P GT2 [Iris Xe Graphics] (rev 0c) Gen 12

[Hello-World-Traveler]

Internal error: failed to load model with internal loader: could not load model: rpc error: code = Canceled desc =

With GPU