Skip to content

Fix intel-mkl-src dependency and use static-linking in Dockerfile #715

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 9, 2025
Merged

Fix intel-mkl-src dependency and use static-linking in Dockerfile #715

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d334824
Add `static-linking` feature in `Dockerfile`
alvarobartt Sep 8, 2025
c648810
Add missing spacing
alvarobartt Sep 8, 2025
6683c25
Adding `features = ["mkl-static-..."]` works
alvarobartt Sep 8, 2025
2d6b431
Add missing `dep:intel-mkl-src` in `router/Cargo.toml`
alvarobartt Sep 8, 2025
e7fa45c
Set `default-features = false` for `intel-mkl-src`
alvarobartt Sep 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,13 @@ serde_json = "1.0"
thiserror = "1.0"
rand = "0.9"
serial_test = "2.0.0"
cudarc = { version = "0.13" , features =["cuda-12020"], default-features = false}
intel-mkl-src = { version = "0.8"}
cudarc = { version = "0.13", features =["cuda-12020"], default-features = false }
intel-mkl-src = { version = "0.8", default-features = false }
candle = { version = "0.8", package = "candle-core" }
candle-nn = { version = "0.8" }
candle-nn = { version = "0.8" }
candle-transformers = { version = "0.8" }
candle-flash-attn = { version = "0.8" }
candle-cublaslt= { version = "0.0.1" }
candle-cublaslt = { version = "0.0.1" }
candle-layer-norm = { version = "0.0.1" }
candle-rotary = { version = "0.0.1" }
candle-flash-attn-v1 = { version = "0.0.1" }
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ COPY --from=planner /usr/src/recipe.json recipe.json

RUN --mount=type=secret,id=actions_results_url,env=ACTIONS_RESULTS_URL \
--mount=type=secret,id=actions_runtime_token,env=ACTIONS_RUNTIME_TOKEN \
cargo chef cook --release --features ort,candle,mkl --no-default-features --recipe-path recipe.json && sccache -s
cargo chef cook --release --features ort,candle,mkl,static-linking --no-default-features --recipe-path recipe.json && sccache -s

COPY backends backends
COPY core core
Expand All @@ -55,7 +55,7 @@ FROM builder AS http-builder

RUN --mount=type=secret,id=actions_results_url,env=ACTIONS_RESULTS_URL \
--mount=type=secret,id=actions_runtime_token,env=ACTIONS_RUNTIME_TOKEN \
cargo build --release --bin text-embeddings-router --features ort,candle,mkl,http --no-default-features && sccache -s
cargo build --release --bin text-embeddings-router --features ort,candle,mkl,static-linking,http --no-default-features && sccache -s

FROM builder AS grpc-builder

Expand All @@ -69,7 +69,7 @@ COPY proto proto

RUN --mount=type=secret,id=actions_results_url,env=ACTIONS_RESULTS_URL \
--mount=type=secret,id=actions_runtime_token,env=ACTIONS_RUNTIME_TOKEN \
cargo build --release --bin text-embeddings-router --features ort,candle,mkl,grpc --no-default-features && sccache -s
cargo build --release --bin text-embeddings-router --features ort,candle,mkl,static-linking,grpc --no-default-features && sccache -s

FROM debian:bookworm-slim AS base

Expand Down
2 changes: 1 addition & 1 deletion backends/candle/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ homepage.workspace = true
[dependencies]
anyhow = { workspace = true }
accelerate-src = { version = "0.3.2", optional = true }
intel-mkl-src = { workspace = true, optional = true }
intel-mkl-src = { workspace = true, optional = true }
candle = { workspace = true }
candle-nn = { workspace = true }
candle-transformers = { workspace = true }
Expand Down
2 changes: 1 addition & 1 deletion router/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ default = ["candle", "http", "dynamic-linking"]
http = ["dep:axum", "dep:axum-tracing-opentelemetry", "dep:base64", "dep:tower-http", "dep:utoipa", "dep:utoipa-swagger-ui"]
grpc = ["metrics-exporter-prometheus/http-listener", "dep:prost", "dep:tonic", "dep:tonic-health", "dep:tonic-reflection", "dep:tonic-build", "dep:async-stream", "dep:tokio-stream"]
metal = ["text-embeddings-backend/metal"]
mkl = ["text-embeddings-backend/mkl"]
mkl = ["text-embeddings-backend/mkl", "dep:intel-mkl-src"]
accelerate = ["text-embeddings-backend/accelerate"]
python = ["text-embeddings-backend/python"]
ort = ["text-embeddings-backend/ort"]
Expand Down
Loading