The MKL dependency error in band_cal_parallel.py

[newplay]

Dear Yang Zhong,

When I run the band_cal_parallel python script, I encounter the following error:

Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.

My LD_LIBRARY_PATH is set as follows:

/home/zjlin/intel/oneapi/ccl/2021.10.0/lib/cpu_gpu_dpcpp:/home/zjlin/intel/oneapi/compiler/2023.2.0/linux/compiler/lib/intel64_lin:/home/zjlin/intel/oneapi/compiler/2023.2.0/linux/lib:/home/zjlin/intel/oneapi/compiler/2023.2.0/linux/lib/oclfpga/host/linux64/lib:/home/zjlin/intel/oneapi/compiler/2023.2.0/linux/lib/x64:/home/zjlin/intel/oneapi/dal/2023.2.0/lib/intel64:/home/zjlin/intel/oneapi/debugger/2023.2.0/dep/lib:/home/zjlin/intel/oneapi/debugger/2023.2.0/gdb/intel64/lib:/home/zjlin/intel/oneapi/debugger/2023.2.0/libipt/intel64/lib:/home/zjlin/intel/oneapi/dnnl/2023.2.0/cpu_dpcpp_gpu_dpcpp/lib:/home/zjlin/intel/oneapi/ipp/2021.9.0/lib/intel64:/home/zjlin/intel/oneapi/ippcp/2021.8.0/lib/intel64:/home/zjlin/intel/oneapi/itac/2021.10.0/slib:/home/zjlin/intel/oneapi/mkl/2023.2.0/lib/intel64:/home/zjlin/intel/oneapi/mpi/2021.10.0//lib:/home/zjlin/intel/oneapi/mpi/2021.10.0//libfabric/lib:/home/zjlin/intel/oneapi/mpi/2021.10.0//lib/release:/home/zjlin/intel/oneapi/tbb/2021.10.0/env/../lib/intel64/gcc4.8

The version of the mkl package is 2023.4.0, and the library contains the following files:

-rwxr-xr-x. 1 zjlin zjlin  50122480  6月 13  2023 libmkl_avx2.so.2
-rwxr-xr-x. 1 zjlin zjlin  66659704  6月 13  2023 libmkl_avx512.so.2
-rwxr-xr-x. 1 zjlin zjlin  53003488  6月 13  2023 libmkl_avx.so.2
-rw-r--r--. 1 zjlin zjlin   1244476  6月 13  2023 libmkl_blacs_intelmpi_ilp64.a
lrwxrwxrwx. 1 zjlin zjlin        32  6月 30  2023 libmkl_blacs_intelmpi_ilp64.so -> libmkl_blacs_intelmpi_ilp64.so.2
-rwxrwxr-x. 1 zjlin zjlin    523704  4月 15 13:49 libmkl_blacs_intelmpi_ilp64.so.1
-rwxr-xr-x. 1 zjlin zjlin    495248  6月 13  2023 libmkl_blacs_intelmpi_ilp64.so.2
-rw-r--r--. 1 zjlin zjlin    739070  6月 13  2023 libmkl_blacs_intelmpi_lp64.a
lrwxrwxrwx. 1 zjlin zjlin        31  6月 30  2023 libmkl_blacs_intelmpi_lp64.so -> libmkl_blacs_intelmpi_lp64.so.2
-rwxr-xr-x. 1 zjlin zjlin    304384  6月 13  2023 libmkl_blacs_intelmpi_lp64.so.2
-rw-r--r--. 1 zjlin zjlin   1263716  6月 13  2023 libmkl_blacs_openmpi_ilp64.a
lrwxrwxrwx. 1 zjlin zjlin        31  6月 30  2023 libmkl_blacs_openmpi_ilp64.so -> libmkl_blacs_openmpi_ilp64.so.2
-rwxr-xr-x. 1 zjlin zjlin    504472  6月 13  2023 libmkl_blacs_openmpi_ilp64.so.2
-rw-r--r--. 1 zjlin zjlin    758310  6月 13  2023 libmkl_blacs_openmpi_lp64.a
lrwxrwxrwx. 1 zjlin zjlin        30  6月 30  2023 libmkl_blacs_openmpi_lp64.so -> libmkl_blacs_openmpi_lp64.so.2
-rwxr-xr-x. 1 zjlin zjlin    309480  6月 13  2023 libmkl_blacs_openmpi_lp64.so.2
-rw-r--r--. 1 zjlin zjlin    659092  6月 13  2023 libmkl_blas95_ilp64.a
-rw-r--r--. 1 zjlin zjlin    657388  6月 13  2023 libmkl_blas95_lp64.a
-rw-r--r--. 1 zjlin zjlin    212330  6月 13  2023 libmkl_cdft_core.a
lrwxrwxrwx. 1 zjlin zjlin        21  6月 30  2023 libmkl_cdft_core.so -> libmkl_cdft_core.so.2
-rwxr-xr-x. 1 zjlin zjlin    164936  6月 13  2023 libmkl_cdft_core.so.2
-rw-r--r--. 1 zjlin zjlin 578357890  6月 13  2023 libmkl_core.a
lrwxrwxrwx. 1 zjlin zjlin        16  6月 30  2023 libmkl_core.so -> libmkl_core.so.2
-rwxrwxr-x. 1 zjlin zjlin  74757224  4月 15 13:48 libmkl_core.so.1
-rwxr-xr-x. 1 zjlin zjlin  73808072  6月 13  2023 libmkl_core.so.2
-rwxr-xr-x. 1 zjlin zjlin  42417640  6月 13  2023 libmkl_def.so.2
-rw-r--r--. 1 zjlin zjlin  33272494  6月 13  2023 libmkl_gf_ilp64.a
lrwxrwxrwx. 1 zjlin zjlin        20  6月 30  2023 libmkl_gf_ilp64.so -> libmkl_gf_ilp64.so.2
-rwxr-xr-x. 1 zjlin zjlin  16908096  6月 13  2023 libmkl_gf_ilp64.so.2
-rw-r--r--. 1 zjlin zjlin  39356660  6月 13  2023 libmkl_gf_lp64.a
lrwxrwxrwx. 1 zjlin zjlin        19  6月 30  2023 libmkl_gf_lp64.so -> libmkl_gf_lp64.so.2
-rwxr-xr-x. 1 zjlin zjlin  20686648  6月 13  2023 libmkl_gf_lp64.so.2
-rw-r--r--. 1 zjlin zjlin  45645264  6月 13  2023 libmkl_gnu_thread.a
lrwxrwxrwx. 1 zjlin zjlin        22  6月 30  2023 libmkl_gnu_thread.so -> libmkl_gnu_thread.so.2
-rwxr-xr-x. 1 zjlin zjlin  32660800  6月 13  2023 libmkl_gnu_thread.so.2
-rw-r--r--. 1 zjlin zjlin  42048900  6月 13  2023 libmkl_intel_ilp64.a
lrwxrwxrwx. 1 zjlin zjlin        23  6月 30  2023 libmkl_intel_ilp64.so -> libmkl_intel_ilp64.so.2
-rwxrwxr-x. 1 zjlin zjlin  12914912  4月 15 13:47 libmkl_intel_ilp64.so.1
-rwxr-xr-x. 1 zjlin zjlin  20492584  6月 13  2023 libmkl_intel_ilp64.so.2
-rw-r--r--. 1 zjlin zjlin  48143118  6月 13  2023 libmkl_intel_lp64.a
lrwxrwxrwx. 1 zjlin zjlin        22  6月 30  2023 libmkl_intel_lp64.so -> libmkl_intel_lp64.so.2
-rwxr-xr-x. 1 zjlin zjlin  24271352  6月 13  2023 libmkl_intel_lp64.so.2
-rw-r--r--. 1 zjlin zjlin  89325670  6月 13  2023 libmkl_intel_thread.a
lrwxrwxrwx. 1 zjlin zjlin        24  6月 30  2023 libmkl_intel_thread.so -> libmkl_intel_thread.so.2
-rwxr-xr-x. 1 zjlin zjlin  63598016  6月 13  2023 libmkl_intel_thread.so.2
-rw-r--r--. 1 zjlin zjlin   7446888  6月 13  2023 libmkl_lapack95_ilp64.a
-rw-r--r--. 1 zjlin zjlin   7379552  6月 13  2023 libmkl_lapack95_lp64.a
-rwxr-xr-x. 1 zjlin zjlin  50216352  6月 13  2023 libmkl_mc3.so.2
-rwxr-xr-x. 1 zjlin zjlin  48727472  6月 13  2023 libmkl_mc.so.2
-rw-r--r--. 1 zjlin zjlin  51404406  6月 13  2023 libmkl_pgi_thread.a
lrwxrwxrwx. 1 zjlin zjlin        22  6月 30  2023 libmkl_pgi_thread.so -> libmkl_pgi_thread.so.2
-rwxr-xr-x. 1 zjlin zjlin  37998288  6月 13  2023 libmkl_pgi_thread.so.2
lrwxrwxrwx. 1 zjlin zjlin        14  6月 30  2023 libmkl_rt.so -> libmkl_rt.so.2
-rwxr-xr-x. 1 zjlin zjlin  18568768  6月 13  2023 libmkl_rt.so.2
-rw-r--r--. 1 zjlin zjlin  12244390  6月 13  2023 libmkl_scalapack_ilp64.a
lrwxrwxrwx. 1 zjlin zjlin        27  6月 30  2023 libmkl_scalapack_ilp64.so -> libmkl_scalapack_ilp64.so.2
-rwxrwxr-x. 1 zjlin zjlin   7718648  4月 15 13:49 libmkl_scalapack_ilp64.so.1
-rwxr-xr-x. 1 zjlin zjlin   7718768  6月 13  2023 libmkl_scalapack_ilp64.so.2
-rw-r--r--. 1 zjlin zjlin  12329916  6月 13  2023 libmkl_scalapack_lp64.a
lrwxrwxrwx. 1 zjlin zjlin        26  6月 30  2023 libmkl_scalapack_lp64.so -> libmkl_scalapack_lp64.so.2
-rwxr-xr-x. 1 zjlin zjlin   7728456  6月 13  2023 libmkl_scalapack_lp64.so.2
-rw-r--r--. 1 zjlin zjlin  38044180  6月 13  2023 libmkl_sequential.a
lrwxrwxrwx. 1 zjlin zjlin        22  6月 30  2023 libmkl_sequential.so -> libmkl_sequential.so.2
-rwxrwxr-x. 1 zjlin zjlin  28992400  4月 15 13:48 libmkl_sequential.so.1
-rwxr-xr-x. 1 zjlin zjlin  28392328  6月 13  2023 libmkl_sequential.so.2
-rw-r--r--. 1 zjlin zjlin 852178088  6月 13  2023 libmkl_sycl.a
lrwxrwxrwx. 1 zjlin zjlin        16  6月 30  2023 libmkl_sycl.so -> libmkl_sycl.so.3
-rwxr-xr-x. 1 zjlin zjlin 650498600  6月 13  2023 libmkl_sycl.so.3
-rw-r--r--. 1 zjlin zjlin 112149460  6月 13  2023 libmkl_tbb_thread.a
lrwxrwxrwx. 1 zjlin zjlin        22  6月 30  2023 libmkl_tbb_thread.so -> libmkl_tbb_thread.so.2
-rwxr-xr-x. 1 zjlin zjlin  40662512  6月 13  2023 libmkl_tbb_thread.so.2
-rwxr-xr-x. 1 zjlin zjlin  14827696  6月 13  2023 libmkl_vml_avx2.so.2
-rwxr-xr-x. 1 zjlin zjlin  14295744  6月 13  2023 libmkl_vml_avx512.so.2
-rwxr-xr-x. 1 zjlin zjlin  15872192  6月 13  2023 libmkl_vml_avx.so.2
-rwxr-xr-x. 1 zjlin zjlin   7739640  6月 13  2023 libmkl_vml_cmpt.so.2
-rwxr-xr-x. 1 zjlin zjlin   8745920  6月 13  2023 libmkl_vml_def.so.2
-rwxr-xr-x. 1 zjlin zjlin  14644304  6月 13  2023 libmkl_vml_mc3.so.2
-rwxr-xr-x. 1 zjlin zjlin  14787656  6月 13  2023 libmkl_vml_mc.so.2
drwxrwxr-x. 3 zjlin zjlin        27 12月 12 11:38 locale

I suspect the problem may be related to the libmkl_blacs_openmpi_lp64 library with reference
Is it correlated with the mkl version? Because I am using Intel oneAPI and there are no .dll files in the library.
Best Regards,
TzuChing

[QuantumLab-ZY]

Dear TzuChing,

The directory of the MKL library I used when compiling mpitool was '/opt/compiler/intel2018u4/compilers_and_libraries_2018.5.274/linux/mkl/lib/intel64/'. Everything went smoothly when using this library at runtime. Recently, I recompiled mpitool with the MKL library from oneAPI2023, but I still encountered the error "Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers" at runtime. I haven't found a solution to this problem yet. It seems that currently mpitool can only be used with the traditional Intel compiler's MKL library instead of oneAPI's MKL library.

Best Regards,
Yang Zhong

[newplay]

Dear Yang Zhong,
Thanks for your reply, I will try use the traditional Intel compiler's mkl to use band_cal_parallel again
TzuChing

[QuantumLab-ZY]

Dear TzuChing,

User flamingoXu seems to have found a solution in Issue #18, you can try it out.

Best wishes,
Yang Zhong

[newplay]

Dear Yang Zhong,

I attempted to install the MKL dependencies using the following command:

mamba install -c intel mkl=2024.1.0 mkl_fft=1.3.1 mkl_random=1.2.2 mkl-service=2.4.0

However, I encountered the following warning and error:

warning  libmamba Added empty dependency for problem type SOLVER_RULE_UPDATE
Could not solve for environment specs
The following packages are incompatible
├─ mkl 2024.1.0**  is requested and can be installed;
└─ mkl_fft 1.3.1**  is installable with the potential options
   ├─ mkl_fft 1.3.1 would require
   │  └─ python >=3.10,<3.11.0a0  with the potential options
   │     ├─ python [3.10.0|3.10.10|...|3.10.9], which can be installed;
   │     └─ python 3.10.14 would require
   │        └─ libsqlite >=3.45.2,<4.0a0 , which does not exist (perhaps a missing channel);
   ├─ mkl_fft 1.3.1 would require
   │  └─ python >=3.7,<3.8.0a0 , which can be installed;
   ├─ mkl_fft 1.3.1 would require
   │  └─ python >=3.8,<3.9.0a0 , which can be installed;
   ├─ mkl_fft 1.3.1 would require
   │  └─ mkl >=2022.1.0,<2023.0a0 , which conflicts with any installable versions previously reported;
   ├─ mkl_fft 1.3.1 would require
   │  └─ mkl >=2022.0.0,<2023.0a0 , which conflicts with any installable versions previously reported;
   ├─ mkl_fft 1.3.1 would require
   │  └─ mkl >=2023.0.0,<2024.0a0 , which conflicts with any installable versions previously reported;
   ├─ mkl_fft 1.3.1 would require
   │  └─ python >=3.11,<3.12.0a0 , which can be installed;
   └─ mkl_fft 1.3.1 would require
      └─ mkl >=2021.3.0,<2022.0a0 , which conflicts with any installable versions previously reported.

Then, I attempted to install the MKL package without specifying the version:

mamba install -c intel mkl mkl_fft=1.3.1 mkl_random=1.2.2 mkl-service=2.4.0

This installation succeeded with the following versions:

mkl                       2023.2.0            intel_49495    intel
mkl-service               2.4.0           py39hae59892_35    intel
mkl_fft                   1.3.1           py39hcab1719_22    intel
mkl_random                1.2.2           py39hbf47bc3_22    intel

Therefore, I believe this dependency combination is more stable.
Best Regards,
TzuChing