More verbose explanation why "Could not find a version that satisfies the requirement"

[stdedos]

Environment

(venv-test) ~/pySAML2/example # pip --version
pip 10.0.1 from /root/pySAML2/example/venv-test/lib/python3.6/site-packages/pip (python 3.6)
(venv-test) ~/pySAML2/example # python --version
Python 3.6.8
(venv-test) ~/pySAML2/example # cat /etc/alpine-release 
3.9.0
(venv-test) ~/pySAML2/example # 

Description

I have spent hours trying to debug, what does this mean:

(venv-test) ~/pySAML2/example # pip install -r requirements.txt  --index-url https://pypi.python.org/simple/
Looking in indexes: https://pypi.python.org/simple/
Looking in links: https://pypi.python.org/simple/
Collecting mako (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/f9/93/63f78c552e4397549499169198698de23b559b52e57f27d967690811d16d/Mako-1.0.10.tar.gz (460kB)
    100% |████████████████████████████████| 460kB 11.8MB/s 
Collecting cherrypy (from -r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/aa/0e/4e353c47789ccb50130a44e765dae55b3e85abca01ff21930533ab36afc9/CherryPy-18.1.1-py2.py3-none-any.whl
Collecting pysaml2<5.0.0,>=4.7.0 (from -r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/7b/20/df88d00a563fe437e9e03cf1b02927b47cea2fb5ed9d696c2820c53d7657/pysaml2-4.7.0-py2.py3-none-any.whl (335kB)
    100% |████████████████████████████████| 337kB 18.3MB/s 
Collecting MarkupSafe>=0.9.2 (from mako->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/b9/2e/64db92e53b86efccfaea71321f597fa2e1b2bd3853d8ce658568f7a13094/MarkupSafe-1.1.1.tar.gz
Collecting cheroot>=6.2.4 (from cherrypy->-r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/3e/50/840039a5350b54fb8efbc3b26c6e4244c9ca24c49ad84fe1f57b1f79ff7d/cheroot-6.5.5-py2.py3-none-any.whl
Collecting portend>=2.1.1 (from cherrypy->-r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/54/18/d288ef3cfcaac40c9c3674d92ef8313bf137deeced810d60d9722c0a327a/portend-2.4-py2.py3-none-any.whl
Collecting more-itertools (from cherrypy->-r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/b3/73/64fb5922b745fc1daee8a2880d907d2a70d9c7bb71eea86fcb9445daab5e/more_itertools-7.0.0-py3-none-any.whl
Collecting zc.lockfile (from cherrypy->-r requirements.txt (line 2))
Collecting pytz (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/3d/73/fe30c2daaaa0713420d0382b16fbb761409f532c56bdcc514bf7b6262bb6/pytz-2019.1-py2.py3-none-any.whl
Collecting defusedxml (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/06/74/9b387472866358ebc08732de3da6dc48e44b0aacd2ddaa5cb85ab7e986a2/defusedxml-0.6.0-py2.py3-none-any.whl
Collecting cryptography>=1.4 (from pysaml2<5.0.0,>=4.7.0->-r requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/07/ca/bc827c5e55918ad223d59d299fff92f3563476c3b00d0a9157d9c0217449/cryptography-2.6.1.tar.gz
  Could not find a version that satisfies the requirement cffi!=1.11.3,>=1.8 (from versions: )
No matching distribution found for cffi!=1.11.3,>=1.8
(venv-test) ~/pySAML2/example # 

While I was able to:

• Run this in a separate system (Ubuntu 16.04)
• With different python3 versions
• Find the required packages (on a manual https://pypi.org/simple/cffi/ search)
• Arch was compatible
• I was able to manually install https://files.pythonhosted.org/packages/93/1a/ab8c62b5838722f29f3daffcc8d4bd61844aa9b5f437341cc890ceee483b/cffi-1.12.3.tar.gz#sha256=041c81822e9f84b1d9c401182e174996f0bae9991f33725d059b771744290774, and yet, pip was pulling my leg: [1]

(venv-example) ~/pySAML2/example # pip install -r /root/pySAML2/example/requirements.txt --extra-index-url https://pypi.python.org/simple/
Looking in indexes: https://pypi.python.org/simple/
Looking in links: https://pypi.python.org/simple/
Collecting mako (from -r /root/pySAML2/example/requirements.txt (line 1))
  Downloading https://pypi.python.org/simple//mako/Mako-1.0.10.tar.gz (460kB)
    100% |████████████████████████████████| 460kB 143kB/s 
Collecting cherrypy (from -r /root/pySAML2/example/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/aa/0e/4e353c47789ccb50130a44e765dae55b3e85abca01ff21930533ab36afc9/CherryPy-18.1.1-py2.py3-none-any.whl
Collecting pysaml2<5.0.0,>=4.7.0 (from -r /root/pySAML2/example/requirements.txt (line 3))
  Downloading https://pypi.python.org/simple//pysaml2/pysaml2-4.7.0-py2.py3-none-any.whl (335kB)
    100% |████████████████████████████████| 337kB 166kB/s 
Collecting MarkupSafe>=0.9.2 (from mako->-r /root/pySAML2/example/requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/b9/2e/64db92e53b86efccfaea71321f597fa2e1b2bd3853d8ce658568f7a13094/MarkupSafe-1.1.1.tar.gz
Collecting cheroot>=6.2.4 (from cherrypy->-r /root/pySAML2/example/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/3e/50/840039a5350b54fb8efbc3b26c6e4244c9ca24c49ad84fe1f57b1f79ff7d/cheroot-6.5.5-py2.py3-none-any.whl
Collecting more-itertools (from cherrypy->-r /root/pySAML2/example/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/b3/73/64fb5922b745fc1daee8a2880d907d2a70d9c7bb71eea86fcb9445daab5e/more_itertools-7.0.0-py3-none-any.whl
Collecting portend>=2.1.1 (from cherrypy->-r /root/pySAML2/example/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/54/18/d288ef3cfcaac40c9c3674d92ef8313bf137deeced810d60d9722c0a327a/portend-2.4-py2.py3-none-any.whl
Collecting zc.lockfile (from cherrypy->-r /root/pySAML2/example/requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/58/c2/d7c89bdad237b4b7837609172be3e8bf5630796c0020494a15b97ece8eb1/zc.lockfile-1.4.tar.gz
Collecting pyOpenSSL (from pysaml2<5.0.0,>=4.7.0->-r /root/pySAML2/example/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/01/c8/ceb170d81bd3941cbeb9940fc6cc2ef2ca4288d0ca8929ea4db5905d904d/pyOpenSSL-19.0.0-py2.py3-none-any.whl
Collecting requests>=1.0.0 (from pysaml2<5.0.0,>=4.7.0->-r /root/pySAML2/example/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting six (from pysaml2<5.0.0,>=4.7.0->-r /root/pySAML2/example/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting cryptography>=1.4 (from pysaml2<5.0.0,>=4.7.0->-r /root/pySAML2/example/requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/07/ca/bc827c5e55918ad223d59d299fff92f3563476c3b00d0a9157d9c0217449/cryptography-2.6.1.tar.gz
  Could not find a version that satisfies the requirement cffi!=1.11.3,>=1.8 (from versions: )
No matching distribution found for cffi!=1.11.3,>=1.8
(venv-example) ~/pySAML2/example # pip list | grep cffi
cffi       1.12.3 

Something started stricking me, when I saw a whole lot of .tar.gz instead of -?py2.?py3-?????-????.whl files getting pulled

After searching the internet without avail, I found #3969 this, and then things started falling in place as to "why" my python so forcefully resisted installing https://files.pythonhosted.org/packages/5f/bf/6aa1925384c23ffeb579e97a5569eb9abce41b6310b329352b8252cee1c3/cffi-1.12.3-cp36-cp36m-manylinux1_x86_64.whl

Expected behavior

I would like to have a very precise indication as to "why" something "is not found".

pip clearly found the version and arch it wanted, it was just not happy with the manylinux1 "tag".

e.g. pip could've said:

  Could not find a version that satisfies the requirement cffi!=1.11.3,>=1.8 
Versions matching requirement "!=1.11.3,>=1.8": [x,y,z, ....] # Descending, truncating if output is too large? Some basic filtering first, THEN the version requirement?
Architectures matching (x86_64): x86_64
"Python Version" matching (p36): [cp36, cp36m]
Distributions matching (linux_x86_64, any): None

How to Reproduce

1. Get package from '...'
2. Then run '...'
3. An error occurs.

Output

# python3 -c "import wheel.pep425tags as w; print(w.get_supported())" | tr '\n' '\0' | sed -E 's/\),/),\n/g'
[('cp36', 'cp36m', 'linux_x86_64'),
 ('cp36', 'abi3', 'linux_x86_64'),
 ('cp36', 'none', 'linux_x86_64'),
 ('cp35', 'abi3', 'linux_x86_64'),
 ('cp34', 'abi3', 'linux_x86_64'),
 ('cp33', 'abi3', 'linux_x86_64'),
 ('cp32', 'abi3', 'linux_x86_64'),
 ('cp36', 'none', 'any'),
 ('cp3', 'none', 'any'),
 ('cp35', 'none', 'any'),
 ('cp34', 'none', 'any'),
 ('cp33', 'none', 'any'),
 ('cp32', 'none', 'any'),
 ('cp31', 'none', 'any'),
 ('cp30', 'none', 'any'),
 ('py3', 'none', 'linux_x86_64'),
 ('py36', 'none', 'any'),
 ('py3', 'none', 'any'),
 ('py35', 'none', 'any'),
 ('py34', 'none', 'any'),
 ('py33', 'none', 'any'),
 ('py32', 'none', 'any'),
 ('py31', 'none', 'any'),
 ('py30', 'none', 'any')]

[1]: I am considering if that would be a bug. What's pips deal, "not" accepting a self-compiled, pre-installed depenedency? Why would it "mandatorily" need to pull it as cffi-1.12.3-cp36-cp36m-manylinux1_x86_64.whl, and not be satisfied by "just" compiling it (or having "me" jump through hoops to make it appear in pip list)?

[stdedos]

I later found the -vvv output (#3642 (comment)). However, it is awfully verbose (as suggested by the -vvv) for tracking down "one" package refusing to install.

The "only two" reasons given were the:

it is not compatible with this Python
No sources permitted for cffi

[stdedos]

Related issues

• Error message when package isn't support by current python version leaves a lot to be desired #5915
• Pip doesn't work in pypy #177
• Redesigning pip's output (for install, wheel and download commands) #4649 (comment)

(?)

[uranusjr]

+1 to this, maybe we can take inspirations from the hash mismatch error. The most difficult parts are probably to draft out the appropriate messages to show in each scenario, and to correctly detect the scenarios (easier if the scenarios are lined out well in the previous part).

[cjerdonek]

However, it is awfully verbose (as suggested by the -vvv) for tracking down "one" package refusing to install.

I happened to write up a comment on another issue with an idea on addressing this: #6121 (comment)
Basically, the idea is to add a new pip tags command (in the sense of PEP 425 tags) that would let one do things like (1) view the effective tags given certain options (--python-version, --platform, etc), and (2) hone in and analyze individual filenames to see whether they would be accepted, and if not, why not.

[cjerdonek]

python3 -c "import wheel.pep425tags as w; print(w.get_supported())"

@stdedos pip uses a vendored version of pep425tags located at pip._internal.pep425tags. Do you get the same output for your above command if you use that?

[stdedos]

I would say I get (almost) the same output, yes (but I am not in front of a computer to verify at the moment)

[stdedos]

Yes, it is exactly the same (I have run that command already, but "a bit differently"):

import pprint; import pip._internal; pprint.pprint(pip._internal.pep425tags.get_supported())