Skip to content

Conversation

oconnor663
Copy link
Contributor

@oconnor663 oconnor663 commented Jul 15, 2025

We currently lint on examples like this (even though it's allowed at runtime, as long as you call f before g):

def f():
    global x
    x = 5
def g():
    print(x)  # error: [unresolved-reference]: Name `x` used when not defined

However, linting only on the print is kind of confusing. This PR adds a lint on the global statement itself (edit: updated after review feedback):

error[unresolved-global]: Invalid global declaration of `x`
 --> test.py:2:5
  |
1 | def f():
2 |     global x
  |     ^^^^^^^^ `x` has no declarations or bindings in the global scope
3 |     x = 42
  |
info: This limits ty's ability to make accurate inferences about the boundness and types of global-scope symbols
info: Consider adding a declaration to the global scope, e.g. `x: int`
info: rule `unresolved-global` is enabled by default

Because this is legal Python, I expect this is going to have a lot of new hits in the ecosystem-analyzer (and I had to fix some of our mdtests). I don't know what would count as an "excessive" number of new failures, though, and I'll need advice from reviewers :)

Copy link
Contributor

github-actions bot commented Jul 15, 2025

mypy_primer results

Changes were detected when running on open source projects
ignite (https://github.com/pytorch/ignite)
+ warning[unresolved-global] tests/ignite/handlers/test_base_logger.py:320:5: Invalid global declaration of `close_counter`: `close_counter` has no declarations or bindings in the global scope
- Found 2125 diagnostics
+ Found 2126 diagnostics

asynq (https://github.com/quora/asynq)
+ warning[unresolved-global] asynq/tests/test_contexts.py:105:5: Invalid global declaration of `expected_change_amount_base`: `expected_change_amount_base` has no declarations or bindings in the global scope
+ warning[unresolved-global] asynq/tests/test_contexts.py:111:9: Invalid global declaration of `expected_change_amount_base`: `expected_change_amount_base` has no declarations or bindings in the global scope
- Found 186 diagnostics
+ Found 188 diagnostics

pywin32 (https://github.com/mhammond/pywin32)
+ warning[unresolved-global] Pythonwin/pywin/Demos/ocx/ocxtest.py:52:5: Invalid global declaration of `calendarParentModule`: `calendarParentModule` has no declarations or bindings in the global scope
+ warning[unresolved-global] Pythonwin/pywin/Demos/ocx/ocxtest.py:119:5: Invalid global declaration of `videoControlModule`: `videoControlModule` has no declarations or bindings in the global scope
+ warning[unresolved-global] Pythonwin/pywin/Demos/ocx/ocxtest.py:119:5: Invalid global declaration of `videoControlFileName`: `videoControlFileName` has no declarations or bindings in the global scope
+ warning[unresolved-global] Pythonwin/pywin/test/test_pywin.py:57:9: Invalid global declaration of `teared_down`: `teared_down` has no declarations or bindings in the global scope
+ warning[unresolved-global] win32/Lib/win32serviceutil.py:600:5: Invalid global declaration of `g_debugService`: `g_debugService` has no declarations or bindings in the global scope
- Found 2008 diagnostics
+ Found 2013 diagnostics

mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ warning[unresolved-global] test/helper_tools/memoryleak.py:32:5: Invalid global declaration of `ssl`: `ssl` has no declarations or bindings in the global scope
- Found 1796 diagnostics
+ Found 1797 diagnostics

pycryptodome (https://github.com/Legrandin/pycryptodome)
+ warning[unresolved-global] lib/Crypto/SelfTest/Cipher/test_pkcs1_oaep.py:329:13: Invalid global declaration of `asked`: `asked` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/Cipher/test_pkcs1_oaep.py:332:17: Invalid global declaration of `asked`: `asked` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/Cipher/test_pkcs1_oaep.py:357:13: Invalid global declaration of `mgfcalls`: `mgfcalls` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/Cipher/test_pkcs1_oaep.py:361:17: Invalid global declaration of `mgfcalls`: `mgfcalls` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/PublicKey/test_DSA.py:71:9: Invalid global declaration of `DSA`: `DSA` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/PublicKey/test_DSA.py:71:9: Invalid global declaration of `Random`: `Random` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/PublicKey/test_DSA.py:71:9: Invalid global declaration of `bytes_to_long`: `bytes_to_long` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/PublicKey/test_DSA.py:71:9: Invalid global declaration of `size`: `size` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/PublicKey/test_RSA.py:90:9: Invalid global declaration of `RSA`: `RSA` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/PublicKey/test_RSA.py:90:9: Invalid global declaration of `Random`: `Random` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/PublicKey/test_RSA.py:90:9: Invalid global declaration of `bytes_to_long`: `bytes_to_long` has no declarations or bindings in the global scope
+ warning[unresolved-global] lib/Crypto/SelfTest/Util/test_Counter.py:33:9: Invalid global declaration of `Counter`: `Counter` has no declarations or bindings in the global scope
- Found 1476 diagnostics
+ Found 1488 diagnostics

openlibrary (https://github.com/internetarchive/openlibrary)
+ warning[unresolved-global] openlibrary/plugins/openlibrary/status.py:107:5: Invalid global declaration of `feature_flags`: `feature_flags` has no declarations or bindings in the global scope
- Found 679 diagnostics
+ Found 680 diagnostics

dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ warning[unresolved-global] ddtrace/vendor/ply/lex.py:868:5: Invalid global declaration of `lexer`: `lexer` has no declarations or bindings in the global scope
+ warning[unresolved-global] ddtrace/vendor/ply/lex.py:874:5: Invalid global declaration of `token`: `token` has no declarations or bindings in the global scope
+ warning[unresolved-global] ddtrace/vendor/ply/lex.py:874:5: Invalid global declaration of `input`: `input` has no declarations or bindings in the global scope
+ warning[unresolved-global] ddtrace/vendor/ply/yacc.py:3224:5: Invalid global declaration of `parse`: `parse` has no declarations or bindings in the global scope
+ warning[unresolved-global] tests/internal/test_forksafe.py:351:9: Invalid global declaration of `service`: `service` has no declarations or bindings in the global scope
- Found 6822 diagnostics
+ Found 6827 diagnostics

materialize (https://github.com/MaterializeInc/materialize)
+ warning[unresolved-global] misc/python/materialize/scalability/executor/benchmark_executor.py:203:9: Invalid global declaration of `next_worker_id`: `next_worker_id` has no declarations or bindings in the global scope
+ warning[unresolved-global] misc/python/materialize/scalability/executor/benchmark_executor.py:300:9: Invalid global declaration of `next_worker_id`: `next_worker_id` has no declarations or bindings in the global scope
- Found 3228 diagnostics
+ Found 3230 diagnostics

scikit-learn (https://github.com/scikit-learn/scikit-learn)
+ warning[unresolved-global] sklearn/linear_model/tests/test_ransac.py:223:5: Invalid global declaration of `cause_skip`: `cause_skip` has no declarations or bindings in the global scope
+ warning[unresolved-global] sklearn/linear_model/tests/test_ransac.py:227:9: Invalid global declaration of `cause_skip`: `cause_skip` has no declarations or bindings in the global scope
- Found 2059 diagnostics
+ Found 2061 diagnostics

scipy (https://github.com/scipy/scipy)
+ warning[unresolved-global] scipy/io/tests/test_mmio.py:29:5: Invalid global declaration of `mminfo`: `mminfo` has no declarations or bindings in the global scope
+ warning[unresolved-global] scipy/io/tests/test_mmio.py:30:5: Invalid global declaration of `mmread`: `mmread` has no declarations or bindings in the global scope
+ warning[unresolved-global] scipy/io/tests/test_mmio.py:31:5: Invalid global declaration of `mmwrite`: `mmwrite` has no declarations or bindings in the global scope
- Found 6649 diagnostics
+ Found 6652 diagnostics

sympy (https://github.com/sympy/sympy)
+ warning[unresolved-global] sympy/ntheory/partitions_.py:13:5: Invalid global declaration of `_factor`: `_factor` has no declarations or bindings in the global scope
+ warning[unresolved-global] sympy/ntheory/partitions_.py:13:5: Invalid global declaration of `_totient`: `_totient` has no declarations or bindings in the global scope
+ warning[unresolved-global] sympy/testing/runtests.py:2233:9: Invalid global declaration of `text`: `text` has no declarations or bindings in the global scope
+ warning[unresolved-global] sympy/testing/runtests.py:2233:9: Invalid global declaration of `linelen`: `linelen` has no declarations or bindings in the global scope
+ warning[unresolved-global] sympy/testing/runtests.py:2238:13: Invalid global declaration of `text`: `text` has no declarations or bindings in the global scope
+ warning[unresolved-global] sympy/testing/runtests.py:2238:13: Invalid global declaration of `linelen`: `linelen` has no declarations or bindings in the global scope
- Found 13502 diagnostics
+ Found 13508 diagnostics
No memory usage changes detected ✅

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This looks pretty good to me. I haven't gone through the mypy_primer hits, but it doesn't look prohibitively large to me.

The main thing I think we need to do before landing this is to improve the diagnostic to make it clearer what the problem is, and why we object to this coding pattern

Copy link
Contributor

ecosystem-analyzer results

Lint rule Added Removed Changed
unresolved-reference 39 0 0
Total 39 0 0

Full report with detailed diff

@oconnor663 oconnor663 requested a review from MichaReiser as a code owner July 15, 2025 16:49
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! (Haven't reviewed the mypy_primer hits other than the quantity -- I'm trusting you to check that they're all as expected!)

@oconnor663
Copy link
Contributor Author

Most of the new lints in the ecosystem are as expected, global statements that are creating a new global variable. But there's another case that shows up a couple times, like this:

class CounterTests(unittest.TestCase):
    def setUp(self):
        global Counter
        from Crypto.Util import Counter

So that's doing a global import, but not in the global scope. If the user is sure they want to do this sort of thing, is there an alternative way for them to do it that will type-check? (Maybe something better than Counter: Any, since that might be bad for completions? Not sure.)

@oconnor663 oconnor663 force-pushed the jack/global_must_refer_to_definition branch from 7ae687d to d6d4914 Compare July 15, 2025 18:56
@oconnor663
Copy link
Contributor Author

Ok, folks who want to do global ... import ... tricks can at least put an Any declaration in the global scope and not be worse off than before. That was the only surprising case (and any other surprises we run into down the road can probably use the same workaround?), so I'm going to go ahead and land this.

@oconnor663 oconnor663 merged commit e73a8ba into main Jul 15, 2025
37 checks passed
@oconnor663 oconnor663 deleted the jack/global_must_refer_to_definition branch July 15, 2025 23:56
oconnor663 added a commit that referenced this pull request Jul 16, 2025
…al` lints

This is a follow-up to #19344 that
improves the error formatting slightly. For example with this program:

```py
def f():
    global foo, bar
```

Before we printed:

```
1 | def f():
2 |     global foo, bar
  |     ^^^^^^^^^^^^^^^ `foo` has no declarations or bindings in the global scope
...
1 | def f():
2 |     global foo, bar
  |     ^^^^^^^^^^^^^^^ `bar` has no declarations or bindings in the global scope
```

Now we print:

```
1 | def f():
2 |     global foo, bar
  |            ^^^ `foo` has no declarations or bindings in the global scope
...
1 | def f():
2 |     global foo, bar
  |                 ^^^ `bar` has no declarations or bindings in the global scope
```
oconnor663 added a commit that referenced this pull request Jul 16, 2025
…`global` lints

This is a follow-up to #19344 that
improves the error formatting slightly. For example with this program:

```py
def f():
    global foo, bar
```

Before we printed:

```
1 | def f():
2 |     global foo, bar
  |     ^^^^^^^^^^^^^^^ `foo` has no declarations or bindings in the global scope
...
1 | def f():
2 |     global foo, bar
  |     ^^^^^^^^^^^^^^^ `bar` has no declarations or bindings in the global scope
```

Now we print:

```
1 | def f():
2 |     global foo, bar
  |            ^^^ `foo` has no declarations or bindings in the global scope
...
1 | def f():
2 |     global foo, bar
  |                 ^^^ `bar` has no declarations or bindings in the global scope
```
oconnor663 added a commit that referenced this pull request Jul 16, 2025
…`global` lints (#19379)

This is a follow-up to #19344 that
improves the error formatting slightly. For example with this program:

```py
def f():
    global foo, bar
```

Before we printed:

```
1 | def f():
2 |     global foo, bar
  |     ^^^^^^^^^^^^^^^ `foo` has no declarations or bindings in the global scope
...
1 | def f():
2 |     global foo, bar
  |     ^^^^^^^^^^^^^^^ `bar` has no declarations or bindings in the global scope
```

Now we print:

```
1 | def f():
2 |     global foo, bar
  |            ^^^ `foo` has no declarations or bindings in the global scope
...
1 | def f():
2 |     global foo, bar
  |                 ^^^ `bar` has no declarations or bindings in the global scope
```

---------

Co-authored-by: Micha Reiser <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ecosystem-analyzer ty Multi-file analysis & type inference
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants