Skip to content

Conversation

sharkdp
Copy link
Contributor

@sharkdp sharkdp commented Jun 27, 2025

Summary

Simplifies literal True and False conditions to ALWAYS_TRUE / ALWAYS_FALSE during semantic index building. This allows us to eagerly evaluate more constraints, which should help with performance (looks like there is a tiny 1% improvement in instrumented benchmarks), but also allows us to eliminate definitely-unreachable branches in control-flow merging. This can lead to better type inference in some cases because it allows us to retain narrowing constraints without solving astral-sh/ty#690 first:

def _(c: int | None):
    if c is None:
        assert False
    
    reveal_type(c)  # int, previously: int | None

closes astral-sh/ty#713

Test Plan

@sharkdp sharkdp added the ty Multi-file analysis & type inference label Jun 27, 2025
Copy link
Contributor

github-actions bot commented Jun 27, 2025

mypy_primer results

Changes were detected when running on open source projects
async-utils (https://github.com/mikeshardmind/async-utils)
- TOTAL MEMORY USAGE: ~49MB
+ TOTAL MEMORY USAGE: ~45MB

paasta (https://github.com/yelp/paasta)
-     memo fields = ~171MB
+     memo fields = ~156MB

alerta (https://github.com/alerta/alerta)
- TOTAL MEMORY USAGE: ~117MB
+ TOTAL MEMORY USAGE: ~106MB

apprise (https://github.com/caronc/apprise)
- warning[possibly-unbound-attribute] test/test_plugin_email.py:369:24: Attribute `parse_url` on type `Unknown | None` is possibly unbound
- warning[possibly-unbound-attribute] test/test_plugin_email.py:370:24: Attribute `parse_url` on type `Unknown | None` is possibly unbound
- warning[possibly-unbound-attribute] test/test_plugin_email.py:371:24: Attribute `parse_url` on type `Unknown | None` is possibly unbound
- Found 4326 diagnostics
+ Found 4323 diagnostics

aiohttp-devtools (https://github.com/aio-libs/aiohttp-devtools)
-     memo fields = ~88MB
+     memo fields = ~97MB

boostedblob (https://github.com/hauntsaninja/boostedblob)
- TOTAL MEMORY USAGE: ~106MB
+ TOTAL MEMORY USAGE: ~97MB

pytest (https://github.com/pytest-dev/pytest)
- warning[possibly-unbound-attribute] src/_pytest/python.py:1310:37: Attribute `stash` on type `Node | None | @Todo(map_with_boundness: intersections with negative contributions)` is possibly unbound
- Found 543 diagnostics
+ Found 542 diagnostics

discord.py (https://github.com/Rapptz/discord.py)
-     memo fields = ~189MB
+     memo fields = ~207MB

freqtrade (https://github.com/freqtrade/freqtrade)
-     memo fields = ~304MB
+     memo fields = ~276MB

django-stubs (https://github.com/typeddjango/django-stubs)
- warning[possibly-unbound-attribute] mypy_django_plugin/lib/helpers.py:153:15: Attribute `names` on type `Unknown | None` is possibly unbound
- warning[possibly-unbound-attribute] mypy_django_plugin/lib/helpers.py:161:11: Attribute `names` on type `Unknown | None` is possibly unbound
- Found 462 diagnostics
+ Found 460 diagnostics

scikit-learn (https://github.com/scikit-learn/scikit-learn)
- TOTAL MEMORY USAGE: ~717MB
+ TOTAL MEMORY USAGE: ~652MB

@sharkdp sharkdp force-pushed the david/eagerly-simplify-true-false branch from f57163a to 698f9a2 Compare June 30, 2025 10:05
@sharkdp sharkdp marked this pull request as ready for review June 30, 2025 10:24
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

Theoretically we could also simplify constraints such as not True not False, or not not True, etc. But it probably wouldn't be worth the added complexity -- I don't know why you'd write code like that?! I can find a few examples on grep.app, but it doesn't seem like a useful pattern...

@sharkdp
Copy link
Contributor Author

sharkdp commented Jun 30, 2025

Theoretically we could also simplify constraints such as not True not False, or not not True, etc. But it probably wouldn't be worth the added complexity -- I don't know why you'd write code like that?!

Agreed.

I also grepped for some common patterns of definitely-true/false conditions and the only other thing that seemed frequently used and feasible to unambiguously detect in semantic index building was the if TYPE_CHECKING and if not TYPE_CHECKING pattern. I seem to remember that we don't look up if it originates from typing anyway? So maybe that's worth doing?

Another thing that could be very valuable for performance would be to detect certain frequently used patterns that will definitely lead to an ambiguous truthiness result later. But I'm not sure what kind of patterns that might be...

@sharkdp sharkdp merged commit db3dcd8 into main Jun 30, 2025
36 checks passed
@sharkdp sharkdp deleted the david/eagerly-simplify-true-false branch June 30, 2025 11:11
@AlexWaygood
Copy link
Member

I seem to remember that we don't look up if it originates from typing anyway? So maybe that's worth doing?

Yeah, I think that could be worth doing!

sharkdp added a commit that referenced this pull request Jul 1, 2025
## Summary

Evaluate `TYPE_CHECKING` to `ALWAYS_TRUE` and `not TYPE_CHECKING` to
`ALWAYS_FALSE` during semantic index building. This is a follow-up to
#18998 and is in principle just a
performance optimization. We see some (favorable) ecosystem changes
because we can eliminate definitely-unreachable branches early now and
retain narrowing constraints without solving
astral-sh/ty#690 first.
iyakushev pushed a commit to iyakushev/ruff that referenced this pull request Jul 1, 2025
## Summary

Evaluate `TYPE_CHECKING` to `ALWAYS_TRUE` and `not TYPE_CHECKING` to
`ALWAYS_FALSE` during semantic index building. This is a follow-up to
astral-sh#18998 and is in principle just a
performance optimization. We see some (favorable) ecosystem changes
because we can eliminate definitely-unreachable branches early now and
retain narrowing constraints without solving
astral-sh/ty#690 first.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ty Multi-file analysis & type inference
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[possibly-unbound-attribute] False positive on result from while loop
2 participants