[ty] fix deferred name loading in PEP695 generic classes/functions #19888
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-08-13 22:34:15.963392901 +0000
+++ new-output.txt 2025-08-13 22:34:16.030393914 +0000
@@ -1,5 +1,5 @@
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
-fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/918d35d/src/function/execute.rs:215:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(104b4)): execute: too many cycle iterations`
+fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/918d35d/src/function/execute.rs:215:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(bcb6)): execute: too many cycle iterations`
_directives_deprecated_library.py:15:31: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
_directives_deprecated_library.py:30:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
_directives_deprecated_library.py:36:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__add__`
@@ -68,9 +68,9 @@
aliases_type_statement.py:48:23: error[invalid-type-form] Boolean operations are not allowed in type expressions
aliases_type_statement.py:49:23: error[fstring-type-annotation] Type expressions cannot use f-strings
aliases_type_statement.py:80:37: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[int, str]`?
-aliases_variance.py:18:24: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
-aliases_variance.py:28:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
-aliases_variance.py:44:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassB[typing.TypeVar("T_co"), typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+aliases_variance.py:18:24: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[typing.TypeVar]'>` with no `__class_getitem__` method
+aliases_variance.py:28:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[typing.TypeVar]'>` with no `__class_getitem__` method
+aliases_variance.py:44:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassB[typing.TypeVar, typing.TypeVar]'>` with no `__class_getitem__` method
annotations_forward_refs.py:22:7: error[unresolved-reference] Name `ClassA` used when not defined
annotations_forward_refs.py:23:12: error[unresolved-reference] Name `ClassA` used when not defined
annotations_forward_refs.py:49:10: error[invalid-type-form] Variable of type `Literal[1]` is not allowed in a type expression
@@ -384,7 +384,7 @@
generics_defaults_referential.py:95:1: error[type-assertion-failure] Argument does not have asserted type `@Todo`
generics_defaults_specialization.py:26:5: error[type-assertion-failure] Argument does not have asserted type `SomethingWithNoDefaults[int, str]`
generics_defaults_specialization.py:27:5: error[type-assertion-failure] Argument does not have asserted type `SomethingWithNoDefaults[int, bool]`
-generics_defaults_specialization.py:30:1: error[non-subscriptable] Cannot subscript object of type `<class 'SomethingWithNoDefaults[int, typing.TypeVar("DefaultStrT", default=str)]'>` with no `__class_getitem__` method
+generics_defaults_specialization.py:30:1: error[non-subscriptable] Cannot subscript object of type `<class 'SomethingWithNoDefaults[int, typing.TypeVar]'>` with no `__class_getitem__` method
generics_defaults_specialization.py:45:1: error[type-assertion-failure] Argument does not have asserted type `@Todo`
generics_paramspec_basic.py:27:38: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
generics_paramspec_components.py:83:18: error[parameter-already-assigned] Multiple values provided for parameter 1 (`x`) of function `foo`
@@ -492,7 +492,6 @@
generics_syntax_infer_variance.py:144:1: error[invalid-assignment] Object of type `ShouldBeInvariant5[int]` is not assignable to `ShouldBeInvariant5[int | float]`
generics_syntax_infer_variance.py:155:1: error[invalid-assignment] Object of type `ShouldBeContravariant1[int]` is not assignable to `ShouldBeContravariant1[int | float]`
generics_syntax_infer_variance.py:156:1: error[invalid-assignment] Object of type `ShouldBeContravariant1[int | float]` is not assignable to `ShouldBeContravariant1[int]`
-generics_syntax_scoping.py:18:26: error[unresolved-reference] Name `T` used when not defined
generics_syntax_scoping.py:35:7: error[unresolved-reference] Name `T` used when not defined
generics_syntax_scoping.py:40:23: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `((...) -> R@decorator1, /) -> (...) -> R@decorator1`
generics_syntax_scoping.py:44:17: error[unresolved-reference] Name `T` used when not defined
@@ -564,20 +563,20 @@
generics_variance.py:14:6: error[invalid-legacy-type-variable] A legacy `typing.TypeVar` cannot be both covariant and contravariant
generics_variance.py:26:27: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Iterator[T_co@ImmutableList]`
generics_variance.py:57:28: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `B_co@func`
-generics_variance.py:175:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:175:35: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
-generics_variance.py:179:29: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:179:39: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:183:21: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
-generics_variance.py:183:27: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
-generics_variance.py:187:25: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
-generics_variance.py:187:31: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:191:33: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:191:43: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
-generics_variance.py:191:49: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:196:5: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:196:15: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
-generics_variance.py:196:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:175:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:175:35: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:179:29: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:179:39: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:183:21: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:183:27: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:187:25: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:187:31: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:191:33: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:191:43: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:191:49: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:196:5: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:196:15: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
+generics_variance.py:196:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar]'>` with no `__class_getitem__` method
generics_variance_inference.py:19:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `T3@ClassA`
generics_variance_inference.py:24:5: error[invalid-assignment] Object of type `ClassA[int | float, int, int]` is not assignable to `ClassA[int, int, int]`
generics_variance_inference.py:25:5: error[invalid-assignment] Object of type `ClassA[int | float, int, int]` is not assignable to `ClassA[int | float, int | float, int]`
@@ -860,5 +859,5 @@
typeddicts_operations.py:60:1: error[type-assertion-failure] Argument does not have asserted type `str | None`
typeddicts_type_consistency.py:101:1: error[invalid-assignment] Object of type `Unknown | None` is not assignable to `str`
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions. Did you mean to use a concrete TypedDict or `collections.abc.Mapping[str, object]` instead?
-Found 861 diagnostics
+Found 860 diagnostics
WARN A fatal error occurred while checking some files. Not all project files were analyzed. See the diagnostics list above for details. |
|
carljm
requested review from
AlexWaygood,
sharkdp,
dcreager and
MichaReiser
as code owners
* main: [ty] Add some additional type safety to `CycleDetector` (#19903)
dcreager
approved these changes
Aug 13, 2025
| info: Attempted to call union type `(def f1() -> int) | (def f2(name: str) -> int) | (def f3(a: int, b: int) -> int) | (def f4[T: str](x: T@f4) -> int) | Literal[5] | (Overload[() -> None, (x: str) -> str]) | (Overload[() -> None, (x: str, y: str) -> str]) | PossiblyNotCallable` | ||
| info: Attempted to call union type `(def f1() -> int) | (def f2(name: str) -> int) | (def f3(a: int, b: int) -> int) | (def f4[T](x: T@f4) -> int) | Literal[5] | (Overload[() -> None, (x: str) -> str]) | (Overload[() -> None, (x: str, y: str) -> str]) | PossiblyNotCallable` |
There was a problem hiding this comment.
This is the place where I'm most leery of not including bounds/constraints/defaults in the rendering of a typevar. Though pyright also does not include them here, so your argument still holds!
There was a problem hiding this comment.
Yes, agreed. Pyrefly and pyright both do not include typevar details here, mypy does.
We can add them back in future. I'm not opposed to that at all, it's just gonna require some extra cycle-handling machinery for type display that I don't think is top priority right now.
AlexWaygood
approved these changes
Aug 13, 2025
dcreager
added a commit
that referenced
this pull request
Aug 14, 2025
* main: Feature/build riscv64 bin (#19819) [ty] Add caching to `CodeGeneratorKind::matches()` (#19912) [ty] Rename `functionArgumentNames` to `callArgumentNames` inlay hint setting (#19911) [ty] Default `ty.inlayHints.*` server settings to true (#19910) [ty] Remove py-fuzzer skips for seeds that are no longer slow (#19906) [ty] fix deferred name loading in PEP695 generic classes/functions (#19888) [ty] Add some additional type safety to `CycleDetector` (#19903) [`flake8-blind-except`] Fix `BLE001` false-positive on `raise ... from None` (#19755) [ty] resolve docstrings for modules (#19898)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
For PEP 695 generic functions and classes, there is an extra "type params scope" (a child of the outer scope, and wrapping the body scope) in which the type parameters are defined; class bases and function parameter/return annotations are resolved in that type-params scope.
This PR fixes some longstanding bugs in how we resolve name loads from inside these PEP 695 type parameter scopes, and also defers type inference of PEP 695 typevar bounds/constraints/default, so we can handle cycles without panicking.
We were previously treating these type-param scopes as lazy nested scopes, which is wrong. In fact they are eager nested scopes; the class
Chere inheritsint, notstr, and previously we got that wrong:But certain syntactic positions within type param scopes (typevar bounds/constraints/defaults) are lazy at runtime, and we should use deferred name resolution for them. This also means they can have cycles; in order to handle that without panicking in type inference, we need to actually defer their type inference until after we have constructed the
TypeVarInstance.PEP 695 does specify that typevar bounds and constraints cannot be generic, and that typevar defaults can only reference prior typevars, not later ones. This reduces the scope of (valid from the type-system perspective) cycles somewhat, although cycles are still possible (e.g.
class C[T: list[C]]). And this is a type-system-only restriction; from the runtime perspective an "invalid" case likeclass C[T: T]actually works fine.I debated whether to implement the PEP 695 restrictions as a way to avoid some cycles up-front, but I ended up deciding against that; I'd rather model the runtime name-resolution semantics accurately, and implement the PEP 695 restrictions as a separate diagnostic on top. (This PR doesn't yet implement those diagnostics, thus some
# TODO: errorin the added tests.)Introducing the possibility of cyclic typevars made typevar display potentially stack overflow. For now I've handled this by simply removing typevar details (bounds/constraints/default) from typevar display. This impacts display of two kinds of types. If you
reveal_type(T)on an unboundTyou now get justtyping.TypeVarinstead oftyping.TypeVar("T", ...)where...is the bound/constraints/default. This matches pyright and mypy; pyrefly usestype[TypeVar[T]]which seems a bit confusing, but does include the name. (We could easily include the name without cycle issues, if there's a syntax we like for that.)It also means that displaying a generic function type like
def f[T: int](x: T) -> T: ...now displays asf[T](x: T) -> Tinstead off[T: int](x: T) -> T. This matches pyright and pyrefly; mypy does include bound/constraints/defaults of typevars in function/callable type display. If we wanted to add this, we would either need to thread a visitor through all the type display code, or add adecycletype transformation that replaced recursive reoccurrence of a type with a marker.Test Plan
Added mdtests and modified existing tests to improve their correctness.
After this PR, there's only a single remaining py-fuzzer seed in the 0-500 range that panics! (Before this PR, there were 10; the fuzzer likes to generate cyclic PEP 695 syntax.)
Ecosystem report
It's all just the changes to
TypeVardisplay.