[ty] Use separate Rust types for bound and unbound type variables #19796
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
primarily for legacy typevar defaults
dcreager
requested review from
carljm,
AlexWaygood,
sharkdp and
MichaReiser
as code owners
|
@carljm and I concurrently realized that we didn't need a new from typing import Generic, TypeVar
T = TypeVar("T")
U = TypeVar("U", default=T)The use of
This was a good suggestion, which I've implemented. |
|
|
Doh! The size of |
* origin/main: [ty] Implemented support for "rename" language server feature (#19551) [ty] Reduce size of member table (#19572) [ty] Move server capabilities creation (#19798) [ty] Repurpose `FunctionType.into_bound_method_type` to return `BoundMethodType` (#19793) [ty] Validate writes to `TypedDict` keys (#19782) [ty] Add support for using the test command emitted when a mdtest fails (#19794)
I made |
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-08-11 17:44:43.096472962 +0000
+++ new-output.txt 2025-08-11 17:44:43.163473111 +0000
@@ -84,9 +84,9 @@
aliases_typealiastype.py:63:42: error[invalid-type-form] Boolean operations are not allowed in type expressions
aliases_typealiastype.py:64:42: error[invalid-type-form] F-strings are not allowed in type expressions
aliases_typealiastype.py:66:47: error[unresolved-reference] Name `BadAlias21` used when not defined
-aliases_variance.py:18:24: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[T_co]'>` with no `__class_getitem__` method
-aliases_variance.py:28:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[T_co]'>` with no `__class_getitem__` method
-aliases_variance.py:44:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassB[T_co, T_contra]'>` with no `__class_getitem__` method
+aliases_variance.py:18:24: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
+aliases_variance.py:28:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassA[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
+aliases_variance.py:44:16: error[non-subscriptable] Cannot subscript object of type `<class 'ClassB[typing.TypeVar("T_co"), typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
annotations_forward_refs.py:22:7: error[unresolved-reference] Name `ClassA` used when not defined
annotations_forward_refs.py:23:12: error[unresolved-reference] Name `ClassA` used when not defined
annotations_forward_refs.py:49:10: error[invalid-type-form] Variable of type `Literal[1]` is not allowed in a type expression
@@ -400,7 +400,7 @@
generics_defaults_referential.py:95:1: error[type-assertion-failure] Argument does not have asserted type `@Todo`
generics_defaults_specialization.py:26:5: error[type-assertion-failure] Argument does not have asserted type `SomethingWithNoDefaults[int, str]`
generics_defaults_specialization.py:27:5: error[type-assertion-failure] Argument does not have asserted type `SomethingWithNoDefaults[int, bool]`
-generics_defaults_specialization.py:30:1: error[non-subscriptable] Cannot subscript object of type `<class 'SomethingWithNoDefaults[int, DefaultStrT]'>` with no `__class_getitem__` method
+generics_defaults_specialization.py:30:1: error[non-subscriptable] Cannot subscript object of type `<class 'SomethingWithNoDefaults[int, typing.TypeVar("DefaultStrT", default=str)]'>` with no `__class_getitem__` method
generics_defaults_specialization.py:45:1: error[type-assertion-failure] Argument does not have asserted type `@Todo`
generics_paramspec_basic.py:27:38: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
generics_paramspec_components.py:83:18: error[parameter-already-assigned] Multiple values provided for parameter 1 (`x`) of function `foo`
@@ -436,16 +436,16 @@
generics_self_advanced.py:18:1: error[type-assertion-failure] Argument does not have asserted type `ParentA`
generics_self_advanced.py:19:1: error[type-assertion-failure] Argument does not have asserted type `ChildA`
generics_self_advanced.py:28:25: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@method1`
-generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self`
-generics_self_advanced.py:36:9: error[type-assertion-failure] Argument does not have asserted type `list[Self]`
-generics_self_advanced.py:37:9: error[type-assertion-failure] Argument does not have asserted type `Self`
-generics_self_advanced.py:38:9: error[type-assertion-failure] Argument does not have asserted type `Self`
-generics_self_advanced.py:43:9: error[type-assertion-failure] Argument does not have asserted type `list[Self]`
-generics_self_advanced.py:44:9: error[type-assertion-failure] Argument does not have asserted type `Self`
-generics_self_advanced.py:45:9: error[type-assertion-failure] Argument does not have asserted type `Self`
-generics_self_attributes.py:26:33: error[invalid-argument-type] Argument is incorrect: Expected `Self | None`, found `LinkedList[int]`
-generics_self_attributes.py:29:5: error[invalid-assignment] Object of type `OrdinalLinkedList` is not assignable to attribute `next` of type `Self | None`
-generics_self_attributes.py:32:5: error[invalid-assignment] Object of type `LinkedList[int]` is not assignable to attribute `next` of type `Self | None`
+generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `typing.Self`
+generics_self_advanced.py:36:9: error[type-assertion-failure] Argument does not have asserted type `list[typing.Self]`
+generics_self_advanced.py:37:9: error[type-assertion-failure] Argument does not have asserted type `typing.Self`
+generics_self_advanced.py:38:9: error[type-assertion-failure] Argument does not have asserted type `typing.Self`
+generics_self_advanced.py:43:9: error[type-assertion-failure] Argument does not have asserted type `list[typing.Self]`
+generics_self_advanced.py:44:9: error[type-assertion-failure] Argument does not have asserted type `typing.Self`
+generics_self_advanced.py:45:9: error[type-assertion-failure] Argument does not have asserted type `typing.Self`
+generics_self_attributes.py:26:33: error[invalid-argument-type] Argument is incorrect: Expected `typing.Self | None`, found `LinkedList[int]`
+generics_self_attributes.py:29:5: error[invalid-assignment] Object of type `OrdinalLinkedList` is not assignable to attribute `next` of type `typing.Self | None`
+generics_self_attributes.py:32:5: error[invalid-assignment] Object of type `LinkedList[int]` is not assignable to attribute `next` of type `typing.Self | None`
generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@set_scale`
generics_self_basic.py:20:16: error[invalid-return-type] Return type does not match returned value: expected `Self@method2`, found `Shape`
generics_self_basic.py:33:16: error[invalid-return-type] Return type does not match returned value: expected `Self@cls_method2`, found `Shape`
@@ -473,7 +473,7 @@
generics_self_usage.py:121:37: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__new__`
generics_self_usage.py:125:37: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `list[Self@__mul__]`
generics_syntax_compatibility.py:23:38: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `V@ClassC | K@method1`
-generics_syntax_compatibility.py:26:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `M@method2 | K`
+generics_syntax_compatibility.py:26:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `M@method2 | K@method2`
generics_syntax_declarations.py:17:1: error[invalid-generic-class] Cannot both inherit from `typing.Generic` and use PEP 695 type variables
generics_syntax_declarations.py:25:20: error[invalid-generic-class] Cannot both inherit from subscripted `Protocol` and use PEP 695 type variables
generics_syntax_declarations.py:32:9: error[unresolved-attribute] Type `T@ClassD` has no attribute `is_integer`
@@ -580,20 +580,20 @@
generics_variance.py:14:6: error[invalid-legacy-type-variable] A legacy `typing.TypeVar` cannot be both covariant and contravariant
generics_variance.py:26:27: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Iterator[T_co@ImmutableList]`
generics_variance.py:57:28: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `B_co@func`
-generics_variance.py:175:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:175:35: error[non-subscriptable] Cannot subscript object of type `<class 'Co[T_co]'>` with no `__class_getitem__` method
-generics_variance.py:179:29: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:179:39: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:183:21: error[non-subscriptable] Cannot subscript object of type `<class 'Co[T_co]'>` with no `__class_getitem__` method
-generics_variance.py:183:27: error[non-subscriptable] Cannot subscript object of type `<class 'Co[T_co]'>` with no `__class_getitem__` method
-generics_variance.py:187:25: error[non-subscriptable] Cannot subscript object of type `<class 'Co[T_co]'>` with no `__class_getitem__` method
-generics_variance.py:187:31: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:191:33: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:191:43: error[non-subscriptable] Cannot subscript object of type `<class 'Co[T_co]'>` with no `__class_getitem__` method
-generics_variance.py:191:49: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:196:5: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:196:15: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
-generics_variance.py:196:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[T_contra]'>` with no `__class_getitem__` method
+generics_variance.py:175:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:175:35: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
+generics_variance.py:179:29: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:179:39: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:183:21: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
+generics_variance.py:183:27: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
+generics_variance.py:187:25: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
+generics_variance.py:187:31: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:191:33: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:191:43: error[non-subscriptable] Cannot subscript object of type `<class 'Co[typing.TypeVar("T_co")]'>` with no `__class_getitem__` method
+generics_variance.py:191:49: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:196:5: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:196:15: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
+generics_variance.py:196:25: error[non-subscriptable] Cannot subscript object of type `<class 'Contra[typing.TypeVar("T_contra")]'>` with no `__class_getitem__` method
generics_variance_inference.py:19:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `T3@ClassA`
generics_variance_inference.py:24:5: error[invalid-assignment] Object of type `ClassA[int | float, int, int]` is not assignable to `ClassA[int, int, int]`
generics_variance_inference.py:25:5: error[invalid-assignment] Object of type `ClassA[int | float, int, int]` is not assignable to `ClassA[int | float, int | float, int]` |
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
unresolved-attribute |
0 | 2,856 | 0 |
invalid-argument-type |
37 | 746 | 21 |
invalid-return-type |
0 | 166 | 1 |
possibly-unbound-attribute |
5 | 43 | 0 |
non-subscriptable |
0 | 0 | 20 |
invalid-assignment |
0 | 14 | 0 |
no-matching-overload |
3 | 0 | 0 |
unused-ignore-comment |
2 | 0 | 0 |
call-non-callable |
1 | 0 | 0 |
not-iterable |
0 | 1 | 0 |
unsupported-operator |
0 | 1 | 0 |
| Total | 48 | 3,827 | 42 |
|
The with some extra debug statements, the two typevars actually have different binding contexts: because the Going to dig into this some more |
The reason for this was interesting! Whenever we use a typevar in a type context, we need to determine if the typevar is bound at that point in the source. We walk through the enclosing scopes, to see if any of them are a generic class/function that binds the typevar in question. That requires knowing the generic context of said classes/functions. For a generic function, we find the infered type for the definition (which should be a This works because we create a EXCEPT! We don't include the implementation when producing the signature of a function! This is almost always what you want. But not here! For a typevar reference inside the implementation body of an overloaded function, we would grab the signature of last non-implementation overload, and use that as the binding context. The fix is to add an additional method on |
|
The remaining new ecosystem diagnostics look to all be from either:
|
carljm
approved these changes
Aug 8, 2025
crates/ty_ide/src/hover.rs
Outdated
Comment on lines
374
to
377
| typing.TypeVar[T: int = bool] | ||
| --------------------------------------------- | ||
| ```python | ||
| T@Alias | ||
| typing.TypeVar[T: int = bool] |
There was a problem hiding this comment.
Does this display change indicate that we now consider T unbound here? Is that right? The type parameter to list is a type expression, so I would expect this use of T to be inferred as Type::TypeVar, and T should be bound to the type alias context.
There was a problem hiding this comment.
It always was unbound, we just couldn't see it before! You're right that it should be bound, but we're not creating GenericContexts for type aliases yet [playground]:
type Alias[T] = list[T]
reveal_type(Alias[int]) # revealed: @TodoOnce type aliases have a generic context, we can update enclosing_generic_scopes to return those, and then this will become bound as it should. I'll add a TODO comment here.
| @@ -102,7 +102,7 @@ def silence[T: type[BaseException]]( | |||
| try: | |||
| func() | |||
| except exception_type as e: | |||
| reveal_type(e) # revealed: T'instance | |||
| reveal_type(e) # revealed: T'instance@silence | |||
There was a problem hiding this comment.
Clearly it pre-exists this PR, but where does the 'instance in this typevar's name come from?
There was a problem hiding this comment.
T is bound to an exception type, but the actual caught expression will be an instance of that type. So it's not correct to reveal T@silence here, since that would be the type of type(e). Type::to_instance will synthesize a new typevar with this modified name. Though a better representation might be letting NominalInstance wrap a typevar?
There was a problem hiding this comment.
Having NominalInstance wrap a typevar seems like an awkward representation, since it would only be valid for a typevar with upper bound of type[...]. I feel like what we're doing here makes sense, just hard to come up with a good name for it. Not something to worry about in this PR.
As a side note, this code example seems quite difficult for type checkers:
from typing import Callable
def silence[T: type[BaseException]](
func: Callable[[], None],
exception_type: T,
) -> T | None:
try:
func()
except exception_type as e:
reveal_type(e)
return e # should be a type error, only returning `type(e)` would be validPyright reveals BaseException*, which is kind of reasonable but what does the * mean? (I think it means "best guess type", which is treated as dynamic.) And then it allows return e, which is clearly a false negative.
Mypy doesn't even allow the except exception_type as e, and then reveals Any.
Pyrefly just panics.
The next interesting bit is to try replacing return e with return type(e), which in principle should be valid. We don't allow that, because we don't track enough metadata to round-trip the T'instance typevar back to T in to_meta_type. (I guess your suggestion of wrapping T rather than synthesizing a new typevar might allow us to fix that.)
Anyway, this is all just musing because it's interesting, not relevant to this PR!
crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md
Outdated
Show resolved
Hide resolved
crates/ty_python_semantic/resources/mdtest/generics/legacy/variables.md
Outdated
Show resolved
Hide resolved
crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md
Outdated
Show resolved
Hide resolved
* main: (31 commits) Add AIR301 rule (#17707) Avoid underflow in default ranges before a BOM (#19839) Update actions/download-artifact digest to de96f46 (#19852) Update docker/login-action action to v3.5.0 (#19860) Update rui314/setup-mold digest to 7344740 (#19853) Update cargo-bins/cargo-binstall action to v1.14.4 (#19855) Update actions/cache action to v4.2.4 (#19854) Update Rust crate hashbrown to v0.15.5 (#19858) Update Rust crate camino to v1.1.11 (#19857) Update Rust crate proc-macro2 to v1.0.96 (#19859) Update dependency ruff to v0.12.8 (#19856) SIM905: Fix handling of U+001C..U+001F whitespace (#19849) RUF064: offer a safe fix for multi-digit zeros (#19847) Clean up unused rendering code in `ruff_linter` (#19832) [ty] Add Salsa caching to `TupleType::to_class_type` (#19840) [ty] Handle cycles when finding implicit attributes (#19833) [ty] fix goto-definition on imports (#19834) [ty] Implement stdlib stub mapping (#19529) [`flake8-comprehensions`] Fix false positive for `C420` with attribute, subscript, or slice assignment targets (#19513) [ty] Implement module-level `__getattr__` support (#19791) ...
Co-authored-by: Carl Meyer <[email protected]>
…eager/bound-typevar * origin/dcreager/bound-typevar: Apply suggestions from code review
dcreager
added a commit
that referenced
this pull request
Aug 11, 2025
* origin/main: [ty] Use separate Rust types for bound and unbound type variables (#19796)
dcreager
added a commit
that referenced
this pull request
Aug 11, 2025
* dcreager/bound-typevar: (41 commits) [ty] Use separate Rust types for bound and unbound type variables (#19796) fix ide tests better unbound typevar rendering Apply suggestions from code review [ty] Add `static-frame` as a walltime benchmark (#19844) add explanatory comment [ty] Update goto range for attribute access to only target the attribute (#19848) remove unneeded ord add TODO for broken hover test better PEP 695 binding context Add AIR301 rule (#17707) Avoid underflow in default ranges before a BOM (#19839) Update actions/download-artifact digest to de96f46 (#19852) Update docker/login-action action to v3.5.0 (#19860) Update rui314/setup-mold digest to 7344740 (#19853) Update cargo-bins/cargo-binstall action to v1.14.4 (#19855) Update actions/cache action to v4.2.4 (#19854) Update Rust crate hashbrown to v0.15.5 (#19858) Update Rust crate camino to v1.1.11 (#19857) Update Rust crate proc-macro2 to v1.0.96 (#19859) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR creates separate Rust types for bound and unbound type variables, as proposed in astral-sh/ty#926.
Closes astral-sh/ty#926