[ty] typing.Self is bound by the method, not the class
#19784
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dcreager
commented
Aug 6, 2025
Comment on lines
+109
to
+110
| TODO: The use of `Self` to annotate the `next_node` attribute should be | ||
| [modeled as a property][self attribute], using `Self` in its parameter and return type. |
There was a problem hiding this comment.
We already have good support for properties, so I would suggest opening up a help wanted issue to tackle this as a follow-on
There was a problem hiding this comment.
That sounds fine (feel free to create the issue), but will that be enough to solve this case? Wouldn't we consider these to be two separate "bindings" of Self (the one on the return type of the next_node property and the one on the return type of next method), and not allow the one to be assignable to the other, even if they share the same upper bound?
There was a problem hiding this comment.
Accessing the attribute would count as a call to the property getter, so my thinking was that we would infer a specialization of {Self@next_node = Self@next} which would make them assignable.
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-08-06 19:18:54.969213042 +0000
+++ new-output.txt 2025-08-06 19:18:55.034213160 +0000
@@ -1,7 +1,7 @@
WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
_directives_deprecated_library.py:15:31: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
_directives_deprecated_library.py:30:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
-_directives_deprecated_library.py:36:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@Spam`
+_directives_deprecated_library.py:36:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__add__`
_directives_deprecated_library.py:41:25: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int | float`
_directives_deprecated_library.py:45:24: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
aliases_explicit.py:41:24: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[str, str]`?
@@ -188,7 +188,7 @@
constructors_call_new.py:113:42: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Class8[list[T@Class8]]`
constructors_call_new.py:116:1: error[type-assertion-failure] Argument does not have asserted type `Class8[list[int]]`
constructors_call_new.py:117:1: error[type-assertion-failure] Argument does not have asserted type `Class8[list[str]]`
-constructors_call_new.py:125:42: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@Class9`
+constructors_call_new.py:125:42: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__new__`
constructors_call_new.py:140:47: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Class11[int]`
constructors_call_type.py:40:5: error[missing-argument] No arguments provided for required parameters `x`, `y` of function `__new__`
constructors_call_type.py:50:5: error[missing-argument] No arguments provided for required parameters `x`, `y` of bound method `__init__`
@@ -197,7 +197,7 @@
constructors_callable.py:37:1: error[type-assertion-failure] Argument does not have asserted type `Class1`
constructors_callable.py:49:13: info[revealed-type] Revealed type: `(...) -> Unknown`
constructors_callable.py:50:1: error[type-assertion-failure] Argument does not have asserted type `Class2`
-constructors_callable.py:57:42: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@Class3`
+constructors_callable.py:57:42: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__new__`
constructors_callable.py:63:13: info[revealed-type] Revealed type: `(...) -> Unknown`
constructors_callable.py:64:1: error[type-assertion-failure] Argument does not have asserted type `Class3`
constructors_callable.py:73:33: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
@@ -432,28 +432,28 @@
generics_scoping.py:15:1: error[type-assertion-failure] Argument does not have asserted type `str`
generics_scoping.py:42:1: error[type-assertion-failure] Argument does not have asserted type `str`
generics_scoping.py:43:1: error[type-assertion-failure] Argument does not have asserted type `bytes`
-generics_self_advanced.py:11:24: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@ParentA`
+generics_self_advanced.py:11:24: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@prop1`
generics_self_advanced.py:18:1: error[type-assertion-failure] Argument does not have asserted type `ParentA`
generics_self_advanced.py:19:1: error[type-assertion-failure] Argument does not have asserted type `ChildA`
-generics_self_advanced.py:28:25: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@ParentB`
-generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self@ChildB`
-generics_self_advanced.py:36:9: error[type-assertion-failure] Argument does not have asserted type `list[Self@ChildB]`
-generics_self_advanced.py:37:9: error[type-assertion-failure] Argument does not have asserted type `Self@ChildB`
-generics_self_advanced.py:38:9: error[type-assertion-failure] Argument does not have asserted type `Self@ChildB`
-generics_self_advanced.py:43:9: error[type-assertion-failure] Argument does not have asserted type `list[Self@ChildB]`
-generics_self_advanced.py:44:9: error[type-assertion-failure] Argument does not have asserted type `Self@ChildB`
-generics_self_advanced.py:45:9: error[type-assertion-failure] Argument does not have asserted type `Self@ChildB`
-generics_self_attributes.py:26:33: error[invalid-argument-type] Argument is incorrect: Expected `Self@LinkedList | None`, found `LinkedList[int]`
-generics_self_attributes.py:29:5: error[invalid-assignment] Object of type `OrdinalLinkedList` is not assignable to attribute `next` of type `Self@LinkedList | None`
-generics_self_attributes.py:32:5: error[invalid-assignment] Object of type `LinkedList[int]` is not assignable to attribute `next` of type `Self@LinkedList | None`
-generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@Shape`
-generics_self_basic.py:20:16: error[invalid-return-type] Return type does not match returned value: expected `Self@Shape`, found `Shape`
-generics_self_basic.py:33:16: error[invalid-return-type] Return type does not match returned value: expected `Self@Shape`, found `Shape`
+generics_self_advanced.py:28:25: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@method1`
+generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self`
+generics_self_advanced.py:36:9: error[type-assertion-failure] Argument does not have asserted type `list[Self]`
+generics_self_advanced.py:37:9: error[type-assertion-failure] Argument does not have asserted type `Self`
+generics_self_advanced.py:38:9: error[type-assertion-failure] Argument does not have asserted type `Self`
+generics_self_advanced.py:43:9: error[type-assertion-failure] Argument does not have asserted type `list[Self]`
+generics_self_advanced.py:44:9: error[type-assertion-failure] Argument does not have asserted type `Self`
+generics_self_advanced.py:45:9: error[type-assertion-failure] Argument does not have asserted type `Self`
+generics_self_attributes.py:26:33: error[invalid-argument-type] Argument is incorrect: Expected `Self | None`, found `LinkedList[int]`
+generics_self_attributes.py:29:5: error[invalid-assignment] Object of type `OrdinalLinkedList` is not assignable to attribute `next` of type `Self | None`
+generics_self_attributes.py:32:5: error[invalid-assignment] Object of type `LinkedList[int]` is not assignable to attribute `next` of type `Self | None`
+generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@set_scale`
+generics_self_basic.py:20:16: error[invalid-return-type] Return type does not match returned value: expected `Self@method2`, found `Shape`
+generics_self_basic.py:33:16: error[invalid-return-type] Return type does not match returned value: expected `Self@cls_method2`, found `Shape`
generics_self_basic.py:51:1: error[type-assertion-failure] Argument does not have asserted type `Shape`
generics_self_basic.py:52:1: error[type-assertion-failure] Argument does not have asserted type `Circle`
generics_self_basic.py:54:1: error[type-assertion-failure] Argument does not have asserted type `Shape`
generics_self_basic.py:55:1: error[type-assertion-failure] Argument does not have asserted type `Circle`
-generics_self_basic.py:64:38: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@Container`
+generics_self_basic.py:64:38: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@set_value`
generics_self_basic.py:67:26: error[invalid-type-form] Special form `typing.Self` expected no type parameter
generics_self_basic.py:74:5: error[type-assertion-failure] Argument does not have asserted type `Container[int]`
generics_self_basic.py:75:5: error[type-assertion-failure] Argument does not have asserted type `Container[str]`
@@ -462,16 +462,16 @@
generics_self_usage.py:73:14: error[invalid-type-form] Variable of type `typing.Self` is not allowed in a type expression
generics_self_usage.py:73:23: error[invalid-type-form] Variable of type `typing.Self` is not allowed in a type expression
generics_self_usage.py:76:6: error[invalid-type-form] Variable of type `typing.Self` is not allowed in a type expression
-generics_self_usage.py:82:54: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@Foo2`
-generics_self_usage.py:86:16: error[invalid-return-type] Return type does not match returned value: expected `Self@Foo3`, found `Foo3`
+generics_self_usage.py:82:54: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@has_existing_self_annotation`
+generics_self_usage.py:86:16: error[invalid-return-type] Return type does not match returned value: expected `Self@return_concrete_type`, found `Foo3`
generics_self_usage.py:98:22: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `T@Bar`
generics_self_usage.py:101:15: error[invalid-type-form] Variable of type `typing.Self` is not allowed in a type expression
generics_self_usage.py:103:12: error[invalid-base] Invalid class base with type `typing.Self`
generics_self_usage.py:106:30: error[invalid-type-form] Variable of type `typing.Self` is not allowed in a type expression
-generics_self_usage.py:111:19: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@Base`
-generics_self_usage.py:116:40: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@Base`
-generics_self_usage.py:121:37: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@MyMetaclass`
-generics_self_usage.py:125:37: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `list[Self@MyMetaclass]`
+generics_self_usage.py:111:19: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@make`
+generics_self_usage.py:116:40: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@return_parameter`
+generics_self_usage.py:121:37: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__new__`
+generics_self_usage.py:125:37: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `list[Self@__mul__]`
generics_syntax_compatibility.py:23:38: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `V@ClassC | K@method1`
generics_syntax_compatibility.py:26:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `M@method2 | K`
generics_syntax_declarations.py:17:1: error[invalid-generic-class] Cannot both inherit from `typing.Generic` and use PEP 695 type variables |
|
|
Most of the ecosystem results are just renaming the binding context that we show when rendering a typevar. This result is interesting: The class's T = TypeVar("T")
class C:
def __init__(self, callback: Callable[[T], None]) -> None:
self.callback = callback
def method(self: Self, u: T) -> None:
self.callback(u)That makes me feel okay about these new ecosystem results, since it means that we're handling |
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
unresolved-attribute |
0 | 0 | 2,830 |
invalid-argument-type |
7 | 1 | 745 |
invalid-return-type |
0 | 1 | 136 |
possibly-unbound-attribute |
0 | 0 | 27 |
invalid-assignment |
0 | 0 | 13 |
invalid-super-argument |
0 | 0 | 5 |
unsupported-operator |
1 | 0 | 1 |
| Total | 8 | 2 | 3,757 |
|
The new diagnostics on altair look like false positives? It looks like casts to It seems like that comes up fairly rarely in practice, though, so I'm fine if we mark that as a TODO and tackle it later! |
That one turns out to be an issue with decorators, similar to #19604 (comment). In this case, the decorator doesn't have a return type annotation, so we assume it returns |
It seems like for a method decorated with an unknown decorator, we should probably still assume the first argument is implicitly typed as |
carljm
reviewed
Aug 6, 2025
Comment on lines
+109
to
+110
| TODO: The use of `Self` to annotate the `next_node` attribute should be | ||
| [modeled as a property][self attribute], using `Self` in its parameter and return type. |
There was a problem hiding this comment.
That sounds fine (feel free to create the issue), but will that be enough to solve this case? Wouldn't we consider these to be two separate "bindings" of Self (the one on the return type of the next_node property and the one on the return type of next method), and not allow the one to be assignable to the other, even if they share the same upper bound?
Comment on lines
+6441
to
+6454
| Type::TypeVar(typevar) if typevar.is_legacy(self.db()) => bind_legacy_typevar( | ||
| self.db(), | ||
| self.context.module(), | ||
| self.index, | ||
| self.scope().file_scope_id(self.db()), | ||
| self.legacy_typevar_binding_context, | ||
| typevar, | ||
| ) | ||
| .map(Type::TypeVar) | ||
| .unwrap_or(ty), | ||
| Type::KnownInstance(KnownInstanceType::TypeVar(typevar)) | ||
| if typevar.is_legacy(self.db()) => | ||
| { | ||
| bind_legacy_typevar( |
There was a problem hiding this comment.
I think this is kind of orthogonal to this PR, and relates instead to a previous PR that landed last week while I was out -- but it doesn't feel right to me that we are handling Type::TypeVar(...) and Type::KnownInstance(KnownInstanceType::TypeVar(...)) in such a parallel way here. It seems like we are missing some invariants that we ought to be enforcing? I think it should be an invariant that Type::TypeVar can never possibly wrap an unbound TypeVarInstance in the first place, so a Type::TypeVar should never need re-binding. Type::TypeVar represents an actual value of the variable type -- this is meaningless for an unbound typevar.
There was a problem hiding this comment.
This came up in last week's PR as well, and I agree. astral-sh/ty#926 tracks handling bound and unbound typevars more separately. KnownInstanceType::TypeVar would only be able to wrap an unbound typevar, and Type::TypeVar would only be able to wrap a bound one.
There was a problem hiding this comment.
Makes sense. That issue proposes one possible fix (split TypeVarInstance into separate Rust types for bound and unbound type variables) which would clearly allow us to enforce the invariant, but even short of that it seems like we could have some asserts/unreachables instead of confusingly handling cases that should be unnecessary/impossible (like here, I don't think we should ever call bind_legacy_typevar on something inside Type::TypeVar, that should always be already bound) to better clarify the semantics. Doesn't have to be in this PR, though.
Or put differently, when we're inferring things inside the function body, we shouldn't care about decorators at all! I was able to resolve this by tracking the undecorated type in the |
Nice, that fix looks good! |
carljm
approved these changes
Aug 6, 2025
dcreager
added a commit
that referenced
this pull request
Aug 16, 2025
`Type::TypeVar` now distinguishes whether the typevar in question is
inferable or not.
A typevar is _not inferable_ inside the body of the generic class or
function that binds it:
```py
def f[T](t: T) -> T:
return t
```
The infered type of `t` in the function body is `TypeVar(T,
NotInferable)`. This represents how e.g. assignability checks need to be
valid for all possible specializations of the typevar. Most of the
existing assignability/etc logic only applies to non-inferable typevars.
Outside of the function body, the typevar is _inferable_:
```py
f(4)
```
Here, the parameter type of `f` is `TypeVar(T, Inferable)`. This
represents how e.g. assignability doesn't need to hold for _all_
specializations; instead, we need to find the constraints under which
this specific assignability check holds.
This is in support of starting to perform specialization inference _as
part of_ performing the assignability check at the call site.
In the [[POPL2015][]] paper, this concept is called _monomorphic_ /
_polymorphic_, but I thought _non-inferable_ / _inferable_ would be
clearer for us.
Depends on #19784
[POPL2015]: https://doi.org/10.1145/2676726.2676991
---------
Co-authored-by: Carl Meyer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes our logic for binding a legacy typevar with its binding context. (To recap, a legacy typevar starts out "unbound" when it is first created, and each time it's used in a generic class or function, we "bind" it with the corresponding
Definition.)We treat
typing.Selfthe same as a legacy typevar, and so we apply this binding logic to it too. Before, we were using the enclosing class as its binding context. But that's not correct — it's the method wheretyping.Selfis used that binds the typevar. (Each invocation of the method will find a new specialization ofSelfbased on the specific instance type containing the invoked method.)This required plumbing through some additional state to the
in_type_expressionmethod.This also revealed that we weren't handling
Self-typed instance attributes correctly (but were coincidentally not getting the expected false positive diagnostics).