Idea: TreeVar (context variables that follow stack discipline)

[oremanj]

A context variable is a convenient way to track something about the scope in which a task executes. Sometimes that "something" effectively constitutes a promise to child tasks that some resource remains usable. For example, trio_asyncio.open_loop() provides the asyncio loop via a contextvar, but it's not much use if the async with block has exited already.

Unfortunately, contextvars are inherited from the environment of the task spawner, not the environment of the nursery in which the task runs. That means they can't reliably provide resources to a task, because the resources might be released before the task finishes. Imagine:

async with trio.open_nursery() as nursery:
    async with trio_asyncio.open_loop() as loop:
        nursery.start_soon(some_asyncio_task)

It would be nice to have a kind of contextvar that is inherited by child tasks based on the value it had when their nursery was opened, rather than the value it had when the child task was spawned. (In theory, cancellation status could use this if it existed. In practice, we probably want to continue to special-case cancellation status because it's so fundamental.)

Implementation idea: each scoped contextvar uses an underlying regular contextvar whose value is a tuple (scoped contextvar's value, task that set it). Also, capture the contextvars context when creating a nursery (this is cheap, since contexts are meant to be shallow-copied all the time). When accessing the value of a scoped contextvar, check whether the underlying regular contextvar was set in the task we're currently in. If so, return the value from the underlying regular contextvar in the current context; otherwise, return its value in our parent nursery's captured context.

[njsmith]

First reactions:

Is this a problem? Answer: Maybe?

I see your point about trio-asyncio. But in your example, I'm not sure it would be so bad if some_asyncio_task instead raised an exception when you exited the loop block... compare to:

async with trio.open_nursery() as nursery:
    async with trio.open_tcp_stream(...) as stream:
        nursery.start_soon(some_stream_task, stream)

Seems like a pretty similar case, it does confuse new users sometimes, but... there's not really anything to do about it except teach folks what things mean.

I think the other way around might be potentially more compelling? If I do:

global aio_supporting_nursery
async with trio_asyncio.open_loop() as loop:
    async with trio.open_nursery() as aio_supporting_nursery:
        await sleep_forever()

# Elsewhere:
aio_supporting_nursery.start_soon(aio_task)

Then currently that won't work, but maybe it would be convenient if it did? I'm not sure whether this example is realistic or contrived :-)

Another option I'll throw out there: trio-asyncio could do some hacky stuff like, find the loop by walking up the task tree and checking for Task.context[trio_asyncio_cvar]. (Hmm. I guess that's similar to your version, except you're pointing out that for this to work 100% accurately, you need to attach the magic value to a nursery, not a task? The really hacky thing would be for trio-asyncio to just jam an attribute on the nursery it creates, like loop_nursery.__trio_asyncio_loop__ = loop.)

Should all contextvars work this way? It's a bit arbitrary whether new tasks inherit their context from the task that called start_soon or the task that holds the nursery. Currently it's from the task that called start_soon, but one could imagine switching this (and you mentioned wishing we could). So we should pause and consider whether the current behavior is ever useful.

I think it actually is. Here's a fairly realistic example: consider a web app, that needs to spawn a background task in response to a request. (And maybe there will be other requests later to check on the status of the background job, etc.) Generally this will require a global nursery to handle the background tasks, and a request handler that spawns a task into this nursery, and then reports back that the task has been started.

Now, let's say you're using a logging library that assigns each request a unique id, and then attaches that to all log messages generated by that request. You probably want that request-id to propagate into the background task, so any logs it generates can be traced back to the originating request. (Or at the least, that's more useful than every background task losing the logging context entirely!)

That's what Trio gives you now, with context propagating along start_soon.

If we do it, how should we implement it? Instead of tuples like (scoped contextvar's value, task that set it), wouldn't it be simpler to just have a Task.scoped_context, and ScopedContextVar.get() does return current_task().scoped_context[self._cvar]? Nurseries would capture it when created, and tasks would copy it from their parent nursery.

[njsmith]

More potential use cases to think about:

• this came up while discussing control-C handling, and the idea that protection should be inherited across tasks and nurseries following the same path that the exception would follow

• in Global deadlock detector #1085, @smurfix pointed out that it would be nice to have the option of marking a task as "not counting as active" for deadlock detection purposes – examples would include the run_sync_soon trio re-entry task, and potentially other system tasks like watchdog heartbeats or global connection pool managers ([discussion] [potential heresy] global connection pools and global tasks hip#125). It would make sense for this state to be inherited along a task tree as well.

Of course both of these cases involve deep involvement with trio's internals, so it would be easy to handle them as special cases, by putting some dedicated attributes on Task or whatever. I'm not sure if that means that this is a more general thing that lots of libraries will need, or if it's something that only comes up in these kinds of special cases.

[oremanj]

I don't know if I'd say that a lot of libraries would need it, but I think it's a feature that will be useful to libraries in at least some cases, and I don't think it costs much to support it in Trio core. This also fits with our principle of "as much of Trio as possible should be implemented in ways that don't require magic access to internals".

[oremanj]

Another use: Trio wants to clean up async generators in one way (#265); asyncio uses a different approach. trio-asyncio should use the right set of async generator semantics for the flavor of code that first iterates the async generator. A scopevar is the natural way to implement this.

I think this pushes me over the edge into "yes, useful".

[njsmith]

Isn't that just a matter of being able to sniff out which mode we're running in at any given moment? Trio-asyncio already has ways to do that, doesn't it?

[oremanj]

I was imagining that the global asyncgen hooks (presumably installed by Trio) would check the scopevar for an optional override of "what hooks to use in this context". That way Trio doesn't have to be directly aware of trio-asyncio. If the hooks were specific to trio-asyncio, then I agree that they could just use the sniffio state, but I think having trio-asyncio reliably install its own global hooks might be tricky in practice.

[smurfix]

trio-asyncio just uses a contextvar to store the loop. That works for applications because you typically have one loop at top level and there is no "outside" to call into trio-asyncio from.

It does not work quite as well for a Trio library which wants to use some asyncio code.
Imagine task A starting the pseudo-asyncio loop, setting up a connection, and exhibiting that to the rest of the program. Now if an "outside" task B comes along and calls that library, you get an interesting error.

I'd recommend to simply add a with_value_of(contextvar,func,*args) method to the nursery. That method would grab the contextvar's value from the nursery's context, sets the contextvar to it, and call the function.
The aforementioned library could then simply use that to restore the nursery's value of trio_asyncio.current_loop in its entry points. That's more efficient than always shlepping two contextvar stacks around, which you'd need to do because the very same library might want to inherit another contextvar, like current_remote or asgi.request or websocket or …, from the caller, not from the nursery.

Untested code:

class Nursery:
    …
    def with_value_of(self, contextvar, func, *args):
        ctx = self.parent_task.context
        val = ctx[contextvar]
        token = contextvar.set(val)
        try:
            return func(*args)
        finally:
            contextvar.reset(token)

[oremanj]

I'd recommend to simply add a with_value_of(contextvar,func,*args) method to the nursery. That method would grab the contextvar's value from the nursery's context, sets the contextvar to it, and call the function.

The problem is that there isn't currently any "nursery's context". There is the nursery's parent task's context, but that might reflect changes that were made after opening the nursery and would otherwise only ever be visible to the body of the nursery's async with block. If you want to know the state when the nursery was created, you have to save it.

That's more efficient than always shlepping two contextvar stacks around, which you'd need to do because the very same library might want to inherit another contextvar, like current_remote or asgi.request or websocket or …, from the caller

I agree that we can't treat all contextvars in this scoped fashion -- I think @njsmith did a good analysis of that above. But I think having one context for traditional contextvars and another for scopevars will work out fine in practice. Copying contexts is really cheap -- that's almost their defining feature, relative to something more mundane like a dictionary. This is important in asyncio where every single callback registration does a context copy. I think Trio can get away with doing two context copies rather than one on each new task creation.

[complyue]

with_value_of(contextvar,func,*args)

If we provide a similar API for frameworks and libraries like:

• with_effect(contextvar, eff_func, business_func, *args)
  define an effectful behavior identified by contextvar, in form of a function, then call a business function with that effect in context

• perform_effect(contextvar, *eff_args)
  call an effect with arguments, either directly from business_func as passed above, or some nested (sync called or awaited) business functions called from business_func, those functions may have been imported from other modular libraries merely aware of the value of contextvar and the expected duck typing rules.

then pretty much an effects system is implemented, relating to conventional algebraic effects construct, we use a contextvar plus duck typed function arguments in place of an Algebra Data Type which specifies the call convention in a well typed way.

data SomEffect = SomeEffect {
    arg1 :: Int
  , arg2 :: String
  };

While functional language implementers tend to use continuation passing to implement effect calls, so even exception handling could be implemented by a library as an effect, we don't have to go that way, with a procedural imperative language where exception handling is already established.

And effect calling don't have to always be about function calls, resolving a value in context is perfect valid use case IMO, and such a pair of APIs (with_effect/perform_effect) is so generic that frameworks and libraries can choose effect identifiers (contextvar as in this example) according to ad-hoc needs, regardless of any implementation details.

[oremanj]

An unresolved question: what should happen around nursery.start() when tasks can move from one scope to another? For some uses of scopevars (like "here is your trio-asyncio loop") you want the task to have access to the ultimate value from the beginning, while for others ("should I raise KeyboardInterrupt?") the behavior should change at the time of the call to task_status.started().

[complyue]

Sorry I'm not familiar with Trio yet. But I'm surprised to hear a task can move between scopes, isn't the idea of structural concurrency to describe tasks with lexical scoping, how can they move ? Or the task @oremanj refers to is actually an asyncio task? Does Trio use asyncio loops/tasks under the hood just as resources ?

[njsmith]

@complyue

with_effect(contextvar, eff_func, business_func, *args)
perform_effect(contextvar, *eff_args)

That's certainly a thing that you can do. But it's not clear to me what the goal is. What problem are you trying to solve?

But I'm surprised to hear a task can move between scopes, isn't the idea of structural concurrency to describe tasks with lexical scoping, how can they move ? Or the task @oremanj refers to is actually an asyncio task? Does Trio use asyncio loops/tasks under the hood just as resources ?

nursery.start is a special feature for tasks that have a complex "startup" phase: https://trio.readthedocs.io/en/stable/reference-core.html#trio.Nursery.start

We don't support moving tasks between scopes in general, but for nursery.start in particular, it basically starts out the task running under the start call, and then once it's successfully started up it moves it into the final nursery.

Trio is completely separate from asyncio; we don't use any asyncio code.

@oremanj

For some uses of scopevars (like "here is your trio-asyncio loop") you want the task to have access to the ultimate value from the beginning, while for others ("should I raise KeyboardInterrupt?") the behavior should change at the time of the call to task_status.started().

My first intuition for nursery.start+KeyboardInterrupt is that during the startup phase you want it to be protected if the starting context OR the final context have protection enabled? Consider the case where start is unprotected but the nursery is protected: I have this image of injecting a KeyboardInterrupt into the task during the startup phase, but then before it propagates out the task calls task_status.started(), switches into the nursery, and then the KeyboardInterrupt arrives in a nursery that was supposed to be protected.... this would require a lot of things to go wrong at once, but still.

This would be pretty straightforward to handle as a one-off in start, no matter how we end up storing the KeyboardInterrupt protection flag. But it feels consistent with the idea that the KeyboardInterrupt problem is a bit of a special case inside Trio, not just an instance of a more general problem.

I was imagining that the global asyncgen hooks (presumably installed by Trio) would check the scopevar for an optional override of "what hooks to use in this context". That way Trio doesn't have to be directly aware of trio-asyncio. If the hooks were specific to trio-asyncio, then I agree that they could just use the sniffio state, but I think having trio-asyncio reliably install its own global hooks might be tricky in practice.

I was thinking that when trio-asyncio would have hooks like:

if in_asyncio_mode:
    ...
else:
    call_original_hook(...)

And every time it starts a loop, it would check if it has its own hooks installed, and if not then install them.

You can't use trio-asyncio unless you're already using trio, and trio will install its hooks unconditionally at the beginning of trio.run, so I think this would be 100% reliable? The only way I can think of where it'd go wrong would be if there's yet a third library also fighting over the hooks in the same thread, but that's a challenge for the scopevar version too :-).

---

Overall, I'm just not quite comfortable that we understand the situation well enough yet to commit to adding a whole new variable mechanism...

I want to think harder about what exactly trio-asyncio needs, since that seems like the on that's potentially really compelling.

I'm also pondering: the critical thing about the ContextVar API specifically is that it's not our API, it's a generic API that gets used by all kinds of things that aren't just Trio specific. So we can provide whatever constellation of semantics we want, but deciding which semantics to apply to ContextVar specifically is important b/c that will by default affect all these libraries that aren't thinking about Trio at all.

One case is logging libraries. I suspect that these prefer the "causal" context propagation along start_soon, as discussed above.

Another case is decimal, which uses a ContextVar to store the decimal context. Say we have code like:

with decimal.localcontext() as ctx:
    ctx.prec = 42
    nursery.start_soon(...)

Would users expect that high-precision setting to be inherited by the child task, or not?

[njsmith]

Also, I'm not sure "scope" is the best name here... a lot of things are scopes :-). Maybe TreeVar?