Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

852a4b5

Problem description

In the free-threaded build, PyThreadState_Clear removes the thread from the biased reference counting table, which can cause finalizers to be called. Those finalizers can themselves destroy the thread state, leaving PyThreadState_Clear with a dangling pointer.

This was reported in CPython in python/cpython#119585 and fixed for PyGILState_Release in python/cpython#119753.

Reproducible example code

repro.cpp

#include <barrier>
#include <pybind11/pybind11.h>
#include <thread>

namespace py = pybind11;

void do_stuff(py::type cls) {
  py::handle obj;

  std::barrier barrier{2};
  std::jthread t1{[&]() {
    py::gil_scoped_acquire gil;
    obj = cls().release();
    barrier.arrive_and_wait();
  }};
  std::jthread t2{[&]() {
    py::gil_scoped_acquire gil;
    barrier.arrive_and_wait();
    // ob_ref_shared becomes negative; transition to the queued state
    obj.dec_ref();
  }};
}

PYBIND11_MODULE(repro, m, py::mod_gil_not_used()) {
  m.def("do_stuff", &do_stuff);
  m.def("finalizer", []() { py::gil_scoped_acquire gil; });
}

main.py

import repro

class Foo:
    def __del__(self):
        repro.finalizer()

repro.do_stuff(Foo)

Is this a regression? Put the last known working version here if it is.

Not a regression