"Tuple field type cannot be Union{}" error in all 1.10 and 1.11 versions

[aplavin]

I've been running some package tests on the upcoming julia version (1.10rc) and noticed this error newly introduced there, compared to 1.9:

julia> Tuple{Union{}}
# 1.9:
Tuple{Union{}}
# 1.10:
ERROR: Tuple field type cannot be Union{}

Not sure if introduced deliberately or as a side-effect of some other change, but it breaks perfectly working code. And I don't see it anywhere in Tuple docs that it shouldn't support all element types.

The above is an MWE, and below is the simplified situation where I actually encountered it:

julia> X = StructArray(a=Union{}[], b=[]);
# 1.10 throws error right here
# 1.9 lets you proceed just fine

# can work with the non-Union{} column:
julia> X.b
Any[]

# can retrieve the Union{} column as well, of course cannot access its elements
julia> X.a |> typeof
Vector{Union{}} (alias for Array{Union{}, 1})

[N5N3]

This is a deliberate change, see #49111.

[vtjnash]

You didn't include a stacktrace, but it looks like this calls collect_structarray, which calls Core.Compiler.return_type, which is a private API in a private package with no documentation or stability guarantees. The closest actual API is Base.promote_op() which should work for this and is documented as such (though using it does not quite accurately meet the AbstractArray or collect APIs--c.f. the implementation of those in Base and how it tries to avoid them).

https://github.com/JuliaArrays/StructArrays.jl/blob/99f05561cab19fb23f478a12a8429764871ccff3/src/collect.jl#L44

help?> Core.Compiler.return_type
  │ Warning
  │
  │  The following bindings may be internal; they may change or be removed in future versions:
  │
  │    •  Core.Compiler
  │
  │    •  Core.Compiler.return_type

  No documentation found for private symbol.

[aplavin]

The MWE doesn't depend on return_type at all. And I don't think the structarray example does either.

See:

julia> arrs = (Union{}[], Int[])

# works on 1.9 – broken on 1.10rc
julia> Tuple{map(eltype, arrs)...}
ERROR: Tuple field type cannot be Union{}

[vtjnash]

Not all apply_type expressions are meaningful. They can return errors

[aplavin]

Creating a Tuple type with arbitrary types inside seems to make total sense, that's one of the most fundamental Julia types. I haven't seen anywhere in the docs that Tuple only supports a subset of Julia types. And this limitation does sound strange, doesn't it?

[aplavin]

So, should this error be removed?
Both:

• this change looks unambiguously breaking, perfectly working code that doesn't rely on internals stops working
• the behavior itself is totally sensible, even aside from the first point; surely one can create a Tuple type with any element types, no exceptions are documented and Union{} as element type does work elsewhere.

[rdboyes]

Running into this error using density() from AlgebraOfGraphics as well - a simple MWE from here: MakieOrg/AlgebraOfGraphics.jl#472 (comment)

julia> draw(
           data((x=randn(100), y=randn(100))) *
           mapping(:x, :y) *
           AlgebraOfGraphics.density() *
           visual(Contour)
       )
ERROR: Tuple field type cannot be Union{}
Stacktrace:
  [1] map(f::Function, d::Dictionaries.Indices{Union{}})
    @ Dictionaries ~/.julia/packages/Dictionaries/7aBxp/src/map.jl:91
  [2] unnest(vs::Vector{@NamedTuple{}}, indices::Dictionaries.Indices{Union{}})
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layer.jl:81
  [3] unnest_dictionaries(vs::Vector{@NamedTuple{}})
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layer.jl:84
  [4] map(f::AlgebraOfGraphics.var"#193#194"{@NamedTuple{…}}, processedlayer::ProcessedLayer)
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layer.jl:101
  [5] (::AlgebraOfGraphics.DensityAnalysis{…})(input::ProcessedLayer)
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/transformations/density.jl:30
  [6] call_composed
    @ Base ./operators.jl:1045 [inlined]
  [7] call_composed
    @ Base ./operators.jl:1044 [inlined]
  [8] (::ComposedFunction{AlgebraOfGraphics.Visual, AlgebraOfGraphics.DensityAnalysis{…}})(x::ProcessedLayer)
    @ Base ./operators.jl:1041
  [9] process(layer::Layer)
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/processing.jl:102
 [10] iterate(g::Base.Generator, s::Vararg{Any})
    @ Base ./generator.jl:47 [inlined]
 [11] collect(itr::Base.Generator{Layers, typeof(AlgebraOfGraphics.process)})
    @ Base ./array.jl:834
 [12] map
    @ ./abstractarray.jl:3310 [inlined]
 [13] ProcessedLayers(a::Layer)
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layers.jl:41
 [14] compute_axes_grid(d::Layer; axis::@NamedTuple{}, palettes::@NamedTuple{})
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layers.jl:114
 [15] compute_axes_grid
    @ ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layers.jl:110 [inlined]
 [16] compute_axes_grid(fig::Figure, d::Layer; axis::@NamedTuple{}, palettes::@NamedTuple{})
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layers.jl:100
 [17] compute_axes_grid
    @ ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/algebra/layers.jl:97 [inlined]
 [18] #241
    @ ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:21 [inlined]
 [19] update
    @ ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:10 [inlined]
 [20] plot!(fig::Figure, d::Layer; axis::@NamedTuple{}, palettes::@NamedTuple{})
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:21
 [21] plot!
    @ ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:16 [inlined]
 [22] (::AlgebraOfGraphics.var"#245#246"{…})(f::Figure)
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:48
 [23] update
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:10 [inlined]
 [24] #draw#244
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:47 [inlined]
 [25] draw(d::Layer)
    @ AlgebraOfGraphics ~/.julia/packages/AlgebraOfGraphics/tbMEb/src/draw.jl:44
 [26] top-level scope
    @ REPL[7]:1
Some type information was truncated. Use `show(err)` to see complete types.

[vtjnash]

As with the previous person, this is due to the Dictionaries package using the disallowed private symbol Core.Compiler.return_type. Open an issue there and link #52385 (comment)?

[aplavin]

This issue (the one discussed here, don't know about AoG) is independent on return_type() though, but due to a breaking change in Julia 1.10. Even more, it's breaking completely sensible behavior of being able to represent Tuple{T} for T = Union{}.

There's nothing fundamentally bad with breaking changes, they should just be clearly communicated. For now, Julia promises no breaking changes in 1.x, which this clearly contradicts.

[nsajko]

The empty union, Union{}, is the type with no instances, thus Tuple{Union{}}, and any other tuple type with an empty union field, is also the type with no instances, so I guess we should have Tuple{Union{}} == Union{} hold. I don't see how it makes sense for Tuple{Union{}} to error.

I admit this might be difficult to fix, though.

[vtjnash]

Having no instances is not the same as having no subtypes. Those are 2 orthogonal properties.

We could try to return Union{} here, as that was the first attempt before making it an error, but much code behaved poorly with that, so it would be a breaking change. Making it an error--for this case which was already buggy--was not a breaking change, since packages had always been told not to call the Core.Compiler functions.

[nsajko]

Both Tuple and Union are part of the public API. So it seems like Tuple{Union{}} is public API, regardless of any Core.Compiler internals, no?

[aplavin]

was not a breaking change, since packages had always been told not to call the Core.Compiler functions.

Not sure why Core.Compiler.return_type is being brought up here over and over, it's completely unrelated to the issue.