gh-153060: Add an empty frozendict singleton

[vstinner]

The singleton is created immortal to avoid refcount contention in Free Threading.

• Add PyInterpreterState.dict_state.empty_frozendict.
• Add _PyDict_Init() and _PyDict_Fini() to create and destroy the empty frozendict singleton.

• Issue: Add an empty frozendict singleton #153060

[vstinner]

Examples:

>>> empty = frozendict()
>>> frozendict() is empty
True
>>> frozendict([]) is empty
True
>>> (empty | empty) is empty
True
>>> (empty | {}) is empty
True
>>> frozendict.fromkeys('') is empty
True

[vstinner]

cc @corona10

[corona10]

Overall looks good to me, but give me enough time to take a look at see the detail.

[vstinner]

I ran a microbenchmark on Linux with CPU isolation on this change:

Benchmark	ref2	singleton2
frozendict()	39.8 ns	19.6 ns: 2.04x faster
frozendict([])	79.8 ns	92.3 ns: 1.16x slower
frozendict((x for x in ()))	230 ns	259 ns: 1.13x slower
frozendict() | {}	73.0 ns	50.0 ns: 1.46x faster
frozendict.fromkeys("")	133 ns	148 ns: 1.11x slower
frozendict({"a": 1, "b": 2, "c": 3, "d": 4})	78.2 ns	85.7 ns: 1.10x slower
frozendict(x=1, y=2) | frozendict(z=3)	251 ns	278 ns: 1.11x slower
frozendict.fromkeys("abdef")	260 ns	266 ns: 1.02x slower
Geometric mean	(ref)	1.06x faster

I'm surprised by the slowdown on these operations:

• frozendict((x for x in ())): 230 ns => 259 ns (+29 ns)
• frozendict.fromkeys(""): 133 ns => 148 ns (+15 ns)
• frozendict(x=1, y=2) | frozendict(z=3): 251 ns => 278 ns (+27 ns)

bench.py script:

Details

import pyperf
runner = pyperf.Runner()

# Get the singleton
runner.timeit('frozendict()',
    stmt='frozendict()')
runner.timeit('frozendict([])',
    setup='iterable = []',
    stmt='frozendict(iterable)')
runner.timeit('frozendict((x for x in ()))',
    stmt='frozendict((x for x in ()))')
runner.timeit('frozendict() | {}',
    stmt='frozendict() | {}')
runner.timeit('frozendict.fromkeys("")',
    stmt='frozendict.fromkeys("")')

# Operations on some small frozen dictionaries
runner.timeit('frozendict({"a": 1, "b": 2, "c": 3, "d": 4})',
    setup='d = {"a": 1, "b": 2, "c": 3, "d": 4}',
    stmt='frozendict(d)')
runner.timeit('frozendict(x=1, y=2) | frozendict(z=3)',
    stmt='frozendict(x=1, y=2) | frozendict(z=3)')
runner.timeit('frozendict.fromkeys("abdef")',
    stmt='frozendict.fromkeys("abdef")')

[ZeroIntensity]

I think it would make sense to also add this to Py_GetConstantBorrowed.

[ZeroIntensity]

Singletons are usually supposed to be stored on _PyRuntime.static_objects.singletons, not on the interpreter state. Is there a reason we're deviating from that convention here?

[vstinner]

Adding the empty frozendict singleton to _PyRuntime.static_objects.singletons would require to initialize a PyFrozenDictObject structure statically which is complicated. PyFrozenDictObject inherits from PyDictObject which has these two members:

• PyDictKeysObject *ma_keys
• PyDictValues *ma_values

We should get access to this empty_keys_struct (Py_EMPTY_KEYS) outside dictobject.c.

static PyDictKeysObject empty_keys_struct = {
        _Py_DICT_IMMORTAL_INITIAL_REFCNT, /* dk_refcnt */
        0, /* dk_log2_size */
        3, /* dk_log2_index_bytes */
        DICT_KEYS_UNICODE, /* dk_kind */
#ifdef Py_GIL_DISABLED
        {0}, /* dk_mutex */
#endif
        1, /* dk_version */
        0, /* dk_usable (immutable) */
        0, /* dk_nentries */
        {DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY,
         DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY, DKIX_EMPTY}, /* dk_indices */
};

Pre-computing hash() is also complicated:

    Py_uhash_t hash = 0;
    hash ^= (1) * 1927868237UL;
    hash ^= (hash >> 11) ^ (hash >> 25);
    hash = hash * 69069U + 907133923UL;
    if (hash == (Py_uhash_t)-1) {
        hash = 590923713UL;
    }

Well. Adding _PyDict_Init() and _PyDict_Fini() to allocate the empty frozendict singleton using dictobject.c code was simpler for me.

[vstinner]

I think it would make sense to also add this to Py_GetConstantBorrowed.

Python had a frozenset singleton, but it was decided to remove it in Python 3.10. frozendict is still very new. I don't know if the empty frozendict singleton will stay forever.

If we add Py_CONSTANT_EMPTY_FROZENDICT, we will have to keep the singleton forever, even if it will become less efficient in the future.

So for now, I would prefer prefer to not add it to Py_GetConstant().