Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Doc/c-api/frame.rst
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ See also :ref:`Reflection <reflection>`.

.. versionadded:: 3.11

.. versionchanged:: 3.16
May return ``NULL`` for a coroutine frame before its first suspension;
see :attr:`frame.f_generator`.


.. c:function:: PyObject* PyFrame_GetGlobals(PyFrameObject *frame)

Expand Down
21 changes: 21 additions & 0 deletions Doc/library/dis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -839,6 +839,27 @@ not have to be) the original ``STACK[-2]``.
separate instructions.


.. opcode:: END_AWAITED_CALL (delta)

Follows a call whose result is awaited right away (``await f()``, and the
calls of :meth:`~object.__aenter__` and :meth:`~object.__aexit__` in
:keyword:`async with`).
The compiler pushes an extra ``NULL``, the *lazy outcome*, below the
callable of such a call, and the call may run the body of a coroutine
function without creating the coroutine object, reporting in the lazy
outcome how the body ended.

If the lazy outcome is ``NULL``, it is removed and execution continues with
:opcode:`GET_AWAITABLE`. If the body returned, it is removed and
execution continues after the :opcode:`END_SEND` of the following
:opcode:`SEND` loop. If the body suspended, the coroutine object is
created and the bytecode counter is incremented by *delta*, to the
:opcode:`YIELD_VALUE` of that loop, with ``STACK[-3:]`` set to the
coroutine, ``NULL`` and the yielded value.

.. versionadded:: 3.16


.. opcode:: GET_AITER

Implements ``STACK[-1] = STACK[-1].__aiter__()``.
Expand Down
4 changes: 3 additions & 1 deletion Doc/library/inspect.rst
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,9 @@ attributes (see :ref:`import-mod-attrs` for module attributes):
| | | coroutine object that |
| | | owns this frame, or |
| | | ``None`` if the frame is |
| | | of a regular function |
| | | of a regular function or |
| | | the generator is not |
| | | materialized yet |
+-----------------+-------------------+---------------------------+
| | f_trace | tracing function for this |
| | | frame, or ``None`` |
Expand Down
14 changes: 13 additions & 1 deletion Doc/reference/datamodel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1736,10 +1736,22 @@ Special read-only attributes

* - .. attribute:: frame.f_generator
- The :term:`generator` or :term:`coroutine` object that owns this frame,
or ``None`` if the frame is a normal function.
or ``None`` if the frame is a normal function or if the generator
is not materialized yet.

.. impl-detail::

A coroutine that is awaited as soon as it is called
(``await coro_func()``) may run without a coroutine object until
it first suspends, so this attribute is ``None`` before that.
Use :attr:`~codeobject.co_flags` of :attr:`frame.f_code` to tell
coroutine frames apart from normal function frames.

.. versionadded:: 3.14

.. versionchanged:: 3.16
May be ``None`` for a coroutine frame before its first suspension.

.. index::
single: f_trace (frame attribute)
single: f_trace_lines (frame attribute)
Expand Down
46 changes: 46 additions & 0 deletions Doc/whatsnew/3.16.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1147,6 +1147,26 @@ datetime and time
(Contributed by Paul Ganssle in :gh:`158252`, based on work by
Stan Ulbrych.)

Interpreter
-----------

* A coroutine that is awaited as soon as it is called
(``await coro_func()``), and an ``async def`` :meth:`~object.__aenter__`
or :meth:`~object.__aexit__` of an :keyword:`async with` statement, now
runs without creating a coroutine object unless it suspends, making awaits
that complete immediately about twice as fast in a micro-benchmark
(``await noop()``: 52 ns to 27 ns on x86-64).
:meth:`~object.__aexit__` still runs with a coroutine object when the body
of the :keyword:`async with` statement raised an exception.
This does not apply while coroutine origin tracking is enabled, while a
profiler or :mod:`sys.monitoring` tool receives call events (for example
:func:`sys.setprofile` or :mod:`profiling.tracing`), while a :pep:`523` frame
evaluation function is set, or inside code compiled by the experimental JIT.
Until it first suspends, :attr:`frame.f_generator` is ``None`` for such a
coroutine, so its value may differ when one of these is active;
see :ref:`whatsnew316-porting`.
(Contributed by Andrew Svetlov in :gh:`161170`.)

re
--

Expand Down Expand Up @@ -1405,12 +1425,26 @@ New deprecations
.. include:: ../deprecations/pending-removal-in-future.rst


.. _whatsnew316-porting:

Porting to Python 3.16
======================

This section lists previously described changes and other bugfixes
that may require changes to your code.

* :attr:`frame.f_generator` and :c:func:`PyFrame_GetGenerator` may now
return ``None``/``NULL`` for a coroutine frame before its first
suspension, because a coroutine that is awaited as soon as it is called
(``await coro_func()``) runs without a coroutine object until then.
Code that used ``frame.f_generator is None`` to detect frames of normal
functions should check ``frame.f_code.co_flags`` for
:data:`~inspect.CO_COROUTINE` instead.
For the same reason, such a coroutine takes its ``__name__`` and
``__qualname__`` from its function when it first suspends, not when it
is called.
(Contributed by Andrew Svetlov in :gh:`161170`.)

* In :mod:`tkinter`, the *name* parameter of the
:meth:`~tkinter.Misc.wait_variable`, :meth:`~tkinter.Misc.setvar` and
:meth:`~tkinter.Misc.getvar` methods and the *value* parameter of
Expand Down Expand Up @@ -1488,6 +1522,18 @@ that may require changes to your code.
(Contributed by An Long in :gh:`86768`.)


CPython bytecode changes
========================

* ``await <call>`` and the calls of :meth:`~object.__aenter__` and
:meth:`~object.__aexit__` in :keyword:`async with` now push an extra
``NULL`` below the callable and
follow the call with the new :opcode:`END_AWAITED_CALL` instruction,
which either goes on to :opcode:`GET_AWAITABLE`, skips the
:opcode:`SEND` loop, or jumps to its :opcode:`YIELD_VALUE`.
(Contributed by Andrew Svetlov in :gh:`161170`.)


Build changes
=============

Expand Down
7 changes: 7 additions & 0 deletions Include/internal/pycore_ceval.h
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,13 @@ extern int _PyEval_EnsureBuiltinsWithModule(
PyObject **p_builtins);

PyAPI_FUNC(PyObject *)_Py_MakeCoro(PyFunctionObject *func);
PyAPI_FUNC(PyObject *) _PyCoro_MaterializeLazy(_PyInterpreterFrame *frame);
PyAPI_FUNC(_PyInterpreterFrame *) _PyEval_SuspendLazyCoro(
PyThreadState *tstate, _PyInterpreterFrame *frame);
PyAPI_FUNC(PyGenObject *) _PyEval_MaterializeLazyCoro(PyThreadState *tstate,
_PyInterpreterFrame *frame);
PyAPI_FUNC(_PyInterpreterFrame *) _PyEval_ThrowIntoLazyCoro(
PyThreadState *tstate, _PyInterpreterFrame *frame, _PyStackRef value);

/* Handle signals, pending calls, GIL drop request
and asynchronous exception */
Expand Down
3 changes: 3 additions & 0 deletions Include/internal/pycore_interpframe.h
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@ static inline void _PyFrame_Copy(_PyInterpreterFrame *src, _PyInterpreterFrame *
dest->stackpointer = dest->localsplus + stacktop;
// visited is GC bookkeeping for the current stack walk, not frame state.
dest->visited = 0;
dest->lazy_coro = 0;
#ifdef Py_DEBUG
dest->stackpointer_valid = src->stackpointer_valid;
dest->lltrace = src->lltrace;
Expand Down Expand Up @@ -208,6 +209,7 @@ _PyFrame_Initialize(
frame->return_offset = 0;
frame->owner = FRAME_OWNED_BY_THREAD;
frame->visited = 0;
frame->lazy_coro = 0;
#ifdef Py_DEBUG
frame->stackpointer_valid = 1;
frame->lltrace = 0;
Expand Down Expand Up @@ -441,6 +443,7 @@ _PyFrame_PushTrampolineUnchecked(PyThreadState *tstate, PyCodeObject *code, int
#endif
frame->owner = FRAME_OWNED_BY_THREAD;
frame->visited = 0;
frame->lazy_coro = 0;
#ifdef Py_DEBUG
frame->stackpointer_valid = 1;
frame->lltrace = 0;
Expand Down
6 changes: 5 additions & 1 deletion Include/internal/pycore_interpframe_structs.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,18 @@ struct _PyInterpreterFrame {
char owner;
uint8_t visited:1;
uint8_t external:1; /* Set when the frame was pushed by something other than the VM */
uint8_t lazy_coro:2; /* LAZY_CORO_REQUESTED or LAZY_CORO_RUNNING */
#ifdef Py_DEBUG
uint8_t stackpointer_valid:1;
uint8_t lltrace:5;
uint8_t lltrace:3;
#endif
/* Locals and stack */
_PyStackRef localsplus[1];
};

#define LAZY_CORO_REQUESTED 1
#define LAZY_CORO_RUNNING 2


/* _PyGenObject_HEAD defines the initial segment of generator
and coroutine objects. */
Expand Down
3 changes: 2 additions & 1 deletion Include/internal/pycore_magic_number.h
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,7 @@ Known values:
Python 3.16a1 3706 (Add INTRINSIC_MATCH_CLASS_ISINSTANCE)
Python 3.16a1 3707 (GET_AWAITABLE also pushes NULL and None for SEND)
Python 3.16a1 3708 (GET_ANEXT also pushes NULL and None for SEND)
Python 3.16a1 3709 (Lazily materialize coroutine objects for awaited calls)

Python 3.17 will start with 3750

Expand All @@ -315,7 +316,7 @@ Known values:

*/

#define PYC_MAGIC_NUMBER 3708
#define PYC_MAGIC_NUMBER 3709
/* This is equivalent to converting PYC_MAGIC_NUMBER to 2 bytes
(little-endian) and then appending b'\r\n'. */
#define PYC_MAGIC_NUMBER_TOKEN \
Expand Down
Loading
Loading