Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 89 additions & 7 deletions Doc/c-api/init_config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1552,6 +1552,15 @@ PyConfig

.. versionadded:: 3.13

.. c:member:: int init_main

If set to ``0``, :c:func:`Py_InitializeFromConfig` stops at the "Core"
initialization phase. See the :c:func:`Py_InitializeMain` function.

Default: ``1``.

.. versionadded:: 3.15

.. c:member:: int isolated

If greater than ``0``, enable isolated mode:
Expand Down Expand Up @@ -2299,13 +2308,86 @@ Py_GetArgcArgv()

See also :c:member:`PyConfig.orig_argv` member.

Delaying main module execution

Multi-Phase Initialization API
==============================

In some embedding use cases, it may be desirable to separate interpreter initialization
from the execution of the main module.
This section is an API introducing multi-phase
initialization, the core feature of :pep:`432`:

* "Core" initialization phase, "bare minimum Python":

* Builtin types;
* Builtin exceptions;
* Builtin and frozen modules;
* The :mod:`sys` module is only partially initialized
(ex: :data:`sys.path` doesn't exist yet).

* "Main" initialization phase, Python is fully initialized:

* Install and configure :mod:`importlib`;
* Apply the :ref:`Path Configuration <init-path-config>`;
* Install signal handlers;
* Finish :mod:`sys` module initialization (ex: create :data:`sys.stdout`
and :data:`sys.path`);
* Enable optional features like :mod:`faulthandler` and :mod:`tracemalloc`;
* Import the :mod:`site` module;
* etc.

This separation can be achieved by setting ``PyConfig.run_command`` to the empty
string during initialization (to prevent the interpreter from dropping into the
interactive prompt), and then subsequently executing the desired main module
code using ``__main__.__dict__`` as the global namespace.
.. c:function:: PyStatus Py_InitializeMain(void)

Move to the "Main" initialization phase, finish the Python initialization.

See the :c:member:`PyConfig.init_main` member.

.. versionadded:: 3.15

No module is imported during the "Core" phase and the ``importlib`` module is
not configured: the :ref:`Path Configuration <init-path-config>` is only
applied during the "Main" phase. It may allow to customize Python in Python to
override or tune the :ref:`Path Configuration <init-path-config>`, maybe
install a custom :data:`sys.meta_path` importer or an import hook, etc.

It may become possible to calculate the :ref:`Path Configuration
<init-path-config>` in Python, after the Core phase and before the Main phase,
which is one of the :pep:`432` motivation.

The "Core" phase is not properly defined: what should be and what should
not be available at this phase is not specified yet.

Example running Python code between "Core" and "Main" initialization
phases::

void init_python(void)
{
PyStatus status;

PyConfig config;
PyConfig_InitPythonConfig(&config);
config.init_main = 0;

/* ... customize 'config' configuration ... */

status = Py_InitializeFromConfig(&config);
PyConfig_Clear(&config);
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}

/* Use sys.stderr because sys.stdout is only created
by Py_InitializeMain() */
int res = PyRun_SimpleString(
"import sys; "
"print('Run Python code before Py_InitializeMain', "
"file=sys.stderr)");
if (res < 0) {
exit(1);
}

/* ... put more configuration code here ... */

status = Py_InitializeMain();
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}
}
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.15.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1220,6 +1220,11 @@ C API changes
New features
------------

* Add :c:func:`Py_InitializeMain` function to move to the "Main" initialization
phase, finish the Python initialization. Rename also
:c:member:`!PyConfig._init_main` to :c:member:`PyConfig.init_main`.
(Contributed by Victor Stinner in :gh:`142417`.)

* Add :c:func:`PySys_GetAttr`, :c:func:`PySys_GetAttrString`,
:c:func:`PySys_GetOptionalAttr`, and :c:func:`PySys_GetOptionalAttrString`
functions as replacements for :c:func:`PySys_GetObject`.
Expand Down
6 changes: 3 additions & 3 deletions Include/cpython/initconfig.h
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,9 @@ typedef struct PyConfig {
int tlbc_enabled;
#endif

// If equal to 0, stop Python initialization before the "main" phase.
int init_main;

/* --- Path configuration inputs ------------ */
int pathconfig_warnings;
wchar_t *program_name;
Expand Down Expand Up @@ -224,9 +227,6 @@ typedef struct PyConfig {
// Needed by freeze_importlib.
int _install_importlib;

// If equal to 0, stop Python initialization before the "main" phase.
int _init_main;

// If non-zero, we believe we're running from a source tree.
int _is_python_build;

Expand Down
2 changes: 2 additions & 0 deletions Include/cpython/pylifecycle.h
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ PyAPI_FUNC(PyStatus) Py_PreInitializeFromArgs(
PyAPI_FUNC(PyStatus) Py_InitializeFromConfig(
const PyConfig *config);

PyAPI_FUNC(PyStatus) Py_InitializeMain(void);

PyAPI_FUNC(int) Py_RunMain(void);


Expand Down
20 changes: 19 additions & 1 deletion Lib/test/test_embed.py
Original file line number Diff line number Diff line change
Expand Up @@ -696,7 +696,7 @@ class InitConfigTests(EmbeddingTestsMixin, unittest.TestCase):
'_install_importlib': True,
'check_hash_pycs_mode': 'default',
'pathconfig_warnings': True,
'_init_main': True,
'init_main': True,
'use_frozen_modules': not support.Py_DEBUG,
'safe_path': False,
'_is_python_build': IGNORE_CONFIG,
Expand Down Expand Up @@ -1314,6 +1314,24 @@ def test_init_run_main(self):
}
self.check_all_configs("test_init_run_main", config, api=API_PYTHON)

def test_init_main(self):
code = ('import _testinternalcapi, json; '
'print(json.dumps(_testinternalcapi.get_configs()))')
config = {
'argv': ['-c', 'arg2'],
'orig_argv': ['python3',
'-c', code,
'arg2'],
'program_name': './python3',
'run_command': code + '\n',
'parse_argv': True,
'init_main': False,
'sys_path_0': '',
}
self.check_all_configs("test_init_main", config,
api=API_PYTHON,
stderr="Run Python code before Py_InitializeMain")

def test_init_parse_argv(self):
config = {
'parse_argv': True,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Add :c:func:`Py_InitializeMain` function to move to the "Main"
initialization phase, finish the Python initialization. Rename also
:c:member:`!PyConfig._init_main` to :c:member:`PyConfig.init_main`. Patch by
Victor Stinner.
2 changes: 1 addition & 1 deletion Programs/_freeze_module.c
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ runtime_init(void)
if (PyInitConfig_SetInt(config, "_install_importlib", 0) < 0) {
goto error;
}
if (PyInitConfig_SetInt(config, "_init_main", 0) < 0) {
if (PyInitConfig_SetInt(config, "init_main", 0) < 0) {
goto error;
}

Expand Down
29 changes: 29 additions & 0 deletions Programs/_testembed.c
Original file line number Diff line number Diff line change
Expand Up @@ -1989,6 +1989,34 @@ static int test_init_run_main(void)
}


static int test_init_main(void)
{
PyConfig config;
PyConfig_InitPythonConfig(&config);

configure_init_main(&config);
config.init_main = 0;
init_from_config_clear(&config);

/* Use stderr: sys.stdout doesn't exist yet,
* it is created by Py_InitializeMain() */
int res = PyRun_SimpleString(
"import sys; "
"print('Run Python code before Py_InitializeMain', "
"file=sys.stderr)");
if (res < 0) {
exit(1);
}

PyStatus status = Py_InitializeMain();
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}

return Py_RunMain();
}


static int test_run_main(void)
{
PyConfig config;
Expand Down Expand Up @@ -2647,6 +2675,7 @@ static struct TestCase TestCases[] = {
{"test_preinit_parse_argv", test_preinit_parse_argv},
{"test_preinit_dont_parse_argv", test_preinit_dont_parse_argv},
{"test_init_run_main", test_init_run_main},
{"test_init_main", test_init_main},
{"test_init_sys_add", test_init_sys_add},
{"test_init_setpath", test_init_setpath},
{"test_init_setpath_config", test_init_setpath_config},
Expand Down
4 changes: 2 additions & 2 deletions Python/initconfig.c
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ static const PyConfigSpec PYCONFIG_SPEC[] = {
// --- Init-only options -----------

SPEC(_config_init, UINT, INIT_ONLY, NO_SYS),
SPEC(_init_main, BOOL, INIT_ONLY, NO_SYS),
SPEC(init_main, BOOL, INIT_ONLY, NO_SYS),
SPEC(_install_importlib, BOOL, INIT_ONLY, NO_SYS),
SPEC(_is_python_build, BOOL, INIT_ONLY, NO_SYS),
SPEC(module_search_paths_set, BOOL, INIT_ONLY, NO_SYS),
Expand Down Expand Up @@ -1036,7 +1036,7 @@ _PyConfig_InitCompatConfig(PyConfig *config)
config->_install_importlib = 1;
config->check_hash_pycs_mode = NULL;
config->pathconfig_warnings = -1;
config->_init_main = 1;
config->init_main = 1;
#ifdef MS_WINDOWS
config->legacy_windows_stdio = -1;
#endif
Expand Down
14 changes: 13 additions & 1 deletion Python/pylifecycle.c
Original file line number Diff line number Diff line change
Expand Up @@ -1462,7 +1462,7 @@ Py_InitializeFromConfig(const PyConfig *config)
}
config = _PyInterpreterState_GetConfig(tstate->interp);

if (config->_init_main) {
if (config->init_main) {
status = pyinit_main(tstate);
if (_PyStatus_EXCEPTION(status)) {
return status;
Expand Down Expand Up @@ -1508,6 +1508,18 @@ Py_Initialize(void)
}


PyStatus
Py_InitializeMain(void)
{
PyStatus status = _PyRuntime_Initialize();
if (_PyStatus_EXCEPTION(status)) {
return status;
}
PyThreadState *tstate = _PyThreadState_GET();
return pyinit_main(tstate);
}


static void
finalize_modules_delete_special(PyThreadState *tstate, int verbose)
{
Expand Down
Loading