Python Initialization Configuration¶
New in version 3.8.
Structures:
Functions:
The preconfiguration (PyPreConfig
type) is stored in
_PyRuntime.preconfig
and the configuration (PyConfig
type) is stored in
PyInterpreterState.config
.
See also Initialization, Finalization, and Threads.
See also
PEP 587 “Python Initialization Configuration”.
PyWideStringList¶
-
PyWideStringList
¶ List of
wchar_t*
strings.If length is non-zero, items must be non-
NULL
and all strings must be non-NULL
.Methods:
-
PyStatus
PyWideStringList_Append
(PyWideStringList *list, const wchar_t *item)¶ Append item to list.
Python must be preinitialized to call this function.
-
PyStatus
PyWideStringList_Insert
(PyWideStringList *list, Py_ssize_t index, const wchar_t *item)¶ Insert item into list at index.
If index is greater than or equal to list length, append item to list.
index must be greater than or equal to 0.
Python must be preinitialized to call this function.
Structure fields:
-
Py_ssize_t
length
¶ List length.
-
wchar_t**
items
¶ List items.
-
PyStatus
PyStatus¶
-
PyStatus
¶ Structure to store an initialization function status: success, error or exit.
For an error, it can store the C function name which created the error.
Structure fields:
-
int
exitcode
¶ Exit code. Argument passed to
exit()
.
-
const char *
err_msg
¶ Error message.
-
const char *
func
¶ Name of the function which created an error, can be
NULL
.
Functions to create a status:
Functions to handle a status:
-
int
PyStatus_Exception
(PyStatus status)¶ Is the status an error or an exit? If true, the exception must be handled; by calling
Py_ExitStatusException()
for example.
-
int
Note
Internally, Python uses macros which set PyStatus.func
,
whereas functions to create a status set func
to NULL
.
Example:
PyStatus alloc(void **ptr, size_t size)
{
*ptr = PyMem_RawMalloc(size);
if (*ptr == NULL) {
return PyStatus_NoMemory();
}
return PyStatus_Ok();
}
int main(int argc, char **argv)
{
void *ptr;
PyStatus status = alloc(&ptr, 16);
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}
PyMem_Free(ptr);
return 0;
}
PyPreConfig¶
-
PyPreConfig
¶ Structure used to preinitialize Python:
Set the Python memory allocator
Configure the LC_CTYPE locale
Set the UTF-8 mode
Function to initialize a preconfiguration:
-
void
PyPreConfig_InitPythonConfig
(PyPreConfig *preconfig)¶ Initialize the preconfiguration with Python Configuration.
-
void
PyPreConfig_InitIsolatedConfig
(PyPreConfig *preconfig)¶ Initialize the preconfiguration with Isolated Configuration.
Structure fields:
-
int
allocator
¶ Name of the memory allocator:
PYMEM_ALLOCATOR_NOT_SET
(0
): don’t change memory allocators (use defaults)PYMEM_ALLOCATOR_DEFAULT
(1
): default memory allocatorsPYMEM_ALLOCATOR_DEBUG
(2
): default memory allocators with debug hooksPYMEM_ALLOCATOR_MALLOC
(3
): force usage ofmalloc()
PYMEM_ALLOCATOR_MALLOC_DEBUG
(4
): force usage ofmalloc()
with debug hooksPYMEM_ALLOCATOR_PYMALLOC
(5
): Python pymalloc memory allocatorPYMEM_ALLOCATOR_PYMALLOC_DEBUG
(6
): Python pymalloc memory allocator with debug hooks
PYMEM_ALLOCATOR_PYMALLOC
andPYMEM_ALLOCATOR_PYMALLOC_DEBUG
are not supported if Python is configured using--without-pymalloc
See Memory Management.
-
int
configure_locale
¶ Set the LC_CTYPE locale to the user preferred locale? If equals to 0, set
coerce_c_locale
andcoerce_c_locale_warn
to 0.
-
int
coerce_c_locale
¶ If equals to 2, coerce the C locale; if equals to 1, read the LC_CTYPE locale to decide if it should be coerced.
-
int
coerce_c_locale_warn
¶ If non-zero, emit a warning if the C locale is coerced.
-
int
dev_mode
¶ See
PyConfig.dev_mode
.
-
int
isolated
¶ See
PyConfig.isolated
.
-
int
legacy_windows_fs_encoding
(Windows only)¶ If non-zero, disable UTF-8 Mode, set the Python filesystem encoding to
mbcs
, set the filesystem error handler toreplace
.Only available on Windows.
#ifdef MS_WINDOWS
macro can be used for Windows specific code.
-
int
parse_argv
¶ If non-zero,
Py_PreInitializeFromArgs()
andPy_PreInitializeFromBytesArgs()
parse theirargv
argument the same way the regular Python parses command line arguments: see Command Line Arguments.
-
int
use_environment
¶
-
int
utf8_mode
¶ If non-zero, enable the UTF-8 mode.
Preinitialization with PyPreConfig¶
Functions to preinitialize Python:
-
PyStatus
Py_PreInitialize
(const PyPreConfig *preconfig)¶ Preinitialize Python from preconfig preconfiguration.
-
PyStatus
Py_PreInitializeFromBytesArgs
(const PyPreConfig *preconfig, int argc, char * const *argv)¶ Preinitialize Python from preconfig preconfiguration and command line arguments (bytes strings).
-
PyStatus
Py_PreInitializeFromArgs
(const PyPreConfig *preconfig, int argc, wchar_t * const * argv)¶ Preinitialize Python from preconfig preconfiguration and command line arguments (wide strings).
The caller is responsible to handle exceptions (error or exit) using
PyStatus_Exception()
and Py_ExitStatusException()
.
For Python Configuration
(PyPreConfig_InitPythonConfig()
), if Python is initialized with
command line arguments, the command line arguments must also be passed to
preinitialize Python, since they have an effect on the pre-configuration
like encodings. For example, the -X utf8
command line option
enables the UTF-8 Mode.
PyMem_SetAllocator()
can be called after Py_PreInitialize()
and
before Py_InitializeFromConfig()
to install a custom memory allocator.
It can be called before Py_PreInitialize()
if
PyPreConfig.allocator
is set to PYMEM_ALLOCATOR_NOT_SET
.
Python memory allocation functions like PyMem_RawMalloc()
must not be
used before Python preinitialization, whereas calling directly malloc()
and
free()
is always safe. Py_DecodeLocale()
must not be called before
the preinitialization.
Example using the preinitialization to enable the UTF-8 Mode:
PyStatus status;
PyPreConfig preconfig;
PyPreConfig_InitPythonConfig(&preconfig);
preconfig.utf8_mode = 1;
status = Py_PreInitialize(&preconfig);
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}
/* at this point, Python will speak UTF-8 */
Py_Initialize();
/* ... use Python API here ... */
Py_Finalize();
PyConfig¶
-
PyConfig
¶ Structure containing most parameters to configure Python.
Structure methods:
-
void
PyConfig_InitPythonConfig
(PyConfig *config)¶ Initialize configuration with Python Configuration.
-
void
PyConfig_InitIsolatedConfig
(PyConfig *config)¶ Initialize configuration with Isolated Configuration.
-
PyStatus
PyConfig_SetString
(PyConfig *config, wchar_t * const *config_str, const wchar_t *str)¶ Copy the wide character string str into
*config_str
.Preinitialize Python if needed.
-
PyStatus
PyConfig_SetBytesString
(PyConfig *config, wchar_t * const *config_str, const char *str)¶ Decode str using
Py_DecodeLocale()
and set the result into*config_str
.Preinitialize Python if needed.
-
PyStatus
PyConfig_SetArgv
(PyConfig *config, int argc, wchar_t * const *argv)¶ Set command line arguments from wide character strings.
Preinitialize Python if needed.
-
PyStatus
PyConfig_SetBytesArgv
(PyConfig *config, int argc, char * const *argv)¶ Set command line arguments: decode bytes using
Py_DecodeLocale()
.Preinitialize Python if needed.
-
PyStatus
PyConfig_SetWideStringList
(PyConfig *config, PyWideStringList *list, Py_ssize_t length, wchar_t **items)¶ Set the list of wide strings list to length and items.
Preinitialize Python if needed.
-
PyStatus
PyConfig_Read
(PyConfig *config)¶ Read all Python configuration.
Fields which are already initialized are left unchanged.
Preinitialize Python if needed.
Most
PyConfig
methods preinitialize Python if needed. In that case, the Python preinitialization configuration in based on thePyConfig
. If configuration fields which are in common withPyPreConfig
are tuned, they must be set before calling aPyConfig
method:Moreover, if
PyConfig_SetArgv()
orPyConfig_SetBytesArgv()
is used, this method must be called first, before other methods, since the preinitialization configuration depends on command line arguments (ifparse_argv
is non-zero).The caller of these methods is responsible to handle exceptions (error or exit) using
PyStatus_Exception()
andPy_ExitStatusException()
.Structure fields:
-
PyWideStringList
argv
¶ Command line arguments,
sys.argv
. Seeparse_argv
to parseargv
the same way the regular Python parses Python command line arguments. Ifargv
is empty, an empty string is added to ensure thatsys.argv
always exists and is never empty.
-
wchar_t*
base_exec_prefix
¶
-
wchar_t*
base_executable
¶ sys._base_executable
:__PYVENV_LAUNCHER__
environment variable value, or copy ofPyConfig.executable
.
-
wchar_t*
base_prefix
¶
-
wchar_t*
platlibdir
¶ sys.platlibdir
: platform library directory name, set at configure time by--with-platlibdir
, overrideable by thePYTHONPLATLIBDIR
environment variable.New in version 3.9.
-
int
buffered_stdio
¶ If equals to 0, enable unbuffered mode, making the stdout and stderr streams unbuffered.
stdin is always opened in buffered mode.
-
int
bytes_warning
¶ If equals to 1, issue a warning when comparing
bytes
orbytearray
withstr
, or comparingbytes
withint
. If equal or greater to 2, raise aBytesWarning
exception.
-
wchar_t*
check_hash_pycs_mode
¶ Control the validation behavior of hash-based
.pyc
files (see PEP 552):--check-hash-based-pycs
command line option value.Valid values:
always
,never
anddefault
.The default value is:
default
.
-
int
configure_c_stdio
¶ If non-zero, configure C standard streams (
stdio
,stdout
,stdout
). For example, set their mode toO_BINARY
on Windows.
-
int
dev_mode
¶ If non-zero, enable the Python Development Mode.
-
int
dump_refs
¶ If non-zero, dump all objects which are still alive at exit.
Py_TRACE_REFS
macro must be defined in build.
-
wchar_t*
exec_prefix
¶
-
wchar_t*
executable
¶
-
int
faulthandler
¶ If non-zero, call
faulthandler.enable()
at startup.
-
wchar_t*
filesystem_encoding
¶ Filesystem encoding,
sys.getfilesystemencoding()
.
-
wchar_t*
filesystem_errors
¶ Filesystem encoding errors,
sys.getfilesystemencodeerrors()
.
-
unsigned long
hash_seed
¶
-
int
use_hash_seed
¶ Randomized hash function seed.
If
use_hash_seed
is zero, a seed is chosen randomly at Pythonstartup, andhash_seed
is ignored.
-
wchar_t*
home
¶ Python home directory.
Initialized from
PYTHONHOME
environment variable value by default.
-
int
import_time
¶ If non-zero, profile import time.
-
int
inspect
¶ Enter interactive mode after executing a script or a command.
-
int
install_signal_handlers
¶ Install signal handlers?
-
int
interactive
¶ Interactive mode.
-
int
isolated
¶ If greater than 0, enable isolated mode:
sys.path
contains neither the script’s directory (computed fromargv[0]
or the current directory) nor the user’s site-packages directory.Python REPL doesn’t import
readline
nor enable default readline configuration on interactive prompts.Set
use_environment
anduser_site_directory
to 0.
-
int
legacy_windows_stdio
¶ If non-zero, use
io.FileIO
instead ofio.WindowsConsoleIO
forsys.stdin
,sys.stdout
andsys.stderr
.Only available on Windows.
#ifdef MS_WINDOWS
macro can be used for Windows specific code.
-
int
malloc_stats
¶ If non-zero, dump statistics on Python pymalloc memory allocator at exit.
The option is ignored if Python is built using
--without-pymalloc
.
-
wchar_t*
pythonpath_env
¶ Module search paths as a string separated by
DELIM
(os.path.pathsep
).Initialized from
PYTHONPATH
environment variable value by default.
-
PyWideStringList
module_search_paths
¶
-
int
module_search_paths_set
¶ sys.path
. Ifmodule_search_paths_set
is equal to 0, themodule_search_paths
is overridden by the function calculating the Path Configuration.
-
int
optimization_level
¶ Compilation optimization level:
0: Peephole optimizer (and
__debug__
is set toTrue
)1: Remove assertions, set
__debug__
toFalse
2: Strip docstrings
-
int
parse_argv
¶ If non-zero, parse
argv
the same way the regular Python command line arguments, and strip Python arguments fromargv
: see Command Line Arguments.
-
int
parser_debug
¶ If non-zero, turn on parser debugging output (for expert only, depending on compilation options).
-
int
pathconfig_warnings
¶ If equal to 0, suppress warnings when calculating the Path Configuration (Unix only, Windows does not log any warning). Otherwise, warnings are written into
stderr
.
-
wchar_t*
prefix
¶
-
wchar_t*
program_name
¶ Program name. Used to initialize
executable
, and in early error messages.
-
wchar_t*
pycache_prefix
¶ sys.pycache_prefix
:.pyc
cache prefix.If
NULL
,sys.pycache_prefix
is set toNone
.
-
int
quiet
¶ Quiet mode. For example, don’t display the copyright and version messages in interactive mode.
-
wchar_t*
run_command
¶ python3 -c COMMAND
argument. Used byPy_RunMain()
.
-
wchar_t*
run_filename
¶ python3 FILENAME
argument. Used byPy_RunMain()
.
-
wchar_t*
run_module
¶ python3 -m MODULE
argument. Used byPy_RunMain()
.
-
int
show_ref_count
¶ Show total reference count at exit?
Set to 1 by
-X showrefcount
command line option.Need a debug build of Python (
Py_REF_DEBUG
macro must be defined).
-
int
skip_source_first_line
¶ Skip the first line of the source?
-
wchar_t*
stdio_encoding
¶
-
wchar_t*
stdio_errors
¶ Encoding and encoding errors of
sys.stdin
,sys.stdout
andsys.stderr
.
-
int
tracemalloc
¶ If non-zero, call
tracemalloc.start()
at startup.
-
int
use_environment
¶ If greater than 0, use environment variables.
-
int
verbose
¶ If non-zero, enable verbose mode.
-
PyWideStringList
warnoptions
¶ sys.warnoptions
: options of thewarnings
module to build warnings filters: lowest to highest priority.The
warnings
module addssys.warnoptions
in the reverse order: the lastPyConfig.warnoptions
item becomes the first item ofwarnings.filters
which is checked first (highest priority).
-
int
write_bytecode
¶ If non-zero, write
.pyc
files.sys.dont_write_bytecode
is initialized to the inverted value ofwrite_bytecode
.
-
PyWideStringList
xoptions
¶
-
int
_use_peg_parser
¶ Enable PEG parser? Default: 1.
Set to 0 by
-X oldparser
andPYTHONOLDPARSER
.See also PEP 617.
Deprecated since version 3.9, will be removed in version 3.10.
-
void
If parse_argv
is non-zero, argv
arguments are parsed the same
way the regular Python parses command line arguments, and Python
arguments are stripped from argv
: see Command Line Arguments.
The xoptions
options are parsed to set other options: see -X
option.
Changed in version 3.9: The show_alloc_count
field has been removed.
Initialization with PyConfig¶
Function to initialize Python:
-
PyStatus
Py_InitializeFromConfig
(const PyConfig *config)¶ Initialize Python from config configuration.
The caller is responsible to handle exceptions (error or exit) using
PyStatus_Exception()
and Py_ExitStatusException()
.
If PyImport_FrozenModules
, PyImport_AppendInittab()
or
PyImport_ExtendInittab()
are used, they must be set or called after Python
preinitialization and before the Python initialization.
Example setting the program name:
void init_python(void)
{
PyStatus status;
PyConfig config;
PyConfig_InitPythonConfig(&config);
/* Set the program name. Implicitly preinitialize Python. */
status = PyConfig_SetString(&config, &config.program_name,
L"/path/to/my_program");
if (PyStatus_Exception(status)) {
goto fail;
}
status = Py_InitializeFromConfig(&config);
if (PyStatus_Exception(status)) {
goto fail;
}
PyConfig_Clear(&config);
return;
fail:
PyConfig_Clear(&config);
Py_ExitStatusException(status);
}
More complete example modifying the default configuration, read the configuration, and then override some parameters:
PyStatus init_python(const char *program_name)
{
PyStatus status;
PyConfig config;
PyConfig_InitPythonConfig(&config);
/* Set the program name before reading the configuration
(decode byte string from the locale encoding).
Implicitly preinitialize Python. */
status = PyConfig_SetBytesString(&config, &config.program_name,
program_name);
if (PyStatus_Exception(status)) {
goto done;
}
/* Read all configuration at once */
status = PyConfig_Read(&config);
if (PyStatus_Exception(status)) {
goto done;
}
/* Append our custom search path to sys.path */
status = PyWideStringList_Append(&config.module_search_paths,
L"/path/to/more/modules");
if (PyStatus_Exception(status)) {
goto done;
}
/* Override executable computed by PyConfig_Read() */
status = PyConfig_SetString(&config, &config.executable,
L"/path/to/my_executable");
if (PyStatus_Exception(status)) {
goto done;
}
status = Py_InitializeFromConfig(&config);
done:
PyConfig_Clear(&config);
return status;
}
Isolated Configuration¶
PyPreConfig_InitIsolatedConfig()
and
PyConfig_InitIsolatedConfig()
functions create a configuration to
isolate Python from the system. For example, to embed Python into an
application.
This configuration ignores global configuration variables, environments
variables, command line arguments (PyConfig.argv
is not parsed)
and user site directory. The C standard streams (ex: stdout
) and the
LC_CTYPE locale are left unchanged. Signal handlers are not installed.
Configuration files are still used with this configuration. Set the Path Configuration (“output fields”) to ignore these configuration files and avoid the function computing the default path configuration.
Python Configuration¶
PyPreConfig_InitPythonConfig()
and PyConfig_InitPythonConfig()
functions create a configuration to build a customized Python which behaves as
the regular Python.
Environments variables and command line arguments are used to configure Python, whereas global configuration variables are ignored.
This function enables C locale coercion (PEP 538) and UTF-8 Mode
(PEP 540) depending on the LC_CTYPE locale, PYTHONUTF8
and
PYTHONCOERCECLOCALE
environment variables.
Example of customized Python always running in isolated mode:
int main(int argc, char **argv)
{
PyStatus status;
PyConfig config;
PyConfig_InitPythonConfig(&config);
config.isolated = 1;
/* Decode command line arguments.
Implicitly preinitialize Python (in isolated mode). */
status = PyConfig_SetBytesArgv(&config, argc, argv);
if (PyStatus_Exception(status)) {
goto fail;
}
status = Py_InitializeFromConfig(&config);
if (PyStatus_Exception(status)) {
goto fail;
}
PyConfig_Clear(&config);
return Py_RunMain();
fail:
PyConfig_Clear(&config);
if (PyStatus_IsExit(status)) {
return status.exitcode;
}
/* Display the error message and exit the process with
non-zero exit code */
Py_ExitStatusException(status);
}
Path Configuration¶
PyConfig
contains multiple fields for the path configuration:
Path configuration inputs:
current working directory: to get absolute paths
PATH
environment variable to get the program full path (fromPyConfig.program_name
)__PYVENV_LAUNCHER__
environment variable(Windows only) Application paths in the registry under “SoftwarePythonPythonCoreX.YPythonPath” of HKEY_CURRENT_USER and HKEY_LOCAL_MACHINE (where X.Y is the Python version).
Path configuration output fields:
If at least one “output field” is not set, Python calculates the path
configuration to fill unset fields. If
module_search_paths_set
is equal to 0,
module_search_paths
is overridden and
module_search_paths_set
is set to 1.
It is possible to completely ignore the function calculating the default
path configuration by setting explicitly all path configuration output
fields listed above. A string is considered as set even if it is non-empty.
module_search_paths
is considered as set if
module_search_paths_set
is set to 1. In this case, path
configuration input fields are ignored as well.
Set pathconfig_warnings
to 0 to suppress warnings when
calculating the path configuration (Unix only, Windows does not log any warning).
If base_prefix
or base_exec_prefix
fields are not set, they inherit their value from prefix
and exec_prefix
respectively.
Py_RunMain()
and Py_Main()
modify sys.path
:
If
run_filename
is set and is a directory which contains a__main__.py
script, prependrun_filename
tosys.path
.If
isolated
is zero:If
run_module
is set, prepend the current directory tosys.path
. Do nothing if the current directory cannot be read.If
run_filename
is set, prepend the directory of the filename tosys.path
.Otherwise, prepend an empty string to
sys.path
.
If site_import
is non-zero, sys.path
can be
modified by the site
module. If
user_site_directory
is non-zero and the user’s
site-package directory exists, the site
module appends the user’s
site-package directory to sys.path
.
The following configuration files are used by the path configuration:
pyvenv.cfg
python._pth
(Windows only)pybuilddir.txt
(Unix only)
The __PYVENV_LAUNCHER__
environment variable is used to set
PyConfig.base_executable
Py_RunMain()¶
-
int
Py_RunMain
(void)¶ Execute the command (
PyConfig.run_command
), the script (PyConfig.run_filename
) or the module (PyConfig.run_module
) specified on the command line or in the configuration.By default and when if
-i
option is used, run the REPL.Finally, finalizes Python and returns an exit status that can be passed to the
exit()
function.
See Python Configuration for an example of
customized Python always running in isolated mode using
Py_RunMain()
.
Py_GetArgcArgv()¶
-
void
Py_GetArgcArgv
(int *argc, wchar_t ***argv)¶ Get the original command line arguments, before Python modified them.
Multi-Phase Initialization Private Provisional API¶
This section is a private provisional API introducing multi-phase initialization, the core feature of the PEP 432:
“Core” initialization phase, “bare minimum Python”:
“Main” initialization phase, Python is fully initialized:
Install and configure
importlib
;Apply the Path Configuration;
Install signal handlers;
Finish
sys
module initialization (ex: createsys.stdout
andsys.path
);Enable optional features like
faulthandler
andtracemalloc
;Import the
site
module;etc.
Private provisional API:
PyConfig._init_main
: if set to 0,Py_InitializeFromConfig()
stops at the “Core” initialization phase.PyConfig._isolated_interpreter
: if non-zero, disallow threads, subprocesses and fork.
-
PyStatus
_Py_InitializeMain
(void)¶ Move to the “Main” initialization phase, finish the Python initialization.
No module is imported during the “Core” phase and the importlib
module is
not configured: the Path Configuration is only
applied during the “Main” phase. It may allow to customize Python in Python to
override or tune the Path Configuration, maybe
install a custom sys.meta_path
importer or an import hook, etc.
It may become possible to calculatin the Path Configuration in Python, after the Core phase and before the Main phase, which is one of the PEP 432 motivation.
The “Core” phase is not properly defined: what should be and what should not be available at this phase is not specified yet. The API is marked as private and provisional: the API can be modified or even be removed anytime until a proper public API is designed.
Example running Python code between “Core” and “Main” initialization phases:
void init_python(void)
{
PyStatus status;
PyConfig config;
PyConfig_InitPythonConfig(&config);
config._init_main = 0;
/* ... customize 'config' configuration ... */
status = Py_InitializeFromConfig(&config);
PyConfig_Clear(&config);
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}
/* Use sys.stderr because sys.stdout is only created
by _Py_InitializeMain() */
int res = PyRun_SimpleString(
"import sys; "
"print('Run Python code before _Py_InitializeMain', "
"file=sys.stderr)");
if (res < 0) {
exit(1);
}
/* ... put more configuration code here ... */
status = _Py_InitializeMain();
if (PyStatus_Exception(status)) {
Py_ExitStatusException(status);
}
}