- Update patch fix_configure_rst.patch
- Update to 3.12.1 (CVE-2023-6507, bsc#1217939):
- Core and Builtins
- gh-112125: Fix None.__ne__(None) returning NotImplemented
instead of False
- gh-112625: Fixes a bug where a bytearray object could be
cleared while iterating over an argument in the
bytearray.join() method that could result in reading memory
after it was freed.
- gh-105967: Workaround a bug in Apple’s macOS platform zlib
library where zlib.crc32() and binascii.crc32() could produce
incorrect results on multi-gigabyte inputs. Including when
using zipfile on zips containing large data.
- gh-112356: Stopped erroneously deleting a LOAD_NULL bytecode
instruction when optimized twice.
- gh-111058: Change coro.cr_frame/gen.gi_frame to return None
after the coroutine/generator has been closed. This fixes a bug
where getcoroutinestate() and getgeneratorstate() return the
wrong state for a closed coroutine/generator.
- gh-112388: Fix an error that was causing the parser to try to
overwrite tokenizer errors. Patch by pablo Galindo
- gh-112387: Fix error positions for decoded strings with
backwards tokenize errors. Patch by Pablo Galindo
- gh-112367: Avoid undefined behaviour when using the perf
trampolines by not freeing the code arenas until shutdown.
Patch by Pablo Galindo
- gh-112243: Don’t include comments in f-string debug
expressions. Patch by Pablo Galindo
- gh-112266: Change docstrings of __dict__ and __weakref__.
- gh-111654: Fix runtime crash when some error happens in opcode
LOAD_FROM_DICT_OR_DEREF.
- gh-109181: Speed up Traceback object creation by lazily compute
the line number. Patch by Pablo Galindo
- gh-102388: Fix a bug where iso2022_jp_3 and iso2022_jp_2004
codecs read out of bounds
- gh-111366: Fix an issue in the codeop that was causing
SyntaxError exceptions raised in the presence of invalid syntax
to not contain precise error messages. Patch by Pablo Galindo
- gh-111380: Fix a bug that was causing SyntaxWarning to appear
twice when parsing if invalid syntax is encountered later.
Patch by Pablo galindo
- gh-94438: Fix a regression that prevented jumping across is
None and is not None when debugging. Patch by Savannah
Ostrowski.
- gh-110938: Fix error messages for indented blocks with
functions and classes with generic type parameters. Patch by
Pablo Galindo
- gh-109894: Fixed crash due to improperly initialized static
MemoryError in subinterpreter.
- gh-110782: Fix crash when typing.TypeVar is constructed with a
keyword argument. Patch by Jelle Zijlstra.
- gh-110696: Fix incorrect error message for invalid argument
unpacking. Patch by Pablo Galindo
- gh-110543: Fix regression in Python 3.12 where
types.CodeType.replace() would produce a broken code object if
called on a module or class code object that contains a
comprehension. Patch by Jelle Zijlstra.
- gh-110514: Add PY_THROW to sys.setprofile() events
- gh-110455: Guard assert(tstate->thread_id > 0) with #ifndef
HAVE_PTHREAD_STUBS. This allows for for pydebug builds to work
under WASI which (currently) lacks thread support.
- gh-110259: Correctly identify the format spec in f-strings
(with single or triple quotes) that have multiple lines in the
expression part and include a formatting spec. Patch by Pablo
Galindo
- gh-110237: Fix missing error checks for calls to PyList_Append
in _PyEval_MatchClass.
- gh-109889: Fix the compiler’s redundant NOP detection algorithm
to skip over NOPs with no line number when looking for the next
instruction’s lineno.
- gh-109853: sys.path[0] is now set correctly for
subinterpreters.
- gh-105716: Subinterpreters now correctly handle the case where
they have threads running in the background. Before, such
threads would interfere with cleaning up and destroying them,
as well as prevent running another script.
- gh-109793: The main thread no longer exits prematurely when a
subinterpreter is cleaned up during runtime finalization. The
bug was a problem particularly because, when triggered, the
Python process would always return with a 0 exitcode, even if
it failed.
- gh-109596: Fix some tokens in the grammar that were incorrectly
marked as soft keywords. Also fix some repeated rule names and
ensure that repeated rules are not allowed. Patch by Pablo
Galindo
- gh-109351: Fix crash when compiling an invalid AST involving a
named (walrus) expression.
- gh-109216: Fix possible memory leak in BUILD_MAP.
- gh-109207: Fix a SystemError in __repr__ of symtable entry
object.
- gh-109179: Fix bug where the C traceback display drops notes
from SyntaxError.
- gh-109052: Use the base opcode when comparing code objects to
avoid interference from instrumentation
- gh-88943: Improve syntax error for non-ASCII character that
follows a numerical literal. It now points on the invalid
non-ASCII character, not on the valid numerical literal.
- gh-106931: Statically allocated string objects are now interned
globally instead of per-interpreter. This fixes a situation
where such a string would only be interned in a single
interpreter. Normal string objects are unaffected.
- Library
- gh-79325: Fix an infinite recursion error in
tempfile.TemporaryDirectory() cleanup on Windows.
- gh-112645: Remove deprecation error on passing onerror to
shutil.rmtree().
- gh-112618: Fix a caching bug relating to typing.Annotated.
Annotated[str, True] is no longer identical to Annotated[str,
1].
- gh-112334: Fixed a performance regression in 3.12’s subprocess
on Linux where it would no longer use the fast-path vfork()
system call when it should have due to a logic bug, instead
always falling back to the safe but slower fork().
- Also fixed a related 3.12 security regression: If a value of
extra_groups=[] was passed to subprocess.Popen or related APIs,
the underlying setgroups(0, NULL) system call to clear the
groups list would not be made in the child process prior to
exec(). This has been assigned CVE-2023-6507.
- This was identified via code inspection in the process of fixing
the first bug.
- gh-110190: Fix ctypes structs with array on Arm platform by
setting MAX_STRUCT_SIZE to 32 in stgdict. Patch by Diego Russo.
- gh-112578: Fix a spurious RuntimeWarning when executing the
zipfile module.
- gh-112509: Fix edge cases that could cause a key to be present
in both the __required_keys__ and __optional_keys__ attributes
of a typing.TypedDict. Patch by Jelle Zijlstra.
- gh-112414: Fix regression in Python 3.12 where calling repr() on
a module that had been imported using a custom loader could fail
with AttributeError. Patch by Alex Waygood.
- gh-112358: Revert change to struct.Struct initialization that
broke some cases of subclassing.
- gh-94722: Fix bug where comparison between instances of DocTest
fails if one of them has None as its lineno.
- gh-112105: Make readline.set_completer_delims() work with
libedit
- gh-111942: Fix SystemError in the TextIOWrapper constructor with
non-encodable “errors” argument in non-debug mode.
- gh-109538: Issue warning message instead of having RuntimeError
be displayed when event loop has already been closed at
StreamWriter.__del__().
- gh-111942: Fix crashes in io.TextIOWrapper.reconfigure() when
pass invalid arguments, e.g. non-string encoding.
- gh-111460: curses: restore wide character support (including
curses.unget_wch() and get_wch()) on macOS, which was
unavailable due to a regression in Python 3.12.
- gh-103791: contextlib.suppress now supports suppressing
exceptions raised as part of a BaseExceptionGroup, in addition
to the recent support for ExceptionGroup.
- gh-111804: Remove posix.fallocate() under WASI as the underlying
posix_fallocate() is not available in WASI preview2.
- gh-111841: Fix truncating arguments on an embedded null
character in os.putenv() and os.unsetenv() on Windows.
- gh-111541: Fix doctest for SyntaxError not-builtin subclasses.
- gh-110894: Call loop exception handler for exceptions in
client_connected_cb of asyncio.start_server() so that
applications can handle it. Patch by Kumar Aditya.
- gh-111531: Fix reference leaks in bind_class() and bind_all()
methods of tkinter widgets.
- gh-111356: Added io.text_encoding(), io.DEFAULT_BUFFER_SIZE, and
io.IncrementalNewlineDecoder to io.__all__.
- gh-111342: Fixed typo in math.sumprod().
- gh-68166: Remove mention of not supported “vsapi” element type
in tkinter.ttk.Style.element_create(). Add tests for
element_create() and other ttk.Style methods. Add examples for
element_create() in the documentation.
- gh-75666: Fix the behavior of tkinter widget’s unbind() method
with two arguments. Previously, widget.unbind(sequence, funcid)
destroyed the current binding for sequence, leaving sequence
unbound, and deleted the funcid command. Now it removes only
funcid from the binding for sequence, keeping other commands,
and deletes the funcid command. It leaves sequence unbound only
if funcid was the last bound command.
- gh-79033: Another attempt at fixing
asyncio.Server.wait_closed(). It now blocks until both
conditions are true: the server is closed, and there are no more
active connections. (This means that in some cases where in
3.12.0 this function would incorrectly have returned
immediately, it will now block; in particular, when there are no
active connections but the server hasn’t been closed yet.)
- gh-111295: Fix time not checking for errors when initializing.
- gh-111253: Add error checking during _socket module init.
- gh-111251: Fix _blake2 not checking for errors when
initializing.
- gh-111174: Fix crash in io.BytesIO.getbuffer() called repeatedly
for empty BytesIO.
- gh-111187: Postpone removal version for
locale.getdefaultlocale() to Python 3.15.
- gh-111159: Fix doctest output comparison for exceptions with
notes.
- gh-110910: Fix invalid state handling in asyncio.TaskGroup and
asyncio.Timeout. They now raise proper RuntimeError if they are
improperly used and are left in consistent state after this.
- gh-111092: Make turtledemo run without default root enabled.
- gh-110488: Fix a couple of issues in
pathlib.PurePath.with_name(): a single dot was incorrectly
considered a valid name, and in PureWindowsPath, a name with an
NTFS alternate data stream, like a:b, was incorrectly considered
invalid.
- gh-110392: Fix tty.setraw() and tty.setcbreak(): previously they
returned partially modified list of the original tty attributes.
tty.cfmakeraw() and tty.cfmakecbreak() now make a copy of the
list of special characters before modifying it.
- gh-110590: Fix a bug in _sre.compile() where TypeError would be
overwritten by OverflowError when the code argument was a list
of non-ints.
- gh-65052: Prevent pdb from crashing when trying to display
undisplayable objects
- gh-110519: Deprecation warning about non-integer number in
gettext now alwais refers to the line in the user code where
gettext function or method is used. Previously it could refer to
a line in gettext code.
- gh-110395: Ensure that select.kqueue() objects correctly appear
as closed in forked children, to prevent operations on an
invalid file descriptor.
- gh-110378: contextmanager() and asynccontextmanager() context
managers now close an invalid underlying generator object that
yields more then one value.
- gh-110365: Fix termios.tcsetattr() bug that was overwritting
existing errors during parsing integers from term list.
- gh-109653: Fix a Python 3.12 regression in the import time of
random. Patch by Alex Waygood.
- gh-110196: Add __reduce__ method to IPv6Address in order to keep
scope_id
- gh-110036: On Windows, multiprocessing Popen.terminate() now
catchs PermissionError and get the process exit code. If the
process is still running, raise again the PermissionError.
Otherwise, the process terminated as expected: store its exit
code. Patch by Victor Stinner.
- gh-110038: Fixed an issue that caused KqueueSelector.select() to
not return all the ready events in some cases when a file
descriptor is registered for both read and write.
- gh-109631: re functions such as re.findall(), re.split(),
re.search() and re.sub() which perform short repeated matches
can now be interrupted by user.
- gh-109747: Improve errors for unsupported look-behind patterns.
Now re.error is raised instead of OverflowError or RuntimeError
for too large width of look-behind pattern.
- gh-109818: Fix reprlib.recursive_repr() not copying
__type_params__ from decorated function.
- gh-109047: concurrent.futures: The executor manager thread now
catches exceptions when adding an item to the call queue. During
Python finalization, creating a new thread can now raise
RuntimeError. Catch the exception and call terminate_broken() in
this case. Patch by Victor Stinner.
- gh-109782: Ensure the signature of os.path.isdir() is identical
on all platforms. Patch by Amin Alaee.
- gh-109590: shutil.which() will prefer files with an extension in
PATHEXT if the given mode includes os.X_OK on win32. If no
PATHEXT match is found, a file without an extension in PATHEXT
can be returned. This change will have shutil.which() act more
similarly to previous behavior in Python 3.11.
- gh-109786: Fix possible reference leaks and crash when re-enter
the __next__() method of itertools.pairwise.
- gh-109593: Avoid deadlocking on a reentrant call to the
multiprocessing resource tracker. Such a reentrant call, though
unlikely, can happen if a GC pass invokes the finalizer for a
multiprocessing object such as SemLock.
- gh-109613: Fix os.stat() and os.DirEntry.stat(): check for
exceptions. Previously, on Python built in debug mode, these
functions could trigger a fatal Python error (and abort the
process) when a function succeeded with an exception set. Patch
by Victor Stinner.
- gh-109375: The pdb alias command now prevents registering
aliases without arguments.
- gh-107219: Fix a race condition in concurrent.futures. When a
process in the process pool was terminated abruptly (while the
future was running or pending), close the connection write end.
If the call queue is blocked on sending bytes to a worker
process, closing the connection write end interrupts the send,
so the queue can be closed. Patch by Victor Stinner.
- gh-50644: Attempts to pickle or create a shallow or deep copy of
codecs streams now raise a TypeError. Previously, copying failed
with a RecursionError, while pickling produced wrong results
that eventually caused unpickling to fail with a RecursionError.
- gh-108987: Fix _thread.start_new_thread() race condition. If a
thread is created during Python finalization, the newly spawned
thread now exits immediately instead of trying to access freed
memory and lead to a crash. Patch by Victor Stinner.
- gh-108791: Improved error handling in pdb command line
interface, making it produce more concise error messages.
- gh-105829: Fix concurrent.futures.ProcessPoolExecutor deadlock
- gh-106584: Fix exit code for unittest if all tests are skipped.
Patch by Egor Eliseev.
- gh-102956: Fix returning of empty byte strings after seek in
zipfile module
- gh-84867: unittest.TestLoader no longer loads test cases from
exact unittest.TestCase and unittest.FunctionTestCase classes.
- gh-91133: Fix a bug in tempfile.TemporaryDirectory cleanup,
which now no longer dereferences symlinks when working around
file system permission errors.
- gh-73561: Omit the interface scope from an IPv6 address when
used as Host header by http.client.
- gh-86826: zipinfo now supports the full range of values in the
TZ string determined by RFC 8536 and detects all invalid
formats. Both Python and C implementations now raise exceptions
of the same type on invalid data.
- bpo-43153: On Windows, tempfile.TemporaryDirectory previously
masked a PermissionError with NotADirectoryError during
directory cleanup. It now correctly raises PermissionError if
errors are not ignored. Patch by Andrei Kulakov and Ken Jin.
- bpo-35332: The shutil.rmtree() function now ignores errors when
calling os.close() when ignore_errors is True, and os.close() no
longer retried after error.
- bpo-41422: Fixed memory leaks of pickle.Pickler and
pickle.Unpickler involving cyclic references via the internal
memo mapping.
- bpo-40262: The ssl.SSLSocket.recv_into() method no longer
requires the buffer argument to implement __len__ and supports
buffers with arbitrary item size.
- Documentation
- gh-111699: Relocate smtpd deprecation notice to its own section
rather than under locale in What’s New in Python 3.12 document
- gh-108826: dis module command-line interface is now mentioned in
documentation. Test- s
- gh-112769: The tests now correctly compare zlib version when
zlib.ZLIB_RUNTIME_VERSION contains non-integer suffixes. For
example zlib-ng defines the version as 1.3.0.zlib-ng.
- gh-110367: Make regrtest --verbose3 option compatible with
--huntrleaks -jN options. The ./python -m test -j1 -R 3:3
--verbose3 command now works as expected. Patch by Victor
Stinner.
- gh-111165: Remove no longer used functions run_unittest() and
run_doctest() from the test.support module.
- gh-110932: Fix regrtest if the SOURCE_DATE_EPOCH environment
variable is defined: use the variable value as the random seed.
Patch by Victor Stinner.
- gh-110995: test_gdb: Fix detection of gdb built without Python
scripting support. Patch by Victor Stinner.
- gh-110918: Test case matching patterns specified by options
--match, --ignore, --matchfile and --ignorefile are now tested
in the order of specification, and the last match determines
whether the test case be run or ignored.
- gh-110647: Fix test_stress_modifying_handlers() of test_signal.
Patch by Victor Stinner.
- gh-103053: Fix test_tools.test_freeze on FreeBSD: run “make
distclean” instead of “make clean” in the copied source
directory to remove also the “python” program. Patch by Victor
Stinner.
- gh-110167: Fix a deadlock in test_socket when server fails with
a timeout but the client is still running in its thread. Don’t
hold a lock to call cleanup functions in doCleanups(). One of
the cleanup function waits until the client completes, whereas
the client could deadlock if it called addCleanup() in such
situation. Patch by Victor Stinner.
- gh-110388: Add tests for tty.
- gh-81002: Add tests for termios.
- gh-110267: Add tests for pickling and copying PyStructSequence
objects. Patched by Xuehai Pan.
- gh-110031: Skip test_threading tests using thread+fork if Python
is built with Address Sanitizer (ASAN). Patch by Victor Stinner.
- gh-110088: Fix test_asyncio timeouts: don’t measure the maximum
duration, a test should not measure a CI performance. Only
measure the minimum duration when a task has a timeout or delay.
Add CLOCK_RES to test_asyncio.utils. Patch by Victor Stinner.
- gh-109974: Fix race conditions in test_threading lock tests.
Wait until a condition is met rather than using time.sleep()
with a hardcoded number of seconds. Patch by Victor Stinner.
- gh-110033: Fix test_interprocess_signal() of test_signal. Make
sure that the subprocess.Popen object is deleted before the test
raising an exception in a signal handler. Otherwise,
Popen.__del__() can get the exception which is logged as
Exception ignored in: ... and the test fails. Patch by Victor
Stinner.
- gh-109594: Fix test_timeout() of
test_concurrent_futures.test_wait. Remove the future which may
or may not complete depending if it takes longer than the
timeout ot not. Keep the second future which does not complete
before wait() timeout. Patch by Victor Stinner.
- gh-109972: Split test_gdb.py file into a test_gdb package made
of multiple tests, so tests can now be run in parallel. Patch by
Victor Stinner.
- gh-103053: Skip test_freeze_simple_script() of
test_tools.test_freeze if Python is built with ./configure
--enable-optimizations, which means with Profile Guided
Optimization (PGO): it just makes the test too slow. The freeze
tool is tested by many other CIs with other (faster) compiler
flags. Patch by Victor Stinner.
- gh-109580: Skip test_perf_profiler if Python is built with ASAN,
MSAN or UBSAN sanitizer. Python does crash randomly in this test
on such build. Patch by Victor Stinner.
- gh-104736: Fix test_gdb on Python built with LLVM clang 16 on
Linux ppc64le (ex: Fedora 38). Search patterns in gdb “bt”
command output to detect when gdb fails to retrieve the
traceback. For example, skip a test if Backtrace stopped: frame
did not save the PC is found. Patch by Victor Stinner.
- gh-108927: Fixed order dependence in running tests in the same
process when a test that has submodules (e.g. test_importlib)
follows a test that imports its submodule (e.g.
test_importlib.util) and precedes a test (e.g. test_unittest or
test_compileall) that uses that submodule.
- Build
- gh-112088: Add Tools/build/regen-configure.sh script to
regenerate the configure with an Ubuntu container image. The
quay.io/tiran/cpython_autoconf:271 container image
(tiran/cpython_autoconf) is no longer used. Patch by Victor
Stinner.
- gh-111046: For wasi-threads, memory is now exported to fix
compatibility issues with some wasm runtimes.
- gh-103053: “make check-clean-src” now also checks if the
“python” program is found in the source directory: fail with an
error if it does exist. Patch by Victor Stinner.
- gh-109191: Fix compile error when building with recent versions
of libedit.
- IDLE
- bpo-35668: Add docstrings to the IDLE debugger module. Fix two
bugs: initialize Idb.botframe (should be in Bdb); in
Idb.in_rpc_code, check whether prev_frame is None before trying
to use it. Greatly expand test_debugger.
- C API
- gh-106560: Fix redundant declarations in the public C API.
Declare PyBool_Type and PyLong_Type only once. Patch by Victor
Stinner.
- gh-112438: Fix support of format units “es”, “et”, “es#”, and
“et#” in nested tuples in PyArg_ParseTuple()-like functions.
- gh-109521: PyImport_GetImporter() now sets RuntimeError if it
fails to get sys.path_hooks or sys.path_importer_cache or they
are not list and dict correspondingly. Previously it could
return NULL without setting error in obscure cases, crash or
raise SystemError if these attributes have wrong type.