greg/cpython
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

gh-135801: Add the module parameter to compile() etc (GH-139652)

Browse Source 
Many functions related to compiling or parsing Python code, such as
compile(), ast.parse(), symtable.symtable(),
and importlib.abc.InspectLoader.source_to_code() now allow to pass
the module name used when filtering syntax warnings.
This commit is contained in:
Serhiy Storchaka
2025-11-13 13:21:32 +02:00
committed by GitHub
parent 63548b3699
commit d8e6bdc0d0
47 changed files with 390 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
Doc/library/ast.rst
View File

@@ -2205,10 +2205,10 @@ Async and await
Apart from the node classes, the :mod:`ast` module defines these utility functions
and classes for traversing abstract syntax trees:
.. function:: parse(source, filename='<unknown>', mode='exec', *, type_comments=False, feature_version=None, optimize=-1)
.. function:: parse(source, filename='<unknown>', mode='exec', *, type_comments=False, feature_version=None, optimize=-1, module=None)
Parse the source into an AST node. Equivalent to ``compile(source,
filename, mode, flags=FLAGS_VALUE, optimize=optimize)``,
filename, mode, flags=FLAGS_VALUE, optimize=optimize, module=module)``,
where ``FLAGS_VALUE`` is ``ast.PyCF_ONLY_AST`` if ``optimize <= 0``
and ``ast.PyCF_OPTIMIZED_AST`` otherwise.
@@ -2261,6 +2261,9 @@ and classes for traversing abstract syntax trees:
The minimum supported version for ``feature_version`` is now ``(3, 7)``.
The ``optimize`` argument was added.
.. versionadded:: next
Added the *module* parameter.
.. function:: unparse(ast_obj)

11
Doc/library/functions.rst
View File

@@ -292,7 +292,9 @@ are always available. They are listed here in alphabetical order.
:func:`property`.
.. function:: compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)
.. function:: compile(source, filename, mode, flags=0, \
dont_inherit=False, optimize=-1, \
*, module=None)
Compile the *source* into a code or AST object. Code objects can be executed
by :func:`exec` or :func:`eval`. *source* can either be a normal string, a
@@ -334,6 +336,10 @@ are always available. They are listed here in alphabetical order.
``__debug__`` is true), ``1`` (asserts are removed, ``__debug__`` is false)
or ``2`` (docstrings are removed too).
The optional argument *module* specifies the module name.
It is needed to unambiguous :ref:`filter <warning-filter>` syntax warnings
by module name.
This function raises :exc:`SyntaxError` if the compiled source is invalid,
and :exc:`ValueError` if the source contains null bytes.
@@ -371,6 +377,9 @@ are always available. They are listed here in alphabetical order.
``ast.PyCF_ALLOW_TOP_LEVEL_AWAIT`` can now be passed in flags to enable
support for top-level ``await``, ``async for``, and ``async with``.
.. versionadded:: next
Added the *module* parameter.
.. class:: complex(number=0, /)
complex(string, /)

10
Doc/library/importlib.rst
View File

@@ -459,7 +459,7 @@ ABC hierarchy::
.. versionchanged:: 3.4
Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
.. staticmethod:: source_to_code(data, path='<string>')
.. staticmethod:: source_to_code(data, path='<string>', fullname=None)
Create a code object from Python source.
@@ -471,11 +471,19 @@ ABC hierarchy::
With the subsequent code object one can execute it in a module by
running ``exec(code, module.__dict__)``.
The optional argument *fullname* specifies the module name.
It is needed to unambiguous :ref:`filter <warning-filter>` syntax
warnings by module name.
.. versionadded:: 3.4
.. versionchanged:: 3.5
Made the method static.
.. versionadded:: next
Added the *fullname* parameter.
.. method:: exec_module(module)
Implementation of :meth:`Loader.exec_module`.

8
Doc/library/symtable.rst
View File

@@ -21,11 +21,17 @@ tables.
Generating Symbol Tables
------------------------
.. function:: symtable(code, filename, compile_type)
.. function:: symtable(code, filename, compile_type, *, module=None)
Return the toplevel :class:`SymbolTable` for the Python source *code*.
*filename* is the name of the file containing the code. *compile_type* is
like the *mode* argument to :func:`compile`.
The optional argument *module* specifies the module name.
It is needed to unambiguous :ref:`filter <warning-filter>` syntax warnings
by module name.
.. versionadded:: next
Added the *module* parameter.
Examining Symbol Tables

7
Doc/whatsnew/3.15.rst
View File

@@ -307,6 +307,13 @@ Other language changes
not only integers or floats, although this does not improve precision.
(Contributed by Serhiy Storchaka in :gh:`67795`.)
* Many functions related to compiling or parsing Python code, such as
:func:`compile`, :func:`ast.parse`, :func:`symtable.symtable`,
and :func:`importlib.abc.InspectLoader.source_to_code`, now allow to pass
the module name. It is needed to unambiguous :ref:`filter <warning-filter>`
syntax warnings by module name.
(Contributed by Serhiy Storchaka in :gh:`135801`.)
New modules
===========

9
Include/internal/pycore_compile.h
View File

@@ -32,7 +32,8 @@ PyAPI_FUNC(PyCodeObject*) _PyAST_Compile(
PyObject *filename,
PyCompilerFlags *flags,
int optimize,
struct _arena *arena);
struct _arena *arena,
PyObject *module);
/* AST preprocessing */
extern int _PyCompile_AstPreprocess(
@@ -41,7 +42,8 @@ extern int _PyCompile_AstPreprocess(
PyCompilerFlags *flags,
int optimize,
struct _arena *arena,
int syntax_check_only);
int syntax_check_only,
PyObject *module);
extern int _PyAST_Preprocess(
struct _mod *,
@@ -50,7 +52,8 @@ extern int _PyAST_Preprocess(
int optimize,
int ff_features,
int syntax_check_only,
int enable_warnings);
int enable_warnings,
PyObject *module);
typedef struct {

3
Include/internal/pycore_parser.h
View File

@@ -48,7 +48,8 @@ extern struct _mod* _PyParser_ASTFromString(
PyObject* filename,
int mode,
PyCompilerFlags *flags,
PyArena *arena);
PyArena *arena,
PyObject *module);
extern struct _mod* _PyParser_ASTFromFile(
FILE *fp,

3
Include/internal/pycore_pyerrors.h
View File

@@ -123,7 +123,8 @@ extern void _PyErr_SetNone(PyThreadState *tstate, PyObject *exception);
extern PyObject* _PyErr_NoMemory(PyThreadState *tstate);
extern int _PyErr_EmitSyntaxWarning(PyObject *msg, PyObject *filename, int lineno, int col_offset,
int end_lineno, int end_col_offset);
int end_lineno, int end_col_offset,
PyObject *module);
extern void _PyErr_RaiseSyntaxError(PyObject *msg, PyObject *filename, int lineno, int col_offset,
int end_lineno, int end_col_offset);

6
Include/internal/pycore_pythonrun.h
View File

@@ -33,6 +33,12 @@ extern const char* _Py_SourceAsString(
PyCompilerFlags *cf,
PyObject **cmd_copy);
extern PyObject * _Py_CompileStringObjectWithModule(
const char *str,
PyObject *filename, int start,
PyCompilerFlags *flags, int optimize,
PyObject *module);
/* Stack size, in "pointers". This must be large enough, so
* no two calls to check recursion depth are more than this far

3
Include/internal/pycore_symtable.h
View File

@@ -188,7 +188,8 @@ extern struct symtable* _Py_SymtableStringObjectFlags(
const char *str,
PyObject *filename,
int start,
PyCompilerFlags *flags);
PyCompilerFlags *flags,
PyObject *module);
int _PyFuture_FromAST(
struct _mod * mod,

5
Lib/ast.py
View File

@@ -24,7 +24,7 @@ from _ast import *
def parse(source, filename='<unknown>', mode='exec', *,
type_comments=False, feature_version=None, optimize=-1):
type_comments=False, feature_version=None, optimize=-1, module=None):
"""
Parse the source into an AST node.
Equivalent to compile(source, filename, mode, PyCF_ONLY_AST).
@@ -44,7 +44,8 @@ def parse(source, filename='<unknown>', mode='exec', *,
feature_version = minor
# Else it should be an int giving the minor version for 3.x.
return compile(source, filename, mode, flags,
_feature_version=feature_version, optimize=optimize)
_feature_version=feature_version, optimize=optimize,
module=module)
def literal_eval(node_or_string):

9
Lib/importlib/_bootstrap_external.py
View File

@@ -819,13 +819,14 @@ class SourceLoader(_LoaderBasics):
name=fullname) from exc
return decode_source(source_bytes)
def source_to_code(self, data, path, *, _optimize=-1):
def source_to_code(self, data, path, fullname=None, *, _optimize=-1):
"""Return the code object compiled from source.
The 'data' argument can be any object type that compile() supports.
"""
return _bootstrap._call_with_frames_removed(compile, data, path, 'exec',
dont_inherit=True, optimize=_optimize)
dont_inherit=True, optimize=_optimize,
module=fullname)
def get_code(self, fullname):
"""Concrete implementation of InspectLoader.get_code.
@@ -894,7 +895,7 @@ class SourceLoader(_LoaderBasics):
source_path=source_path)
if source_bytes is None:
source_bytes = self.get_data(source_path)
code_object = self.source_to_code(source_bytes, source_path)
code_object = self.source_to_code(source_bytes, source_path, fullname)
_bootstrap._verbose_message('code object from {}', source_path)
if (not sys.dont_write_bytecode and bytecode_path is not None and
source_mtime is not None):
@@ -1186,7 +1187,7 @@ class NamespaceLoader:
return ''
def get_code(self, fullname):
return compile('', '<string>', 'exec', dont_inherit=True)
return compile('', '<string>', 'exec', dont_inherit=True, module=fullname)
def create_module(self, spec):
"""Use default semantics for module creation."""

11
Lib/importlib/abc.py
View File

@@ -108,7 +108,7 @@ class InspectLoader(Loader):
source = self.get_source(fullname)
if source is None:
return None
return self.source_to_code(source)
return self.source_to_code(source, '<string>', fullname)
@abc.abstractmethod
def get_source(self, fullname):
@@ -120,12 +120,12 @@ class InspectLoader(Loader):
raise ImportError
@staticmethod
def source_to_code(data, path='<string>'):
def source_to_code(data, path='<string>', fullname=None):
"""Compile 'data' into a code object.
The 'data' argument can be anything that compile() can handle. The'path'
argument should be where the data was retrieved (when applicable)."""
return compile(data, path, 'exec', dont_inherit=True)
return compile(data, path, 'exec', dont_inherit=True, module=fullname)
exec_module = _bootstrap_external._LoaderBasics.exec_module
load_module = _bootstrap_external._LoaderBasics.load_module
@@ -163,9 +163,8 @@ class ExecutionLoader(InspectLoader):
try:
path = self.get_filename(fullname)
except ImportError:
return self.source_to_code(source)
else:
return self.source_to_code(source, path)
path = '<string>'
return self.source_to_code(source, path, fullname)
_register(
ExecutionLoader,

2
Lib/modulefinder.py
View File

@@ -334,7 +334,7 @@ class ModuleFinder:
self.msgout(2, "load_module ->", m)
return m
if type == _PY_SOURCE:
co = compile(fp.read(), pathname, 'exec')
co = compile(fp.read(), pathname, 'exec', module=fqname)
elif type == _PY_COMPILED:
try:
data = fp.read()

2
Lib/profiling/sampling/_sync_coordinator.py
View File

@@ -182,7 +182,7 @@ def _execute_script(script_path: str, script_args: List[str], cwd: str) -> None:
try:
# Compile and execute the script
code = compile(source_code, script_path, 'exec')
code = compile(source_code, script_path, 'exec', module='__main__')
exec(code, {'__name__': '__main__', '__file__': script_path})
except SyntaxError as e:
raise TargetError(f"Syntax error in script {script_path}: {e}") from e

2
Lib/profiling/tracing/__init__.py
View File

@@ -185,7 +185,7 @@ def main():
progname = args[0]
sys.path.insert(0, os.path.dirname(progname))
with io.open_code(progname) as fp:
code = compile(fp.read(), progname, 'exec')
code = compile(fp.read(), progname, 'exec', module='__main__')
spec = importlib.machinery.ModuleSpec(name='__main__', loader=None,
origin=progname)
module = importlib.util.module_from_spec(spec)

6
Lib/runpy.py
View File

@@ -247,7 +247,7 @@ def _get_main_module_details(error=ImportError):
sys.modules[main_name] = saved_main
def _get_code_from_file(fname):
def _get_code_from_file(fname, module):
# Check for a compiled file first
from pkgutil import read_code
code_path = os.path.abspath(fname)
@@ -256,7 +256,7 @@ def _get_code_from_file(fname):
if code is None:
# That didn't work, so try it as normal source code
with io.open_code(code_path) as f:
code = compile(f.read(), fname, 'exec')
code = compile(f.read(), fname, 'exec', module=module)
return code
def run_path(path_name, init_globals=None, run_name=None):
@@ -283,7 +283,7 @@ def run_path(path_name, init_globals=None, run_name=None):
if isinstance(importer, type(None)):
# Not a valid sys.path entry, so run the code directly
# execfile() doesn't help as we want to allow compiled files
code = _get_code_from_file(path_name)
code = _get_code_from_file(path_name, run_name)
return _run_module_code(code, init_globals, run_name,
pkg_name=pkg_name, script_name=path_name)
else:

4
Lib/symtable.py
View File

@@ -17,13 +17,13 @@ from enum import StrEnum
__all__ = ["symtable", "SymbolTableType", "SymbolTable", "Class", "Function", "Symbol"]
def symtable(code, filename, compile_type):
def symtable(code, filename, compile_type, *, module=None):
""" Return the toplevel *SymbolTable* for the source code.
*filename* is the name of the file with the code
and *compile_type* is the *compile()* mode argument.
"""
top = _symtable.symtable(code, filename, compile_type)
top = _symtable.symtable(code, filename, compile_type, module=module)
return _newSymbolTable(top, filename)
class SymbolTableFactory:

10
Lib/test/test_ast/test_ast.py
View File

@@ -1083,6 +1083,16 @@ class AST_Tests(unittest.TestCase):
self.assertEqual(wm.filename, '<unknown>')
self.assertIs(wm.category, SyntaxWarning)
with warnings.catch_warnings(record=True) as wlog:
warnings.simplefilter('error')
warnings.filterwarnings('always', module=r'package\.module\z')
warnings.filterwarnings('error', module=r'<unknown>')
ast.parse(source, filename, module='package.module')
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10])
for wm in wlog:
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
class CopyTests(unittest.TestCase):
"""Test copying and pickling AST nodes."""

3
Lib/test/test_builtin.py
View File

@@ -1103,7 +1103,8 @@ class BuiltinTest(ComplexesAreIdenticalMixin, unittest.TestCase):
with warnings.catch_warnings(record=True) as wlog:
warnings.simplefilter('error')
warnings.filterwarnings('always', module=r'<string>\z')
warnings.filterwarnings('always', module=r'package.module\z')
warnings.filterwarnings('error', module=r'<string>')
exec(source, {'__name__': 'package.module', '__file__': filename})
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10, 13, 14, 21])
for wm in wlog:

23
Lib/test/test_cmd_line_script.py
View File

@@ -814,15 +814,26 @@ class CmdLineTest(unittest.TestCase):
filename = support.findfile('test_import/data/syntax_warnings.py')
rc, out, err = assert_python_ok(
'-Werror',
'-Walways:::test.test_import.data.syntax_warnings',
'-Walways:::__main__',
'-Werror:::test.test_import.data.syntax_warnings',
'-Werror:::syntax_warnings',
filename)
self.assertEqual(err.count(b': SyntaxWarning: '), 6)
rc, out, err = assert_python_ok(
'-Werror',
'-Walways:::syntax_warnings',
filename)
self.assertEqual(err.count(b': SyntaxWarning: '), 6)
def test_zipfile_run_filter_syntax_warnings_by_module(self):
filename = support.findfile('test_import/data/syntax_warnings.py')
with open(filename, 'rb') as f:
source = f.read()
with os_helper.temp_dir() as script_dir:
zip_name, _ = make_zip_pkg(
script_dir, 'test_zip', 'test_pkg', '__main__', source)
rc, out, err = assert_python_ok(
'-Werror',
'-Walways:::__main__',
'-Werror:::test_pkg.__main__',
os.path.join(zip_name, 'test_pkg')
)
self.assertEqual(err.count(b': SyntaxWarning: '), 12)
def tearDownModule():

10
Lib/test/test_compile.py
View File

@@ -1759,6 +1759,16 @@ class TestSpecifics(unittest.TestCase):
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
with warnings.catch_warnings(record=True) as wlog:
warnings.simplefilter('error')
warnings.filterwarnings('always', module=r'package\.module\z')
warnings.filterwarnings('error', module=module_re)
compile(source, filename, 'exec', module='package.module')
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10, 13, 14, 21])
for wm in wlog:
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
@support.subTests('src', [
textwrap.dedent("""
def f():

15
Lib/test/test_import/__init__.py
View File

@@ -1259,20 +1259,7 @@ os.does_not_exist
warnings.catch_warnings(record=True) as wlog):
warnings.simplefilter('error')
warnings.filterwarnings('always', module=module_re)
import test.test_import.data.syntax_warnings
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10, 13, 14, 21])
filename = test.test_import.data.syntax_warnings.__file__
for wm in wlog:
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
module_re = r'syntax_warnings\z'
unload('test.test_import.data.syntax_warnings')
with (os_helper.temp_dir() as tmpdir,
temporary_pycache_prefix(tmpdir),
warnings.catch_warnings(record=True) as wlog):
warnings.simplefilter('error')
warnings.filterwarnings('always', module=module_re)
warnings.filterwarnings('error', module='syntax_warnings')
import test.test_import.data.syntax_warnings
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10, 13, 14, 21])
filename = test.test_import.data.syntax_warnings.__file__

43
Lib/test/test_runpy.py
View File

@@ -20,9 +20,11 @@ from test.support import (
requires_subprocess,
verbose,
)
from test import support
from test.support.import_helper import forget, make_legacy_pyc, unload
from test.support.os_helper import create_empty_file, temp_dir, FakePath
from test.support.script_helper import make_script, make_zip_script
from test.test_importlib.util import temporary_pycache_prefix
import runpy
@@ -763,6 +765,47 @@ s = "non-ASCII: h\xe9"
result = run_path(filename)
self.assertEqual(result['s'], "non-ASCII: h\xe9")
def test_run_module_filter_syntax_warnings_by_module(self):
module_re = r'test\.test_import\.data\.syntax_warnings\z'
with (temp_dir() as tmpdir,
temporary_pycache_prefix(tmpdir),
warnings.catch_warnings(record=True) as wlog):
warnings.simplefilter('error')
warnings.filterwarnings('always', module=module_re)
warnings.filterwarnings('error', module='syntax_warnings')
ns = run_module('test.test_import.data.syntax_warnings')
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10, 13, 14, 21])
filename = ns['__file__']
for wm in wlog:
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
def test_run_path_filter_syntax_warnings_by_module(self):
filename = support.findfile('test_import/data/syntax_warnings.py')
with warnings.catch_warnings(record=True) as wlog:
warnings.simplefilter('error')
warnings.filterwarnings('always', module=r'<run_path>\z')
warnings.filterwarnings('error', module='test')
warnings.filterwarnings('error', module='syntax_warnings')
warnings.filterwarnings('error',
module=r'test\.test_import\.data\.syntax_warnings')
run_path(filename)
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10, 13, 14, 21])
for wm in wlog:
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
with warnings.catch_warnings(record=True) as wlog:
warnings.simplefilter('error')
warnings.filterwarnings('always', module=r'package\.script\z')
warnings.filterwarnings('error', module='<run_path>')
warnings.filterwarnings('error', module='test')
warnings.filterwarnings('error', module='syntax_warnings')
warnings.filterwarnings('error',
module=r'test\.test_import\.data\.syntax_warnings')
run_path(filename, run_name='package.script')
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10, 13, 14, 21])
@force_not_colorized_test_class
class TestExit(unittest.TestCase):

10
Lib/test/test_symtable.py
View File

@@ -601,6 +601,16 @@ class SymtableTest(unittest.TestCase):
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
with warnings.catch_warnings(record=True) as wlog:
warnings.simplefilter('error')
warnings.filterwarnings('always', module=r'package\.module\z')
warnings.filterwarnings('error', module=module_re)
symtable.symtable(source, filename, 'exec', module='package.module')
self.assertEqual(sorted(wm.lineno for wm in wlog), [4, 7, 10])
for wm in wlog:
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
class ComprehensionTests(unittest.TestCase):
def get_identifiers_recursive(self, st, res):

23
Lib/test/test_zipimport_support.py
View File

@@ -13,9 +13,12 @@ import doctest
import inspect
import linecache
import unittest
import warnings
from test import support
from test.support import os_helper
from test.support.script_helper import (spawn_python, kill_python, assert_python_ok,
make_script, make_zip_script)
from test.support import import_helper
verbose = test.support.verbose
@@ -236,6 +239,26 @@ class ZipSupportTests(unittest.TestCase):
# bdb/pdb applies normcase to its filename before displaying
self.assertIn(os.path.normcase(run_name.encode('utf-8')), data)
def test_import_filter_syntax_warnings_by_module(self):
filename = support.findfile('test_import/data/syntax_warnings.py')
with (os_helper.temp_dir() as tmpdir,
import_helper.DirsOnSysPath()):
zip_name, _ = make_zip_script(tmpdir, "test_zip",
filename, 'test_pkg/test_mod.py')
sys.path.insert(0, zip_name)
import_helper.unload('test_pkg.test_mod')
with warnings.catch_warnings(record=True) as wlog:
warnings.simplefilter('error')
warnings.filterwarnings('always', module=r'test_pkg\.test_mod\z')
warnings.filterwarnings('error', module='test_mod')
import test_pkg.test_mod
self.assertEqual(sorted(wm.lineno for wm in wlog),
sorted([4, 7, 10, 13, 14, 21]*2))
filename = test_pkg.test_mod.__file__
for wm in wlog:
self.assertEqual(wm.filename, filename)
self.assertIs(wm.category, SyntaxWarning)
def tearDownModule():
test.support.reap_children()

6
Lib/zipimport.py
View File

@@ -742,9 +742,9 @@ def _normalize_line_endings(source):
# Given a string buffer containing Python source code, compile it
# and return a code object.
def _compile_source(pathname, source):
def _compile_source(pathname, source, module):
source = _normalize_line_endings(source)
return compile(source, pathname, 'exec', dont_inherit=True)
return compile(source, pathname, 'exec', dont_inherit=True, module=module)
# Convert the date/time values found in the Zip archive to a value
# that's compatible with the time stamp stored in .pyc files.
@@ -815,7 +815,7 @@ def _get_module_code(self, fullname):
except ImportError as exc:
import_error = exc
else:
code = _compile_source(modpath, data)
code = _compile_source(modpath, data, fullname)
if code is None:
# bad magic number or non-matching mtime
# in byte code, try next

6
Misc/NEWS.d/next/Core_and_Builtins/2025-10-06-14-19-47.gh-issue-135801.OhxEZS.rst Normal file
View File

@@ -0,0 +1,6 @@
Many functions related to compiling or parsing Python code, such as
:func:`compile`, :func:`ast.parse`, :func:`symtable.symtable`, and
:func:`importlib.abc.InspectLoader.source_to_code` now allow to specify
the module name.
It is needed to unambiguous :ref:`filter <warning-filter>` syntax warnings
by module name.

58
Modules/clinic/symtablemodule.c.h generated
View File

@@ -2,30 +2,67 @@
preserve
[clinic start generated code]*/
#include "pycore_modsupport.h" // _PyArg_CheckPositional()
#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
# include "pycore_gc.h" // PyGC_Head
# include "pycore_runtime.h" // _Py_ID()
#endif
#include "pycore_modsupport.h" // _PyArg_UnpackKeywords()
PyDoc_STRVAR(_symtable_symtable__doc__,
"symtable($module, source, filename, startstr, /)\n"
"symtable($module, source, filename, startstr, /, *, module=None)\n"
"--\n"
"\n"
"Return symbol and scope dictionaries used internally by compiler.");
#define _SYMTABLE_SYMTABLE_METHODDEF \
{"symtable", _PyCFunction_CAST(_symtable_symtable), METH_FASTCALL, _symtable_symtable__doc__},
{"symtable", _PyCFunction_CAST(_symtable_symtable), METH_FASTCALL|METH_KEYWORDS, _symtable_symtable__doc__},
static PyObject *
_symtable_symtable_impl(PyObject *module, PyObject *source,
PyObject *filename, const char *startstr);
PyObject *filename, const char *startstr,
PyObject *modname);
static PyObject *
_symtable_symtable(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
_symtable_symtable(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{
PyObject *return_value = NULL;
#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
#define NUM_KEYWORDS 1
static struct {
PyGC_Head _this_is_not_used;
PyObject_VAR_HEAD
Py_hash_t ob_hash;
PyObject *ob_item[NUM_KEYWORDS];
} _kwtuple = {
.ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
.ob_hash = -1,
.ob_item = { &_Py_ID(module), },
};
#undef NUM_KEYWORDS
#define KWTUPLE (&_kwtuple.ob_base.ob_base)
#else // !Py_BUILD_CORE
# define KWTUPLE NULL
#endif // !Py_BUILD_CORE
static const char * const _keywords[] = {"", "", "", "module", NULL};
static _PyArg_Parser _parser = {
.keywords = _keywords,
.fname = "symtable",
.kwtuple = KWTUPLE,
};
#undef KWTUPLE
PyObject *argsbuf[4];
Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 3;
PyObject *source;
PyObject *filename = NULL;
const char *startstr;
PyObject *modname = Py_None;
if (!_PyArg_CheckPositional("symtable", nargs, 3, 3)) {
args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser,
/*minpos*/ 3, /*maxpos*/ 3, /*minkw*/ 0, /*varpos*/ 0, argsbuf);
if (!args) {
goto exit;
}
source = args[0];
@@ -45,7 +82,12 @@ _symtable_symtable(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
PyErr_SetString(PyExc_ValueError, "embedded null character");
goto exit;
}
return_value = _symtable_symtable_impl(module, source, filename, startstr);
if (!noptargs) {
goto skip_optional_kwonly;
}
modname = args[3];
skip_optional_kwonly:
return_value = _symtable_symtable_impl(module, source, filename, startstr, modname);
exit:
/* Cleanup for filename */
@@ -53,4 +95,4 @@ exit:
return return_value;
}
/*[clinic end generated code: output=7a8545d9a1efe837 input=a9049054013a1b77]*/
/*[clinic end generated code: output=0137be60c487c841 input=a9049054013a1b77]*/

19
Modules/symtablemodule.c
View File

@@ -16,14 +16,17 @@ _symtable.symtable
filename: unicode_fs_decoded
startstr: str
/
*
module as modname: object = None
Return symbol and scope dictionaries used internally by compiler.
[clinic start generated code]*/
static PyObject *
_symtable_symtable_impl(PyObject *module, PyObject *source,
PyObject *filename, const char *startstr)
/*[clinic end generated code: output=59eb0d5fc7285ac4 input=436ffff90d02e4f6]*/
PyObject *filename, const char *startstr,
PyObject *modname)
/*[clinic end generated code: output=235ec5a87a9ce178 input=fbf9adaa33c7070d]*/
{
struct symtable *st;
PyObject *t;
@@ -50,7 +53,17 @@ _symtable_symtable_impl(PyObject *module, PyObject *source,
Py_XDECREF(source_copy);
return NULL;
}
st = _Py_SymtableStringObjectFlags(str, filename, start, &cf);
if (modname == Py_None) {
modname = NULL;
}
else if (!PyUnicode_Check(modname)) {
PyErr_Format(PyExc_TypeError,
"symtable() argument 'module' must be str or None, not %T",
modname);
Py_XDECREF(source_copy);
return NULL;
}
st = _Py_SymtableStringObjectFlags(str, filename, start, &cf, modname);
Py_XDECREF(source_copy);
if (st == NULL) {
return NULL;

2
Parser/lexer/state.c
View File

@@ -43,6 +43,7 @@ _PyTokenizer_tok_new(void)
tok->encoding = NULL;
tok->cont_line = 0;
tok->filename = NULL;
tok->module = NULL;
tok->decoding_readline = NULL;
tok->decoding_buffer = NULL;
tok->readline = NULL;
@@ -91,6 +92,7 @@ _PyTokenizer_Free(struct tok_state *tok)
Py_XDECREF(tok->decoding_buffer);
Py_XDECREF(tok->readline);
Py_XDECREF(tok->filename);
Py_XDECREF(tok->module);
if ((tok->readline != NULL || tok->fp != NULL ) && tok->buf != NULL) {
PyMem_Free(tok->buf);
}

1
Parser/lexer/state.h
View File

@@ -102,6 +102,7 @@ struct tok_state {
int parenlinenostack[MAXLEVEL];
int parencolstack[MAXLEVEL];
PyObject *filename;
PyObject *module;
/* Stuff for checking on different tab sizes */
int altindstack[MAXINDENT]; /* Stack of alternate indents */
/* Stuff for PEP 0263 */

6
Parser/peg_api.c
View File

@@ -4,13 +4,15 @@
mod_ty
_PyParser_ASTFromString(const char *str, PyObject* filename, int mode,
PyCompilerFlags *flags, PyArena *arena)
PyCompilerFlags *flags, PyArena *arena,
PyObject *module)
{
if (PySys_Audit("compile", "yO", str, filename) < 0) {
return NULL;
}
mod_ty result = _PyPegen_run_parser_from_string(str, mode, filename, flags, arena);
mod_ty result = _PyPegen_run_parser_from_string(str, mode, filename, flags,
arena, module);
return result;
}

8
Parser/pegen.c
View File

@@ -1010,6 +1010,11 @@ _PyPegen_run_parser_from_file_pointer(FILE *fp, int start_rule, PyObject *filena
// From here on we need to clean up even if there's an error
mod_ty result = NULL;
tok->module = PyUnicode_FromString("__main__");
if (tok->module == NULL) {
goto error;
}
int parser_flags = compute_parser_flags(flags);
Parser *p = _PyPegen_Parser_New(tok, start_rule, parser_flags, PY_MINOR_VERSION,
errcode, NULL, arena);
@@ -1036,7 +1041,7 @@ error:
mod_ty
_PyPegen_run_parser_from_string(const char *str, int start_rule, PyObject *filename_ob,
PyCompilerFlags *flags, PyArena *arena)
PyCompilerFlags *flags, PyArena *arena, PyObject *module)
{
int exec_input = start_rule == Py_file_input;
@@ -1054,6 +1059,7 @@ _PyPegen_run_parser_from_string(const char *str, int start_rule, PyObject *filen
}
// This transfers the ownership to the tokenizer
tok->filename = Py_NewRef(filename_ob);
tok->module = Py_XNewRef(module);
// We need to clear up from here on
mod_ty result = NULL;

2
Parser/pegen.h
View File

@@ -378,7 +378,7 @@ mod_ty _PyPegen_run_parser_from_file_pointer(FILE *, int, PyObject *, const char
const char *, const char *, PyCompilerFlags *, int *, PyObject **,
PyArena *);
void *_PyPegen_run_parser(Parser *);
mod_ty _PyPegen_run_parser_from_string(const char *, int, PyObject *, PyCompilerFlags *, PyArena *);
mod_ty _PyPegen_run_parser_from_string(const char *, int, PyObject *, PyCompilerFlags *, PyArena *, PyObject *);
asdl_stmt_seq *_PyPegen_interactive_exit(Parser *);
// Generated function in parse.c - function definition in python.gram

2
Parser/string_parser.c
View File

@@ -88,7 +88,7 @@ warn_invalid_escape_sequence(Parser *p, const char* buffer, const char *first_in
}
if (PyErr_WarnExplicitObject(category, msg, p->tok->filename,
lineno, NULL, NULL) < 0) {
lineno, p->tok->module, NULL) < 0) {
if (PyErr_ExceptionMatches(category)) {
/* Replace the Syntax/DeprecationWarning exception with a SyntaxError
to get a more accurate error report */

4
Parser/tokenizer/helpers.c
View File

@@ -127,7 +127,7 @@ _PyTokenizer_warn_invalid_escape_sequence(struct tok_state *tok, int first_inval
}
if (PyErr_WarnExplicitObject(PyExc_SyntaxWarning, msg, tok->filename,
tok->lineno, NULL, NULL) < 0) {
tok->lineno, tok->module, NULL) < 0) {
Py_DECREF(msg);
if (PyErr_ExceptionMatches(PyExc_SyntaxWarning)) {
@@ -166,7 +166,7 @@ _PyTokenizer_parser_warn(struct tok_state *tok, PyObject *category, const char *
}
if (PyErr_WarnExplicitObject(category, errmsg, tok->filename,
tok->lineno, NULL, NULL) < 0) {
tok->lineno, tok->module, NULL) < 0) {
if (PyErr_ExceptionMatches(category)) {
/* Replace the DeprecationWarning exception with a SyntaxError
to get a more accurate error report */

2
Programs/_freeze_module.py
View File

@@ -23,7 +23,7 @@ def read_text(inpath: str) -> bytes:
def compile_and_marshal(name: str, text: bytes) -> bytes:
filename = f"<frozen {name}>"
# exec == Py_file_input
code = compile(text, filename, "exec", optimize=0, dont_inherit=True)
code = compile(text, filename, "exec", optimize=0, dont_inherit=True, module=name)
return marshal.dumps(code)

2
Programs/freeze_test_frozenmain.py
View File

@@ -24,7 +24,7 @@ def dump(fp, filename, name):
with tokenize.open(filename) as source_fp:
source = source_fp.read()
code = compile(source, code_filename, 'exec')
code = compile(source, code_filename, 'exec', module=name)
data = marshal.dumps(code)
writecode(fp, name, data)

8
Python/ast_preprocess.c
View File

@@ -16,6 +16,7 @@ typedef struct {
typedef struct {
PyObject *filename;
PyObject *module;
int optimize;
int ff_features;
int syntax_check_only;
@@ -71,7 +72,8 @@ control_flow_in_finally_warning(const char *kw, stmt_ty n, _PyASTPreprocessState
}
int ret = _PyErr_EmitSyntaxWarning(msg, state->filename, n->lineno,
n->col_offset + 1, n->end_lineno,
n->end_col_offset + 1);
n->end_col_offset + 1,
state->module);
Py_DECREF(msg);
return ret < 0 ? 0 : 1;
}
@@ -969,11 +971,13 @@ astfold_type_param(type_param_ty node_, PyArena *ctx_, _PyASTPreprocessState *st
int
_PyAST_Preprocess(mod_ty mod, PyArena *arena, PyObject *filename, int optimize,
int ff_features, int syntax_check_only, int enable_warnings)
int ff_features, int syntax_check_only, int enable_warnings,
PyObject *module)
{
_PyASTPreprocessState state;
memset(&state, 0, sizeof(_PyASTPreprocessState));
state.filename = filename;
state.module = module;
state.optimize = optimize;
state.ff_features = ff_features;
state.syntax_check_only = syntax_check_only;

25
Python/bltinmodule.c
View File

@@ -751,6 +751,7 @@ compile as builtin_compile
dont_inherit: bool = False
optimize: int = -1
*
module as modname: object = None
_feature_version as feature_version: int = -1
Compile source into a code object that can be executed by exec() or eval().
@@ -770,8 +771,8 @@ in addition to any features explicitly specified.
static PyObject *
builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
const char *mode, int flags, int dont_inherit,
int optimize, int feature_version)
/*[clinic end generated code: output=b0c09c84f116d3d7 input=8f0069edbdac381b]*/
int optimize, PyObject *modname, int feature_version)
/*[clinic end generated code: output=9a0dce1945917a86 input=ddeae1e0253459dc]*/
{
PyObject *source_copy;
const char *str;
@@ -800,6 +801,15 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
"compile(): invalid optimize value");
goto error;
}
if (modname == Py_None) {
modname = NULL;
}
else if (!PyUnicode_Check(modname)) {
PyErr_Format(PyExc_TypeError,
"compile() argument 'module' must be str or None, not %T",
modname);
goto error;
}
if (!dont_inherit) {
PyEval_MergeCompilerFlags(&cf);
@@ -845,8 +855,9 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
goto error;
}
int syntax_check_only = ((flags & PyCF_OPTIMIZED_AST) == PyCF_ONLY_AST); /* unoptiomized AST */
if (_PyCompile_AstPreprocess(mod, filename, &cf, optimize,
arena, syntax_check_only) < 0) {
if (_PyCompile_AstPreprocess(mod, filename, &cf, optimize, arena,
syntax_check_only, modname) < 0)
{
_PyArena_Free(arena);
goto error;
}
@@ -859,7 +870,7 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
goto error;
}
result = (PyObject*)_PyAST_Compile(mod, filename,
&cf, optimize, arena);
&cf, optimize, arena, modname);
}
_PyArena_Free(arena);
goto finally;
@@ -877,7 +888,9 @@ builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
tstate->suppress_co_const_immortalization++;
#endif
result = Py_CompileStringObject(str, filename, start[compile_mode], &cf, optimize);
result = _Py_CompileStringObjectWithModule(str, filename,
start[compile_mode], &cf,
optimize, modname);
#ifdef Py_GIL_DISABLED
tstate->suppress_co_const_immortalization--;

26
Python/clinic/bltinmodule.c.h generated
View File

@@ -238,7 +238,8 @@ PyDoc_STRVAR(builtin_chr__doc__,
PyDoc_STRVAR(builtin_compile__doc__,
"compile($module, /, source, filename, mode, flags=0,\n"
" dont_inherit=False, optimize=-1, *, _feature_version=-1)\n"
" dont_inherit=False, optimize=-1, *, module=None,\n"
" _feature_version=-1)\n"
"--\n"
"\n"
"Compile source into a code object that can be executed by exec() or eval().\n"
@@ -260,7 +261,7 @@ PyDoc_STRVAR(builtin_compile__doc__,
static PyObject *
builtin_compile_impl(PyObject *module, PyObject *source, PyObject *filename,
const char *mode, int flags, int dont_inherit,
int optimize, int feature_version);
int optimize, PyObject *modname, int feature_version);
static PyObject *
builtin_compile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
@@ -268,7 +269,7 @@ builtin_compile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObj
PyObject *return_value = NULL;
#if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
#define NUM_KEYWORDS 7
#define NUM_KEYWORDS 8
static struct {
PyGC_Head _this_is_not_used;
PyObject_VAR_HEAD
@@ -277,7 +278,7 @@ builtin_compile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObj
} _kwtuple = {
.ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
.ob_hash = -1,
.ob_item = { &_Py_ID(source), &_Py_ID(filename), &_Py_ID(mode), &_Py_ID(flags), &_Py_ID(dont_inherit), &_Py_ID(optimize), &_Py_ID(_feature_version), },
.ob_item = { &_Py_ID(source), &_Py_ID(filename), &_Py_ID(mode), &_Py_ID(flags), &_Py_ID(dont_inherit), &_Py_ID(optimize), &_Py_ID(module), &_Py_ID(_feature_version), },
};
#undef NUM_KEYWORDS
#define KWTUPLE (&_kwtuple.ob_base.ob_base)
@@ -286,14 +287,14 @@ builtin_compile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObj
# define KWTUPLE NULL
#endif // !Py_BUILD_CORE
static const char * const _keywords[] = {"source", "filename", "mode", "flags", "dont_inherit", "optimize", "_feature_version", NULL};
static const char * const _keywords[] = {"source", "filename", "mode", "flags", "dont_inherit", "optimize", "module", "_feature_version", NULL};
static _PyArg_Parser _parser = {
.keywords = _keywords,
.fname = "compile",
.kwtuple = KWTUPLE,
};
#undef KWTUPLE
PyObject *argsbuf[7];
PyObject *argsbuf[8];
Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 3;
PyObject *source;
PyObject *filename = NULL;
@@ -301,6 +302,7 @@ builtin_compile(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObj
int flags = 0;
int dont_inherit = 0;
int optimize = -1;
PyObject *modname = Py_None;
int feature_version = -1;
args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser,
@@ -359,12 +361,18 @@ skip_optional_pos:
if (!noptargs) {
goto skip_optional_kwonly;
}
feature_version = PyLong_AsInt(args[6]);
if (args[6]) {
modname = args[6];
if (!--noptargs) {
goto skip_optional_kwonly;
}
}
feature_version = PyLong_AsInt(args[7]);
if (feature_version == -1 && PyErr_Occurred()) {
goto exit;
}
skip_optional_kwonly:
return_value = builtin_compile_impl(module, source, filename, mode, flags, dont_inherit, optimize, feature_version);
return_value = builtin_compile_impl(module, source, filename, mode, flags, dont_inherit, optimize, modname, feature_version);
exit:
/* Cleanup for filename */
@@ -1277,4 +1285,4 @@ builtin_issubclass(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
exit:
return return_value;
}
/*[clinic end generated code: output=7eada753dc2e046f input=a9049054013a1b77]*/
/*[clinic end generated code: output=06500bcc9a341e68 input=a9049054013a1b77]*/

30
Python/compile.c
View File

@@ -104,11 +104,13 @@ typedef struct _PyCompiler {
* (including instructions for nested code objects)
*/
int c_disable_warning;
PyObject *c_module;
} compiler;
static int
compiler_setup(compiler *c, mod_ty mod, PyObject *filename,
PyCompilerFlags *flags, int optimize, PyArena *arena)
PyCompilerFlags *flags, int optimize, PyArena *arena,
PyObject *module)
{
PyCompilerFlags local_flags = _PyCompilerFlags_INIT;
@@ -126,6 +128,7 @@ compiler_setup(compiler *c, mod_ty mod, PyObject *filename,
if (!_PyFuture_FromAST(mod, filename, &c->c_future)) {
return ERROR;
}
c->c_module = Py_XNewRef(module);
if (!flags) {
flags = &local_flags;
}
@@ -136,7 +139,9 @@ compiler_setup(compiler *c, mod_ty mod, PyObject *filename,
c->c_optimize = (optimize == -1) ? _Py_GetConfig()->optimization_level : optimize;
c->c_save_nested_seqs = false;
if (!_PyAST_Preprocess(mod, arena, filename, c->c_optimize, merged, 0, 1)) {
if (!_PyAST_Preprocess(mod, arena, filename, c->c_optimize, merged,
0, 1, module))
{
return ERROR;
}
c->c_st = _PySymtable_Build(mod, filename, &c->c_future);
@@ -156,6 +161,7 @@ compiler_free(compiler *c)
_PySymtable_Free(c->c_st);
}
Py_XDECREF(c->c_filename);
Py_XDECREF(c->c_module);
Py_XDECREF(c->c_const_cache);
Py_XDECREF(c->c_stack);
PyMem_Free(c);
@@ -163,13 +169,13 @@ compiler_free(compiler *c)
static compiler*
new_compiler(mod_ty mod, PyObject *filename, PyCompilerFlags *pflags,
int optimize, PyArena *arena)
int optimize, PyArena *arena, PyObject *module)
{
compiler *c = PyMem_Calloc(1, sizeof(compiler));
if (c == NULL) {
return NULL;
}
if (compiler_setup(c, mod, filename, pflags, optimize, arena) < 0) {
if (compiler_setup(c, mod, filename, pflags, optimize, arena, module) < 0) {
compiler_free(c);
return NULL;
}
@@ -1221,7 +1227,8 @@ _PyCompile_Warn(compiler *c, location loc, const char *format, ...)
return ERROR;
}
int ret = _PyErr_EmitSyntaxWarning(msg, c->c_filename, loc.lineno, loc.col_offset + 1,
loc.end_lineno, loc.end_col_offset + 1);
loc.end_lineno, loc.end_col_offset + 1,
c->c_module);
Py_DECREF(msg);
return ret;
}
@@ -1476,10 +1483,10 @@ _PyCompile_OptimizeAndAssemble(compiler *c, int addNone)
PyCodeObject *
_PyAST_Compile(mod_ty mod, PyObject *filename, PyCompilerFlags *pflags,
int optimize, PyArena *arena)
int optimize, PyArena *arena, PyObject *module)
{
assert(!PyErr_Occurred());
compiler *c = new_compiler(mod, filename, pflags, optimize, arena);
compiler *c = new_compiler(mod, filename, pflags, optimize, arena, module);
if (c == NULL) {
return NULL;
}
@@ -1492,7 +1499,8 @@ _PyAST_Compile(mod_ty mod, PyObject *filename, PyCompilerFlags *pflags,
int
_PyCompile_AstPreprocess(mod_ty mod, PyObject *filename, PyCompilerFlags *cf,
int optimize, PyArena *arena, int no_const_folding)
int optimize, PyArena *arena, int no_const_folding,
PyObject *module)
{
_PyFutureFeatures future;
if (!_PyFuture_FromAST(mod, filename, &future)) {
@@ -1502,7 +1510,9 @@ _PyCompile_AstPreprocess(mod_ty mod, PyObject *filename, PyCompilerFlags *cf,
if (optimize == -1) {
optimize = _Py_GetConfig()->optimization_level;
}
if (!_PyAST_Preprocess(mod, arena, filename, optimize, flags, no_const_folding, 0)) {
if (!_PyAST_Preprocess(mod, arena, filename, optimize, flags,
no_const_folding, 0, module))
{
return -1;
}
return 0;
@@ -1627,7 +1637,7 @@ _PyCompile_CodeGen(PyObject *ast, PyObject *filename, PyCompilerFlags *pflags,
return NULL;
}
compiler *c = new_compiler(mod, filename, pflags, optimize, arena);
compiler *c = new_compiler(mod, filename, pflags, optimize, arena, NULL);
if (c == NULL) {
_PyArena_Free(arena);
return NULL;

7
Python/errors.c
View File

@@ -1960,10 +1960,11 @@ _PyErr_RaiseSyntaxError(PyObject *msg, PyObject *filename, int lineno, int col_o
*/
int
_PyErr_EmitSyntaxWarning(PyObject *msg, PyObject *filename, int lineno, int col_offset,
int end_lineno, int end_col_offset)
int end_lineno, int end_col_offset,
PyObject *module)
{
if (PyErr_WarnExplicitObject(PyExc_SyntaxWarning, msg,
filename, lineno, NULL, NULL) < 0)
if (PyErr_WarnExplicitObject(PyExc_SyntaxWarning, msg, filename, lineno,
module, NULL) < 0)
{
if (PyErr_ExceptionMatches(PyExc_SyntaxWarning)) {
/* Replace the SyntaxWarning exception with a SyntaxError

38
Python/pythonrun.c
View File

@@ -1252,12 +1252,19 @@ _PyRun_StringFlagsWithName(const char *str, PyObject* name, int start,
} else {
name = &_Py_STR(anon_string);
}
PyObject *module = NULL;
if (globals && PyDict_GetItemStringRef(globals, "__name__", &module) < 0) {
goto done;
}
mod = _PyParser_ASTFromString(str, name, start, flags, arena);
mod = _PyParser_ASTFromString(str, name, start, flags, arena, module);
Py_XDECREF(module);
if (mod != NULL) {
if (mod != NULL) {
ret = run_mod(mod, name, globals, locals, flags, arena, source, generate_new_source);
}
done:
Py_XDECREF(source);
_PyArena_Free(arena);
return ret;
@@ -1407,8 +1414,17 @@ run_mod(mod_ty mod, PyObject *filename, PyObject *globals, PyObject *locals,
return NULL;
}
}
PyObject *module = NULL;
if (globals && PyDict_GetItemStringRef(globals, "__name__", &module) < 0) {
if (interactive_src) {
Py_DECREF(interactive_filename);
}
return NULL;
}
PyCodeObject *co = _PyAST_Compile(mod, interactive_filename, flags, -1, arena);
PyCodeObject *co = _PyAST_Compile(mod, interactive_filename, flags, -1,
arena, module);
Py_XDECREF(module);
if (co == NULL) {
if (interactive_src) {
Py_DECREF(interactive_filename);
@@ -1507,6 +1523,14 @@ error:
PyObject *
Py_CompileStringObject(const char *str, PyObject *filename, int start,
PyCompilerFlags *flags, int optimize)
{
return _Py_CompileStringObjectWithModule(str, filename, start,
flags, optimize, NULL);
}
PyObject *
_Py_CompileStringObjectWithModule(const char *str, PyObject *filename, int start,
PyCompilerFlags *flags, int optimize, PyObject *module)
{
PyCodeObject *co;
mod_ty mod;
@@ -1514,14 +1538,16 @@ Py_CompileStringObject(const char *str, PyObject *filename, int start,
if (arena == NULL)
return NULL;
mod = _PyParser_ASTFromString(str, filename, start, flags, arena);
mod = _PyParser_ASTFromString(str, filename, start, flags, arena, module);
if (mod == NULL) {
_PyArena_Free(arena);
return NULL;
}
if (flags && (flags->cf_flags & PyCF_ONLY_AST)) {
int syntax_check_only = ((flags->cf_flags & PyCF_OPTIMIZED_AST) == PyCF_ONLY_AST); /* unoptiomized AST */
if (_PyCompile_AstPreprocess(mod, filename, flags, optimize, arena, syntax_check_only) < 0) {
if (_PyCompile_AstPreprocess(mod, filename, flags, optimize, arena,
syntax_check_only, module) < 0)
{
_PyArena_Free(arena);
return NULL;
}
@@ -1529,7 +1555,7 @@ Py_CompileStringObject(const char *str, PyObject *filename, int start,
_PyArena_Free(arena);
return result;
}
co = _PyAST_Compile(mod, filename, flags, optimize, arena);
co = _PyAST_Compile(mod, filename, flags, optimize, arena, module);
_PyArena_Free(arena);
return (PyObject *)co;
}

4
Python/symtable.c
View File

@@ -3137,7 +3137,7 @@ symtable_raise_if_not_coroutine(struct symtable *st, const char *msg, _Py_Source
struct symtable *
_Py_SymtableStringObjectFlags(const char *str, PyObject *filename,
int start, PyCompilerFlags *flags)
int start, PyCompilerFlags *flags, PyObject *module)
{
struct symtable *st;
mod_ty mod;
@@ -3147,7 +3147,7 @@ _Py_SymtableStringObjectFlags(const char *str, PyObject *filename,
if (arena == NULL)
return NULL;
mod = _PyParser_ASTFromString(str, filename, start, flags, arena);
mod = _PyParser_ASTFromString(str, filename, start, flags, arena, module);
if (mod == NULL) {
_PyArena_Free(arena);
return NULL;

4
Tools/peg_generator/peg_extension/peg_extension.c
View File

@@ -8,7 +8,7 @@ _build_return_object(mod_ty module, int mode, PyObject *filename_ob, PyArena *ar
PyObject *result = NULL;
if (mode == 2) {
result = (PyObject *)_PyAST_Compile(module, filename_ob, NULL, -1, arena);
result = (PyObject *)_PyAST_Compile(module, filename_ob, NULL, -1, arena, NULL);
} else if (mode == 1) {
result = PyAST_mod2obj(module);
} else {
@@ -93,7 +93,7 @@ parse_string(PyObject *self, PyObject *args, PyObject *kwds)
PyCompilerFlags flags = _PyCompilerFlags_INIT;
mod_ty res = _PyPegen_run_parser_from_string(the_string, Py_file_input, filename_ob,
&flags, arena);
&flags, arena, NULL);
if (res == NULL) {
goto error;
}