The Python language is designed to make complex tasks simple, so updates iterate relatively quickly and require us to keep up with them!
The release of a new version is always accompanied by new features and functions, and the first thing we need to understand and attribute these points before upgrading the version, so that we may use them flexibly in our programming later. Can’t wait, ready to go, so let’s start now!
1. PEP 604
New Type Union Operator
In previous versions when you wanted to declare a type to contain multiple types, you needed to use Union[]
to contain multiple types, now you can just use the |
symbol to do it.
-
Old Version
-
New Version
-
can be used in
isinstance
andissubclass
.
2. PEP 613
TypeAlias
Sometimes we want to customize a type, so we can create an alias (Alias
) to do so. However, for the type checker (Type Checker
), it can’t tell if it’s a type alias or a normal assignment, and now we can easily tell by introducing TypeAlias
.
-
The previous writing style
1
Board = List[Tuple[str, str]]
-
The new version of the writing style
3. PEP 647
User-Defined Type Guards
In contemporary static checking tools (such as typescript
, mypy
, etc.) there is a feature called Type narrowing
. This is when a parameter type could have matched more than one type, but under certain conditions the type range can be narrowed down to a smaller range of type(s).
|
|
-
Problems identified - Deficiencies exist
A more accurate knowledge of the object’s type is very friendly to
mypy
, and the conclusions of the check will be more accurate; type narrowing can be problematic in some scenarios.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
In : def is_str(obj: object) -> bool: ...: return isinstance(obj, str) ...: ...: ...: def to_list(obj: object) -> list[str]: ...: if is_str(obj): ...: return list(obj) ...: return [] ...: In : def is_str(obj: object) -> bool: ...: return isinstance(obj, str) ...: ...: ...: def to_list(obj: object) -> list[str]: ...: if is_str(obj): ...: return list(obj) ...: return [] ...: In : to_list('aaa') Out: ['a', 'a', 'a'] In : to_list(111) Out: []
This code is simpler than the one mentioned in
PEP
, they are both correct codes and the type comments are problematic. But runningmypy
prompts an error message. In the2
functionsobj
is used because it is not sure of the object type, soobject
is used, and in factto_list
will only handleobj
as typestr
. Originallyif is_str(obj)
would have narrowed the type, but because it was split into functions,isinstance
didn’t succeed in narrowing it here.1 2 3 4 5 6
➜ mypy wrong_to_list.py wrong_to_list.py:7: error: No overload variant of "list" matches argument type "object" wrong_to_list.py:7: note: Possible overload variants: wrong_to_list.py:7: note: def [_T] list(self) -> List[_T] wrong_to_list.py:7: note: def [_T] list(self, Iterable[_T]) -> List[_T] Found 1 error in 1 file (checked 1 source file)
-
User-defined
Type Guards
are provided in the new version to resolveOriginally the type of the return value was
bool
, now we specify it asTypeGuard[str]
so thatmypy
can understand its type. Actually, to put it another way, you can readTypeGuard[str]
as an alias forbool
with a type declaration, so please understand this carefully.
4. PEP 612
Parameter Specification Variables
The type system in Python
has limited support for types of Callable
(such as functions), it can only specify the type of the Callable
, but it cannot be propagated for arguments to function calls.
-
This problem exists mainly in the usage of decorators
The argument value received by
join
is supposed to be a list of strings, butmypy
does not validate this lastprint(join([1, 2]))
correctly. Because the type ofargs
andkwargs
in theinner
function in thelog
decorator isAny
, this causes the type of the arguments chosen for the call to be unverified, and frankly, it can be written any way.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
from collections.abc import Callable from typing import Any, TypeVar R = TypeVar('R') def log(func: Callable[..., R]) -> Callable[..., R]: def inner(*args: Any, **kwargs: Any) -> R: print('In') return func(*args, **kwargs) return inner @log def join(items: list[str]): return ','.join(items) print(join(['1', '2'])) # Correct usage print(join([1, 2])) # Wrong usage, mypy should prompt type error
-
Newer versions can use
ParamSpec
to resolveBy using
typing.ParamSpec
, the argument type ofinner
is passed directly throughP.args
andP.kwargs
for validation purposes.typing.ParamSpec
helps us to facilitate [referencing] positional and keyword arguments, and thisPEP
addition of `typing. is to provide an ability to add, remove or convert parameters of another callable object.The more common added arguments I can think of are the [injected] type of decorators. The
join
function has2
arguments, but since the first argumentlogger
is [injected] in thewith_logger
decorator, you only need to pass the value of theitems
argument when you use it.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
import logging from collections.abc import Callable from typing import TypeVar, ParamSpec, Concatenate logging.basicConfig(level=logging.NOTSET) R = TypeVar('R') P = ParamSpec('P') def with_logger(func: Callable[Concatenate[logging.Logger, P], R]) -> Callable[P, R]: def inner(*args: P.args, **kwargs: P.kwargs) -> R: logger = logging.getLogger(func.__name__) return func(logger, *args, **kwargs) return inner @with_logger def join(logger: logging.Logger, items: list[str]): logger.info('Info') return ','.join(items) print(join(['1', '2'])) print(join([1, 2]))
In addition to adding, removing and converting parameters can also be done with
Concatenate
, see another example of removing parameters. With theremove_first
decorator, the first argument passed in is ignored, soadd(1, 2, 3)
is actually computingadd(2, 3)
. Be careful to understand where thisConcatenate
is, if it is added, thenConcatenate
is added to the type declaration of theCallable
of the decorator argument, if it is removed, it is added to the type declaration of the returnedCallable
.
5. PEP 618
Parameters of the zip function
I believe that every Python
developer with much working experience has experienced this trap, when the elements within the argument are of different lengths, the part with the longer length is ignored, without any hint, when in fact it was originally mentioned in the documentation.
But for most developers no one really pays attention to this documentation, the problem is very implicit and requires developers to understand the zip
problem. The newer versions of strict=True
will report an error when the length of the elements within the argument is different.
|
|
6. PEP 626
Better error messages
There are some examples of error messages listed in the official that were not obvious and friendly enough in the previous version, this version deals with them centrally. Here they are: Same SyntaxError
Isn’t the hint in the new version easier to understand? Not only does it suggest the type of error, it also gives hints.
-
The old way of writing
-
New writing style
-
The old way of writing
-
New writing style
7. PEP 634-636
Match-Case
Many Python
core developers believe that Python
does not need to add switch-case
syntax, because the same effect can be achieved with if/elif/else
. This new syntax is called Structural Pattern Matching
in Chinese, and there are three PEP
s to introduce it because there are a lot of new features
- PEP 634: Introduces the
match
syntax and supported patterns. - PEP 635: Explains the reason for the syntax being designed this way
- PEP 636: a tutorial introducing the concepts, syntax and semantics
match
is followed by the variable to be matched, case
is followed by a different condition, and then the statement to be executed if the condition is met. The last case
is underlined to indicate a default match, so if the previous condition does not match, it will be executed in this case
, which is equivalent to the previous else
.
But the match-case
syntax can do much more than the switch-case
syntax in C
/ Go
languages, it is actually the match-case
syntax in Scala
/ Erlang
languages, which supports complex pattern matching, and I will demonstrate the flexibility of this new syntax and pythonic
in detail with several pattern examples.
-
[1] Literal mode
The above example is a literal pattern that uses the basic data structures that come with
Python
, such as strings, numbers, booleans, andNone
. -
[2] Capture mode
Can match the assignment target of a single expression. If
greeting
is not null, it will be assigned toname
, but note that ifgreeting
is null aNameError
orUnboundLocalError
error will be thrown, sincename
has not been defined before.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
In : capture('Alex') Hi Alex! In : capture('Santa') Hi Santa! Match In : capture('') Hello! --------------------------------------------------------------------------- UnboundLocalError Traceback (most recent call last) Input In [4], in <cell line: 1>() ----> 1 capture('') Input In [1], in capture(greeting) 1 def capture(greeting): 2 match greeting: 3 case "": 4 print("Hello!") 5 case name: 6 print(f"Hi {name}!") ----> 7 if name == "Santa": 8 print('Match') UnboundLocalError: local variable 'name' referenced before assignment
-
[3] Sequence mode
The results can be used in
match
in either list or tuple format. You can also use thefirst, *rest = seq
pattern to unpack as described in PEP 3132 - Extended Iterable Unpacking.The first element of this
match
condition needs to be1
, otherwise the match fails. The firstcase
uses a list and unpacking, the secondcase
uses a tuple, which is actually the same semantics as a list, and the third is still a list.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
In : def sequence(collection): ...: match collection: ...: case 1, [x, *others]: ...: print(f"Got 1 and a nested sequence: {x=}, {others=}") ...: case (1, x): ...: print(f"Got 1 and {x}") ...: case [x, y, z]: ...: print(f"{x=}, {y=}, {z=}") ...: In : sequence([1]) In : sequence([1, 2]) Got 1 and 2 In : sequence([1, 2, 3]) x=1, y=2, z=3 In : sequence([1, [2, 3]]) Got 1 and a nested sequence: x=2, others=[3] In : sequence([1, [2, 3, 4]]) Got 1 and a nested sequence: x=2, others=[3, 4] In : sequence([2, 3]) In : sequence((1, 2)) Got 1 and 2
If the pattern after
case
is a single item, you can remove the parentheses and write it that way. However, note thatcase 1, [x, *others]
cannot be written without the parentheses, and the logic of the decomposition will change if the parentheses are removed. -
[4] Wildcard pattern
Use the single underscore
_
to match any result, but not bound (not assigned to a variable or variables), the finalcase _
is the wildcard pattern, but of course there can be multiple matches, and the sequence pattern mentioned earlier also supports_
. The use of wildcards requires attention to the logical order, putting the small range in front and the large range in the back to prevent it from not meeting expectations.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
In : def sequence2(collection): ...: match collection: ...: case ["a", *_, "z"]: ...: print('matches any sequence of length two or more that starts with "a" and ends with "z".') ...: case (_, _, *_): ...: print('matches any sequence of length two or more.') ...: case [*_]: ...: print('matches a sequence of any length.') ...: In : sequence2(['a', 2, 3, 'z']) matches any sequence of length two or more that starts with "a" and ends with "z". In : sequence2(['a', 2, 3, 'b']) matches any sequence of length two or more. In : sequence2(['a', 'b']) matches any sequence of length two or more. In : sequence2(['a']) matches a sequence of any length.
-
[5] Constant value mode
This pattern, mainly matches constants or enumeration values of the
enum
module.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
In : class Color(Enum): ...: RED = 1 ...: GREEN = 2 ...: BLUE = 3 ...: In : class NewColor: ...: YELLOW = 4 ...: In : def constant_value(color): ...: match color: ...: case Color.RED: ...: print('Red') ...: case NewColor.YELLOW: ...: print('Yellow') ...: case new_color: ...: print(new_color) ...: In : constant_value(Color.RED) # Match the first case Red In : constant_value(NewColor.YELLOW) # Match the second case Yellow In : constant_value(Color.GREEN) # Match the third case Color.GREEN In : constant_value(4) # The constant values match the second case as well Yellow In : constant_value(10) # Other constants 10
Note here that since
case
has a binding effect, you cannot use constants likeYELLOW
directly, which is wrong. -
[6] Mapping mode
It’s actually
case
followed by support for using a dictionary to do the matching.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
In : def mapping(config): ...: match config: ...: case {'sub': sub_config, **rest}: ...: print(f'Sub: {sub_config}') ...: print(f'OTHERS: {rest}') ...: case {'route': route}: ...: print(f'ROUTE: {route}') ...: In : mapping({}) In : mapping({'route': '/auth/login'}) # Match the first case ROUTE: /auth/login # Match dictionaries with sub keys, values are bound to sub_config and the rest of the dictionary is bound to rest In : mapping({'route': '/auth/login', 'sub': {'a': 1}}) # Match the second case Sub: {'a': 1} OTHERS: {'route': '/auth/login'}
-
[7] Class mode
After
case
supports any object to do matching, this is because for matching the position needs to be determined , so the position parameter needs to be used to identify it.Another solution to this custom class that doesn’t use positional arguments for matching is to use
__match_args__
to return an array of positional arguments, like this. HerePoint2
uses the standard library’sdataclasses.dataclass
decorator, which will provide the__match_args__
property, so it can be used directly.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
In : class Point: ...: __match_args__ = ('x', 'y') ...: ...: def __init__(self, x, y): ...: self.x = x ...: self.y = y ...: In : from dataclasses import dataclass In : @dataclass ...: class Point2: ...: x: int ...: y: int ...: In : def class_pattern(obj): ...: match obj: ...: case Point(x, y): ...: print(f'Point({x=},{y=})') ...: case Point2(x, y): ...: print(f'Point2({x=},{y=})') ...: In : class_pattern(Point(1, 2)) Point(x=1,y=2) In : class_pattern(Point2(1, 2)) Point2(x=1,y=2)
-
[8] Combination (OR) mode
Multiple literals can be combined to represent an or relation using
|
, and|
can exist more than one within acase
condition to represent multiple or relations.1 2 3 4 5 6 7 8 9 10 11 12
def or_pattern(obj): match obj: case 0 | 1 | 2: # 0,1,2 three numbers match print('small number') case list() | set(): # List or set matching print('list or set') case str() | bytes(): # String or bytes match print('str or bytes') case Point(x, y) | Point2(x, y): # Borrowing from the previous 2 classes, one of them can match print(f'{x=},{y=}') case [x] | x: # list and only one element or single value matches the print(f'{x=}')
Note that the
[x]
incase [x] | x
is not triggered because of the matching order, andx
cannot be a set, string,byte
, etc., because it will not be matched in the previous condition. Also, there is nocase
syntax inPython
for theAND
relationship.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
In : or_pattern(1) small number In : or_pattern(2) small number In : or_pattern([1]) list or set In : or_pattern({1, 2}) list or set In : or_pattern('sss') str or bytes In : or_pattern(b'sd') str or bytes In : or_pattern(Point(1, 2)) x=1,y=2 In : or_pattern(Point2(1, 2)) x=1,y=2 In : or_pattern(4) x=4 In : or_pattern({}) x={}
-
[9] AS mode
The
AS
pattern was actually a walrus pattern in the early days, but it was later discussed that the use of theas
keyword would give the syntax an edge. It should be noted that[0, int() as i]
in this case is a subpattern, that is, it contains patterns within patterns:[0, int() as i]
is the sequence pattern matched bycase
, and whereint() as i
is the subpattern, it is theAS
pattern.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
In : def as_pattern(obj): ...: match obj: ...: case str() as s: ...: print(f'Got str: {s=}') ...: case [0, int() as i]: ...: print(f'Got int: {i=}') ...: case [tuple() as tu]: ...: print(f'Got tuple: {tu=}') ...: case list() | set() | dict() as iterable: ...: print(f'Got iterable: {iterable=}') ...: ...: In : as_pattern('sss') Got str: s='sss' In : as_pattern([0, 1]) Got int: i=1 In : as_pattern([(1,)]) Got tuple: tu=(1,) In : as_pattern([1, 2, 3]) Got iterable: iterable=[1, 2, 3] In : as_pattern({'a': 1}) Got iterable: iterable={'a': 1}
-
[10] Add conditions to the schema
The pattern also supports adding an
if
judgment (calledguard
), which allows the match to be further judged, equivalent to achieving some degree ofAND
effect.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
In : def go(obj): ...: match obj: ...: case ['go', direction] if direction in ['east', 'north']: ...: print('Right way') ...: case direction if direction == 'west': ...: print('Wrong way') ...: case ['go', _] | _: ...: print('Other way') ...: In : go(['go', 'east']) # Matching condition 1 Right way In : go('west') # Matching condition 2 Wrong way In : go('north') # Match default conditions Other way In : go(['go', 'west']) # Match default conditions Other way
8. BPO 12782
New Context Manager syntax
In previous versions, sometimes we wanted to put multiple contexts inside a with
on a long line, and a common way to do this was to use a backslash, which would align open
. But writing it this way caused code checking tools like pep8
and black
to report an error: they all thought that backslashes should not be used to explicitly continue a line.
-
The previous writing style
-
New writing style
Now you can add a bracket to these context managers and it’s perfect, in fact you can use
with
to enclose multiple context managers (Context Manager
) in brackets.
9. Note
Introduce the new features of the module and analyze their use and usage.
- [1] itertools.pairwise
Its a way to return iterators in the form of elements of an iterable object in order, with two elements placed next to each other in a group. The package more-itertools was available in previous versions and has now been added to the standard library.
- [2] contextlib.aclosing
The standard contextlib.closing
decorator has been introduced before, and contextlib.closing
is actually just a version of async
. As officially described, its role is to implement the following logic.
A good habit with Python
is to close the file handle when the operation is done, and the built-in open
with with
is a good practice. It works by calling ff.__exit__
to automatically close the file handle when the block is finished.
But note that not all open
methods natively support with
(or provide __exit__
and __enter__
methods), and of course there are other types of operations that need to be made sure to be closed (even if an exception is thrown). In fact, the with
syntax is generally supported by standard libraries or well-known projects, but may not be available when writing your own or corporate projects, for example, I define a class like this.
|
|
So, you can wrap it with contextlib.closing
. And contextlib.closing
is actually a version of asyncio
.
-
[3] contextlib.AsyncContextDecorator
contextlibContext.Decorator
was added inPython 3.2
, which is the basis forcontextlib.contextmanager
, as expressed in the class name, and which serves to allow the context manager to be used as a decorator.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
# The most basic context manager without parameters class mycontext: def __enter__(self): print('Starting') return self def __exit__(self, *exc): print('Finishing') return False # You can use the with syntax like this In : with mycontext(): ...: print('Inner') ...: Starting Inner Finishing
What if you want to change the management of this context to a decorator? Just make
mycontext
inherit from the base classcontextlib.ContextDecorator
, isn’t that convenient? Socontextlib.AsyncContextDecorator
is actually the version ofasyncio
.1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
In : class mycontext(contextlib.ContextDecorator): # Note the inheritance here ...: def __enter__(self): ...: print('Starting') ...: return self ...: ...: def __exit__(self, *exc): ...: print('Finishing') ...: return False ...: In : @mycontext() ...: def p(): ...: print('Inner') ...: In : p() Starting Inner Finishing
10. Reference
- https://www.escapelife.site/posts/584a5e6a.html
- # What’s New In Python 3.10
- Python 3.10 adds four new features related to the type system
- https://peps.python.org/pep-0604/
- https://peps.python.org/pep-0612/
- https://peps.python.org/pep-0647/
- https://peps.python.org/pep-0613/
- Python 3.10 Match-Case Syntax Explained