Dylan  Iqbal

Dylan Iqbal

1620281933

What’s New In Python 3.10

This article explains the new features in Python 3.10, compared to 3.9.

For full details, see the changelog.

Note: Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.10 moves towards release, so it’s worth checking back even after reading earlier versions.

Summary – Release highlights

  • PEP 644, require OpenSSL 1.1.1 or newer

New Features

Parenthesized context managers

Using enclosing parentheses for continuation across multiple lines in context managers is now supported. This allows formatting a long collection of context managers in multiple lines in a similar way as it was previously possible with import statements. For instance, all these examples are now valid:

with (CtxManager() as example):
    ...

with (
    CtxManager1(),
    CtxManager2()
):
    ...

with (CtxManager1() as example,
      CtxManager2()):
    ...

with (CtxManager1(),
      CtxManager2() as example):
    ...

with (
    CtxManager1() as example1,
    CtxManager2() as example2
):
    ...

it is also possible to use a trailing comma at the end of the enclosed group:

with (
    CtxManager1() as example1,
    CtxManager2() as example2,
    CtxManager3() as example3,
):
    ...

This new syntax uses the non LL(1) capacities of the new parser. Check PEP 617 for more details.

(Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaou in bpo-12782 and bpo-40334.)

Better error messages

SyntaxErrors

When parsing code that contains unclosed parentheses or brackets the interpreter now includes the location of the unclosed bracket of parentheses instead of displaying SyntaxError: unexpected EOF while parsing or pointing to some incorrect location. For instance, consider the following code (notice the unclosed ‘{‘):

expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
            38: 4, 39: 4, 45: 5, 46: 5, 47: 5, 48: 5, 49: 5, 54: 6,
some_other_code = foo()

Previous versions of the interpreter reported confusing places as the location of the syntax error:

File "example.py", line 3
    some_other_code = foo()
                    ^
SyntaxError: invalid syntax

but in Python 3.10 a more informative error is emitted:

File "example.py", line 1
    expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
               ^
SyntaxError: '{' was never closed

In a similar way, errors involving unclosed string literals (single and triple quoted) now point to the start of the string instead of reporting EOF/EOL.

These improvements are inspired by previous work in the PyPy interpreter.

(Contributed by Pablo Galindo in bpo-42864 and Batuhan Taskaya in bpo-40176.)

SyntaxError exceptions raised by the interpreter will now highlight the full error range of the expression that consistutes the syntax error itself, instead of just where the problem is detected. In this way, instead of displaying (before Python 3.10):

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

now Python 3.10 will display the exception as:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized

This improvement was contributed by Pablo Galindo in bpo-43914.

A considerable amount of new specialized messages for SyntaxError exceptions have been incorporated. Some of the most notable ones:

  • Missing : before blocks:
>>> if rocket.position > event_horizon
  File "<stdin>", line 1
    if rocket.position > event_horizon
                                      ^
SyntaxError: expected ':'

(Contributed by Pablo Galindo in bpo-42997)

  • Unparenthesised tuples in comprehensions targets:
>>> {x,y for x,y in range(100)}
  File "<stdin>", line 1
    {x,y for x,y in range(100)}
     ^
SyntaxError: did you forget parentheses around the comprehension target?

(Contributed by Pablo Galindo in bpo-43017)

  • Missing commas in collection literals and between expressions:
>>> items = {
... x: 1,
... y: 2
... z: 3,
  File "<stdin>", line 3
    y: 2
       ^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

(Contributed by Pablo Galindo in bpo-43822)

  • Exception groups without parentheses:
>>> try:
...     build_dyson_sphere()
... except NotEnoughScienceError, NotEnoughResourcesError:
  File "<stdin>", line 3
    except NotEnoughScienceError, NotEnoughResourcesError:
           ^
SyntaxError: exception group must be parenthesized

(Contributed by Pablo Galindo in bpo-43149)

  • Missing : and values in dictionary literals:
>>> values = {
... x: 1,
... y: 2,
... z:
... }
  File "<stdin>", line 4
    z:
     ^
SyntaxError: expression expected after dictionary key and ':'

>>> values = {x:1, y:2, z w:3}
  File "<stdin>", line 1
    values = {x:1, y:2, z w:3}
                        ^
SyntaxError: ':' expected after dictionary key

(Contributed by Pablo Galindo in bpo-43823)

  • Usage of = instead of == in comparisons:
>>> if rocket.position = event_horizon:
  File "<stdin>", line 1
    if rocket.position = event_horizon:
                       ^
SyntaxError: cannot assign to attribute here. Maybe you meant '==' instead of '='?

(Contributed by Pablo Galindo in bpo-43797)

  • Usage of * in f-strings:
>>> f"Black holes {*all_black_holes} and revelations"
  File "<stdin>", line 1
    (*all_black_holes)
     ^
SyntaxError: f-string: cannot use starred expression here

(Contributed by Pablo Galindo in bpo-41064)

IndentationErrors

Many IndentationError exceptions now have more context regarding what kind of block was expecting an indentation, including the location of the statement:

>>> def foo():
...    if lel:
...    x = 2
  File "<stdin>", line 3
    x = 2
    ^
IndentationError: expected an indented block after 'if' statement in line 2
AttributeErrors

When printing AttributeError, PyErr_Display() will offer suggestions of similar attribute names in the object that the exception was raised from:

>>> collections.namedtoplo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'collections' has no attribute 'namedtoplo'. Did you mean: namedtuple?

(Contributed by Pablo Galindo in bpo-38530.)

NameErrors

When printing NameErrorraised by the interpreter,PyErr_Display() will offer suggestions of similar variable names in the function that the exception was raised from:

>>> schwarzschild_black_hole = None
>>> schwarschild_black_hole
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?

(Contributed by Pablo Galindo in bpo-38530.)

PEP 626: Precise line numbers for debugging and other tools

PEP 626 brings more precise and reliable line numbers for debugging, profiling and coverage tools. Tracing events, with the correct line number, are generated for all lines of code executed and only for lines of code that are executed.

The f_lineno attribute of frame objects will always contain the expected line number.

The co_lnotab attribute of code objects is deprecated and will be removed in 3.12. Code that needs to convert from offset to line number should use the new co_lines() method instead.

PEP 634: Structural Pattern Matching

Structural pattern matching has been added in the form of a match statement and case statements of patterns with associated actions. Patterns consist of sequences, mappings, primitive data types as well as class instances. Pattern matching enables programs to extract information from complex data types, branch on the structure of data, and apply specific actions based on different forms of data.

Syntax and operations

The generic syntax of pattern matching is:

match subject:
    case <pattern_1>:
        <action_1>
    case <pattern_2>:
        <action_2>
    case <pattern_3>:
        <action_3>
    case _:
        <action_wildcard>

A match statement takes an expression and compares its value to successive patterns given as one or more case blocks. Specifically, pattern matching operates by:

  1. using data with type and shape (the subject)
  2. evaluating the subject in the match statement
  3. comparing the subject with each pattern in a case statement from top to bottom until a match is confirmed.
  4. executing the action associated with the pattern of the confirmed match
  5. If an exact match is not confirmed, the last case, a wildcard _, if provided, will be used as the matching case. If an exact match is not confirmed and a wildcard case does not exist, the entire match block is a no-op.
Declarative approach

Readers may be aware of pattern matching through the simple example of matching a subject (data object) to a literal (pattern) with the switch statement found in C, Java or JavaScript (and many other languages). Often the switch statement is used for comparison of an object/expression with case statements containing literals.

More powerful examples of pattern matching can be found in languages such as Scala and Elixir. With structural pattern matching, the approach is “declarative” and explicitly states the conditions (the patterns) for data to match.

While an “imperative” series of instructions using nested “if” statements could be used to accomplish something similar to structural pattern matching, it is less clear than the “declarative” approach. Instead the “declarative” approach states the conditions to meet for a match and is more readable through its explicit patterns. While structural pattern matching can be used in its simplest form comparing a variable to a literal in a case statement, its true value for Python lies in its handling of the subject’s type and shape.

Simple pattern: match to a literal

Let’s look at this example as pattern matching in its simplest form: a value, the subject, being matched to several literals, the patterns. In the example below, status is the subject of the match statement. The patterns are each of the case statements, where literals represent request status codes. The associated action to the case is executed after a match:

def http_error(status):
    match status:
        case 400:
            return "Bad request"
        case 404:
            return "Not found"
        case 418:
            return "I'm a teapot"
        case _:
            return "Something's wrong with the Internet"

If the above function is passed a status of 418, “I’m a teapot” is returned. If the above function is passed a status of 500, the case statement with _ will match as a wildcard, and “Something’s wrong with the Internet” is returned. Note the last block: the variable name, _, acts as a wildcard and insures the subject will always match. The use of _ is optional.

You can combine several literals in a single pattern using | (“or”):

case 401 | 403 | 404:
    return "Not allowed"
Behavior without the wildcard

If we modify the above example by removing the last case block, the example becomes:

def http_error(status):
    match status:
        case 400:
            return "Bad request"
        case 404:
            return "Not found"
        case 418:
            return "I'm a teapot"

Without the use of _ in a case statement, a match may not exist. If no match exists, the behavior is a no-op. For example, if status of 500 is passed, a no-op occurs.

Patterns with a literal and variable

Patterns can look like unpacking assignments, and a pattern may be used to bind variables. In this example, a data point can be unpacked to its x-coordinate and y-coordinate:

## point is an (x, y) tuple
match point:
    case (0, 0):
        print("Origin")
    case (0, y):
        print(f"Y={y}")
    case (x, 0):
        print(f"X={x}")
    case (x, y):
        print(f"X={x}, Y={y}")
    case _:
        raise ValueError("Not a point")

The first pattern has two literals, (0, 0), and may be thought of as an extension of the literal pattern shown above. The next two patterns combine a literal and a variable, and the variable binds a value from the subject (point). The fourth pattern captures two values, which makes it conceptually similar to the unpacking assignment (x, y) = point.

Patterns and classes

If you are using classes to structure your data, you can use as a pattern the class name followed by an argument list resembling a constructor. This pattern has the ability to capture class attributes into variables:

class Point:
    x: int
    y: int

def location(point):
    match point:
        case Point(x=0, y=0):
            print("Origin is the point's location.")
        case Point(x=0, y=y):
            print(f"Y={y} and the point is on the y-axis.")
        case Point(x=x, y=0):
            print(f"X={x} and the point is on the x-axis.")
        case Point():
            print("The point is located somewhere else on the plane.")
        case _:
            print("Not a point")
Patterns with positional parameters

You can use positional parameters with some builtin classes that provide an ordering for their attributes (e.g. dataclasses). You can also define a specific position for attributes in patterns by setting the __match_args__ special attribute in your classes. If it’s set to (“x”, “y”), the following patterns are all equivalent (and all bind the y attribute to the var variable):

Point(1, var)
Point(1, y=var)
Point(x=1, y=var)
Point(y=var, x=1)
Nested patterns

Patterns can be arbitrarily nested. For example, if our data is a short list of points, it could be matched like this:

match points:
    case []:
        print("No points in the list.")
    case [Point(0, 0)]:
        print("The origin is the only point in the list.")
    case [Point(x, y)]:
        print(f"A single point {x}, {y} is in the list.")
    case [Point(0, y1), Point(0, y2)]:
        print(f"Two points on the Y axis at {y1}, {y2} are in the list.")
    case _:
        print("Something else is found in the list.")
Complex patterns and the wildcard

To this point, the examples have used _ alone in the last case statement. A wildcard can be used in more complex patterns, such as ('error', code, _). For example:

match test_variable:
    case ('warning', code, 40):
        print("A warning has been received.")
    case ('error', code, _):
        print(f"An error {code} occurred.")

In the above case, test_variable will match for (‘error’, code, 100) and (‘error’, code, 800).

Guard

We can add an if clause to a pattern, known as a “guard”. If the guard is false, match goes on to try the next case block. Note that value capture happens before the guard is evaluated:

match point:
    case Point(x, y) if x == y:
        print(f"The point is located on the diagonal Y=X at {x}.")
    case Point(x, y):
        print(f"Point is not on the diagonal.")
Other Key Features

Several other key features:

  • Like unpacking assignments, tuple and list patterns have exactly the same meaning and actually match arbitrary sequences. Technically, the subject must be an instance of collections.abc.Sequence. Therefore, an important exception is that patterns don’t match iterators. Also, to prevent a common mistake, sequence patterns don’t match strings.
  • Sequence patterns support wildcards: [x, y, *rest] and (x, y, *rest) work similar to wildcards in unpacking assignments. The name after * may also be _, so (x, y, *_) matches a sequence of at least two items without binding the remaining items.
  • Mapping patterns: {"bandwidth": b, "latency": l} captures the "bandwidth" and "latency" values from a dict. Unlike sequence patterns, extra keys are ignored. A wildcard **rest is also supported. (But **_ would be redundant, so is not allowed.)
  • Subpatterns may be captured using the as keyword:
case (Point(x1, y1), Point(x2, y2) as p2): ...
  • This binds x1, y1, x2, y2 like you would expect without the as clause, and p2 to the entire second item of the subject.
  • Most literals are compared by equality. However, the singletons True, False and None are compared by identity.
  • Named constants may be used in patterns. These named constants must be dotted names to prevent the constant from being interpreted as a capture variable:
from enum import Enum
class Color(Enum):
    RED = 0
    GREEN = 1
    BLUE = 2

match color:
    case Color.RED:
        print("I see red!")
    case Color.GREEN:
        print("Grass is green")
    case Color.BLUE:
        print("I'm feeling the blues :(")

For the full specification see PEP 634. Motivation and rationale are in PEP 635, and a longer tutorial is in PEP 636.

Optional EncodingWarning and encoding="locale" option

The default encoding of TextIOWrapper and open() is platform and locale dependent. Since UTF-8 is used on most Unix platforms, omitting encoding option when opening UTF-8 files (e.g. JSON, YAML, TOML, Markdown) is a very common bug. For example:

## BUG: "rb" mode or encoding="utf-8" should be used.
with open("data.json") as f:
    data = json.load(f)

To find this type of bug, an optional EncodingWarning is added. It is emitted when sys.flags.warn_default_encoding is true and locale-specific default encoding is used.

-X warn_default_encoding option and PYTHONWARNDEFAULTENCODING are added to enable the warning.

See Text Encoding for more information.

New Features Related to Type Hints

This section covers major changes affecting PEP 484 type hints and the typing module.

PEP 604: New Type Union Operator

A new type union operator was introduced which enables the syntax X | Y. This provides a cleaner way of expressing ‘either type X or type Y’ instead of using typing.Union, especially in type hints.

In previous versions of Python, to apply a type hint for functions accepting arguments of multiple types, typing.Union was used:

def square(number: Union[int, float]) -> Union[int, float]:
    return number ** 2

Type hints can now be written in a more succinct manner:

def square(number: int | float) -> int | float:
    return number ** 2

This new syntax is also accepted as the second argument to isinstance()andissubclass():

>>> isinstance(1, int | str)
True

See Union Type and PEP 604 for more details.

(Contributed by Maggie Moss and Philippe Prados in bpo-41428.)

PEP 612: Parameter Specification Variables

Two new options to improve the information provided to static type checkers for PEP 484‘s Callable have been added to the typing module.

The first is the parameter specification variable. They are used to forward the parameter types of one callable to another callable – a pattern commonly found in higher order functions and decorators. Examples of usage can be found in typing.ParamSpec. Previously, there was no easy way to type annotate dependency of parameter types in such a precise manner.

The second option is the new Concatenate operator. It’s used in conjunction with parameter specification variables to type annotate a higher order callable which adds or removes parameters of another callable. Examples of usage can be found in typing.Concatenate.

See typing.Callable, typing.ParamSpec, typing.Concatenate, typing.ParamSpecArgs, typing.ParamSpecKwargs, and PEP 612 for more details.

(Contributed by Ken Jin in bpo-41559, with minor enhancements by Jelle Zijlstra in bpo-43783. PEP written by Mark Mendoza.)

PEP 613: TypeAlias

PEP 484 introduced the concept of type aliases, only requiring them to be top-level unannotated assignments. This simplicity sometimes made it difficult for type checkers to distinguish between type aliases and ordinary assignments, especially when forward references or invalid types were involved. Compare:

StrCache = 'Cache[str]'  ## a type alias
LOG_PREFIX = 'LOG[DEBUG]'  ## a module constant

Now the typing module has a special value TypeAlias which lets you declare type aliases more explicitly:

StrCache: TypeAlias = 'Cache[str]'  ## a type alias
LOG_PREFIX = 'LOG[DEBUG]'  ## a module constant

See PEP 613 for more details.

(Contributed by Mikhail Golubev in bpo-41923.)

PEP 647: User-Defined Type Guards

TypeGuard has been added to the typing module to annotate type guard functions and improve information provided to static type checkers during type narrowing. For more information, please see TypeGuard‘s documentation, and PEP 647.

(Contributed by Ken Jin and Guido van Rossum in bpo-43766. PEP written by Eric Traut.)

Other Language Changes

  • The int type has a new method int.bit_count(), returning the number of ones in the binary expansion of a given integer, also known as the population count. (Contributed by Niklas Fiekas in bpo-29882.)
  • The views returned by dict.keys(), dict.values() and dict.items() now all have a mapping attribute that gives a types.MappingProxyType object wrapping the original dictionary. (Contributed by Dennis Sweeney in bpo-40890.)
  • PEP 618: The zip() function now has an optional strict flag, used to require that all the iterables have an equal length.
  • Builtin and extension functions that take integer arguments no longer accept Decimals, Fractions and other objects that can be converted to integers only with a loss (e.g. that have the int() method but do not have the index() method). (Contributed by Serhiy Storchaka in bpo-37999.)
  • If object.ipow() returns NotImplemented, the operator will correctly fall back to object.pow() and object.rpow() as expected. (Contributed by Alex Shkop in bpo-38302.)
  • Assignment expressions can now be used unparenthesized within set literals and set comprehensions, as well as in sequence indexes (but not slices).
  • Functions have a new __builtins__ attribute which is used to look for builtin symbols when a function is executed, instead of looking into __globals__[__builtins__]. The attribute is initialized from __globals__["__builtins__"] if it exists, else from the current builtins. (Contributed by Mark Shannon in bpo-42990.)
  • Two new builtin functions – aiter() and anext() have been added to provide asynchronous counterparts to iter() and next(), respectively. (Contributed by Joshua Bronson, Daniel Pope, and Justin Wang in bpo-31861.)
  • Static methods (@staticmethod) and class methods (@classmethod) now inherit the method attributes (__module__, __name__, __qualname__, __doc__, __annotations__) and have a new __wrapped__ attribute. Moreover, static methods are now callable as regular functions. (Contributed by Victor Stinner in bpo-43682.)
  • Annotations for complex targets (everything beside simple name targets defined by PEP 526) no longer cause any runtime effects with from __future__ import annotations. (Contributed by Batuhan Taskaya in bpo-42737.)
  • Class and module objects now lazy-create empty annotations dicts on demand. The annotations dicts are stored in the object’s __dict__ for backwards compatibility. This improves the best practices for working with __annotations__; for more information, please see Annotations Best Practices. (Contributed by Larry Hastings in bpo-43901.)

New Modules

  • None yet.

Improved Modules

asyncio

Add missing connect_accepted_socket() method. (Contributed by Alex Grönholm in bpo-41332.)

argparse

Misleading phrase “optional arguments” was replaced with “options” in argparse help. Some tests might require adaptation if they rely on exact output match. (Contributed by Raymond Hettinger in bpo-9694.)

array

The index() method of array.array now has optional start and stop parameters. (Contributed by Anders Lorentsen and Zackery Spytz in bpo-31956.)

base64

Add base64.b32hexencode() and base64.b32hexdecode() to support the Base32 Encoding with Extended Hex Alphabet.

bdb

Add clearBreakpoints() to reset all set breakpoints. (Contributed by Irit Katriel in bpo-24160.)

codecs

Add a codecs.unregister() function to unregister a codec search function. (Contributed by Hai Shi in bpo-41842.)

collections.abc

The __args__ of the parameterized generic for collections.abc.Callable are now consistent with typing.Callable. collections.abc.Callable generic now flattens type parameters, similar to what typing.Callable currently does. This means that collections.abc.Callable[[int, str], str] will have __args__ of (int, str, str); previously this was ([int, str], str). To allow this change, types.GenericAlias can now be subclassed, and a subclass will be returned when subscripting the collections.abc.Callable type. Note that a TypeError may be raised for invalid forms of parameterizing collections.abc.Callable which may have passed silently in Python 3.9. (Contributed by Ken Jin in bpo-42195.)

contextlib

Add a contextlib.aclosing() context manager to safely close async generators and objects representing asynchronously released resources. (Contributed by Joongi Kim and John Belmonte in bpo-41229.)

Add asynchronous context manager support to contextlib.nullcontext(). (Contributed by Tom Gringauz in bpo-41543.)

Add AsyncContextDecorator, for supporting usage of async context managers as decorators.

curses

The extended color functions added in ncurses 6.1 will be used transparently by curses.color_content(), curses.init_color(), curses.init_pair(), and curses.pair_content(). A new function, curses.has_extended_color_support(), indicates whether extended color support is provided by the underlying ncurses library. (Contributed by Jeffrey Kintscher and Hans Petter Jansson in bpo-36982.)

The BUTTON5_* constants are now exposed in the curses module if they are provided by the underlying curses library. (Contributed by Zackery Spytz in bpo-39273.)

dataclasses

Add slots parameter in dataclasses.dataclass() decorator. (Contributed by Yurii Karabas in bpo-42269)

distutils

The entire distutils package is deprecated, to be removed in Python 3.12. Its functionality for specifying package builds has already been completely replaced by third-party packages setuptools and packaging, and most other commonly used APIs are available elsewhere in the standard library (such as platform, shutil, subprocess or sysconfig). There are no plans to migrate any other functionality from distutils, and applications that are using other functions should plan to make private copies of the code. Refer to PEP 632 for discussion.

The bdist_wininst command deprecated in Python 3.8 has been removed. The bdist_wheel command is now recommended to distribute binary packages on Windows. (Contributed by Victor Stinner in bpo-42802.)

doctest

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in bpo-42133.)

encodings

encodings.normalize_encoding() now ignores non-ASCII characters. (Contributed by Hai Shi in bpo-39337.)

enum

Enum __repr__() now returns enum_name.member_name and __str__() now returns member_name. Stdlib enums available as module constants have a repr() of module_name.member_name. (Contributed by Ethan Furman in bpo-40066.)

Add enum.StrEnum for enums where all members are strings. (Contributed by Ethan Furman in bpo-41816.)

fileinput

Add encoding and errors parameters in fileinput.input() and fileinput.FileInput. (Contributed by Inada Naoki in bpo-43712.)

fileinput.hook_compressed() now returns TextIOWrapper object when mode is “r” and file is compressed, like uncompressed files. (Contributed by Inada Naoki in bpo-5758.)

gc

Add audit hooks for gc.get_objects(), gc.get_referrers() and gc.get_referents(). (Contributed by Pablo Galindo in bpo-43439.)

glob

Add the root_dir and dir_fd parameters in glob() and iglob() which allow to specify the root directory for searching. (Contributed by Serhiy Storchaka in bpo-38144.)

hashlib

The hashlib module requires OpenSSL 1.1.1 or newer. (Contributed by Christian Heimes in PEP 644 and bpo-43669.)

The hashlib module has preliminary support for OpenSSL 3.0.0. (Contributed by Christian Heimes in bpo-38820 and other issues.)

The pure-Python fallback of pbkdf2_hmac() is deprecated. In the future PBKDF2-HMAC will only be available when Python has been built with OpenSSL support. (Contributed by Christian Heimes in bpo-43880.)

hmac

The hmac module now uses OpenSSL’s HMAC implementation internally. (Contributed by Christian Heimes in bpo-40645.)

IDLE and idlelib

Make IDLE invoke sys.excepthook() (when started without ‘-n’). User hooks were previously ignored. (Patch by Ken Hilton in bpo-43008.)

This change was backported to a 3.9 maintenance release.

Add a Shell sidebar. Move the primary prompt (‘>>>’) to the sidebar. Add secondary prompts (‘…’) to the sidebar. Left click and optional drag selects one or more lines of text, as with the editor line number sidebar. Right click after selecting text lines displays a context menu with ‘copy with prompts’. This zips together prompts from the sidebar with lines from the selected text. This option also appears on the context menu for the text. (Contributed by Tal Einat in bpo-37903.)

Use spaces instead of tabs to indent interactive code. This makes interactive code entries ‘look right’. Making this feasible was a major motivation for adding the shell sidebar. Contributed by Terry Jan Reedy in bpo-37892.)

We expect to backport these shell changes to a future 3.9 maintenance release.

importlib.metadata

Feature parity with importlib_metadata 3.7.

importlib.metadata.entry_points() now provides a nicer experience for selecting entry points by group and name through a new importlib.metadata.EntryPoints class.

Added importlib.metadata.packages_distributions() for resolving top-level Python modules and packages to their importlib.metadata.Distribution.

inspect

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in bpo-42133.)

Add inspect.get_annotations(), which safely computes the annotations defined on an object. It works around the quirks of accessing the annotations on various types of objects, and makes very few assumptions about the object it examines. inspect.get_annotations() can also correctly un-stringize stringized annotations. inspect.get_annotations() is now considered best practice for accessing the annotations dict defined on any Python object; for more information on best practices for working with annotations, please see Annotations Best Practices. Relatedly, inspect.signature(), inspect.Signature.from_callable(), and inspect.Signature.from_function() now call inspect.get_annotations() to retrieve annotations. This means inspect.signature() and inspect.Signature.from_callable() can also now un-stringize stringized annotations. (Contributed by Larry Hastings in bpo-43817.)

linecache

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in bpo-42133.)

os

Add os.cpu_count() support for VxWorks RTOS. (Contributed by Peixing Xin in bpo-41440.)

Add a new function os.eventfd() and related helpers to wrap the eventfd2 syscall on Linux. (Contributed by Christian Heimes in bpo-41001.)

Add os.splice() that allows to move data between two file descriptors without copying between kernel address space and user address space, where one of the file descriptors must refer to a pipe. (Contributed by Pablo Galindo in bpo-41625.)

Add O_EVTONLY, O_FSYNC, O_SYMLINK and O_NOFOLLOW_ANY for macOS. (Contributed by Dong-hee Na in bpo-43106.)

pathlib

Add slice support to PurePath.parents. (Contributed by Joshua Cannon in bpo-35498)

Add negative indexing support to PurePath.parents. (Contributed by Yaroslav Pankovych in bpo-21041)

Add Path.hardlink_to method that supersedes link_to(). The new method has the same argument order as symlink_to(). (Contributed by Barney Gale in bpo-39950.)

platform

Add platform.freedesktop_os_release() to retrieve operation system identification from freedesktop.org os-release standard file. (Contributed by Christian Heimes in bpo-28468)

pprint

pprint can now pretty-print dataclasses.dataclass instances. (Contributed by Lewis Gaul in bpo-43080.)

py_compile

Add --quiet option to command-line interface of py_compile. (Contributed by Gregory Schevchenko in bpo-38731.)

pyclbr

Add an end_lineno attribute to the Function and Class objects in the tree returned by pyclbr.readline() and pyclbr.readline_ex(). It matches the existing (start) lineno. (Contributed by Aviral Srivastava in bpo-38307.)

shelve

The shelve module now uses pickle.DEFAULT_PROTOCOL by default instead of pickle protocol 3 when creating shelves. (Contributed by Zackery Spytz in bpo-34204.)

statistics

Add covariance(), Pearson’s correlation(), and simple linear_regression() functions. (Contributed by Tymoteusz Wołodźko in bpo-38490.)

site

When a module does not define __loader__, fall back to __spec__.loader. (Contributed by Brett Cannon in bpo-42133.)

socket

The exception socket.timeout is now an alias of TimeoutError. (Contributed by Christian Heimes in bpo-42413.)

Add option to create MPTCP sockets with IPPROTO_MPTCP (Contributed by Rui Cunha in bpo-43571.)

ssl

The ssl module requires OpenSSL 1.1.1 or newer. (Contributed by Christian Heimes in PEP 644 and bpo-43669.)

The ssl module has preliminary support for OpenSSL 3.0.0 and new option OP_IGNORE_UNEXPECTED_EOF. (Contributed by Christian Heimes in bpo-38820, bpo-43794, bpo-43788, bpo-43791, bpo-43799, bpo-43920, bpo-43789, and bpo-43811.)

Deprecated function and use of deprecated constants now result in a DeprecationWarning. The following features have been deprecated since Python 3.6, Python 3.7, or OpenSSL 1.1.0: OP_NO_SSLv2, OP_NO_SSLv3, OP_NO_TLSv1, OP_NO_TLSv1_1, OP_NO_TLSv1_2, OP_NO_TLSv1_3, PROTOCOL_SSLv2, PROTOCOL_SSLv3, PROTOCOL_SSLv23, PROTOCOL_TLSv1, PROTOCOL_TLSv1_1, PROTOCOL_TLSv1_2, PROTOCOL_TLS, wrap_socket(), match_hostname(), RAND_pseudo_bytes(), RAND_egd(), ssl.SSLSocket.selected_npn_protocol(), ssl.SSLContext.set_npn_protocols(). (Contributed by Christian Heimes in bpo-43880.)

The ssl module now has more secure default settings. Ciphers without forward secrecy or SHA-1 MAC are disabled by default. Security level 2 prohibits weak RSA, DH, and ECC keys with less than 112 bits of security. SSLContext defaults to minimum protocol version TLS 1.2. Settings are based on Hynek Schlawack’s research. (Contributed by Christian Heimes in bpo-43998.)

The deprecated protocols SSL 3.0, TLS 1.0, and TLS 1.1 are no longer officially supported. Python does not block them actively. However OpenSSL build options, distro configurations, vendor patches, and cipher suites may prevent a successful handshake.

Add a timeout parameter to the ssl.get_server_certificate() function. (Contributed by Zackery Spytz in bpo-31870.)

The ssl module uses heap-types and multi-phase initialization. (Contributed by Christian Heimes in bpo-42333.)

A new verify flag VERIFY_X509_PARTIAL_CHAIN has been added. (Contributed by l0x in bpo-40849.)

sqlite3

Add audit events for connect/handle(), enable_load_extension(), and load_extension(). (Contributed by Erlend E. Aasland in bpo-43762.)

sys

Add sys.orig_argv attribute: the list of the original command line arguments passed to the Python executable. (Contributed by Victor Stinner in bpo-23427.)

Add sys.stdlib_module_names, containing the list of the standard library module names. (Contributed by Victor Stinner in bpo-42955.)

_thread

_thread.interrupt_main() now takes an optional signal number to simulate (the default is still signal.SIGINT). (Contributed by Antoine Pitrou in bpo-43356.)

threading

Add threading.gettrace() and threading.getprofile() to retrieve the functions set by threading.settrace() and threading.setprofile() respectively. (Contributed by Mario Corchero in bpo-42251.)

Add threading.excepthook to allow retrieving the original value of threading.excepthook() in case it is set to a broken or a different value. (Contributed by Mario Corchero in bpo-42308.)

traceback

The format_exception(), format_exception_only(), and print_exception() functions can now take an exception object as a positional-only argument. (Contributed by Zackery Spytz and Matthias Bussonnier in bpo-26389.)

types

Reintroduce the types.EllipsisType, types.NoneType and types.NotImplementedType classes, providing a new set of types readily interpretable by type checkers. (Contributed by Bas van Beek in bpo-41810.)

typing

For major changes, see New Features Related to Type Hints.

The behavior of typing.Literal was changed to conform with PEP 586 and to match the behavior of static type checkers specified in the PEP.

  1. Literal now de-duplicates parameters.
  2. Equality comparisons between Literal objects are now order independent.
  3. Literal comparisons now respects types. For example, Literal[0] == Literal[False] previously evaluated to True. It is now False. To support this change, the internally used type cache now supports differentiating types.
  4. Literal objects will now raise a TypeError exception during equality comparisons if any of their parameters are not hashable. Note that declaring Literal with unhashable parameters will not throw an error:
>>> from typing import Literal
>>> Literal[{0}]
>>> Literal[{0}] == Literal[{False}]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'set'

(Contributed by Yurii Karabas in bpo-42345.)

Add new function typing.is_typeddict() to introspect if an annotation is a typing.TypedDict. (Contributed by Patrick Reader in bpo-41792)

unittest

Add new method assertNoLogs() to complement the existing assertLogs(). (Contributed by Kit Yan Choi in bpo-39385.)

urllib.parse

Python versions earlier than Python 3.10 allowed using both ; and & as query parameter separators in urllib.parse.parse_qs() and urllib.parse.parse_qsl(). Due to security concerns, and to conform with newer W3C recommendations, this has been changed to allow only a single separator key, with & as the default. This change also affects cgi.parse()andcgi.parse_multipart() as they use the affected functions internally. For more details, please see their respective documentation. (Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in bpo-42967.)

xml

Add a LexicalHandler class to the xml.sax.handler module. (Contributed by Jonathan Gossage and Zackery Spytz in bpo-35018.)

zipimport

Add methods related to PEP 451: find_spec(), zipimport.zipimporter.create_module(), and zipimport.zipimporter.exec_module(). (Contributed by Brett Cannon in bpo-42131.)

Add invalidate_caches() method. (Contributed by Desmond Cheong in bpo-14678.)

Optimizations

  • Constructors str(), bytes() and bytearray() are now faster (around 30–40% for small objects). (Contributed by Serhiy Storchaka in bpo-41334.)
  • The runpy module now imports fewer modules. The python3 -m module-name command startup time is 1.4x faster in average. On Linux, python3 -I -m module-name imports 69 modules on Python 3.9, whereas it only imports 51 modules (-18) on Python 3.10. (Contributed by Victor Stinner in bpo-41006 and bpo-41718.)
  • The LOAD_ATTR instruction now uses new “per opcode cache” mechanism. It is about 36% faster now for regular attributes and 44% faster for slots. (Contributed by Pablo Galindo and Yury Selivanov in bpo-42093 and Guido van Rossum in bpo-42927, based on ideas implemented originally in PyPy and MicroPython.)
  • When building Python with –enable-optimizations now -fno-semantic-interposition is added to both the compile and link line. This speeds builds of the Python interpreter created with –enable-shared with gcc by up to 30%. See this article for more details. (Contributed by Victor Stinner and Pablo Galindo in bpo-38980.)
  • Use a new output buffer management code for bz2 / lzma / zlib modules, and add .readall() function to _compression.DecompressReader class. bz2 decompression is now 1.09x ~ 1.17x faster, lzma decompression 1.20x ~ 1.32x faster, `GzipFile.read(-1) 1.11x ~ 1.18x faster. (Contributed by Ma Lin, reviewed by Gregory P. Smith, in bpo-41486)
  • When using stringized annotations, annotations dicts for functions are no longer created when the function is created. Instead, they are stored as a tuple of strings, and the function object lazily converts this into the annotations dict on demand. This optimization cuts the CPU time needed to define an annotated function by half. (Contributed by Yurii Karabas and Inada Naoki in bpo-42202)
  • Substring search functions such as str1 in str2 and str2.find(str1) now sometimes use Crochemore & Perrin’s “Two-Way” string searching algorithm to avoid quadratic behavior on long strings. (Contributed by Dennis Sweeney in bpo-41972)
  • Add micro-optimizations to _PyType_Lookup() to improve type attribute cache lookup performance in the common case of cache hits. This makes the interpreter 1.04 times faster on average. (Contributed by Dino Viehland in bpo-43452)
  • The following built-in functions now support the faster PEP 590 vectorcall calling convention: map(), filter(), reversed(), bool() and float(). (Contributed by Dong-hee Na and Jeroen Demeyer in bpo-43575, bpo-43287, bpo-41922, bpo-41873 and bpo-41870)
  • BZ2File performance is improved by removing internal RLock. This makes BZ2File thread unsafe in the face of multiple simultaneous readers or writers, just like its equivalent classes in gzip and lzma have always been. (Contributed by Inada Naoki in bpo-43785).

Deprecated

Removed

  • Removed special methods __int__, __float__, __floordiv__, __mod__, __divmod__, __rfloordiv__, __rmod__ and __rdivmod__ of the complexclass. They always raised aTypeError. (Contributed by Serhiy Storchaka in bpo-41974.)
  • The ParserBase.error() method from the private and undocumented _markupbase module has been removed. html.parser.HTMLParser is the only subclass of ParserBase and its `error() implementation was already removed in Python 3.5. (Contributed by Berker Peksag in bpo-31844.)
  • Removed the unicodedata.ucnhash_CAPI attribute which was an internal PyCapsule object. The related private _PyUnicode_Name_CAPI structure was moved to the internal C API. (Contributed by Victor Stinner in bpo-42157.)
  • Removed the parser module, which was deprecated in 3.9 due to the switch to the new PEG parser, as well as all the C source and header files that were only being used by the old parser, including node.h, parser.h, graminit.h and grammar.h.
  • Removed the Public C API functions PyParser_SimpleParseStringFlags(), PyParser_SimpleParseStringFlagsFilename(), PyParser_SimpleParseFileFlags() and PyNode_Compile() that were deprecated in 3.9 due to the switch to the new PEG parser.
  • Removed the formatter module, which was deprecated in Python 3.4. It is somewhat obsolete, little used, and not tested. It was originally scheduled to be removed in Python 3.6, but such removals were delayed until after Python 2.7 EOL. Existing users should copy whatever classes they use into their code. (Contributed by Dong-hee Na and Terry J. Reedy in bpo-42299.)
  • Removed the PyModule_GetWarningsModule() function that was useless now due to the _warnings module was converted to a builtin module in 2.6. (Contributed by Hai Shi in bpo-42599.)
  • Remove deprecated aliases to Collections Abstract Base Classes from the collections module. (Contributed by Victor Stinner in bpo-37324.)
  • The loop parameter has been removed from most of asyncio‘s high-level API following deprecation in Python 3.8. The motivation behind this change is multifold:
  1. This simplifies the high-level API.
  2. The functions in the high-level API have been implicitly getting the current thread’s running event loop since Python 3.7. There isn’t a need to pass the event loop to the API in most normal use cases.
  3. Event loop passing is error-prone especially when dealing with loops running in different threads.
  • Note that the low-level API will still accept loop. See Changes in the Python API for examples of how to replace existing code.
  • (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley in bpo-42392.)

Porting to Python 3.10

This section lists previously described changes and other bugfixes that may require changes to your code.

Changes in the Python API

async def foo(loop):
    await asyncio.sleep(1, loop=loop)
  • Should be replaced with this:
async def foo():
    await asyncio.sleep(1)
  • If foo() was specifically designed not to run in the current thread’s running event loop (e.g. running in another thread’s event loop), consider using asyncio.run_coroutine_threadsafe() instead.
  • (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley in bpo-42392.)
  • The types.FunctionType constructor now inherits the current builtins if the globals dictionary has no "__builtins__" key, rather than using {"None": None} as builtins: same behavior as eval() and exec() functions. Defining a function with def function(...): ... in Python is not affected, globals cannot be overriden with this syntax: it also inherits the current builtins. (Contributed by Victor Stinner in bpo-42990.)

CPython bytecode changes

  • The MAKE_FUNCTION instruction now accepts either a dict or a tuple of strings as the function’s annotations. (Contributed by Yurii Karabas and Inada Naoki in bpo-42202)

Build Changes

  • PEP 644: Python now requires OpenSSL 1.1.1 or newer. OpenSSL 1.0.2 is no longer supported. (Contributed by Christian Heimes in bpo-43669.)
  • The C99 functions snprintf() and vsnprintf() are now required to build Python. (Contributed by Victor Stinner in bpo-36020.)
  • sqlite3 requires SQLite 3.7.15 or higher. (Contributed by Sergey Fedoseev and Erlend E. Aasland bpo-40744 and bpo-40810.)
  • The atexit module must now always be built as a built-in module. (Contributed by Victor Stinner in bpo-42639.)
  • Add –disable-test-modules option to the configure script: don’t build nor install test modules. (Contributed by Xavier de Gaye, Thomas Petazzoni and Peixing Xin in bpo-27640.)
  • Add –with-wheel-pkg-dir=PATH option to the ./configure script. If specified, the ensurepip module looks for setuptools and pip wheel packages in this directory: if both are present, these wheel packages are used instead of ensurepip bundled wheel packages.
  • Some Linux distribution packaging policies recommend against bundling dependencies. For example, Fedora installs wheel packages in the /usr/share/python-wheels/ directory and don’t install the ensurepip._bundled package.
  • (Contributed by Victor Stinner in bpo-42856.)
  • Add a new configure --without-static-libpython option to not build the libpythonMAJOR.MINOR.a static library and not install the python.o object file.
  • (Contributed by Victor Stinner in bpo-43103.)
  • The configure script now uses the pkg-config utility, if available, to detect the location of Tcl/Tk headers and libraries. As before, those locations can be explicitly specified with the –with-tcltk-includes and –with-tcltk-libs configuration options. (Contributed by Manolis Stamatogiannakis in bpo-42603.)
  • Add –with-openssl-rpath option to configure script. The option simplifies building Python with a custom OpenSSL installation, e.g. ./configure --with-openssl=/path/to/openssl --with-openssl-rpath=auto. (Contributed by Christian Heimes in bpo-43466.)

C API Changes

New Features

Porting to Python 3.10

#if PY_VERSION_HEX < 0x030900A4
##  define Py_SET_REFCNT(obj, refcnt) ((Py_REFCNT(obj) = (refcnt)), (void)0)
#endif

Deprecated

  • The PyUnicode_InternImmortal() function is now deprecated and will be removed in Python 3.12: use PyUnicode_InternInPlace() instead. (Contributed by Victor Stinner in bpo-41692.)

Removed

  • PyObject_AsCharBuffer(), PyObject_AsReadBuffer(), PyObject_CheckReadBuffer(), and PyObject_AsWriteBuffer() are removed. Please migrate to new buffer protocol; PyObject_GetBuffer()andPyBuffer_Release(). (Contributed by Inada Naoki in bpo-41103.)
  • Removed Py_UNICODE_str* functions manipulating Py_UNICODE* strings. (Contributed by Inada Naoki in bpo-41123.)
  • Py_UNICODE_strlen: use PyUnicode_GetLength() or PyUnicode_GET_LENGTH
  • Py_UNICODE_strcat: use PyUnicode_CopyCharacters() or PyUnicode_FromFormat()
  • Py_UNICODE_strcpy, Py_UNICODE_strncpy: use PyUnicode_CopyCharacters() or PyUnicode_Substring()
  • Py_UNICODE_strcmp: use PyUnicode_Compare()
  • Py_UNICODE_strncmp: use PyUnicode_Tailmatch()
  • Py_UNICODE_strchr, Py_UNICODE_strrchr: use PyUnicode_FindChar()
  • Removed PyUnicode_GetMax(). Please migrate to new (PEP 393) APIs. (Contributed by Inada Naoki in bpo-41103.)
  • Removed PyLong_FromUnicode(). Please migrate to PyLong_FromUnicodeObject(). (Contributed by Inada Naoki in bpo-41103.)
  • Removed PyUnicode_AsUnicodeCopy(). Please use PyUnicode_AsUCS4Copy() or PyUnicode_AsWideCharString() (Contributed by Inada Naoki in bpo-41103.)
  • Removed _Py_CheckRecursionLimit variable: it has been replaced by ceval.recursion_limit of the PyInterpreterState structure. (Contributed by Victor Stinner in bpo-41834.)
  • Removed undocumented macros Py_ALLOW_RECURSION and Py_END_ALLOW_RECURSION and the recursion_critical field of the PyInterpreterState structure. (Contributed by Serhiy Storchaka in bpo-41936.)
  • Removed the undocumented PyOS_InitInterrupts() function. Initializing Python already implicitly installs signal handlers: see PyConfig.install_signal_handlers. (Contributed by Victor Stinner in bpo-41713.)
  • Remove the PyAST_Validate() function. It is no longer possible to build a AST object (mod_ty type) with the public C API. The function was already excluded from the limited C API (PEP 384). (Contributed by Victor Stinner in bpo-43244.)
  • Remove the symtable.h header file and the undocumented functions:
  • PyST_GetScope()
  • PySymtable_Build()
  • PySymtable_BuildObject()
  • PySymtable_Free()
  • Py_SymtableString()
  • Py_SymtableStringObject()
  • The Py_SymtableString() function was part the stable ABI by mistake but it could not be used, because the symtable.h header file was excluded from the limited C API.
  • Use Python symtable module instead. (Contributed by Victor Stinner in bpo-43244.)
  • Remove PyOS_ReadlineFunctionPointer() from the limited C API headers and from python3.dll, the library that provides the stable ABI on Windows. Since the function takes a FILE* argument, its ABI stability cannot be guaranteed. (Contributed by Petr Viktorin in bpo-43868.)
  • Remove ast.h, asdl.h, and Python-ast.h header files. These functions were undocumented and excluded from the limited C API. Most names defined by these header files were not prefixed by Py and so could create names conflicts. For example, Python-ast.h defined a Yield macro which was conflict with the Yield name used by the Windows <winbase.h> header. Use the Python ast module instead. (Contributed by Victor Stinner in bpo-43244.)
  • Remove the compiler and parser functions using struct _mod type, because the public AST C API was removed:
  • PyAST_Compile()
  • PyAST_CompileEx()
  • PyAST_CompileObject()
  • PyFuture_FromAST()
  • PyFuture_FromASTObject()
  • PyParser_ASTFromFile()
  • PyParser_ASTFromFileObject()
  • PyParser_ASTFromFilename()
  • PyParser_ASTFromString()
  • PyParser_ASTFromStringObject()
  • These functions were undocumented and excluded from the limited C API. (Contributed by Victor Stinner in bpo-43244.)
  • Remove the pyarena.h header file with functions:
  • PyArena_New()
  • PyArena_Free()
  • PyArena_Malloc()
  • PyArena_AddPyObject()
  • These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. (Contributed by Victor Stinner in bpo-43244.)

The Original Article can be found on python.org

#python #machine-learning #web-development #programming #developer

What is GEEK

Buddha Community

What’s New In Python 3.10
Veronica  Roob

Veronica Roob

1653475560

A Pure PHP Implementation Of The MessagePack Serialization Format

msgpack.php

A pure PHP implementation of the MessagePack serialization format.

Features

Installation

The recommended way to install the library is through Composer:

composer require rybakit/msgpack

Usage

Packing

To pack values you can either use an instance of a Packer:

$packer = new Packer();
$packed = $packer->pack($value);

or call a static method on the MessagePack class:

$packed = MessagePack::pack($value);

In the examples above, the method pack automatically packs a value depending on its type. However, not all PHP types can be uniquely translated to MessagePack types. For example, the MessagePack format defines map and array types, which are represented by a single array type in PHP. By default, the packer will pack a PHP array as a MessagePack array if it has sequential numeric keys, starting from 0 and as a MessagePack map otherwise:

$mpArr1 = $packer->pack([1, 2]);               // MP array [1, 2]
$mpArr2 = $packer->pack([0 => 1, 1 => 2]);     // MP array [1, 2]
$mpMap1 = $packer->pack([0 => 1, 2 => 3]);     // MP map {0: 1, 2: 3}
$mpMap2 = $packer->pack([1 => 2, 2 => 3]);     // MP map {1: 2, 2: 3}
$mpMap3 = $packer->pack(['a' => 1, 'b' => 2]); // MP map {a: 1, b: 2}

However, sometimes you need to pack a sequential array as a MessagePack map. To do this, use the packMap method:

$mpMap = $packer->packMap([1, 2]); // {0: 1, 1: 2}

Here is a list of type-specific packing methods:

$packer->packNil();           // MP nil
$packer->packBool(true);      // MP bool
$packer->packInt(42);         // MP int
$packer->packFloat(M_PI);     // MP float (32 or 64)
$packer->packFloat32(M_PI);   // MP float 32
$packer->packFloat64(M_PI);   // MP float 64
$packer->packStr('foo');      // MP str
$packer->packBin("\x80");     // MP bin
$packer->packArray([1, 2]);   // MP array
$packer->packMap(['a' => 1]); // MP map
$packer->packExt(1, "\xaa");  // MP ext

Check the "Custom types" section below on how to pack custom types.

Packing options

The Packer object supports a number of bitmask-based options for fine-tuning the packing process (defaults are in bold):

NameDescription
FORCE_STRForces PHP strings to be packed as MessagePack UTF-8 strings
FORCE_BINForces PHP strings to be packed as MessagePack binary data
DETECT_STR_BINDetects MessagePack str/bin type automatically
  
FORCE_ARRForces PHP arrays to be packed as MessagePack arrays
FORCE_MAPForces PHP arrays to be packed as MessagePack maps
DETECT_ARR_MAPDetects MessagePack array/map type automatically
  
FORCE_FLOAT32Forces PHP floats to be packed as 32-bits MessagePack floats
FORCE_FLOAT64Forces PHP floats to be packed as 64-bits MessagePack floats

The type detection mode (DETECT_STR_BIN/DETECT_ARR_MAP) adds some overhead which can be noticed when you pack large (16- and 32-bit) arrays or strings. However, if you know the value type in advance (for example, you only work with UTF-8 strings or/and associative arrays), you can eliminate this overhead by forcing the packer to use the appropriate type, which will save it from running the auto-detection routine. Another option is to explicitly specify the value type. The library provides 2 auxiliary classes for this, Map and Bin. Check the "Custom types" section below for details.

Examples:

// detect str/bin type and pack PHP 64-bit floats (doubles) to MP 32-bit floats
$packer = new Packer(PackOptions::DETECT_STR_BIN | PackOptions::FORCE_FLOAT32);

// these will throw MessagePack\Exception\InvalidOptionException
$packer = new Packer(PackOptions::FORCE_STR | PackOptions::FORCE_BIN);
$packer = new Packer(PackOptions::FORCE_FLOAT32 | PackOptions::FORCE_FLOAT64);

Unpacking

To unpack data you can either use an instance of a BufferUnpacker:

$unpacker = new BufferUnpacker();

$unpacker->reset($packed);
$value = $unpacker->unpack();

or call a static method on the MessagePack class:

$value = MessagePack::unpack($packed);

If the packed data is received in chunks (e.g. when reading from a stream), use the tryUnpack method, which attempts to unpack data and returns an array of unpacked messages (if any) instead of throwing an InsufficientDataException:

while ($chunk = ...) {
    $unpacker->append($chunk);
    if ($messages = $unpacker->tryUnpack()) {
        return $messages;
    }
}

If you want to unpack from a specific position in a buffer, use seek:

$unpacker->seek(42); // set position equal to 42 bytes
$unpacker->seek(-8); // set position to 8 bytes before the end of the buffer

To skip bytes from the current position, use skip:

$unpacker->skip(10); // set position to 10 bytes ahead of the current position

To get the number of remaining (unread) bytes in the buffer:

$unreadBytesCount = $unpacker->getRemainingCount();

To check whether the buffer has unread data:

$hasUnreadBytes = $unpacker->hasRemaining();

If needed, you can remove already read data from the buffer by calling:

$releasedBytesCount = $unpacker->release();

With the read method you can read raw (packed) data:

$packedData = $unpacker->read(2); // read 2 bytes

Besides the above methods BufferUnpacker provides type-specific unpacking methods, namely:

$unpacker->unpackNil();   // PHP null
$unpacker->unpackBool();  // PHP bool
$unpacker->unpackInt();   // PHP int
$unpacker->unpackFloat(); // PHP float
$unpacker->unpackStr();   // PHP UTF-8 string
$unpacker->unpackBin();   // PHP binary string
$unpacker->unpackArray(); // PHP sequential array
$unpacker->unpackMap();   // PHP associative array
$unpacker->unpackExt();   // PHP MessagePack\Type\Ext object

Unpacking options

The BufferUnpacker object supports a number of bitmask-based options for fine-tuning the unpacking process (defaults are in bold):

NameDescription
BIGINT_AS_STRConverts overflowed integers to strings [1]
BIGINT_AS_GMPConverts overflowed integers to GMP objects [2]
BIGINT_AS_DECConverts overflowed integers to Decimal\Decimal objects [3]

1. The binary MessagePack format has unsigned 64-bit as its largest integer data type, but PHP does not support such integers, which means that an overflow can occur during unpacking.

2. Make sure the GMP extension is enabled.

3. Make sure the Decimal extension is enabled.

Examples:

$packedUint64 = "\xcf"."\xff\xff\xff\xff"."\xff\xff\xff\xff";

$unpacker = new BufferUnpacker($packedUint64);
var_dump($unpacker->unpack()); // string(20) "18446744073709551615"

$unpacker = new BufferUnpacker($packedUint64, UnpackOptions::BIGINT_AS_GMP);
var_dump($unpacker->unpack()); // object(GMP) {...}

$unpacker = new BufferUnpacker($packedUint64, UnpackOptions::BIGINT_AS_DEC);
var_dump($unpacker->unpack()); // object(Decimal\Decimal) {...}

Custom types

In addition to the basic types, the library provides functionality to serialize and deserialize arbitrary types. This can be done in several ways, depending on your use case. Let's take a look at them.

Type objects

If you need to serialize an instance of one of your classes into one of the basic MessagePack types, the best way to do this is to implement the CanBePacked interface in the class. A good example of such a class is the Map type class that comes with the library. This type is useful when you want to explicitly specify that a given PHP array should be packed as a MessagePack map without triggering an automatic type detection routine:

$packer = new Packer();

$packedMap = $packer->pack(new Map([1, 2, 3]));
$packedArray = $packer->pack([1, 2, 3]);

More type examples can be found in the src/Type directory.

Type transformers

As with type objects, type transformers are only responsible for serializing values. They should be used when you need to serialize a value that does not implement the CanBePacked interface. Examples of such values could be instances of built-in or third-party classes that you don't own, or non-objects such as resources.

A transformer class must implement the CanPack interface. To use a transformer, it must first be registered in the packer. Here is an example of how to serialize PHP streams into the MessagePack bin format type using one of the supplied transformers, StreamTransformer:

$packer = new Packer(null, [new StreamTransformer()]);

$packedBin = $packer->pack(fopen('/path/to/file', 'r+'));

More type transformer examples can be found in the src/TypeTransformer directory.

Extensions

In contrast to the cases described above, extensions are intended to handle extension types and are responsible for both serialization and deserialization of values (types).

An extension class must implement the Extension interface. To use an extension, it must first be registered in the packer and the unpacker.

The MessagePack specification divides extension types into two groups: predefined and application-specific. Currently, there is only one predefined type in the specification, Timestamp.

Timestamp

The Timestamp extension type is a predefined type. Support for this type in the library is done through the TimestampExtension class. This class is responsible for handling Timestamp objects, which represent the number of seconds and optional adjustment in nanoseconds:

$timestampExtension = new TimestampExtension();

$packer = new Packer();
$packer = $packer->extendWith($timestampExtension);

$unpacker = new BufferUnpacker();
$unpacker = $unpacker->extendWith($timestampExtension);

$packedTimestamp = $packer->pack(Timestamp::now());
$timestamp = $unpacker->reset($packedTimestamp)->unpack();

$seconds = $timestamp->getSeconds();
$nanoseconds = $timestamp->getNanoseconds();

When using the MessagePack class, the Timestamp extension is already registered:

$packedTimestamp = MessagePack::pack(Timestamp::now());
$timestamp = MessagePack::unpack($packedTimestamp);

Application-specific extensions

In addition, the format can be extended with your own types. For example, to make the built-in PHP DateTime objects first-class citizens in your code, you can create a corresponding extension, as shown in the example. Please note, that custom extensions have to be registered with a unique extension ID (an integer from 0 to 127).

More extension examples can be found in the examples/MessagePack directory.

To learn more about how extension types can be useful, check out this article.

Exceptions

If an error occurs during packing/unpacking, a PackingFailedException or an UnpackingFailedException will be thrown, respectively. In addition, an InsufficientDataException can be thrown during unpacking.

An InvalidOptionException will be thrown in case an invalid option (or a combination of mutually exclusive options) is used.

Tests

Run tests as follows:

vendor/bin/phpunit

Also, if you already have Docker installed, you can run the tests in a docker container. First, create a container:

./dockerfile.sh | docker build -t msgpack -

The command above will create a container named msgpack with PHP 8.1 runtime. You may change the default runtime by defining the PHP_IMAGE environment variable:

PHP_IMAGE='php:8.0-cli' ./dockerfile.sh | docker build -t msgpack -

See a list of various images here.

Then run the unit tests:

docker run --rm -v $PWD:/msgpack -w /msgpack msgpack

Fuzzing

To ensure that the unpacking works correctly with malformed/semi-malformed data, you can use a testing technique called Fuzzing. The library ships with a help file (target) for PHP-Fuzzer and can be used as follows:

php-fuzzer fuzz tests/fuzz_buffer_unpacker.php

Performance

To check performance, run:

php -n -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php

Example output

Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000

=============================================
Test/Target            Packer  BufferUnpacker
---------------------------------------------
nil .................. 0.0030 ........ 0.0139
false ................ 0.0037 ........ 0.0144
true ................. 0.0040 ........ 0.0137
7-bit uint #1 ........ 0.0052 ........ 0.0120
7-bit uint #2 ........ 0.0059 ........ 0.0114
7-bit uint #3 ........ 0.0061 ........ 0.0119
5-bit sint #1 ........ 0.0067 ........ 0.0126
5-bit sint #2 ........ 0.0064 ........ 0.0132
5-bit sint #3 ........ 0.0066 ........ 0.0135
8-bit uint #1 ........ 0.0078 ........ 0.0200
8-bit uint #2 ........ 0.0077 ........ 0.0212
8-bit uint #3 ........ 0.0086 ........ 0.0203
16-bit uint #1 ....... 0.0111 ........ 0.0271
16-bit uint #2 ....... 0.0115 ........ 0.0260
16-bit uint #3 ....... 0.0103 ........ 0.0273
32-bit uint #1 ....... 0.0116 ........ 0.0326
32-bit uint #2 ....... 0.0118 ........ 0.0332
32-bit uint #3 ....... 0.0127 ........ 0.0325
64-bit uint #1 ....... 0.0140 ........ 0.0277
64-bit uint #2 ....... 0.0134 ........ 0.0294
64-bit uint #3 ....... 0.0134 ........ 0.0281
8-bit int #1 ......... 0.0086 ........ 0.0241
8-bit int #2 ......... 0.0089 ........ 0.0225
8-bit int #3 ......... 0.0085 ........ 0.0229
16-bit int #1 ........ 0.0118 ........ 0.0280
16-bit int #2 ........ 0.0121 ........ 0.0270
16-bit int #3 ........ 0.0109 ........ 0.0274
32-bit int #1 ........ 0.0128 ........ 0.0346
32-bit int #2 ........ 0.0118 ........ 0.0339
32-bit int #3 ........ 0.0135 ........ 0.0368
64-bit int #1 ........ 0.0138 ........ 0.0276
64-bit int #2 ........ 0.0132 ........ 0.0286
64-bit int #3 ........ 0.0137 ........ 0.0274
64-bit int #4 ........ 0.0180 ........ 0.0285
64-bit float #1 ...... 0.0134 ........ 0.0284
64-bit float #2 ...... 0.0125 ........ 0.0275
64-bit float #3 ...... 0.0126 ........ 0.0283
fix string #1 ........ 0.0035 ........ 0.0133
fix string #2 ........ 0.0094 ........ 0.0216
fix string #3 ........ 0.0094 ........ 0.0222
fix string #4 ........ 0.0091 ........ 0.0241
8-bit string #1 ...... 0.0122 ........ 0.0301
8-bit string #2 ...... 0.0118 ........ 0.0304
8-bit string #3 ...... 0.0119 ........ 0.0315
16-bit string #1 ..... 0.0150 ........ 0.0388
16-bit string #2 ..... 0.1545 ........ 0.1665
32-bit string ........ 0.1570 ........ 0.1756
wide char string #1 .. 0.0091 ........ 0.0236
wide char string #2 .. 0.0122 ........ 0.0313
8-bit binary #1 ...... 0.0100 ........ 0.0302
8-bit binary #2 ...... 0.0123 ........ 0.0324
8-bit binary #3 ...... 0.0126 ........ 0.0327
16-bit binary ........ 0.0168 ........ 0.0372
32-bit binary ........ 0.1588 ........ 0.1754
fix array #1 ......... 0.0042 ........ 0.0131
fix array #2 ......... 0.0294 ........ 0.0367
fix array #3 ......... 0.0412 ........ 0.0472
16-bit array #1 ...... 0.1378 ........ 0.1596
16-bit array #2 ........... S ............. S
32-bit array .............. S ............. S
complex array ........ 0.1865 ........ 0.2283
fix map #1 ........... 0.0725 ........ 0.1048
fix map #2 ........... 0.0319 ........ 0.0405
fix map #3 ........... 0.0356 ........ 0.0665
fix map #4 ........... 0.0465 ........ 0.0497
16-bit map #1 ........ 0.2540 ........ 0.3028
16-bit map #2 ............. S ............. S
32-bit map ................ S ............. S
complex map .......... 0.2372 ........ 0.2710
fixext 1 ............. 0.0283 ........ 0.0358
fixext 2 ............. 0.0291 ........ 0.0371
fixext 4 ............. 0.0302 ........ 0.0355
fixext 8 ............. 0.0288 ........ 0.0384
fixext 16 ............ 0.0293 ........ 0.0359
8-bit ext ............ 0.0302 ........ 0.0439
16-bit ext ........... 0.0334 ........ 0.0499
32-bit ext ........... 0.1845 ........ 0.1888
32-bit timestamp #1 .. 0.0337 ........ 0.0547
32-bit timestamp #2 .. 0.0335 ........ 0.0560
64-bit timestamp #1 .. 0.0371 ........ 0.0575
64-bit timestamp #2 .. 0.0374 ........ 0.0542
64-bit timestamp #3 .. 0.0356 ........ 0.0533
96-bit timestamp #1 .. 0.0362 ........ 0.0699
96-bit timestamp #2 .. 0.0381 ........ 0.0701
96-bit timestamp #3 .. 0.0367 ........ 0.0687
=============================================
Total                  2.7618          4.0820
Skipped                     4               4
Failed                      0               0
Ignored                     0               0

With JIT:

php -n -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.jit_buffer_size=64M -dopcache.jit=tracing -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php

Example output

Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000

=============================================
Test/Target            Packer  BufferUnpacker
---------------------------------------------
nil .................. 0.0005 ........ 0.0054
false ................ 0.0004 ........ 0.0059
true ................. 0.0004 ........ 0.0059
7-bit uint #1 ........ 0.0010 ........ 0.0047
7-bit uint #2 ........ 0.0010 ........ 0.0046
7-bit uint #3 ........ 0.0010 ........ 0.0046
5-bit sint #1 ........ 0.0025 ........ 0.0046
5-bit sint #2 ........ 0.0023 ........ 0.0046
5-bit sint #3 ........ 0.0024 ........ 0.0045
8-bit uint #1 ........ 0.0043 ........ 0.0081
8-bit uint #2 ........ 0.0043 ........ 0.0079
8-bit uint #3 ........ 0.0041 ........ 0.0080
16-bit uint #1 ....... 0.0064 ........ 0.0095
16-bit uint #2 ....... 0.0064 ........ 0.0091
16-bit uint #3 ....... 0.0064 ........ 0.0094
32-bit uint #1 ....... 0.0085 ........ 0.0114
32-bit uint #2 ....... 0.0077 ........ 0.0122
32-bit uint #3 ....... 0.0077 ........ 0.0120
64-bit uint #1 ....... 0.0085 ........ 0.0159
64-bit uint #2 ....... 0.0086 ........ 0.0157
64-bit uint #3 ....... 0.0086 ........ 0.0158
8-bit int #1 ......... 0.0042 ........ 0.0080
8-bit int #2 ......... 0.0042 ........ 0.0080
8-bit int #3 ......... 0.0042 ........ 0.0081
16-bit int #1 ........ 0.0065 ........ 0.0095
16-bit int #2 ........ 0.0065 ........ 0.0090
16-bit int #3 ........ 0.0056 ........ 0.0085
32-bit int #1 ........ 0.0067 ........ 0.0107
32-bit int #2 ........ 0.0066 ........ 0.0106
32-bit int #3 ........ 0.0063 ........ 0.0104
64-bit int #1 ........ 0.0072 ........ 0.0162
64-bit int #2 ........ 0.0073 ........ 0.0174
64-bit int #3 ........ 0.0072 ........ 0.0164
64-bit int #4 ........ 0.0077 ........ 0.0161
64-bit float #1 ...... 0.0053 ........ 0.0135
64-bit float #2 ...... 0.0053 ........ 0.0135
64-bit float #3 ...... 0.0052 ........ 0.0135
fix string #1 ....... -0.0002 ........ 0.0044
fix string #2 ........ 0.0035 ........ 0.0067
fix string #3 ........ 0.0035 ........ 0.0077
fix string #4 ........ 0.0033 ........ 0.0078
8-bit string #1 ...... 0.0059 ........ 0.0110
8-bit string #2 ...... 0.0063 ........ 0.0121
8-bit string #3 ...... 0.0064 ........ 0.0124
16-bit string #1 ..... 0.0099 ........ 0.0146
16-bit string #2 ..... 0.1522 ........ 0.1474
32-bit string ........ 0.1511 ........ 0.1483
wide char string #1 .. 0.0039 ........ 0.0084
wide char string #2 .. 0.0073 ........ 0.0123
8-bit binary #1 ...... 0.0040 ........ 0.0112
8-bit binary #2 ...... 0.0075 ........ 0.0123
8-bit binary #3 ...... 0.0077 ........ 0.0129
16-bit binary ........ 0.0096 ........ 0.0145
32-bit binary ........ 0.1535 ........ 0.1479
fix array #1 ......... 0.0008 ........ 0.0061
fix array #2 ......... 0.0121 ........ 0.0165
fix array #3 ......... 0.0193 ........ 0.0222
16-bit array #1 ...... 0.0607 ........ 0.0479
16-bit array #2 ........... S ............. S
32-bit array .............. S ............. S
complex array ........ 0.0749 ........ 0.0824
fix map #1 ........... 0.0329 ........ 0.0431
fix map #2 ........... 0.0161 ........ 0.0189
fix map #3 ........... 0.0205 ........ 0.0262
fix map #4 ........... 0.0252 ........ 0.0205
16-bit map #1 ........ 0.1016 ........ 0.0927
16-bit map #2 ............. S ............. S
32-bit map ................ S ............. S
complex map .......... 0.1096 ........ 0.1030
fixext 1 ............. 0.0157 ........ 0.0161
fixext 2 ............. 0.0175 ........ 0.0183
fixext 4 ............. 0.0156 ........ 0.0185
fixext 8 ............. 0.0163 ........ 0.0184
fixext 16 ............ 0.0164 ........ 0.0182
8-bit ext ............ 0.0158 ........ 0.0207
16-bit ext ........... 0.0203 ........ 0.0219
32-bit ext ........... 0.1614 ........ 0.1539
32-bit timestamp #1 .. 0.0195 ........ 0.0249
32-bit timestamp #2 .. 0.0188 ........ 0.0260
64-bit timestamp #1 .. 0.0207 ........ 0.0281
64-bit timestamp #2 .. 0.0212 ........ 0.0291
64-bit timestamp #3 .. 0.0207 ........ 0.0295
96-bit timestamp #1 .. 0.0222 ........ 0.0358
96-bit timestamp #2 .. 0.0228 ........ 0.0353
96-bit timestamp #3 .. 0.0210 ........ 0.0319
=============================================
Total                  1.6432          1.9674
Skipped                     4               4
Failed                      0               0
Ignored                     0               0

You may change default benchmark settings by defining the following environment variables:

NameDefault
MP_BENCH_TARGETSpure_p,pure_u, see a list of available targets
MP_BENCH_ITERATIONS100_000
MP_BENCH_DURATIONnot set
MP_BENCH_ROUNDS3
MP_BENCH_TESTS-@slow, see a list of available tests

For example:

export MP_BENCH_TARGETS=pure_p
export MP_BENCH_ITERATIONS=1000000
export MP_BENCH_ROUNDS=5
# a comma separated list of test names
export MP_BENCH_TESTS='complex array, complex map'
# or a group name
# export MP_BENCH_TESTS='-@slow' // @pecl_comp
# or a regexp
# export MP_BENCH_TESTS='/complex (array|map)/'

Another example, benchmarking both the library and the PECL extension:

MP_BENCH_TARGETS=pure_p,pure_u,pecl_p,pecl_u \
php -n -dextension=msgpack.so -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php

Example output

Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000

===========================================================================
Test/Target            Packer  BufferUnpacker  msgpack_pack  msgpack_unpack
---------------------------------------------------------------------------
nil .................. 0.0031 ........ 0.0141 ...... 0.0055 ........ 0.0064
false ................ 0.0039 ........ 0.0154 ...... 0.0056 ........ 0.0053
true ................. 0.0038 ........ 0.0139 ...... 0.0056 ........ 0.0044
7-bit uint #1 ........ 0.0061 ........ 0.0110 ...... 0.0059 ........ 0.0046
7-bit uint #2 ........ 0.0065 ........ 0.0119 ...... 0.0042 ........ 0.0029
7-bit uint #3 ........ 0.0054 ........ 0.0117 ...... 0.0045 ........ 0.0025
5-bit sint #1 ........ 0.0047 ........ 0.0103 ...... 0.0038 ........ 0.0022
5-bit sint #2 ........ 0.0048 ........ 0.0117 ...... 0.0038 ........ 0.0022
5-bit sint #3 ........ 0.0046 ........ 0.0102 ...... 0.0038 ........ 0.0023
8-bit uint #1 ........ 0.0063 ........ 0.0174 ...... 0.0039 ........ 0.0031
8-bit uint #2 ........ 0.0063 ........ 0.0167 ...... 0.0040 ........ 0.0029
8-bit uint #3 ........ 0.0063 ........ 0.0168 ...... 0.0039 ........ 0.0030
16-bit uint #1 ....... 0.0092 ........ 0.0222 ...... 0.0049 ........ 0.0030
16-bit uint #2 ....... 0.0096 ........ 0.0227 ...... 0.0042 ........ 0.0046
16-bit uint #3 ....... 0.0123 ........ 0.0274 ...... 0.0059 ........ 0.0051
32-bit uint #1 ....... 0.0136 ........ 0.0331 ...... 0.0060 ........ 0.0048
32-bit uint #2 ....... 0.0130 ........ 0.0336 ...... 0.0070 ........ 0.0048
32-bit uint #3 ....... 0.0127 ........ 0.0329 ...... 0.0051 ........ 0.0048
64-bit uint #1 ....... 0.0126 ........ 0.0268 ...... 0.0055 ........ 0.0049
64-bit uint #2 ....... 0.0135 ........ 0.0281 ...... 0.0052 ........ 0.0046
64-bit uint #3 ....... 0.0131 ........ 0.0274 ...... 0.0069 ........ 0.0044
8-bit int #1 ......... 0.0077 ........ 0.0236 ...... 0.0058 ........ 0.0044
8-bit int #2 ......... 0.0087 ........ 0.0244 ...... 0.0058 ........ 0.0048
8-bit int #3 ......... 0.0084 ........ 0.0241 ...... 0.0055 ........ 0.0049
16-bit int #1 ........ 0.0112 ........ 0.0271 ...... 0.0048 ........ 0.0045
16-bit int #2 ........ 0.0124 ........ 0.0292 ...... 0.0057 ........ 0.0049
16-bit int #3 ........ 0.0118 ........ 0.0270 ...... 0.0058 ........ 0.0050
32-bit int #1 ........ 0.0137 ........ 0.0366 ...... 0.0058 ........ 0.0051
32-bit int #2 ........ 0.0133 ........ 0.0366 ...... 0.0056 ........ 0.0049
32-bit int #3 ........ 0.0129 ........ 0.0350 ...... 0.0052 ........ 0.0048
64-bit int #1 ........ 0.0145 ........ 0.0254 ...... 0.0034 ........ 0.0025
64-bit int #2 ........ 0.0097 ........ 0.0214 ...... 0.0034 ........ 0.0025
64-bit int #3 ........ 0.0096 ........ 0.0287 ...... 0.0059 ........ 0.0050
64-bit int #4 ........ 0.0143 ........ 0.0277 ...... 0.0059 ........ 0.0046
64-bit float #1 ...... 0.0134 ........ 0.0281 ...... 0.0057 ........ 0.0052
64-bit float #2 ...... 0.0141 ........ 0.0281 ...... 0.0057 ........ 0.0050
64-bit float #3 ...... 0.0144 ........ 0.0282 ...... 0.0057 ........ 0.0050
fix string #1 ........ 0.0036 ........ 0.0143 ...... 0.0066 ........ 0.0053
fix string #2 ........ 0.0107 ........ 0.0222 ...... 0.0065 ........ 0.0068
fix string #3 ........ 0.0116 ........ 0.0245 ...... 0.0063 ........ 0.0069
fix string #4 ........ 0.0105 ........ 0.0253 ...... 0.0083 ........ 0.0077
8-bit string #1 ...... 0.0126 ........ 0.0318 ...... 0.0075 ........ 0.0088
8-bit string #2 ...... 0.0121 ........ 0.0295 ...... 0.0076 ........ 0.0086
8-bit string #3 ...... 0.0125 ........ 0.0293 ...... 0.0130 ........ 0.0093
16-bit string #1 ..... 0.0159 ........ 0.0368 ...... 0.0117 ........ 0.0086
16-bit string #2 ..... 0.1547 ........ 0.1686 ...... 0.1516 ........ 0.1373
32-bit string ........ 0.1558 ........ 0.1729 ...... 0.1511 ........ 0.1396
wide char string #1 .. 0.0098 ........ 0.0237 ...... 0.0066 ........ 0.0065
wide char string #2 .. 0.0128 ........ 0.0291 ...... 0.0061 ........ 0.0082
8-bit binary #1 ........... I ............. I ........... F ............. I
8-bit binary #2 ........... I ............. I ........... F ............. I
8-bit binary #3 ........... I ............. I ........... F ............. I
16-bit binary ............. I ............. I ........... F ............. I
32-bit binary ............. I ............. I ........... F ............. I
fix array #1 ......... 0.0040 ........ 0.0129 ...... 0.0120 ........ 0.0058
fix array #2 ......... 0.0279 ........ 0.0390 ...... 0.0143 ........ 0.0165
fix array #3 ......... 0.0415 ........ 0.0463 ...... 0.0162 ........ 0.0187
16-bit array #1 ...... 0.1349 ........ 0.1628 ...... 0.0334 ........ 0.0341
16-bit array #2 ........... S ............. S ........... S ............. S
32-bit array .............. S ............. S ........... S ............. S
complex array ............. I ............. I ........... F ............. F
fix map #1 ................ I ............. I ........... F ............. I
fix map #2 ........... 0.0345 ........ 0.0391 ...... 0.0143 ........ 0.0168
fix map #3 ................ I ............. I ........... F ............. I
fix map #4 ........... 0.0459 ........ 0.0473 ...... 0.0151 ........ 0.0163
16-bit map #1 ........ 0.2518 ........ 0.2962 ...... 0.0400 ........ 0.0490
16-bit map #2 ............. S ............. S ........... S ............. S
32-bit map ................ S ............. S ........... S ............. S
complex map .......... 0.2380 ........ 0.2682 ...... 0.0545 ........ 0.0579
fixext 1 .................. I ............. I ........... F ............. F
fixext 2 .................. I ............. I ........... F ............. F
fixext 4 .................. I ............. I ........... F ............. F
fixext 8 .................. I ............. I ........... F ............. F
fixext 16 ................. I ............. I ........... F ............. F
8-bit ext ................. I ............. I ........... F ............. F
16-bit ext ................ I ............. I ........... F ............. F
32-bit ext ................ I ............. I ........... F ............. F
32-bit timestamp #1 ....... I ............. I ........... F ............. F
32-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #1 ....... I ............. I ........... F ............. F
64-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #3 ....... I ............. I ........... F ............. F
96-bit timestamp #1 ....... I ............. I ........... F ............. F
96-bit timestamp #2 ....... I ............. I ........... F ............. F
96-bit timestamp #3 ....... I ............. I ........... F ............. F
===========================================================================
Total                  1.5625          2.3866        0.7735          0.7243
Skipped                     4               4             4               4
Failed                      0               0            24              17
Ignored                    24              24             0               7

With JIT:

MP_BENCH_TARGETS=pure_p,pure_u,pecl_p,pecl_u \
php -n -dextension=msgpack.so -dzend_extension=opcache.so \
-dpcre.jit=1 -dopcache.jit_buffer_size=64M -dopcache.jit=tracing -dopcache.enable=1 -dopcache.enable_cli=1 \
tests/bench.php

Example output

Filter: MessagePack\Tests\Perf\Filter\ListFilter
Rounds: 3
Iterations: 100000

===========================================================================
Test/Target            Packer  BufferUnpacker  msgpack_pack  msgpack_unpack
---------------------------------------------------------------------------
nil .................. 0.0001 ........ 0.0052 ...... 0.0053 ........ 0.0042
false ................ 0.0007 ........ 0.0060 ...... 0.0057 ........ 0.0043
true ................. 0.0008 ........ 0.0060 ...... 0.0056 ........ 0.0041
7-bit uint #1 ........ 0.0031 ........ 0.0046 ...... 0.0062 ........ 0.0041
7-bit uint #2 ........ 0.0021 ........ 0.0043 ...... 0.0062 ........ 0.0041
7-bit uint #3 ........ 0.0022 ........ 0.0044 ...... 0.0061 ........ 0.0040
5-bit sint #1 ........ 0.0030 ........ 0.0048 ...... 0.0062 ........ 0.0040
5-bit sint #2 ........ 0.0032 ........ 0.0046 ...... 0.0062 ........ 0.0040
5-bit sint #3 ........ 0.0031 ........ 0.0046 ...... 0.0062 ........ 0.0040
8-bit uint #1 ........ 0.0054 ........ 0.0079 ...... 0.0062 ........ 0.0050
8-bit uint #2 ........ 0.0051 ........ 0.0079 ...... 0.0064 ........ 0.0044
8-bit uint #3 ........ 0.0051 ........ 0.0082 ...... 0.0062 ........ 0.0044
16-bit uint #1 ....... 0.0077 ........ 0.0094 ...... 0.0065 ........ 0.0045
16-bit uint #2 ....... 0.0077 ........ 0.0094 ...... 0.0063 ........ 0.0045
16-bit uint #3 ....... 0.0077 ........ 0.0095 ...... 0.0064 ........ 0.0047
32-bit uint #1 ....... 0.0088 ........ 0.0119 ...... 0.0063 ........ 0.0043
32-bit uint #2 ....... 0.0089 ........ 0.0117 ...... 0.0062 ........ 0.0039
32-bit uint #3 ....... 0.0089 ........ 0.0118 ...... 0.0063 ........ 0.0044
64-bit uint #1 ....... 0.0097 ........ 0.0155 ...... 0.0063 ........ 0.0045
64-bit uint #2 ....... 0.0095 ........ 0.0153 ...... 0.0061 ........ 0.0045
64-bit uint #3 ....... 0.0096 ........ 0.0156 ...... 0.0063 ........ 0.0047
8-bit int #1 ......... 0.0053 ........ 0.0083 ...... 0.0062 ........ 0.0044
8-bit int #2 ......... 0.0052 ........ 0.0080 ...... 0.0062 ........ 0.0044
8-bit int #3 ......... 0.0052 ........ 0.0080 ...... 0.0062 ........ 0.0043
16-bit int #1 ........ 0.0089 ........ 0.0097 ...... 0.0069 ........ 0.0046
16-bit int #2 ........ 0.0075 ........ 0.0093 ...... 0.0063 ........ 0.0043
16-bit int #3 ........ 0.0075 ........ 0.0094 ...... 0.0062 ........ 0.0046
32-bit int #1 ........ 0.0086 ........ 0.0122 ...... 0.0063 ........ 0.0044
32-bit int #2 ........ 0.0087 ........ 0.0120 ...... 0.0066 ........ 0.0046
32-bit int #3 ........ 0.0086 ........ 0.0121 ...... 0.0060 ........ 0.0044
64-bit int #1 ........ 0.0096 ........ 0.0149 ...... 0.0060 ........ 0.0045
64-bit int #2 ........ 0.0096 ........ 0.0157 ...... 0.0062 ........ 0.0044
64-bit int #3 ........ 0.0096 ........ 0.0160 ...... 0.0063 ........ 0.0046
64-bit int #4 ........ 0.0097 ........ 0.0157 ...... 0.0061 ........ 0.0044
64-bit float #1 ...... 0.0079 ........ 0.0153 ...... 0.0056 ........ 0.0044
64-bit float #2 ...... 0.0079 ........ 0.0152 ...... 0.0057 ........ 0.0045
64-bit float #3 ...... 0.0079 ........ 0.0155 ...... 0.0057 ........ 0.0044
fix string #1 ........ 0.0010 ........ 0.0045 ...... 0.0071 ........ 0.0044
fix string #2 ........ 0.0048 ........ 0.0075 ...... 0.0070 ........ 0.0060
fix string #3 ........ 0.0048 ........ 0.0086 ...... 0.0068 ........ 0.0060
fix string #4 ........ 0.0050 ........ 0.0088 ...... 0.0070 ........ 0.0059
8-bit string #1 ...... 0.0081 ........ 0.0129 ...... 0.0069 ........ 0.0062
8-bit string #2 ...... 0.0086 ........ 0.0128 ...... 0.0069 ........ 0.0065
8-bit string #3 ...... 0.0086 ........ 0.0126 ...... 0.0115 ........ 0.0065
16-bit string #1 ..... 0.0105 ........ 0.0137 ...... 0.0128 ........ 0.0068
16-bit string #2 ..... 0.1510 ........ 0.1486 ...... 0.1526 ........ 0.1391
32-bit string ........ 0.1517 ........ 0.1475 ...... 0.1504 ........ 0.1370
wide char string #1 .. 0.0044 ........ 0.0085 ...... 0.0067 ........ 0.0057
wide char string #2 .. 0.0081 ........ 0.0125 ...... 0.0069 ........ 0.0063
8-bit binary #1 ........... I ............. I ........... F ............. I
8-bit binary #2 ........... I ............. I ........... F ............. I
8-bit binary #3 ........... I ............. I ........... F ............. I
16-bit binary ............. I ............. I ........... F ............. I
32-bit binary ............. I ............. I ........... F ............. I
fix array #1 ......... 0.0014 ........ 0.0059 ...... 0.0132 ........ 0.0055
fix array #2 ......... 0.0146 ........ 0.0156 ...... 0.0155 ........ 0.0148
fix array #3 ......... 0.0211 ........ 0.0229 ...... 0.0179 ........ 0.0180
16-bit array #1 ...... 0.0673 ........ 0.0498 ...... 0.0343 ........ 0.0388
16-bit array #2 ........... S ............. S ........... S ............. S
32-bit array .............. S ............. S ........... S ............. S
complex array ............. I ............. I ........... F ............. F
fix map #1 ................ I ............. I ........... F ............. I
fix map #2 ........... 0.0148 ........ 0.0180 ...... 0.0156 ........ 0.0179
fix map #3 ................ I ............. I ........... F ............. I
fix map #4 ........... 0.0252 ........ 0.0201 ...... 0.0214 ........ 0.0167
16-bit map #1 ........ 0.1027 ........ 0.0836 ...... 0.0388 ........ 0.0510
16-bit map #2 ............. S ............. S ........... S ............. S
32-bit map ................ S ............. S ........... S ............. S
complex map .......... 0.1104 ........ 0.1010 ...... 0.0556 ........ 0.0602
fixext 1 .................. I ............. I ........... F ............. F
fixext 2 .................. I ............. I ........... F ............. F
fixext 4 .................. I ............. I ........... F ............. F
fixext 8 .................. I ............. I ........... F ............. F
fixext 16 ................. I ............. I ........... F ............. F
8-bit ext ................. I ............. I ........... F ............. F
16-bit ext ................ I ............. I ........... F ............. F
32-bit ext ................ I ............. I ........... F ............. F
32-bit timestamp #1 ....... I ............. I ........... F ............. F
32-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #1 ....... I ............. I ........... F ............. F
64-bit timestamp #2 ....... I ............. I ........... F ............. F
64-bit timestamp #3 ....... I ............. I ........... F ............. F
96-bit timestamp #1 ....... I ............. I ........... F ............. F
96-bit timestamp #2 ....... I ............. I ........... F ............. F
96-bit timestamp #3 ....... I ............. I ........... F ............. F
===========================================================================
Total                  0.9642          1.0909        0.8224          0.7213
Skipped                     4               4             4               4
Failed                      0               0            24              17
Ignored                    24              24             0               7

Note that the msgpack extension (v2.1.2) doesn't support ext, bin and UTF-8 str types.

License

The library is released under the MIT License. See the bundled LICENSE file for details.

Author: rybakit
Source Code: https://github.com/rybakit/msgpack.php
License: MIT License

#php 

Ray  Patel

Ray Patel

1619518440

top 30 Python Tips and Tricks for Beginners

Welcome to my Blog , In this article, you are going to learn the top 10 python tips and tricks.

1) swap two numbers.

2) Reversing a string in Python.

3) Create a single string from all the elements in list.

4) Chaining Of Comparison Operators.

5) Print The File Path Of Imported Modules.

6) Return Multiple Values From Functions.

7) Find The Most Frequent Value In A List.

8) Check The Memory Usage Of An Object.

#python #python hacks tricks #python learning tips #python programming tricks #python tips #python tips and tricks #python tips and tricks advanced #python tips and tricks for beginners #python tips tricks and techniques #python tutorial #tips and tricks in python #tips to learn python #top 30 python tips and tricks for beginners

Ray  Patel

Ray Patel

1623406860

What’s New in Python 3.10?

A rundown of the coolest features

Python 3.10 development has stabilized and we can finally test out all of the new features that will be included in the final release.

We’ll cover some of the most interesting additions to Python — structural pattern matching, parenthesized context managers, _more _typing, and the new and improved error messages.

Check out the video version of the article here:

Structural Pattern Matching

Parenthesized Context Managers

More Typing

#data-science #programming #programming-languages #python #what’s new in python 3.10 #python 3.10

Ray  Patel

Ray Patel

1619510796

Lambda, Map, Filter functions in python

Welcome to my Blog, In this article, we will learn python lambda function, Map function, and filter function.

Lambda function in python: Lambda is a one line anonymous function and lambda takes any number of arguments but can only have one expression and python lambda syntax is

Syntax: x = lambda arguments : expression

Now i will show you some python lambda function examples:

#python #anonymous function python #filter function in python #lambda #lambda python 3 #map python #python filter #python filter lambda #python lambda #python lambda examples #python map

Shardul Bhatt

Shardul Bhatt

1626775355

Why use Python for Software Development

No programming language is pretty much as diverse as Python. It enables building cutting edge applications effortlessly. Developers are as yet investigating the full capability of end-to-end Python development services in various areas. 

By areas, we mean FinTech, HealthTech, InsureTech, Cybersecurity, and that's just the beginning. These are New Economy areas, and Python has the ability to serve every one of them. The vast majority of them require massive computational abilities. Python's code is dynamic and powerful - equipped for taking care of the heavy traffic and substantial algorithmic capacities. 

Programming advancement is multidimensional today. Endeavor programming requires an intelligent application with AI and ML capacities. Shopper based applications require information examination to convey a superior client experience. Netflix, Trello, and Amazon are genuine instances of such applications. Python assists with building them effortlessly. 

5 Reasons to Utilize Python for Programming Web Apps 

Python can do such numerous things that developers can't discover enough reasons to admire it. Python application development isn't restricted to web and enterprise applications. It is exceptionally adaptable and superb for a wide range of uses.

Robust frameworks 

Python is known for its tools and frameworks. There's a structure for everything. Django is helpful for building web applications, venture applications, logical applications, and mathematical processing. Flask is another web improvement framework with no conditions. 

Web2Py, CherryPy, and Falcon offer incredible capabilities to customize Python development services. A large portion of them are open-source frameworks that allow quick turn of events. 

Simple to read and compose 

Python has an improved sentence structure - one that is like the English language. New engineers for Python can undoubtedly understand where they stand in the development process. The simplicity of composing allows quick application building. 

The motivation behind building Python, as said by its maker Guido Van Rossum, was to empower even beginner engineers to comprehend the programming language. The simple coding likewise permits developers to roll out speedy improvements without getting confused by pointless subtleties. 

Utilized by the best 

Alright - Python isn't simply one more programming language. It should have something, which is the reason the business giants use it. Furthermore, that too for different purposes. Developers at Google use Python to assemble framework organization systems, parallel information pusher, code audit, testing and QA, and substantially more. Netflix utilizes Python web development services for its recommendation algorithm and media player. 

Massive community support 

Python has a steadily developing community that offers enormous help. From amateurs to specialists, there's everybody. There are a lot of instructional exercises, documentation, and guides accessible for Python web development solutions. 

Today, numerous universities start with Python, adding to the quantity of individuals in the community. Frequently, Python designers team up on various tasks and help each other with algorithmic, utilitarian, and application critical thinking. 

Progressive applications 

Python is the greatest supporter of data science, Machine Learning, and Artificial Intelligence at any enterprise software development company. Its utilization cases in cutting edge applications are the most compelling motivation for its prosperity. Python is the second most well known tool after R for data analytics.

The simplicity of getting sorted out, overseeing, and visualizing information through unique libraries makes it ideal for data based applications. TensorFlow for neural networks and OpenCV for computer vision are two of Python's most well known use cases for Machine learning applications.

Summary

Thinking about the advances in programming and innovation, Python is a YES for an assorted scope of utilizations. Game development, web application development services, GUI advancement, ML and AI improvement, Enterprise and customer applications - every one of them uses Python to its full potential. 

The disadvantages of Python web improvement arrangements are regularly disregarded by developers and organizations because of the advantages it gives. They focus on quality over speed and performance over blunders. That is the reason it's a good idea to utilize Python for building the applications of the future.

#python development services #python development company #python app development #python development #python in web development #python software development