Python etc

2022-08-30 17:01:01 The method __del__ is called on the object by the garbage collector when the last reference to the object is removed:

class A:
def __del__(self):
print('destroying')

a = b = A()
del a
del b
# destroying

def f():
a = A()

f()
# destroying

The method is used by Python's file object to close the descriptor when you don't need it anymore:

def f():
a_file = open('a_file.txt')
...

However, you cannot safely rely on that the destructor (this is how it's called in other languages, like C) will be ever called. For instance, it can be not true in PyPy, MicroPython, or just if the garbage collector is disabled using gc.disable().

The thumb-up rule is to use the destructor only for unimportant things. For example, aiohttp.ClientSession uses __del__ to warn about an unclosed session:

def __del__(self) -> None:
if not self.closed:
warnings.warn(
f"Unclosed client session {self!r}", ResourceWarning
) Open / Comment

2022-08-23 17:01:01 The del statement is used to delete things. It has a few distinct behaviors, depending on what is the specified target.

If a variable specified, it will be removed from the scope in which it is defined:

a = []
del a
a
# NameError: name 'a' is not defined

If the target has a form target[index], target.__delitem__(index) will be called. It is defined for built-in collections to remove items from them:

a = [1, 2, 3]
del a[0]
a # [2, 3]

d = {1: 2, 3: 4}
del d[3]
d # {1: 2}

Slices are also supported:

a = [1, 2, 3, 4]
del a[2:]
a # [1, 2]

And the last behavior, if target.attr is specified, target.__delattr__(attr) will be called. It is defined for object:

class A:
b = 'default'
a = A()
a.b = 'overwritten'
a.b # 'overwritten'
del a.b
a.b # 'default'
del a.b # AttributeError Open / Comment

2022-08-16 17:01:01 The operator is checks if the two given objects are the same object in the memory:

{} is {} # False
d = {}
d is d # True

Since types are also objects, you can use it to compare types:

type(1) is int # True
type(1) is float # False
type(1) is not float # True

And you can also use == for comparing types:

type(1) == int # True

So, when to use is and when to use ==? There are some best practices:

+ Use is to compare with None: var is None.

+ Use is to compare with True and False. However, don't explicitly check for True and False in conditions, prefer just if user.admin instead of if user.admin is True. Still, the latter can be useful in tests: assert actual is True.

+ Use isinstance to compare types: if isinstance(user, LoggedInUser). The big difference is that it allows subclasses. So if you have a class Admin which is subclass of LoggedInUser, it will pass isinstance check.

+ Use is in some rare cases when you explicitly want to allow only the given type without subclasses: type(user) is Admin. Keep in mind, that mypy will refine the type only for isinstance but not for type is.

+ Use is to compare enum members: color is Color.RED.

+ Use == in ORMs and query builders like sqlalchemy: session.query(User).filter(User.admin == True). The reason is that is behavior cannot be redefined using magic methods but == can (using __eq__).

+ Use == in all other cases. In particular, always use == to compare values: answer == 42. Open / Comment

2022-08-09 17:01:01 LookupError is a base class for IndexError and KeyError:

LookupError.__subclasses__()
# [IndexError, KeyError, encodings.CodecRegistryError]

KeyError.mro()
# [KeyError, LookupError, Exception, BaseException, object]

IndexError.mro()
# [IndexError, LookupError, Exception, BaseException, object]

The main purpose of this intermediate exception is to simplify a bit lookup for deeply nested structures when any of these two exceptions may occur:

try:
username = resp['posts'][-1]['authors'][0]['name']
except LookupError:
username = None Open / Comment

2022-08-02 17:01:02 PEP-604 (landed in Python 3.10) introduced a new short syntax for typing.Union (as I predicted, but I messed up union with intersection, shame on me):

def greet(name: str) -> str | None:
if not name:
return None
return f"Hello, {name}"

You already can use it in older Python versions by adding from __future__ import annotations, type checkers will understand you. Open / Comment

2022-07-26 17:01:01 Often, your type annotations will have circular dependencies. For example, Article has an attribute category: Category, and Category has attribute articles: list[Article]. If both classes are in the same file, adding from __future__ import annotations would solve the issue. But what if they are in different modules? Then you can hide imports that you need only for type annotations inside of the if TYPE_CHECKING block:

from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING

if TYPE_CHECKING:
from .category import Category

@dataclass
class Article:
category: Category

Fun fact: this constant is defined as TYPE_CHECKING = False. It won't be executed at runtime, but the type checker is a static analyzer, it doesn't care. Open / Comment

2022-07-19 17:01:01 PEP-563 (landed in Python 3.7) introduced postponed evaluation of type annotations. That means, all your type annotations aren't executed at runtime but rather considered strings.

The initial idea was to make it the default behavior in Python 3.10 but it was postponed after a negative reaction from the community. In short, it would be in some cases impossible to get type information at runtime which is crucial for some tools like pydantic or typeguard. For example, see pydantic#2678.

Either way, starting from Python 3.7, you can activate this behavior by adding from __future__ import annotations at the beginning of a file. It will improve the import time and allow you to use in annotations objects that aren't defined yet.

For example:

class A:
@classmethod
def create(cls) -> A:
return cls()

This code will fail at import time:

Traceback (most recent call last):
File "tmp.py", line 1, in
class A:
File "tmp.py", line 3, in A
def create(cls) -> A:
NameError: name 'A' is not defined

Now add the magic import, and it will work:

from __future__ import annotations

class A:
@classmethod
def create(cls) -> A:
return cls()

Another solution is to manually make annotations strings. So, instead of -> A: you could write -> 'A':.

6.0K views14:01

Open / Comment

2022-07-14 17:01:01 Now, let's see how to dump stack trace when a specific signal is received. We will use SIGUSR1 but you can do the same for any signal.

import faulthandler
from signal import SIGUSR1
from time import sleep

faulthandler.register(SIGUSR1)
sleep(60)

Now, in a new terminal, find out the PID of the interpreter. If the file is named tmp.py, this is how you can do it (we add [] in grep to exclude the grep itself from the output):

ps -ax | grep '[t]mp.py'

The first number in the output is the PID. Now, use it to send the signal for PID 12345:

kill -SIGUSR1 12345

And back in the terminal with the running script. You will see the stack trace:

Current thread 0x00007f22edb29740 (most recent call first):
File "tmp.py", line 6 in

This trick can help you to see where your program has frozen without adding logs to every line. However, a better alternative can be something like py-spy which allows you to dump the current stack trace without any changes in the code.

5.0K views14:01

Open / Comment

2022-07-12 17:01:01 The module faulthandler allows registering a handler that will dump the current stack trace in a specific file (stderr by default) upon receiving a specific signal or every N seconds.

For example, dump stack trace every 2 seconds:

import faulthandler
from time import sleep

faulthandler.dump_traceback_later(
timeout=2,
repeat=True,
)
for i in range(5):
print(f"iteration {i}")
sleep(1)

Output:

iteration 0
iteration 1
Timeout (0:00:02)!
Thread 0x00007f8289147740 (most recent call first):
File "tmp.py", line 10 in
iteration 2
iteration 3
Timeout (0:00:02)!
Thread 0x00007f8289147740 (most recent call first):
File "tmp.py", line 10 in
iteration 4

4.3K views14:01

Open / Comment

2022-07-05 17:01:01 The module atexit allows registering hooks that will be executed when the program terminates.

There are only a few cases when it is NOT executed:

+ When os._exit (don't confuse with sys.exit) is called.
+ When the interpreter failed with a fatal error.
+ When the process is hard-killed. For example, someone executed kill -9 or the system is ran out of memory.

In all other cases, like an unhandled exception or sys.exit, the registered hooks will be executed.

A few use cases:

+ Finish pending jobs
+ Send pending log messages into the log system
+ Save interactive interpreter history

However, keep in mind that there is no way to handle unhandled exceptions using atexit because it is executed after the exception is printed and discarded.

import atexit

atexit.register(print, 'FINISHED')
1/0

Output:

Traceback (most recent call last):
File "example.py", line 4, in
1/0
ZeroDivisionError: division by zero
FINISHED

5.5K views14:01

Open / Comment