dataparser

Description

This module provides a flexible and extensible system for parsing and converting data between Python objects and SQLite database representations. It offers tools for creating custom parsers, handling various data types, and managing the conversion process for database operations.

Warning

If you create custom parsers or using initialized built-in ones, you should always set the connection’s parameter detect_types to sqlite3.PARSE_DECLTYPES.

Key Components

• Parser: Class for creating and managing custom data parsers.

• Parsable: Abstract base class for creating custom parsable objects.

• BuiltinParsers: Container for default and additional pre-defined parsers.

Other Components

• DefaultDataType: Type alias for basic types natively supported by SQLite.

• allowed_types(): Function for querying types, supported by the database.

• allowed_typenames(): Function for querying typenames (sqlite column types), supported by the database.

Usage Examples

• If you need to work with booleans, date, time, datetime, lists, tuples, sets, dicts, json objects, you don’t need to create custom parsers for them. Call BuiltinParsers.init() method and you will be able to work with these types.

from sl3aio import BuiltinParsers

BuiltinParsers.init()

• If you need to work with a custom data type, you can create a custom parser for it. Here is an example.

from sl3aio import Parser
from dataclasses import dataclass

@dataclass
class Point:
    x: float
    y: float

# Here we defining the parser for the point's class.
# 'types' is a set of types associated with this parser.
# '_typenames' is a set of typenames (column types in sqlite) associated with this parser.
# 'loads' is a function that converts bytes to the Point.
# 'dumps' is a function that converts the Point to bytes or other parsable object.
point_parser = Parser(
    types={Point},
    _typenames={'POINT'},
    loads=lambda data: Point(*map(float, data.decode('ascii').split())),
    dumps=lambda obj: f'{obj.x} {obj.y}'.encode('ascii')
).register()
# Note that the 'register()' method must be called in order for sqlite to know about the parser.

• This example demonstrates how to create a custom parsable object and register it with the parser system.

from sl3aio import Parser, Parsable


class CustomObject(Parsable):
    def __init__(self, value):
        self.value = value

    @classmethod
    def from_data(cls, data: bytes):
        return cls(int.from_bytes(data, 'big'))

    def to_data(self):
        return self.value.to_bytes(4, 'big')


custom_parser = Parser.from_parsable(CustomObject, ['CUSTOM'])
custom_parser.register()

sl3aio.dataparser.DefaultDataType: TypeAlias = bytes | str | int | float | None

Types that are supported by sqlite3 natively.

sl3aio.dataparser.allowed_types() → set[type]

List types that can be written into database.

Returns:

Set of writable types.

Return type:

set [type]

sl3aio.dataparser.allowed_typenames() → set[str]

List names that can be used as column type in database.

Note

For default data types, this list includes only their affinities. So there are no such typenames as DOUBLE, TINYINT, VARCHAR(...) and etc. in the result set.

Returns:

Set of allowed columns types.

Return type:

set [str]

class sl3aio.dataparser.Parser(types: set[type[T]], _typenames: set[str], loads: Callable[[bytes | str | int | float | None], T], dumps: Callable[[T], bytes | str | int | float | None | Any])

Class for creating custom parsers.

Automates the registration of converters and adapters in sqlite3. Provides convinient access to the already registered parsers.

Attention

Every single parser must have at least one supported type and at least one supported typename, otherwise instantiation will raise the AssertionError.

See also

Parsable, BuiltinParsers

instances: ClassVar[set[Self]] = {Parser(types={<class 'bytes'>}, _typenames={'BYTES', 'BLOB'}), Parser(types={<class 'float'>}, _typenames={'FLOAT', 'DOUBLE', 'REAL'}), Parser(types={<class 'int'>}, _typenames={'INTEGER', 'INT'}), Parser(types={<class 'str'>}, _typenames={'CHAR', 'STRING', 'STR', 'VARCHAR', 'TEXT'})}

Container for all of the parsers that were created.

types: set[type[T]]

Set of types corresponding to the parser.

loads: Callable[[bytes | str | int | float | None], T]

Method to parse a data recieved from the table.

Note

The type of the data will be the same as the return type of the dumps method or, if the return type is an other object that has a parser, corresponding to it, the return type will be the same as the return type of the dumps method of the other object (and so on until the return type of dumps won’t be one of the DefaultDataType)

dumps: Callable[[T], bytes | str | int | float | None | Any]

Method to convert an object to the object of the allowed type, listed in allowed_types().

property typenames: set[str]

Set of names, corresponding to the parser.

classmethod from_parsable(parsable: type[T], typenames: Iterable[str] = ()) → Parser

Construct a new instance from a parsable object and optional typenames.

Parameters:

• parsable (type [Parsable]) – Object that can be loaded and dumped using its own converters.

• typenames (Iterable [str], optional) – Optional typenames. If not provided name of the class will be used instead. Defaults to empty tuple.

Returns:

New instance of the parser.

Return type:

Self

classmethod get_by_type(_type: T) → Parser | None

Get an instance of a parser from its registry by the type it supports.

Parameters:

_type (T) – Type that must be supported by required parser.

Returns:

Instance of the parser or None if no parser corresponding to the given type was found.

Return type:

Parser [T] | None

classmethod get_by_typename(_typename: str) → Parser | None

Get an instance of a parser from its registry by the typename of type which it supports.

Parameters:

_typename (str) – Name of the type that must be supported by required parser.

Returns:

Instance of the parser or None if no parser corresponding to the given typename was found.

Return type:

Parser [T] | None

register() → Self

Register converters and adapters in sqlite3.

Returns:

Self for chaining.

Return type:

‘Self’

unregister() → Self

Unregister converters and adapters in sqlite3.

Returns:

Self for chaining.

Return type:

Self

class sl3aio.dataparser.Parsable

Base class for custom parsable objects.

See also

Parser

abstract classmethod from_data(data: bytes | str | int | float | None) → T

Create an instance from a data recieved from the table.

Parameters:

data (bytes) – Incoming data from the sqlite database.

Returns:

Instance of the parsable class.

Return type:

Parsable

See also

Parser.loads

abstract to_data() → bytes | str | int | float | None | Any

Converts self to the any object of the allowed type, listed in allowed_types().

Returns:

Object that can be written into sqlite database.

Return type:

DefaultDataType | Any

See also

Parser.dumps

final class sl3aio.dataparser.BuiltinParsers

Container for default and some extra parsers.

Attention

Before using BuiltionParsers.BOOL, BuiltionParsers.SET, BuiltionParsers.TUPLE, BuiltionParsers.JSON, BuiltionParsers.TIME, BuiltionParsers.DATE and BuiltionParsers.DATETIME parsers, you must call BuiltinParsers.init() method.

Warning

Do not registrate BuiltionParsers.BLOB, BuiltionParsers.INT, BuiltionParsers.REAL and BuiltionParsers.TEXT parsers using their’s Parser.register() method.

See also

Parser, Parsable

BLOB: ClassVar[Parser[bytes]] = Parser(types={<class 'bytes'>}, _typenames={'BYTES', 'BLOB'})

Parser for binary data.

INT: ClassVar[Parser[int]] = Parser(types={<class 'int'>}, _typenames={'INTEGER', 'INT'})

Parser for integers.

REAL: ClassVar[Parser[float]] = Parser(types={<class 'float'>}, _typenames={'FLOAT', 'DOUBLE', 'REAL'})

Parser for floating-point and real numbers.

TEXT: ClassVar[Parser[str]] = Parser(types={<class 'str'>}, _typenames={'CHAR', 'STRING', 'STR', 'VARCHAR', 'TEXT'})

Parser for strings.

BOOL: ClassVar[Parser[bool]]

Parser for boolean values.

JSON: ClassVar[Parser[dict | list]]

Parser for both python dictionaries and lists (JSON objects).

TUPLE: ClassVar[Parser[tuple]]

Parser for python tuples.

SET: ClassVar[Parser[set]]

Parser for python sets.

TIME: ClassVar[Parser[time]]

Parser for time in one of the iso 8601 formats.

DATE: ClassVar[Parser[date]]

Parser for date in iso 8601 format.

DATETIME: ClassVar[Parser[datetime]]

Parser for date and time in iso 8601 format.

static init() → None

Creates and registrates all builtin parsers except BLOB, INT, REAL and TEXT (those were created automatically).