Custom data types

Create custom data types and use them in your databases.

Introduction

sl3aio’s dataparser module uses sqlite3’s adapters and converters system to create loaders and dumpers for the data types. In this library object that contains load and dump methods for other objects is Parser.

Warning

If you create custom parsers or using initialized built-in ones, you should always set the connection’s parameter detect_types to sqlite3.PARSE_DECLTYPES.

from sqlite3 import PARSE_DECLTYPES

connector = Connector(..., detect_types=PARSE_DECLTYPES, ...)

Listing allowed types and typenames

In some cases, you may need a list of allowed types or typenames. (For example, you want to check whether your custom type is allowed to be used as a column type)

Allowed types

You can access the list of allowed types with allowed_types():

from sl3aio import allowed_types

allowed_types = allowed_types()

Note

This method returns a set of types that can be written into the database.

Allowed typenames

You can access the list of allowed typenames with allowed_typenames():

from sl3aio import allowed_typenames

allowed_typenames = allowed_typenames()

Note

This method returns a set of strings that can be column’s type in the database.

For default data types, this list includes only their affinities. So there are no such typenames as DOUBLE, TINYINT, VARCHAR(...) and etc. in the result set.

Custom parsers

You can define your own parsers for custom types using the Parser class. Let’s create a parser for the 2D point for example.

First import the Parser class:

from sl3aio import Parser

Then create a type for the 2D point:

class Point2D:
    def __init__(self, x: float, y: float) -> None:
        self.x = x
        self.y = y

Now create loads (converts data of DefaultDataType, recieved from the table, to python object) and dumps (converts python object to any of the allowed types, listed in allowed_types() method) methods for this type:

def loads(data: str) -> Point2D:
    point = data.split()
    return float(point[0]), float(point[1])


def dumps(point: Point2D) -> str:
    return f'{point.x} {point.y}'

Note

The type of data, recieved from the table by loads method will be the same as the return type of the dumps method.

If the dumps method returns an other type, that has its own parser, then the loads method will receive date the same type as the return type of this other type (and so on until the return type of dumps won’t be one of the DefaultDataType).

Finally create and register the parser:

point_parser = Parser(
    types={Point2D},
    _typenames={'Point2D', '2dpoint'},
    loads=loads,
    dumps=dumps
).register()

Hint

  • The Parser constructor takes the following parameters:

    1. types: Set of the types corresponding to the parser.

    2. _typenames: Set of the typenames (column types) corresponding to the parser. Every given typename will be converted to uppercase during initialization.

    3. loads: Method for converting data from the table to python object.

    4. dumps: Method for converting python object to the type corresponding to the parser.

  • The Parser.register() method registrates loads and dumps methods as the sqlite3’s converters and adapters.

  • Use the Parser.unregister() method to remove the converters and adapters from sqlite3.

Now you can use Point2D type in your database.

Tip

You can obtain the parser later by the desired type or typename using the following methods:

# Using type
point_parser = Parser.get_by_type(Point2D)

# Using typename
# (the given typename will be converted to uppercase automatically)
point_parser = Parser.get_by_typename('Point2D')

Parsable objects

You can also create a parser from the Parsable subclasses instances that must implement the Parsable.from_data() abstract classmethod that represents the loads and the Parsable.to_data() abstract method that represents the dumps.

Import the Parser and Parsable classes:

from sl3aio import Parsable, Parser

Create a type for the 2D point inherited from the Parsable and implement its abstract methods:

class Point2D(Parsable):
    def __init__(self, x: float, y: float) -> None:
        self.x = x
        self.y = y

    @classmethod
    def from_data(cls, data: str) -> 'Point2D':
        return cls(*map(float, data.split()))

    def to_data(self) -> str:
        return f'{self.x} {self.y}'

Now registrate the parser for the Point2D class using the Parser.from_parsable() classmethod:

point_parser = Parser.from_parsable(Point2D, typenames=['Point2D', '2dpoint']).register()

Hint

  • The Parser.from_parsable() method takes the following parameters:

    1. parsable: The subclass of the Parsable class.

    2. typenames: An any iterable of strings that represent the typenames for this parsable, optional, defaults to the empty tuple. If not provided or empty, the uppercase name of the parsable class is used.

Now you can use Point2D type in your database.

Built-in parsers

You can find several ready-made parsers in BuiltinParsers. Some of them are are available only after initialization.

Attention

Before using BuiltionParsers.BOOL, BuiltionParsers.SET, BuiltionParsers.TUPLE, BuiltionParsers.JSON, BuiltionParsers.TIME, BuiltionParsers.DATE and BuiltionParsers.DATETIME parsers, you must call BuiltinParsers.init() method that creates and registrates all these parsers.

Warning

Do not registrate BuiltionParsers.BLOB, BuiltionParsers.INT, BuiltionParsers.REAL and BuiltionParsers.TEXT parsers using their’s Parser.register() method.

So, if you want to use boolean values in your database, you can call the BuiltinParsers.init() method before accessing the tables, and then simply use them.

from sl3aio import BuiltinParsers

BuiltinParsers.init()
await table.insert(some_bool_value=True)

If you want to change the load/dump method of a parser after initialization, you can do so:

from sl3aio import BuiltinParsers


def new_date_dumps(obj):
    # Your new dumps logic here


def new_date_loads(data):
    # Your new loads logic here


BuiltinParsers.DATE.dumps = new_date_dumps

BuiltinParsers.init()
BuiltinParsers.DATE.dumps = new_date_dumps
BuiltinParsers.DATE.loads = new_date_loads
BuiltinParsers.DATE.register()  # Reregister the dumps/loads methods.