dlt.destinations.impl.lancedb.schema

Utilities for creating arrow schemas from table schemas.

NULL_SCHEMA

Empty pyarrow Schema with no fields.

make_arrow_field_schema

def make_arrow_field_schema(column_name: str, column: TColumnSchema,
                            type_mapper: DataTypeMapper) -> TArrowField

View source on GitHub

Creates a PyArrow field from a dlt column schema.

make_arrow_table_schema

def make_arrow_table_schema(
        table_name: str,
        schema: Schema,
        type_mapper: DataTypeMapper,
        vector_field_name: Optional[str] = None,
        embedding_fields: Optional[List[str]] = None,
        embedding_model_func: Optional[TextEmbeddingFunction] = None,
        embedding_model_dimensions: Optional[int] = None) -> TArrowSchema

View source on GitHub

Creates a PyArrow schema from a dlt schema.

add_vector_column

def add_vector_column(records: TArrowData, table_schema: TArrowSchema,
                      vector_column: str) -> TArrowData

View source on GitHub

Inserts a null vector_column at the index the table holds it at, when records omit it.

A merge matches the payload schema against the table by name, order and type.