dlt.destinations.impl.lancedb.configuration

PUBLIC_SCHEMA_NAME

Name the SQL endpoint gives to the root namespace.

LanceDBCredentials Objects

@configspec
class LanceDBCredentials(CredentialsConfiguration)

View source on GitHub

Credentials for a managed LanceDB Enterprise or Cloud cluster.

database

Name of the database that holds every dataset, connected to as db://<database>. Leave empty to let each dataset be its own database

api_key

API key authenticating to the cluster.

host_override

Cluster endpoint. Required for Enterprise, leave empty for LanceDB Cloud.

region

Region of the LanceDB Cloud database.

flightsql_host

Host of the Arrow Flight SQL endpoint, which Enterprise serves from a separate load balancer. Leave empty to disable SQL access.

flightsql_port

Port of the Arrow Flight SQL endpoint.

flightsql_tls

Whether the Arrow Flight SQL endpoint requires TLS.

headers

Extra HTTP headers sent to the cluster.

read_consistency_interval_seconds

(Enterprise cluster does not honor this) How stale a read of the managed client can be, in seconds.

get_conn

def get_conn(
    database: str,
    read_consistency_interval_seconds: Optional[float] = None
) -> "RemoteDBConnection"

View source on GitHub

Returns a connection to database on the managed cluster, at the requested staleness.

Arguments:

• database str - Name of the database, which is the dataset being read or written.
• read_consistency_interval_seconds Optional[float] - Overrides read_consistency_interval_seconds. Pass 0 to always read the latest version.

Returns:

• RemoteDBConnection - Connection shared by all callers asking for the same database and staleness.

flightsql_uri

def flightsql_uri() -> str

View source on GitHub

Returns the gRPC URI of the Arrow Flight SQL endpoint.

flightsql_headers

def flightsql_headers(database: str) -> List[Tuple[bytes, bytes]]

View source on GitHub

Returns call headers authenticating and routing Flight SQL requests to database.

LanceDBClientConfiguration Objects

@configspec
class LanceDBClientConfiguration(DestinationClientDwhConfiguration)

View source on GitHub

commit_tag

Tag applied to every table of the dataset after a successful load, so the whole dataset can be read back as one tagged version.

embeddings

Embeddings config. Adds a vector column when set.

dataset_sentinel_namespace_name

Namespace marking a dataset as created.

normalize_dataset_name

def normalize_dataset_name(schema: Schema) -> str

View source on GitHub

Returns the database where the schema tables of schema are materialized, which is the dataset itself.

Raises:

• ConfigurationValueError - If credentials.database configures a database and dataset_name names a different one.

fingerprint

def fingerprint() -> str

View source on GitHub

Returns a fingerprint of the cluster.

data_location

def data_location() -> str

View source on GitHub

Returns the cluster, which is where every dataset of this destination lives.

The location excludes the database: one Arrow Flight SQL endpoint accesses every database of the cluster, so two datasets of one cluster must compare equal.

can_write_from

def can_write_from(other: DestinationClientConfiguration) -> bool

View source on GitHub

LanceDB has no engine that can execute SQL or run a model job, so dlt is that engine and materializes eagerly.