Python API

Get and set runtime settings

terracotta.get_settings()

Returns the current set of global runtime settings.

Return type:

TerracottaSettings

Example

>>> import terracotta as tc
>>> tc.get_settings().DEBUG
False

terracotta.update_settings(**new_config)

Update the global Terracotta runtime settings.

Parameters:

new_config (Any) – Options to override. Have to be valid Terracotta settings.

Return type:

None

Example

>>> import terracotta as tc
>>> tc.get_settings().DEFAULT_TILE_SIZE
(256, 256)
>>> tc.update_settings(DEFAULT_TILE_SIZE=[512, 512])
>>> tc.get_settings().DEFAULT_TILE_SIZE
(512, 512)

Get a driver instance

terracotta.get_driver(url_or_path, provider=None)

Retrieve Terracotta driver instance for the given path.

This function always returns the same instance for identical inputs.

Warning

Always retrieve Driver instances through this function instead of instantiating them directly to prevent caching issues.

Parameters:

• url_or_path (Union[str, Path]) – A path identifying the database to connect to. The expected format depends on the driver provider.

• provider (Optional[str]) – Driver provider to use (one of sqlite, sqlite-remote, mysql, postgresql; default: auto-detect).

Return type:

TerracottaDriver

Example

>>> import terracotta as tc
>>> tc.get_driver('tc.sqlite')
TerracottaDriver(
    meta_store=SQLiteDriver('/home/terracotta/tc.sqlite'),
    raster_store=GeoTiffRasterStore()
)
>>> tc.get_driver('mysql://root@localhost/tc')
TerracottaDriver(
    meta_store=MySQLDriver('mysql+pymysql://localhost:3306/tc'),
    raster_store=GeoTiffRasterStore()
)
>>> # pass provider if path is given in a non-standard way
>>> tc.get_driver('root@localhost/tc', provider='mysql')
TerracottaDriver(
    meta_store=MySQLDriver('mysql+pymysql://localhost:3306/tc'),
    raster_store=GeoTiffRasterStore()
)

TerracottaDriver

class terracotta.drivers.TerracottaDriver(meta_store, raster_store)

Terracotta driver object used to retrieve raster tiles and metadata.

Do not instantiate directly, use terracotta.get_driver() instead.

compute_metadata(path, *, extra_metadata=None, use_chunks=None, max_shape=None)

Compute metadata for a dataset.

Parameters:

• path (str) – Path identifing the dataset.

• extra_metadata (Optional[Any]) – Any additional metadata that will be returned as is in the result, under the metadata key.

• use_chunks (Optional[bool]) – Whether to load the dataset in chunks, when computing. Useful if the dataset is too large to fit in memory. Mutually exclusive with max_shape.

• max_shape (Optional[Sequence[int]]) – If dataset is larger than this shape, it will be downsampled while loading. Useful if the dataset is too large to fit in memory. Mutually exclusive with use_chunks.

Return type:

Dict[str, Any]

Returns:

A dict with the values

• range: global minimum and maximum value in dataset

• bounds: physical bounds covered by dataset in latitude-longitude projection

• convex_hull: GeoJSON shape specifying total data coverage in latitude-longitude projection

• percentiles: array of pre-computed percentiles from 1% through 99%

• mean: global mean

• stdev: global standard deviation

• metadata: any additional client-relevant metadata

connect(verify=True)

Context manager to connect to the metastore and clean up on exit.

This allows you to pool interactions with the metastore to prevent possibly expensive reconnects, or to roll back several interactions if one of them fails.

Parameters:

verify (bool) – Whether to verify the metastore (primarily its version) when connecting. Should be true unless absolutely necessary, such as when instantiating the metastore during creation of it.

Return type:

AbstractContextManager

Note

Make sure to call create() on a fresh metastore before using this method.

Example

>>> import terracotta as tc
>>> driver = tc.get_driver('tc.sqlite')
>>> with driver.connect():
...     for keys, dataset in datasets.items():
...         # connection will be kept open between insert operations
...         driver.insert(keys, dataset)

create(keys, *, key_descriptions=None)

Create a new, empty metadata store.

Parameters:

• keys (Sequence[str]) – A sequence defining the key names and order.

• key_descriptions (Optional[Mapping[str, str]]) – A mapping from key name to a human-readable description of what the key encodes.

Return type:

None

property db_version: str

Terracotta version used to create the meta store.

Returns:

A str specifying the version of Terracotta that was used to create the meta store.

delete(keys)

Remove a dataset from the meta store.

Parameters:

keys (Union[Sequence[str], Mapping[str, str]]) – Keys of the dataset. Can either be given as a sequence of key values, or as a mapping {key_name: key_value}.

Return type:

None

get_datasets(where=None, page=0, limit=None)

Get all known dataset key combinations matching the given constraints, and a path to retrieve the data (dependent on the raster store).

Parameters:

• where (Union[Sequence[str], Mapping[str, Union[str, List[str]]], None]) – A mapping from key name to key value constraint(s)

• page (int) – A pagination parameter, skips first page * limit results

• limit (Optional[int]) – A pagination parameter, max number of results to return

Return type:

Dict[Tuple[str, ...], Any]

Returns:

A dict mapping from key sequence tuple to dataset path.

get_keys()

Get all known keys and their fulltext descriptions.

Return type:

OrderedDict

Returns:

An OrderedDict in the form {key_name: key_description}

get_metadata(keys)

Return all stored metadata for given keys.

Parameters:

keys (Union[Sequence[str], Mapping[str, str]]) – Keys of the requested dataset. Can either be given as a sequence of key values, or as a mapping {key_name: key_value}.

Return type:

Dict[str, Any]

Returns:

A dict with the values

• range: global minimum and maximum value in dataset

• bounds: physical bounds covered by dataset in latitude-longitude projection

• convex_hull: GeoJSON shape specifying total data coverage in latitude-longitude projection

• percentiles: array of pre-computed percentiles from 1% through 99%

• mean: global mean

• stdev: global standard deviation

• metadata: any additional client-relevant metadata

get_raster_tile(keys, *, tile_bounds=None, tile_size=(256, 256), preserve_values=False, asynchronous=False)

Load a raster tile with given keys and bounds.

Parameters:

• keys (Union[Sequence[str], Mapping[str, str]]) – Key sequence identifying the dataset to load tile from.

• tile_bounds (Optional[Sequence[float]]) – Physical bounds of the tile to read, in Web Mercator projection (EPSG3857). Reads the whole dataset if not given.

• tile_size (Sequence[int]) – Shape of the output array to return. Must be two-dimensional. Defaults to DEFAULT_TILE_SIZE.

• preserve_values (bool) – Whether to preserve exact numerical values (e.g. when reading categorical data). Sets all interpolation to nearest neighbor.

• asynchronous (bool) – If given, the tile will be read asynchronously in a separate thread. This function will return immediately with a Future that can be used to retrieve the result.

Return type:

Any

Returns:

Requested tile as MaskedArray of shape tile_size if asynchronous=False, otherwise a Future containing the result.

insert(keys, path, *, override_path=None, metadata=None, skip_metadata=False)

Register a new dataset. Used to populate meta store.

Parameters:

• keys (Union[Sequence[str], Mapping[str, str]]) – Keys of the dataset. Can either be given as a sequence of key values, or as a mapping {key_name: key_value}.

• path (str) – Path to access dataset (driver dependent).

• override_path (Optional[str]) – If given, this path will be inserted into the meta store instead of the one used to load the dataset.

• metadata (Optional[Mapping[str, Any]]) – Metadata dict for the dataset. If not given, metadata will be computed via compute_metadata().

• skip_metadata (bool) – If True, will skip metadata computation (will be computed during first request instead). Has no effect if metadata argument is given.

Return type:

None

property key_names: Tuple[str, ...]

Get names of all keys defined by the meta store.

Returns:

A tuple defining the key names and order.

Supported metadata stores

SQLite metadata store

class terracotta.drivers.sqlite_meta_store.SQLiteMetaStore(path)

An SQLite-backed metadata driver.

Stores metadata and paths to raster files in SQLite.

This is the simplest Terracotta driver, as it requires no additional infrastructure. The SQLite database is simply a file that can e.g. be stored together with the actual raster files.

Note

This driver requires the SQLite database to be physically present on the server. For remote SQLite databases hosted on S3, use RemoteSQLiteDriver.

The SQLite database consists of 4 different tables:

• terracotta: Metadata about the database itself.

• key_names: Contains two columns holding all available keys and their description.

• datasets: Maps key values to physical raster path.

• metadata: Contains actual metadata as separate columns. Indexed via key values.

This driver caches key names, but not metadata.

Warning

This driver is not thread-safe. It is not possible to connect to the database outside the main thread.

Remote SQLite metadata store

class terracotta.drivers.sqlite_remote_meta_store.RemoteSQLiteMetaStore(remote_path)

An SQLite-backed metadata driver, where the database file is stored remotely on S3.

Stores metadata and paths to raster files in SQLite.

See also

SQLiteDriver for the local version of this driver.

The SQLite database is simply a file that can be stored e.g. together with the actual raster files on S3. Before handling the first request, this driver will download a temporary copy of the remote database file. It is thus not feasible for large databases.

The local database copy will be updated in regular intervals defined by REMOTE_DB_CACHE_TTL.

Warning

This driver is read-only. Any attempts to use the create, insert, or delete methods will throw a DatabaseNotWritableError.

MySQL metadata store

class terracotta.drivers.mysql_meta_store.MySQLMetaStore(mysql_path)

A MySQL-backed metadata driver.

Stores metadata and paths to raster files in MySQL.

Requires a running MySQL server.

The MySQL database consists of 4 different tables:

• terracotta: Metadata about the database itself.

• key_names: Contains two columns holding all available keys and their description.

• datasets: Maps key values to physical raster path.

• metadata: Contains actual metadata as separate columns. Indexed via key values.

This driver caches key names.