Python · TardisDevParser on GitHub

What it does

Tardis.dev sells historical market data for crypto exchanges: trades, order book snapshots, and more. This tool downloads that data and saves it to disk. Nothing else. It does not parse the data, clean it, or store it in a database. It is the first step in a larger pipeline, and it only does that first step.

The tool has two commands. tardis-fetch downloads data. tardis-inspect checks your API key and shows what data is available, without downloading anything.

Installation

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

You need a Tardis.dev API key. Set it once as an environment variable. 

cp .env.example .env   # edit .env with your real key
export TARDIS_API_KEY=your-tardis-dev-api-key-here

Checking your key and browsing data

Before downloading anything, use tardis-inspect to confirm your key works and to see what is available. It makes small, read-only requests, so nothing is downloaded. 

# Confirm the API key works
tardis-inspect check-key

# List every exchange Tardis.dev has data for
tardis-inspect exchanges

# See data types, symbols, and date ranges for one exchange
tardis-inspect describe binance

Fetching data

A fetch job needs five things: an exchange, one or more data types, one or more symbols, a start date, and an end date. You can set these with command-line flags, or put them in a YAML config file. 

tardis-fetch \
  --exchange binance \
  --data-types trades,book_snapshot_25 \
  --symbols BTCUSDT,ETHUSDT \
  --from-date 2024-01-01 \
  --to-date 2024-01-07 \
  --output-dir ./data

Or with a config file, which is easier to reuse across runs: 

exchange: binance
data_types:
  - trades
  - book_snapshot_25
symbols:
  - BTCUSDT
  - ETHUSDT
from_date: "2024-01-01"
to_date: "2024-01-07"
output_dir: ./data
format: csv
overwrite: false
tardis-fetch --config config/example.yaml

If you use both, command-line flags win. A flag overrides the config file, and the config file overrides the TARDIS_API_KEY environment variable. This lets you keep one reusable config file and change just one or two values per run.

Downloaded files are written to <output_dir>/<exchange>/<data_type>/<symbol>/<date>.csv.gz. If a file already exists, the fetch skips it. Pass --overwrite to re-download it anyway.

How it works

FetchOptions, in config.py, holds the settings for one fetch job. It validates them: an exchange must be set, at least one data type and one symbol must be given, the start date can't be after the end date, and an API key must be present from one of the three sources. If any of that is wrong, it raises TardisConfigError with a message saying what is missing. 

@dataclass
class FetchOptions:
    exchange: str
    data_types: list[str]
    symbols: list[str]
    from_date: date
    to_date: date
    output_dir: Path = Path("./data")
    api_key: str = ""
    format: str = "csv"
    overwrite: bool = False

    def __post_init__(self) -> None:
        if not self.exchange:
            raise TardisConfigError("exchange is required")
        if not self.data_types:
            raise TardisConfigError("at least one data_type is required")
        # ... symbols, dates, and api_key are checked the same way

TardisClient, in client.py, does the actual downloading. For each combination of data type, symbol, and day in the date range, it builds a URL, sends a GET request with the API key as a bearer token, and streams the response to a temporary file. Once the download finishes, it renames the temporary file to its final name. This way a failed or interrupted download never leaves a partial file at the path a later run would treat as "already downloaded". 

def fetch_one(self, data_type: str, symbol: str, day: date) -> Path:
    dest = self.local_path(data_type, symbol, day)
    if dest.exists() and not self.options.overwrite:
        return dest

    url = self.build_url(data_type, symbol, day)
    response = self.session.get(url, stream=True, timeout=60)
    self._raise_for_status(response, url)

    dest.parent.mkdir(parents=True, exist_ok=True)
    tmp_path = dest.with_suffix(dest.suffix + ".part")
    with open(tmp_path, "wb") as f:
        for chunk in response.iter_content(chunk_size=1024 * 256):
            if chunk:
                f.write(chunk)
    tmp_path.replace(dest)
    return dest

The client also retries failed requests. If Tardis.dev responds with a 429 (rate limit) or a 5xx (server error), it waits and tries again, up to five times, with a growing delay between attempts. A 401 or 403 response means the API key was rejected, and raises TardisAuthError right away instead of retrying, since retrying a bad key will never succeed.

TardisInspector, in inspector.py, is the read-only companion used by tardis-inspect. check_credentials() confirms an API key works by sending a HEAD request for a symbol and date that is known to be available, so it can check the key without downloading real data. list_exchanges() and describe_exchange() call Tardis.dev's public metadata API to list exchanges and show what data types, symbols, and date ranges each exchange has.

Errors

Three exception types cover the ways a fetch can fail, so a caller can tell them apart.

  • TardisConfigError — the fetch options are invalid, for example a missing API key or an end date before the start date.
  • TardisAuthError — the API key was rejected by Tardis.dev.
  • TardisAPIError — some other request failed, for example no data at the requested URL, or an unexpected server response.

tardis-fetch catches all three at the top level and exits with a different status code for each, so the error type is visible from the shell as well as the message.

Project structure

src/tardis_reader/
    config.py          # FetchOptions: loads, merges, and validates fetch settings
    client.py          # TardisClient: downloads data from the Datasets API
    inspector.py        # TardisInspector: checks the API key, lists available data
    exceptions.py        # TardisConfigError / TardisAuthError / TardisAPIError
    cli.py              # tardis-fetch entrypoint
    inspect_cli.py       # tardis-inspect entrypoint

config/
    example.yaml        # Sample fetch config

tests/
    test_config.py
    test_client.py
    test_inspector.py

Tests cover option parsing and validation, the download/skip/retry/error paths in the client, and the credential check and metadata lookups in the inspector. Run them with pytest.