finlab.data: data.get Data Download API

finlab.data is the core module for downloading FinLab datasets. The usual workflow is to select the market with data.set_market(), find dataset keys with data.search(), download them with data.get(), then narrow the stock universe with data.us_universe() or data.universe().

Task	API	Common usage
Select the U.S. market	`data.set_market()`	`data.set_market('us')`
Download prices, fundamentals, revenue, and trading data	`data.get()`	`data.get('price:adj_close')`
Find table and field names	`data.search()`	`data.search('equity', market='us')`
Restrict index, sector, exchange, or Taiwan market universes	`data.us_universe()` / `data.universe()`	`with data.us_universe(index='SP500'):`
Control download size and memory usage	`truncate_start`, cache settings	`data.truncate_start = '2020-01-01'`

First-time users should read Data Download. To turn downloaded data into stock-selection conditions, continue with finlab.dataframe.

Use Cases

Download historical data such as stock prices, market value, and quarterly financial statements
Filter data by index, sector, or exchange
Search for available data tables and fields
Configure data caching strategies
Limit data download range to save memory

Quick Examples

Basic Usage: Download Data

from finlab import data

data.set_market('us')

# Download adjusted closing prices
close = data.get('price:adj_close')
raw_close = data.get('price:close')  # unadjusted price for valuation

# Download quarterly revenue (aligned to SEC filing dates)
revenue = data.get('us_income_statement:revenue')

# Trailing-12-month P/E from diluted EPS
eps = data.get('us_income_statement:eps_diluted')
pe_ratio = raw_close / eps.rolling(4).sum()

English price keys in both markets

price:open, price:high, price:low, price:close, and price:volume work in both markets. After data.set_market("us") they return U.S. prices; under the default Taiwan market or data.set_market("tw") they return Taiwan daily prices and share the cache with the original Taiwan keys. The Taiwan aliases also work with data.gets(), lazy=True, and start/end date filters. volume means shares traded.

data.set_market("us")
us_close = data.get("price:close")
data.set_market("tw")
tw_close = data.get("price:close")

Taiwan fields outside these five aliases keep their original keys.

Search Available Fields

# Search U.S. stock data
data.search('close', market='us')

# Search U.S. ETF/fund data
data.search('close', market='us_fund')

# List Taiwan stock fields
data.search(market='tw')

Restrict Market Scope

data.set_market('us')

# Only fetch the latest S&P 500 constituents
with data.us_universe(index='SP500'):
    close = data.get('price:adj_close')

# Only fetch one sector
with data.us_universe(sector='Technology'):
    close = data.get('price:adj_close')

# Switch to US ETF/fund data
data.set_market('us_fund')
fund_close = data.get('price:adj_close')

Remove a universe filter

Use data.reset_universe() to disable the filter. data.set_universe(), data.universe(), and data.us_universe() without any filter do the same and drop no identifiers:

data.set_market('us')
data.set_universe(sector='Technology')
filtered = data.get('price:adj_close')
data.reset_universe()
unfiltered = data.get('price:adj_close')

The reset applies to subsequent data.get() and data.gets() calls, including cached reads. It preserves cached datasets, the active market, date limits, and already returned DataFrames. Inside data.universe() or data.us_universe(), a reset lasts until another selection or context exit; exiting still restores the selection from before that context. Repeated resets are safe.

Detailed Guide

See Data Download Details for: - Complete data download tutorial - Data table structure explanation - Advanced filtering techniques - Error handling methods

Global Configuration

Force Cloud/Local Data

from finlab import data

# Force cloud download (re-download every time)
data.force_cloud_download = True

# Or use only the local cache (offline environment); never both at once
# data.use_local_data_only = True

Limit Data Time Range

# Only download data from 2020-2023 (saves memory)
data.truncate_start = '2020-01-01'
data.truncate_end = '2023-12-31'

# All subsequent data.get() calls will use this range
close = data.get('price:adj_close')

Recommended Usage

Development phase: Use truncate_start to limit data range for faster testing
Production backtesting: Remove truncate limits, use full historical data
Low memory: Set truncate_start or use use_local_data_only

API Reference

data.get()

finlab.data.get

Shared fetch pipeline plus data.get() single-dataset wrapper.

DownloadedFrame `module-attribute`

DownloadedFrame = tuple[pd.DataFrame, bytes | None]

(df, raw_feather_bytes) — bytes is None when payload was inlined.

get

get(
    dataset,
    save_to_storage=True,
    force_download=False,
    lazy=False,
    progress='silent',
    progress_callback=None,
    *,
    start=None,
    end=None,
    as_of=None,
    version=None,
)

Download historical data.

See https://ai.finlab.tw/database for the full data catalogue.

PARAMETER	DESCRIPTION
`dataset`	The name of dataset. `tw_tick:SYMBOL` and `tw_minute:SYMBOL` return after-market long-form pandas frames and require both dates (at most 31 calendar days). See the intraday data guide. TYPE: `str`
`save_to_storage`	Whether to cache intraday data or retained versions (`as_of`/`version`). `False` raises `ValueError` for ordinary reads; use `data.set_storage(data.CacheStorage())` instead. TYPE: `bool` DEFAULT: `True`
`force_download`	Whether to force download the dataset from cloud. TYPE: `bool` DEFAULT: `False`
`lazy`	Return a lazy computation wrapper after fetching the dataset. TYPE: `bool` DEFAULT: `False`
`progress`	`'display'`, `'event'`, or `'silent'`. TYPE: `ProgressMode` DEFAULT: `'silent'`
`progress_callback`	Receives structured events in event mode. TYPE: `Callable[[ProgressEvent], None] \| None` DEFAULT: `None`
`start`	Optional inclusive lower bound on the `date` column. Accepts any date-like value (string or `Timestamp`). Pushed down to the Arrow read so only in-range rows are materialised. Requires the dataset to have a `date` column/index, otherwise raises `ValueError`. TYPE: `DateBound` DEFAULT: `None`
`end`	Optional inclusive upper bound on the `date` column. Same rules as `start`. When both `start` and `end` are `None` the result is byte-identical to a plain read. TYPE: `DateBound` DEFAULT: `None`
`as_of`	Return the dataset exactly as it was published at this time (naive values are Asia/Taipei, for every market; a bare date means 00:00 that day, before that day's publications). Only versions the server still retains are available; list them with `data.versions(dataset)`. Requires a paid data plan and counts toward the download quota; a version already downloaded is read from local storage. `financial_statement` keeps permanent vintages instead (`financial_statement_versions`). TYPE: `DateBound` DEFAULT: `None`
`version`	Return exactly this generation from `data.versions(dataset)`. Mutually exclusive with `as_of`. TYPE: `str \| int \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`DataFrame`	financial data as a DataFrame.

get_role

get_role()

Return the current user role (e.g. 'free' or 'vip').

is_tradable

is_tradable(market=None)

Whether loaded data completed its final scheduled refresh before next open.

Daily TW data is judged by publication evidence, not expiry; use data.readiness() for the reason. Markets without publication evidence and runs without a trading calendar fall back to the published expiry.

is_vip

is_vip()

Return True if the current user is a VIP member.

next_future_expiry

next_future_expiry()

Next future expiry time among cached datasets (UTC).

Returns only expiry times in the future, skipping already expired data.

RETURNS	DESCRIPTION
`datetime \| None`	datetime in UTC representing the next future expiry, or None if
`datetime \| None`	no future expiries exist or no data is available.

plan_download

plan_download(dataset)

Paid ordinary-read plan: candidate sizes and estimates in bytes.

process_data

process_data(dataset, df)

Process raw data into standardized format.

suggested_tradable_time

suggested_tradable_time(market=None)

Latest required scheduled source check, not a guaranteed arrival time.

Returns None if readiness is unknown or every required slot has passed. Rerun the strategy after the scheduled check and confirm readiness again.

Common U.S. Data Tables

Call data.set_market('us') first; generic price:* keys then resolve to us_price:*.

Price Data: - price:adj_close - Adjusted daily close (use for signals and returns) - price:close - Raw daily close - price:adj_open / price:adj_high / price:adj_low - Adjusted open/high/low - price:volume - Shares traded - etl:us_market_value - Daily market value (USD)

Quarterly Fundamentals (aligned to each SEC filing date): - us_income_statement:revenue, us_income_statement:net_income, us_income_statement:eps_diluted - us_balance_sheet:total_assets, us_balance_sheet:total_stockholders_equity - us_cash_flow:operating_cash_flow, us_cash_flow:free_cash_flow

Universe and Benchmarks: - us_index_constituents:sp500 / us_index_constituents:nasdaq100 - Dated index membership - etl:us_common_stock_filter, etl:us_liquid_stock_filter - Common-stock and liquidity filters - world_index:adj_close - Index levels such as ^GSPC (S&P 500), ^IXIC, ^DJI

Ratios such as P/E, P/B and ROE are computed from these tables (for example P/E as unadjusted price:close divided by eps.rolling(4).sum()). The us_ratios:* and us_key_metrics:* tables hold only recent daily snapshots, so use them for current screening, not historical backtests. Browse everything in the Database Catalog.

Common Errors

DatasetNotFoundError: Data table name is wrong; the message lists the closest valid keys
Empty DataFrame: Query conditions too strict or data does not exist
Out of memory: Too much data downloaded, use truncate_start to limit range

data.search()

data.search(keyword: str | None = None, market: str | None = None, *, details: bool = False) -> pd.Series | pd.DataFrame

Searches the FinLab data catalog. The keyword matches dataset keys and, when the catalog provides them, each dataset's name, description, aliases and table columns. Matching ignores case and full-width/half-width differences. No dataset is downloaded.

Parameter	Type	Description
`keyword`	`str` or `None`	Text to look for. `None` lists every dataset of the market. A market name on its own (`data.search('us')`) is treated as `market`.
`market`	`str` or `None`	`'us'`, `'us_fund'`, `'tw'`, `'rotc'`, `'hk'`, `'jp'`, `'kr'`, `'uk'`, or `'all'`. Defaults to the market set with `data.set_market()` (Taiwan if none).
`details`	`bool`	Return a DataFrame with `dataset`, `auth` (`free` or `vip`) and `free_until` (inclusive free-data cutoff, or `None`) instead of names only.

Returns: a pd.Series of dataset names (name="dataset"), or a pd.DataFrame when details=True. When results include VIP-only datasets, a hint to use details=True is printed. Missing catalog permissions stay unknown; search does not grant access.

Examples:

from finlab import data

# U.S. fields about equity: dataset names, descriptions and columns are matched
data.search('equity', market='us')

# Price fields of U.S. ETFs and funds
data.search('close', market='us_fund')

# Which U.S. datasets are free and which need VIP
data.search('income', market='us', details=True)

# Every U.S. stock field
all_fields = data.search(market='us')
print(f"Total {len(all_fields)} fields")

Hints for mistyped keys

When data.get() cannot find a dataset it raises finlab.exceptions.DatasetNotFoundError (a RuntimeError subclass) listing the closest valid keys. If the key is a registered equivalent legacy name, it emits a UserWarning and fetches the correct key instead.

Download failures

When data.get() fails to download data it raises finlab.exceptions.SignedDownloadError (a DataError subclass). The message names the dataset, the HTTP status or kind of connection problem, and what to do; it never contains the download URL. To branch on the cause, read e.kind ("http", "timeout", "connection", "ssl", "redirects", "browser", "invalid_reply", "other") and e.status (the HTTP status, None when no response arrived) instead of matching the message. For compatibility it also keeps the requests type raised before (such as requests.HTTPError or requests.Timeout) and is a requests.RequestException and OSError; new code should use except DataError.

Daily download quota

Free and VIP accounts receive one warning per session when a download response reports daily usage at or above 80%. When the quota is exhausted, data.get() / data.gets() raise finlab.exceptions.QuotaExceededError, a subclass of both DataError and RuntimeError. The message includes the limit and the daily reset time, 08:00 UTC+8 (00:00 UTC). In batch backtests, catch this exception before general backtest errors to distinguish exhausted quota from a failed strategy.

data.describe()

finlab.data.describe

describe(dataset, market=None)

Inspect an exact dataset key or category using the cached search catalog.

No dataset is downloaded. layout describes the published dataset (pivot/table/long), whereas source_layout is only populated if the catalog explicitly describes upstream storage. frequency is never inferred from the crawler's cron schedule.

index_end is the catalog's last index label, which can be a future period label. It is not the latest available observation date. source_created_time is the published artifact's creation time, not a row's availability time. latest_data_date is None unless explicitly published under that name. Missing metadata stays None.

An unlisted category:key_date returns category context with catalogued=False and unknown layout/availability. Absence from the catalog does not prove that get() cannot download it, nor that its values represent announcement dates. Check the category description for semantics. Other unknown keys raise KeyError; an unavailable/empty catalog raises RuntimeError. Markets follow search(), except that "all" is not supported.

data.describe('us_price:close', market='us')
data.describe('us_income_statement:revenue', market='us')

Pass a table name such as us_income_statement instead of a full key to get the table's columns and description. This query shares the search catalog cache and downloads no dataset. layout describes the published dataset; source_layout is the upstream storage format only when explicitly documented. Missing metadata returns None. The update schedule does not establish observation frequency. index_end is the last index label and may be a future period date; source_created_time is the artifact creation time. Neither establishes row availability or latest_data_date.

columns lists fields published in the catalog. An unlisted key_date returns catalogued=False, key_date_available=None, and category_description. This does not establish whether it can be downloaded or whether its values are announcement dates. Other unknown names raise KeyError; an unavailable or empty catalog raises RuntimeError.

data.us_universe()

finlab.data.us_universe

us_universe(
    sector='ALL',
    industry='ALL',
    exchange='ALL',
    security_type='ALL',
    index=None,
)

Context manager to set a global stock universe filter for US market data retrieval.

This context manager limits the set of US stocks returned by data functions to a specific sector, industry, exchange, and/or index selection. The filter is applied globally within the context and is restored after the context exits.

Parameters

sector : str | list[str], default 'ALL' Sector name(s) to include. Supports regex-like substring matching. industry : str | list[str], default 'ALL' Industry name(s) to include. Supports regex-like substring matching. exchange : str | list[str], default 'ALL' Exchange name(s) to include. Common values: 'NASDAQ', 'NYSE', 'AMEX'. security_type : str | list[str], default 'ALL' Security type(s) to include. Use 'common_stock' for clean US equities. index : str | None, default None Market index to filter by. Supported values: 'SP500'/'SPX' (S&P 500), 'NASDAQ100'/'NDX' (NASDAQ 100). Can be combined with sector/industry/exchange for further filtering.

Notes

index uses the latest row of the constituent table to select columns for the entire price history. Backtesting these columns introduces survivorship and look-ahead bias; later index changes can also change results for earlier dates.

For historical selection, read unfiltered prices and apply the dated constituent records to scores before ranking, as shown below. Dates before the first record and price symbols absent from the table are False. The example matches symbols exactly: spelling differences, renamed stocks and missing price history can exclude constituents. Source delays or later corrections remain in the records; this is not an immutable snapshot of what was known at the time. Use each date's mask after that day's close.

Examples

from finlab import data with data.us_universe(sector='Technology', exchange='NASDAQ'): ... close = data.get('us_price:close')

Filter to common stocks only:

with data.us_universe(security_type='common_stock'): ... close = data.get('us_price:close')

Filter to the latest S&P 500 constituents (all historical dates):

with data.us_universe(index='SP500'): ... close = data.get('us_price:close')

Filter to the latest NASDAQ 100 Technology stocks (all historical dates):

with data.us_universe(index='NDX', sector='Technology'): ... close = data.get('us_price:close')

Historical index backtest:

with data.us_universe(): # Temporarily clear any existing universe filter. ... close = data.get('us_price:close') ... constituents = data.get('us_index_constituents:sp500') in_index = constituents.reindex(close.index, method='ffill').eq(True) score = close.pct_change(20) position = score.where(in_index).is_largest(3)

For NASDAQ 100, replace us_index_constituents:sp500 with us_index_constituents:nasdaq100.

U.S. Market Filtering:

data.set_market('us')

# Get the full price history of the latest S&P 500 constituents
with data.us_universe(index='SP500'):
    close = data.get('price:close')

# Get Technology stocks among the latest NASDAQ 100 constituents
with data.us_universe(index='NDX', sector='Technology'):
    close = data.get('price:close')

# Get a clean U.S. common-stock universe
with data.us_universe(security_type='common_stock'):
    close = data.get('price:close')

# Can also use data.universe() with set_market
with data.universe(index='SP500', exchange='NASDAQ'):
    close = data.get('price:close')

Supported index parameter values:

Index	Accepted aliases
S&P 500	`'SP500'`, `'SPX'`, `'S&P500'`, `'S&P 500'`
NASDAQ 100	`'NASDAQ100'`, `'NDX'`, `'NASDAQ 100'`

index can be combined with sector, industry, exchange, and security_type; the result is the intersection of all filters. U.S. sector names: Technology, Healthcare, Financial Services, Consumer Cyclical, Industrials, Communication Services, Consumer Defensive, Energy, Basic Materials, Real Estate, Utilities.

Historical index backtests

index='SP500' or 'NDX' selects columns for the entire history using the last row of the constituent table. Backtesting these columns introduces survivorship and look-ahead bias: later additions appear before they joined, while removed stocks disappear from the entire history. Index changes can therefore change earlier results when the same strategy is rerun. The index parameter of data.universe(), data.set_universe() and data.set_us_universe() works the same way.

For historical selection, read unfiltered U.S. prices, apply the constituent record dated on or before each price date to scores, then rank stocks:

from finlab import data

with data.us_universe():  # Temporarily clear the existing universe; restore it on exit.
    close = data.get('us_price:close')
    constituents = data.get('us_index_constituents:sp500')
in_index = constituents.reindex(close.index, method='ffill').eq(True)
score = close.pct_change(20)
position = score.where(in_index).is_largest(3)

For NASDAQ 100, use us_index_constituents:nasdaq100. Dates before the first record and price symbols absent from the table are False; use each date's mask after that day's close. This example matches symbols exactly, so spelling differences, renamed stocks or missing price history can still exclude constituents. The mask is rebuilt from the records currently stored: source delays, gaps and later corrections still affect results. It is not a complete snapshot of what was known at the time.

data.universe()

data.universe(
    exchange='ALL', sector='ALL', exclude_sector=None, industry='ALL',
    asset_type=None, security_type='ALL', index=None, *,
    whitelist=None, market=None, category=None, exclude_category=None,
)

Context manager that limits which securities data.get() returns, and restores the previous selection on exit. It dispatches on the active market: with data.set_market('us') it filters U.S. stocks like data.us_universe(); under the default Taiwan market it filters Taiwan listings.

Parameter	Type	Description
`exchange`	`str` or `list[str]`	U.S. and other international markets: `'NASDAQ'`, `'NYSE'`, `'AMEX'`, `'HKEX'`, ... Taiwan: `'TWSE'` / `'TPEx'` (legacy `'TSE'` / `'OTC'`).
`sector`	`str` or `list[str]`	Sector name(s), matched as regular expressions.
`exclude_sector`	`str`, `list[str]` or `None`	Sector(s) to exclude (Taiwan only).
`industry`	`str` or `list[str]`	Industry filter (international markets).
`asset_type`	`str` or `None`	Taiwan only: `'ETF'` or `'STOCK_FUTURE'`. `'STOCK_FUTURE'` uses the latest underlying list for every date; see Historical stock-futures backtests.
`security_type`	`str` or `list[str]`	Security type filter (international markets), e.g. `'common_stock'`.
`index`	`str` or `None`	U.S. index: `'SP500'` / `'SPX'` or `'NASDAQ100'` / `'NDX'`. Uses the latest constituent list for every date; see Historical index backtests.
`whitelist`	`str`, iterable of `str`, or `None`	Explicit security IDs (a string is one ID). Alone, it bypasses the current market catalog, so delisted IDs can be kept; with other filters, the intersection is used. Empty or non-overlapping selections return empty data. It does not encode listing dates.
`market`, `category`, `exclude_category`		Legacy Taiwan aliases for `exchange` + `asset_type`, `sector`, and `exclude_sector`.

Examples:

from finlab import data

data.set_market('us')

# NASDAQ-listed stocks only
with data.universe(exchange='NASDAQ'):
    close = data.get('price:adj_close')

# Two sectors at once
with data.universe(sector=['Technology', 'Healthcare']):
    close = data.get('price:adj_close')

# A fixed list of tickers
with data.universe(whitelist=['AAPL', 'MSFT', 'NVDA']):
    close = data.get('price:adj_close')

Under the Taiwan market, exchange selects TWSE or TPEx listings and sector takes Taiwan industry names, which are written in Chinese in the Taiwan data.

Historical stock-futures backtests

asset_type='STOCK_FUTURE' filters every date with today's list of stock-futures underlyings. Stocks appear years before their futures listed, and former underlyings disappear from the whole history. To select by date, mask scores from each stock's first futures trade (standard or mini contract):

The Taiwan dataset and column names below use Unicode escapes; Python resolves them to their original names.

from finlab import data

with data.universe():  # Temporarily clear any existing universe filter.
    close = data.get('price:\u6536\u76e4\u50f9')
    underlying = data.get('single_stock_futures_and_equity_options_underlying')
    volume = data.get('futures_price:\u6210\u4ea4\u91cf')
underlying = underlying[
    underlying['\u662f\u5426\u70ba\u80a1\u7968\u671f\u8ca8\u6a19\u7684'] == 'Y'
]
contracts = (
    underlying[
        '\u80a1\u7968\u671f\u8ca8\u3001\u9078\u64c7\u6b0a\u5546\u54c1\u4ee3\u78bc'
    ].astype(str)
    + 'F\u4e00\u822c'
)
traded = volume.reindex(columns=contracts).gt(0)
traded.columns = underlying['symbol'].astype(str)
listed = traded.T.groupby(level=0).any().T.cummax()
has_futures = listed.reindex(close.index, method='ffill', fill_value=False)
score = close.pct_change(20)
position = score.where(has_futures).is_largest(10)

The mask is built from today's underlying list, so former underlyings are still left out.

data.indicator()

data.indicator(indname: str, adjust_price: bool = False, resample: str = 'D', *, strict: bool = False, **kwargs) -> pd.DataFrame | tuple[pd.DataFrame, ...]

Computes a TA-Lib or pandas-ta indicator for every stock of the active market from its open, high, low, close and volume. Install TA-Lib or pandas-ta first.

Parameter	Type	Description
`indname`	`str`	Indicator name: TA-Lib functions such as `'SMA'`, `'RSI'`, `'STOCH'`, `'MACD'` (TA-Lib docs), or pandas-ta indicators such as `'supertrend'`.
`adjust_price`	`bool`	Compute from adjusted prices instead of raw prices.
`resample`	`str`	Price bar period: `'D'` daily, `'W'` weekly, `'M'` monthly.
`strict`	`bool`	Accept only TA-Lib's numeric parameter names and reject price selectors, e.g. `data.indicator('RSI', strict=True, timeperiod=14)`. Not supported for pandas-ta (raises `NotImplementedError`). Only names are checked; TA-Lib validates the values.
`market`	`str`	Optional keyword: `'US_STOCK'` or `'TW_STOCK'`, overriding the active market.
`**kwargs`		Indicator parameters, e.g. `timeperiod=14` for TA-Lib's RSI.

Returns: one FinlabDataFrame (dates by symbols) per indicator output; a tuple when the indicator has several outputs (for example STOCH returns k, d).

Examples:

from finlab import data

data.set_market('us')

# 14-day RSI of every U.S. stock
rsi = data.indicator('RSI', timeperiod=14)

# MACD has three outputs
macd, macd_signal, macd_hist = data.indicator('MACD')

# Weekly stochastic oscillator
k, d = data.indicator('STOCH', resample='W')

Cache Management

data.set_storage()

data.set_storage(storage: CacheStorage | FileStorage) -> None

Sets where downloaded data is kept locally so repeated data.get() calls do not download it again. FinLab provides two storages: CacheStorage keeps data in memory, and FileStorage writes it to files. By default FinLab uses FileStorage in the operating system's temporary folder.

Parameter	Type	Description
`storage`	`CacheStorage` or `FileStorage`	The storage interface to use from now on.

data.CacheStorage

data.CacheStorage()

Keeps downloaded data in memory only, for the current Python session.

Method	Description
`read_dataframe_date_range(name, start, end)`	Cached rows whose `date` lies in `[start, end]` (both inclusive).
`get_time_created(name)`	Revision time of the cached frame, or `None`.
`get_time_expired(name)`	Published expiry known for the cached version, if any.
`cache_version(name)`	Identifier that changes on every write, even at the same revision.
`clear()`	Remove all in-memory data and timestamps.
`release_memory()`	Release cached data; with no disk copy this clears everything.

data.FileStorage

data.FileStorage(path: str | None = None, use_cache: bool = True)

Writes downloaded data as Feather files, one file per dataset, with : in the dataset key replaced by # (for example us_price#adj_close.feather).

Parameter	Type	Description
`path`	`str` or `None`	Folder for the data files. Defaults to FinLab's temporary folder.
`use_cache`	`bool`	Also keep a copy of each dataset in memory.

Method	Description
`read_dataframe_date_range(name, start, end)`	Read only the on-disk rows whose `date` lies in `[start, end]`, without loading the whole table.
`get_dataframe_row_count(name)` / `get_dataframe_index(name)`	Row count or index of the local file.
`get_time_created(name)`	Revision of the cache file (its modification time, set by the last download).
`release_memory()`	Release the in-memory copies, keeping the files.
`clear()`	Delete the local data files.
`diagnose(dataset=None)`	Print the local storage status for one dataset, or list all local data.

Custom Cache Strategy:

import pandas as pd

from finlab import data

# Use a custom directory
data.set_storage(data.FileStorage('/path/to/custom/cache'))

data.set_market('us')
close = data.get('price:adj_close')

# The file can be read directly with pandas
close_from_disk = pd.read_feather('/path/to/custom/cache/us_price#adj_close.feather')

# Check what is stored locally
data._storage.diagnose()

User Role

finlab.data.get_role

get_role()

Return the current user role (e.g. 'free' or 'vip').

finlab.data.is_vip

is_vip()

Return True if the current user is a VIP member.

Example:

from finlab import data

# Query user role (requires at least one data.get() call first)
data.set_market('us')
close = data.get('price:adj_close')

role = data.get_role()  # 'free', 'vip', ...
print(f"Current role: {role}")

if data.is_vip():
    print("VIP member: full data access enabled")

Other Utilities

data.get_strategies()

data.get_strategies(api_token: str | None = None) -> dict

Returns the data of the strategies you uploaded to FinLab, the same numbers shown on your strategy dashboard, so several strategies can be combined in your own code.

Parameter	Type	Description
`api_token`	`str` or `None`	Your FinLab API token. When omitted, the token of the current `finlab.login()` session is used.

Returns: a dict keyed by strategy name (an empty dict when authentication fails). Each value contains, among others:

Key	Content
`returns`	Equity curve: `{'time': [...], 'value': [...]}`
`stats`	Statistics: `daily_mean` (annualized mean return), `daily_sharpe`, `max_drawdown`, `win_ratio`, plus `drawdown_details` (each drawdown's start, end, length and depth)
`return_table`	Monthly returns by year, with `YTD`
`ndays_return`	Returns over the last 1, 5, 10, 20 and 60 days
`positions`	Current holdings with entry date and price, weights and status
`fee_ratio`, `tax_ratio`, `trade_at`	Trading assumptions of the strategy
`last_trading_date`, `next_trading_date`, `update_date`	Schedule information

from finlab import data

strategies = data.get_strategies()
for name, info in strategies.items():
    print(name, info['stats']['daily_sharpe'])

FAQ

Q: How do I find out what data is available for download?

Method 1: Use search()

# List all U.S. stock fields
all_fields = data.search(market='us')
for field in all_fields[:10]:
    print(field)

Method 2: Check the online database Visit the FinLab Database Catalog to browse the full list.

Q: Data download is slow, what can I do?

data.set_market('us')

# Method 1: Limit time range
data.truncate_start = '2020-01-01'

# Method 2: Use cache (second call will be fast)
close = data.get('price:adj_close')  # First call downloads
close = data.get('price:adj_close')  # Second call reads the local cache

# Method 3: Limit the stock universe
with data.us_universe(index='SP500'):
    close = data.get('price:adj_close')  # Only S&P 500 columns

Q: DatasetNotFoundError or KeyError - what should I do?

Possible causes: 1. Not logged in - Run finlab.login() and complete login 2. Incorrect field name - Use data.search('close', market='us') to verify the correct name; the error message also lists the closest valid keys 3. Wrong market - Generic keys such as price:adj_close resolve through the active market; call data.set_market('us') first, or use the full key us_price:adj_close 4. Insufficient data permission - Check whether the account can access this dataset with data.search(..., details=True)

import finlab

# Check if logged in
try:
    token, token_type = finlab.get_token()
    print(f"Logged in ({token_type})")
except Exception:
    print("Not logged in, please run finlab.login()")

Q: How do I download U.S. ETF data?

from finlab import data

# ETFs and funds are a separate market
data.set_market('us_fund')
etf_close = data.get('price:adj_close')[['SPY', 'QQQ']]

# Search U.S. ETF/fund fields
fund_fields = data.search(market='us_fund')

Q: What about missing values in the data?

data.set_market('us')
close = data.get('price:adj_close')

# Check missing values
print(f"Missing value ratio: {close.isna().sum().sum() / close.size:.2%}")

# Fill missing values
close_filled = close.ffill()  # Forward fill

# Or drop stocks with too many missing values
close_clean = close.dropna(
    axis=1, thresh=int(len(close) * 0.8)
)  # Keep stocks with 80%+ data

Q: How can I save memory?

# Method 1: Limit time range
data.truncate_start = '2020-01-01'

# Method 2: Process in batches
all_stocks = close.columns
for batch in [all_stocks[i : i + 100] for i in range(0, len(all_stocks), 100)]:
    batch_close = close[batch]
    # Process 100 stocks at a time...

# Method 3: Only download the fields you need
# Avoid calling data.get() for too many tables at once

Resources

Data Download Detailed Tutorial - Complete usage guide
FinLab Database Catalog - All available data tables
Quick Start Guide - Getting started
FAQ - More troubleshooting

Historical Taiwan listing eligibility

Taiwan market only. universe.membership() returns a boolean FinlabDataFrame on the Taiwan close-price dates and columns. It supports TSE, OTC, and TSE_OTC without changing existing universe contexts or ranking methods.

from finlab import data
from finlab.data import universe

data.set_market('tw')
# Fetch scores without an existing snapshot universe removing securities.
close = data.get("price:close")
score = close.pct_change(20)
eligible = universe.membership("TSE_OTC")
position = score[eligible].is_largest(20)

Apply eligibility before every cross-sectional ranking. The mask cannot restore securities previously removed from score data. Membership reads bypass the snapshot universe without changing it and respect the active date window.

Intervals use current company records, historical delisted-company records, and exchange termination lists. Listing dates are inclusive; termination dates are exclusive. Multiple listing periods and exchange transfers are supported. Suspensions and missing trades do not terminate listing eligibility; apply trading conditions separately. Indicator warm-up data is not erased.

Unverified historical periods remain False with a warning. Inspect eligible.attrs["membership_missing_history"] for incomplete company histories. Securities without company listing records, such as ETFs, are not included merely because they have price columns; this is not a complete ETF universe.

This reconstructs effective-date eligibility from currently available records. It does not provide universe versions or as-known-at-the-time snapshots. Historical corrections can still change results, so it cannot guarantee reproduction of old backtests.

Download preflight and per-item usage

Paid plans can use data.plan_download(key) for a quota-free ordinary whole-dataset check. Each full/recent/delta candidate has a row with the planned route (cache/recent/delta/full), needs_update (null when unknown), size_bytes and planned_bytes. For a list:

import pandas as pd
from finlab import data

keys = ['us_price:close', 'us_price:volume']
plan = pd.concat([data.plan_download(key) for key in keys], ignore_index=True)
plan.to_csv('download-plan.csv', index=False)
with data.download_usage() as rows:
    for key in keys:
        data.get(key)
receipts = pd.DataFrame(rows)
receipts.to_csv('download-usage.csv', index=False)
receipts.to_json('download-usage.json', orient='records')

Preflight never signs a download URL. Older servers reject it without falling back to a chargeable download probe. This is a snapshot: hash/merge fallback may add a full download and concurrent publication may change sizes. Unknown values stay null, never an inferred zero or guaranteed usage ceiling. Planning excludes intraday, retained versions and start/end optimisations.

For event='provider', billed_bytes is the provider's per-item charge. Signing already counts even if transfer or merge fails. quota is the shared daily account total in MiB, not a per-item bill. Fallback rows explain the reason; cache rows report local reads. Attempts begin with unknown bytes: lost/retried signing replies and older providers may leave them unknown. A ledger with unknown provider amounts is not a complete bill. Rows survive an exception; export pd.DataFrame(rows) in finally. Collection adds no requests and includes ordinary get/gets workers in this scope, excluding other processes.

finlab.data: data.get Data Download API

Use Cases

Quick Examples

Basic Usage: Download Data

English price keys in both markets

Search Available Fields

Restrict Market Scope

Remove a universe filter

Detailed Guide

Global Configuration

Force Cloud/Local Data

Limit Data Time Range

API Reference

data.get()

finlab.data.get

DownloadedFrame module-attribute

get

get_role

is_tradable

is_vip

next_future_expiry

plan_download

process_data

suggested_tradable_time

data.search()

data.describe()

finlab.data.describe

data.us_universe()

finlab.data.us_universe

Parameters

Notes

Examples

Historical index backtests

data.universe()

Historical stock-futures backtests

data.indicator()

Cache Management

data.set_storage()

data.CacheStorage

data.FileStorage

User Role

finlab.data.get_role

finlab.data.is_vip

Other Utilities

data.get_strategies()

FAQ

Q: How do I find out what data is available for download?

Q: Data download is slow, what can I do?

Q: DatasetNotFoundError or KeyError - what should I do?

Q: How do I download U.S. ETF data?

Q: What about missing values in the data?

Q: How can I save memory?

Resources

Historical Taiwan listing eligibility

Download preflight and per-item usage

DownloadedFrame `module-attribute`