finlab.data: data.get Data Download API
finlab.data is the core module for downloading FinLab datasets. The usual workflow is to find dataset keys with data.search(), download them with data.get(), then control market scope with data.universe() or data.set_market().
| Task | API | Common usage |
|---|---|---|
| Download prices, fundamentals, revenue, and trading data | data.get() |
data.get('price:收盤價') |
| Find table and field names | data.search() |
data.search('ROE') |
| Restrict listed, OTC, sector, or US universes | data.universe() / data.us_universe() |
with data.universe(market='TSE'): |
| Control download size and memory usage | truncate_start, cache settings |
data.truncate_start = '2020-01-01' |
First-time users should read Data Download. To turn downloaded data into stock-selection conditions, continue with finlab.dataframe.
Use Cases
- Download historical data such as stock prices, financial statements, and institutional trading
- Filter data by market or industry sector
- Search for available data tables and fields
- Configure data caching strategies
- Limit data download range to save memory
Quick Examples
Basic Usage: Download Data
from finlab import data
# Download closing prices
close = data.get('price:收盤價')
# Download P/E ratio
pe_ratio = data.get('price_earning_ratio:本益比')
# Download monthly revenue
revenue = data.get('monthly_revenue:當月營收')
Search Available Fields
# Search for fields containing "收盤" (close)
data.search('收盤')
# Output: ['price:收盤價', 'etl:不含除權息收盤價', ...]
# Search US stock data
data.search('close', market='us')
# Search US ETF/fund data
data.search('close', market='us_fund')
Restrict Market Scope
# Only fetch listed (TSE) company data
with data.universe(market='TSE'):
close = data.get('price:收盤價')
# Only fetch specific industry sectors
with data.universe(category=['水泥工業', '食品工業']):
close = data.get('price:收盤價')
# Switch to US ETF/fund data
data.set_market('us_fund')
fund_close = data.get('price:adj_close')
Detailed Guide
See Data Download Details for: - Complete data download tutorial - Data table structure explanation - Advanced filtering techniques - Error handling methods
Global Configuration
Force Cloud/Local Data
from finlab import data
# Force cloud download (re-download every time)
data.force_cloud_download = True
# Force local cache only (offline environment)
data.use_local_data_only = True
Limit Data Time Range
# Only download data from 2020-2023 (saves memory)
data.truncate_start = '2020-01-01'
data.truncate_end = '2023-12-31'
# All subsequent data.get() calls will use this range
close = data.get('price:收盤價')
Recommended Usage
- Development phase: Use
truncate_startto limit data range for faster testing - Production backtesting: Remove truncate limits, use full historical data
- Low memory: Set
truncate_startor useuse_local_data_only
API Reference
data.get()
finlab.data.get
Shared fetch pipeline plus data.get() single-dataset wrapper.
DownloadedFrame
module-attribute
(df, raw_feather_bytes) — bytes is None when payload was inlined.
get
Download historical data.
See https://ai.finlab.tw/database for the full data catalogue.
| PARAMETER | DESCRIPTION |
|---|---|
dataset
|
The name of dataset.
TYPE:
|
save_to_storage
|
Whether to save the dataset to storage for later use.
TYPE:
|
force_download
|
Whether to force download the dataset from cloud.
TYPE:
|
lazy
|
Return a lazy computation wrapper after fetching the dataset.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
DataFrame
|
financial data as a DataFrame. |
is_tradable
Check if cached data is safe for trading.
True when
- All data not expired (now < expiry for all datasets)
- Before market open only: no more crawl updates before today's open (expiry > today's market open time)
| PARAMETER | DESCRIPTION |
|---|---|
market
|
A Market instance (TWMarket/USMarket). If None, defaults to TWMarket.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
bool
|
True if data is fresh enough for trading. |
next_future_expiry
Next future expiry time among cached datasets (UTC).
Returns only expiry times in the future, skipping already expired data.
| RETURNS | DESCRIPTION |
|---|---|
datetime | None
|
datetime in UTC representing the next future expiry, or None if |
datetime | None
|
no future expiries exist or no data is available. |
suggested_tradable_time
Estimate when is_tradable() will become True.
If already tradable, returns None. Otherwise: 1. If any loaded dataset is already expired, returns latest expired expiry + buffer. 2. Else (typically before today's market open), returns latest expiry that is still before today's open, plus buffer.
| PARAMETER | DESCRIPTION |
|---|---|
market
|
A Market instance. If None, defaults to TWMarket.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
datetime | None
|
datetime in UTC, or None if already tradable or no data. |
Common Data Tables
Price Data:
- price:收盤價 - Daily closing price
- price:開盤價 - Daily opening price
- price:最高價 / price:最低價 - Daily high/low
- price:成交股數 - Trading volume
Fundamental Data:
- price_earning_ratio:本益比 - P/E ratio
- price_earning_ratio:股價淨值比 - P/B ratio
- fundamental_features:股東權益報酬率 - ROE
- financial_statement:每股盈餘 - EPS
Institutional Trading Data:
- institutional_investors_trading_summary:投信買賣超股數
- margin_transactions:融資使用率
- etl:外資持股比例
Monthly Revenue:
- monthly_revenue:當月營收
- monthly_revenue:去年同月增減(%)
See the full list at the Database Catalog.
Common Errors
- KeyError: Data table name is wrong or API token is not set
- Empty DataFrame: Query conditions too strict or data does not exist
- Out of memory: Too much data downloaded, use
truncate_startto limit range
data.search()
finlab.data.search
Search available data fields in the finlab database.
Examples:
# Search for fields containing "股東" (shareholder)
data.search('股東')
# ['fundamental_features:股東權益報酬率', 'internal_equity_pledge:百分之十以上大股東持有股數', ...]
# Search for US stock PE ratio
data.search('pe', market='us')
# List all Taiwan stock fields
all_fields = data.search()
print(f"Total {len(all_fields)} fields")
data.universe()
finlab.data.universe
universe
universe(
exchange='ALL',
sector='ALL',
exclude_sector=None,
industry='ALL',
asset_type=None,
security_type='ALL',
index=None,
*,
market=None,
category=None,
exclude_category=None,
)
Context manager to set a global stock universe filter for data retrieval.
Auto-dispatches TW vs international logic based on the active data context.
Parameters
exchange : str | list[str], default 'ALL'
TW: 'TWSE'/'TPEx' (or legacy 'TSE'/'OTC').
International: 'NASDAQ'/'NYSE'/'AMEX'/'HKEX'/'TSE'/etc.
sector : str | list[str], default 'ALL'
Sector name(s) with regex matching.
exclude_sector : str | list[str] | None, default None
Sector(s) to exclude (TW only).
industry : str | list[str], default 'ALL'
Industry filter (international markets).
asset_type : str | None, default None
TW only: 'ETF' or 'STOCK_FUTURE'.
security_type : str | list[str], default 'ALL'
Security type filter (international markets), e.g. 'common_stock'.
index : str | None, default None
Market index to filter by (international markets only).
For US market: 'SP500'/'SPX', 'NASDAQ100'/'NDX'.
market : str | None
Legacy alias for TW exchange+asset_type.
category : str | list[str] | None
Legacy alias for sector.
exclude_category : str | list[str] | None
Legacy alias for exclude_sector.
Examples
TW market (default):
from finlab import data with data.universe(exchange=['TWSE', 'TPEx'], sector=['鋼鐵工業', '航運業']): ... close = data.get('price:收盤價')
Legacy TW usage (still works):
with data.universe(market='TSE_OTC', sector='水泥', exclude_sector='ETF'): ... close = data.get('price:收盤價')
US market (S&P 500):
data.set_market('us') with data.universe(index='SP500'): ... close = data.get('price:close')
US market (NASDAQ 100 Technology stocks):
data.set_market('us') with data.universe(index='NDX', sector='Technology'): ... close = data.get('price:close')
JP market:
data.set_market('jp') with data.universe(sector='Technology'): ... close = data.get('price:close')
us_universe
Context manager to set a global stock universe filter for US market data retrieval.
This context manager limits the set of US stocks returned by data functions to a specific sector, industry, exchange, and/or index selection. The filter is applied globally within the context and is restored after the context exits.
Parameters
sector : str | list[str], default 'ALL' Sector name(s) to include. Supports regex-like substring matching. industry : str | list[str], default 'ALL' Industry name(s) to include. Supports regex-like substring matching. exchange : str | list[str], default 'ALL' Exchange name(s) to include. Common values: 'NASDAQ', 'NYSE', 'AMEX'. security_type : str | list[str], default 'ALL' Security type(s) to include. Use 'common_stock' for clean US equities. index : str | None, default None Market index to filter by. Supported values: 'SP500'/'SPX' (S&P 500), 'NASDAQ100'/'NDX' (NASDAQ 100). Can be combined with sector/industry/exchange for further filtering.
Examples
from finlab import data with data.us_universe(sector='Technology', exchange='NASDAQ'): ... close = data.get('us_price:close')
Filter to common stocks only:
with data.us_universe(security_type='common_stock'): ... close = data.get('us_price:close')
Filter to S&P 500 constituents:
with data.us_universe(index='SP500'): ... close = data.get('us_price:close')
Filter to NASDAQ 100 Technology stocks:
with data.us_universe(index='NDX', sector='Technology'): ... close = data.get('us_price:close')
set_universe
set_universe(
exchange='ALL',
sector='ALL',
exclude_sector=None,
industry='ALL',
asset_type=None,
security_type='ALL',
index=None,
*,
market=None,
category=None,
exclude_category=None,
)
Set global stock universe filter. Auto-dispatches based on current market.
When data.set_market('tw') (or no market set), uses TW logic
(security_categories). For any other market (us, hk, jp, kr, uk),
loads {market}_company_profile and filters by available columns.
Parameters
exchange : str | list[str], default 'ALL'
TW: 'TWSE', 'TPEx' (or legacy 'TSE', 'OTC').
International: 'NASDAQ', 'NYSE', 'AMEX', 'HKEX', 'TSE', etc.
sector : str | list[str], default 'ALL'
Sector name(s) with regex matching.
exclude_sector : str | list[str] | None, default None
Sector(s) to exclude (TW only).
industry : str | list[str], default 'ALL'
Industry filter (international markets).
asset_type : str | None, default None
TW only: 'ETF' or 'STOCK_FUTURE'.
security_type : str | list[str], default 'ALL'
Security type filter (international markets), e.g. 'common_stock'.
index : str | None, default None
Market index to filter by (international markets only).
For US market: 'SP500'/'SPX', 'NASDAQ100'/'NDX'.
market : str | None
Legacy alias for TW exchange+asset_type (e.g. 'TSE_OTC', 'ETF').
category : str | list[str] | None
Legacy alias for sector.
exclude_category : str | list[str] | None
Legacy alias for exclude_sector.
set_us_universe
Set global US stock universe filter.
Thin wrapper around _set_intl_universe_impl for backward compatibility.
Parameters
sector : str | list[str], default 'ALL' Sector filter with regex-like substring matching. industry : str | list[str], default 'ALL' Industry filter with regex-like substring matching. exchange : str | list[str], default 'ALL' Exchange filter (e.g., 'NASDAQ', 'NYSE', 'AMEX'). security_type : str | list[str], default 'ALL' Security type filter. Use 'common_stock' for clean US equities. index : str | None, default None Market index to filter by. Supported values: 'SP500'/'SPX' (S&P 500), 'NASDAQ100'/'NDX' (NASDAQ 100).
Examples:
# Example 1: Only listed companies
with data.universe(market='TSE'):
close = data.get('price:收盤價')
print(f"Number of listed companies: {len(close.columns)}")
# Example 2: Specific industry sectors
with data.universe(category=['半導體業']):
close = data.get('price:收盤價')
# Example 3: Top 100 by market cap
with data.universe(size=100):
close = data.get('price:收盤價')
# Example 4: Combined conditions
with data.universe(market='TSE_OTC', category=['電子工業'], size=50):
close = data.get('price:收盤價')
Available market parameter values:
- 'TSE' - Listed (Taiwan Stock Exchange)
- 'OTC' - OTC (Taipei Exchange)
- 'TSE_OTC' - Listed + OTC
- 'ALL' - All (including Emerging Stock Board)
data.us_universe()
finlab.data.us_universe
Context manager to set a global stock universe filter for US market data retrieval.
This context manager limits the set of US stocks returned by data functions to a specific sector, industry, exchange, and/or index selection. The filter is applied globally within the context and is restored after the context exits.
Parameters
sector : str | list[str], default 'ALL' Sector name(s) to include. Supports regex-like substring matching. industry : str | list[str], default 'ALL' Industry name(s) to include. Supports regex-like substring matching. exchange : str | list[str], default 'ALL' Exchange name(s) to include. Common values: 'NASDAQ', 'NYSE', 'AMEX'. security_type : str | list[str], default 'ALL' Security type(s) to include. Use 'common_stock' for clean US equities. index : str | None, default None Market index to filter by. Supported values: 'SP500'/'SPX' (S&P 500), 'NASDAQ100'/'NDX' (NASDAQ 100). Can be combined with sector/industry/exchange for further filtering.
Examples
from finlab import data with data.us_universe(sector='Technology', exchange='NASDAQ'): ... close = data.get('us_price:close')
Filter to common stocks only:
with data.us_universe(security_type='common_stock'): ... close = data.get('us_price:close')
Filter to S&P 500 constituents:
with data.us_universe(index='SP500'): ... close = data.get('us_price:close')
Filter to NASDAQ 100 Technology stocks:
with data.us_universe(index='NDX', sector='Technology'): ... close = data.get('us_price:close')
US Market Filtering:
data.set_market('us')
# Get S&P 500 constituents
with data.us_universe(index='SP500'):
close = data.get('price:close')
# Get NASDAQ 100 Technology stocks
with data.us_universe(index='NDX', sector='Technology'):
close = data.get('price:close')
# Get a clean US common-stock universe
with data.us_universe(security_type='common_stock'):
close = data.get('price:close')
# Can also use data.universe() with set_market
with data.universe(index='SP500', exchange='NASDAQ'):
close = data.get('price:close')
Supported index parameter values:
| Index | Accepted aliases |
|---|---|
| S&P 500 | 'SP500', 'SPX', 'S&P500', 'S&P 500' |
| NASDAQ 100 | 'NASDAQ100', 'NDX', 'NASDAQ 100' |
index can be combined with sector, industry, exchange, and security_type — the result is the intersection of all filters.
data.indicator()
finlab.data.indicator
Technical indicator computation: data.indicator().
get_strategies
取得已上傳量化平台的策略回傳資料。
可取得自己策略儀表板上的數據,例如每個策略的報酬率曲線、報酬率統計、夏普率、近期部位、近期換股日..., 這些數據可以用來進行多策略彙整的應用喔!
| PARAMETER | DESCRIPTION |
|---|---|
api_token
|
若未帶入finlab模組的api_token,會自動跳出GUI頁面, 複製網頁內的api_token貼至輸入欄位即可。
TYPE:
|
Returns: (dict): strategies data Response detail:
``` py
{
strategy1:{
'asset_type': '',
'drawdown_details': {
'2015-06-04': {
'End': '2015-11-03',
'Length': 152,
'drawdown': -0.19879090089478024
},
...
},
'fee_ratio': 0.000475,
'last_trading_date': '2022-06-10',
'last_updated': 'Sun, 03 Jul 2022 12:02:27 GMT',
'ndays_return': {
'1': -0.01132480035770611,
'10': -0.0014737286933147464,
'20': -0.06658015749110646,
'5': -0.002292995729485159,
'60': -0.010108700314771735
},
'next_trading_date': '2022-06-10',
'positions': {
'1413 宏洲': {
'entry_date': '2022-05-10',
'entry_price': 10.05,
'exit_date': '',
'next_weight': 0.1,
'return': -0.010945273631840613,
'status': '買進',
'weight': 0.1479332345384493
},
'last_updated': 'Sun, 03 Jul 2022 12:02:27 GMT',
'next_trading_date': '2022-06-10',
'trade_at': 'open',
'update_date': '2022-06-10'
},
'return_table': {
'2014': {
'Apr': 0.0,
'Aug': 0.06315180932606546,
'Dec': 0.0537589857541485,
'Feb': 0.0,
'Jan': 0.0,
'Jul': 0.02937490104459939,
'Jun': 0.01367930162104769,
'Mar': 0.0,
'May': 0.0,
'Nov': -0.0014734320286596825,
'Oct': -0.045082529665408266,
'Sep': 0.04630906972509852,
'YTD': 0.16626214846456966
},
...
},
'returns': {
'time': [
'2014-06-10',
'2014-06-11',
'2014-06-12',
...
],
'value': [
100,
99.9,
100.2,
...
]
},
'stats': {
'avg_down_month': -0.03304015302646822,
'avg_drawdown': -0.0238021414698247,
'avg_drawdown_days': 19.77952755905512,
'avg_up_month': 0.05293384465715908,
'cagr': 0.33236021285588846,
'calmar': 1.65261094975066,
'daily_kurt': 4.008888367138843,
'daily_mean': 0.3090784769257415,
'daily_sharpe': 1.747909002374217,
'daily_skew': -0.6966018726321078,
'daily_sortino': 2.8300677082214034,
...
},
'tax_ratio': 0.003,
'trade_at': 'open',
'update_date': '2022-06-10'
},
strategy2:{...},
...}
```
indicator
支援 Talib 和 pandas_ta 上百種技術指標,計算 2000 檔股票、10年的所有資訊。
在使用這個函式前,需要安裝計算技術指標的 Packages
| PARAMETER | DESCRIPTION |
|---|---|
indname
|
指標名稱, 以 TA-Lib 舉例,例如 SMA, STOCH, RSI 等,可以參考 talib 文件。 以 Pandas-ta 舉例,例如 supertrend, ssf 等,可以參考 Pandas-ta 文件。
TYPE:
|
adjust_price
|
是否使用還原股價計算。
TYPE:
|
resample
|
技術指標價格週期,ex:
TYPE:
|
market
|
市場選擇,ex:
TYPE:
|
**kwargs
|
技術指標的參數設定,TA-Lib 中的 RSI 為例,調整項為計算週期
TYPE:
|
建議使用者可以先參考以下範例,並且搭配 talib官方文件,就可以掌握製作技術指標的方法了。
Technical Indicator Examples:
from finlab import data
# Get MACD indicator
macd = data.indicator('macd', data.get('price:收盤價'))
# Get RSI indicator
rsi = data.indicator('rsi', data.get('price:收盤價'), period=14)
Cache Management
finlab.data.set_storage
設定本地端儲存歷史資料的方式
假設使用 data.get 獲取歷史資料則,在預設情況下,程式會自動在本地複製一份,以避免重複下載大量數據。
storage 就是用來儲存歷史資料的接口。我們提供兩種 storage 接口,分別是 finlab.data.CacheStorage (預設) 以及
finlab.data.FileStorage。前者是直接存在記憶體中,後者是存在檔案中。詳情請參考 CacheStorage 和 FileStorage 來獲得更詳細的資訊。
在預設情況下,程式會自動使用 finlab.data.FileStorage 並將重複索取之歷史資料存在作業系統預設「暫時資料夾」。
| PARAMETER | DESCRIPTION |
|---|---|
storage
|
The interface of storage
TYPE:
|
Examples:
欲切換成以檔案方式儲存,可以用以下之方式:
可以在本地端的 ./finlab_db/price#收盤價.pickle 中,看到下載的資料,
可以使用 pickle 調閱歷史資料:
finlab.data.CacheStorage
finlab.data.FileStorage
將歷史資料儲存於檔案中
| PARAMETER | DESCRIPTION |
|---|---|
path
|
資料儲存的路徑
TYPE:
|
use_cache
|
是否額外使用快取,將資料複製一份到記憶體中。
TYPE:
|
Examples:
欲切換成以檔案方式儲存,可以用以下之方式:
可以在本地端的 ./finlab_db/price#收盤價.feather 中,看到下載的資料,
可以使用 pandas.read_feather 調閱歷史資料:
diagnose
Custom Cache Strategy:
from finlab.data import set_storage, FileStorage
# Use a custom directory
storage = FileStorage('/path/to/custom/cache')
set_storage(storage)
# All subsequent data will be cached to the specified location
close = data.get('price:收盤價')
User Role
Example:
from finlab import data
# Query user role (requires at least one data.get() call first)
close = data.get('price:收盤價')
role = data.get_role() # 'free', 'vip', ...
print(f"Current role: {role}")
if data.is_vip():
print("VIP member — full data access enabled")
Other Utilities
finlab.data.get_strategies
取得已上傳量化平台的策略回傳資料。
可取得自己策略儀表板上的數據,例如每個策略的報酬率曲線、報酬率統計、夏普率、近期部位、近期換股日..., 這些數據可以用來進行多策略彙整的應用喔!
| PARAMETER | DESCRIPTION |
|---|---|
api_token
|
若未帶入finlab模組的api_token,會自動跳出GUI頁面, 複製網頁內的api_token貼至輸入欄位即可。
TYPE:
|
Returns: (dict): strategies data Response detail:
``` py
{
strategy1:{
'asset_type': '',
'drawdown_details': {
'2015-06-04': {
'End': '2015-11-03',
'Length': 152,
'drawdown': -0.19879090089478024
},
...
},
'fee_ratio': 0.000475,
'last_trading_date': '2022-06-10',
'last_updated': 'Sun, 03 Jul 2022 12:02:27 GMT',
'ndays_return': {
'1': -0.01132480035770611,
'10': -0.0014737286933147464,
'20': -0.06658015749110646,
'5': -0.002292995729485159,
'60': -0.010108700314771735
},
'next_trading_date': '2022-06-10',
'positions': {
'1413 宏洲': {
'entry_date': '2022-05-10',
'entry_price': 10.05,
'exit_date': '',
'next_weight': 0.1,
'return': -0.010945273631840613,
'status': '買進',
'weight': 0.1479332345384493
},
'last_updated': 'Sun, 03 Jul 2022 12:02:27 GMT',
'next_trading_date': '2022-06-10',
'trade_at': 'open',
'update_date': '2022-06-10'
},
'return_table': {
'2014': {
'Apr': 0.0,
'Aug': 0.06315180932606546,
'Dec': 0.0537589857541485,
'Feb': 0.0,
'Jan': 0.0,
'Jul': 0.02937490104459939,
'Jun': 0.01367930162104769,
'Mar': 0.0,
'May': 0.0,
'Nov': -0.0014734320286596825,
'Oct': -0.045082529665408266,
'Sep': 0.04630906972509852,
'YTD': 0.16626214846456966
},
...
},
'returns': {
'time': [
'2014-06-10',
'2014-06-11',
'2014-06-12',
...
],
'value': [
100,
99.9,
100.2,
...
]
},
'stats': {
'avg_down_month': -0.03304015302646822,
'avg_drawdown': -0.0238021414698247,
'avg_drawdown_days': 19.77952755905512,
'avg_up_month': 0.05293384465715908,
'cagr': 0.33236021285588846,
'calmar': 1.65261094975066,
'daily_kurt': 4.008888367138843,
'daily_mean': 0.3090784769257415,
'daily_sharpe': 1.747909002374217,
'daily_skew': -0.6966018726321078,
'daily_sortino': 2.8300677082214034,
...
},
'tax_ratio': 0.003,
'trade_at': 'open',
'update_date': '2022-06-10'
},
strategy2:{...},
...}
```
FAQ
Q: How do I find out what data is available for download?
Method 1: Use search()
Method 2: Check the online database Visit the FinLab Database Catalog to browse the full list.
Q: Data download is slow, what can I do?
# Method 1: Limit time range
data.truncate_start = '2020-01-01'
# Method 2: Use cache (second call will be fast)
close = data.get('price:收盤價') # First call is slow
close = data.get('price:收盤價') # Second call is fast (uses cache)
# Method 3: Use universe to limit stock count
with data.universe(size=100):
close = data.get('price:收盤價') # Only downloads 100 stocks
Q: KeyError: 'price:收盤價' - what should I do?
Possible causes:
1. Not logged in - Run finlab.login() and complete login
2. Incorrect field name - Use data.search('收盤') to verify the correct name
3. Insufficient data permission - Check whether the account can access this dataset
import finlab
# Check if logged in
try:
token, token_type = finlab.get_token()
print(f"Logged in ({token_type})")
except:
print("Not logged in, please run finlab.login()")
Q: How do I download US stock data?
from finlab import data
# US stock closing prices
us_close = data.get('price:close', market='us')
# Search US stock fields
us_fields = data.search(market='us')
Q: What about missing values in the data?
close = data.get('price:收盤價')
# Check missing values
print(f"Missing value ratio: {close.isna().sum().sum() / close.size:.2%}")
# Fill missing values
close_filled = close.fillna(method='ffill') # Forward fill
# Or drop stocks with too many missing values
close_clean = close.dropna(axis=1, thresh=len(close)*0.8) # Keep stocks with 80%+ data
Q: How can I save memory?
# Method 1: Limit time range
data.truncate_start = '2020-01-01'
# Method 2: Process in batches
all_stocks = close.columns
for batch in [all_stocks[i:i+100] for i in range(0, len(all_stocks), 100)]:
batch_close = close[batch]
# Process 100 stocks at a time...
# Method 3: Only download the fields you need
# Avoid calling data.get() for too many tables at once
Resources
- Data Download Detailed Tutorial - Complete usage guide
- FinLab Database Catalog - All available data tables
- Quick Start Guide - Getting started
- FAQ - More troubleshooting