Skip to content

FinLab Module Architecture & Relationship Diagram

This document provides visual diagrams illustrating the relationships, data flow, and workflows among FinLab's modules, helping you quickly understand the system architecture.

Overall Architecture Diagram

FinLab uses a layered architecture, from the Data Layer at the bottom to the Trading Layer at the top:

graph TB
    subgraph "Trading Layer"
        portfolio[finlab.portfolio<br/>Portfolio Management]
        online[finlab.online<br/>Live Trading]
    end

    subgraph "Analysis Layer"
        analysis[finlab.analysis<br/>Strategy Analysis]
        optimize[finlab.optimize<br/>Parameter Optimization]
        plot[finlab.plot<br/>Data Visualization]
    end

    subgraph "Backtest Layer"
        backtest[finlab.backtest<br/>Backtest Engine]
        report[finlab.report<br/>Report Generation]
        market[finlab.market<br/>Market Objects]
    end

    subgraph "Strategy Layer"
        ml_feature[finlab.ml.feature<br/>Feature Engineering]
        ml_label[finlab.ml.label<br/>Label Generation]
        ml_model[finlab.ml.qlib<br/>Model Training]
    end

    subgraph "Data Layer"
        data[finlab.data<br/>Data Access]
        dataframe[finlab.dataframe<br/>FinlabDataFrame]
    end

    %% Data flow
    data --> dataframe
    dataframe --> ml_feature
    dataframe --> backtest
    ml_feature --> ml_label
    ml_label --> ml_model
    ml_model --> backtest
    backtest --> report
    report --> analysis
    report --> optimize
    analysis --> portfolio
    optimize --> backtest
    portfolio --> online
    market --> backtest
    dataframe --> plot

    style data fill:#e1f5ff
    style backtest fill:#fff4e1
    style analysis fill:#e8f5e9
    style online fill:#fce4ec

Core Workflows

Workflow 1: Basic Strategy Development

The basic flow from data loading to report generation:

sequenceDiagram
    participant User as User
    participant Data as finlab.data
    participant DF as FinlabDataFrame
    participant Backtest as finlab.backtest
    participant Report as Report object

    User->>Data: data.get('price:收盤價')
    Data-->>User: DataFrame
    User->>DF: Compute indicator .average(20)
    DF-->>User: FinlabDataFrame
    User->>DF: Generate signal close > ma20
    DF-->>User: bool DataFrame
    User->>Backtest: sim(position, resample='M')
    Backtest->>Report: Build Report object
    Report-->>User: Backtest results
    User->>Report: report.display()
    Report-->>User: Show performance charts

Workflow 2: Strategy Optimization

Use sim_conditions() to test multiple condition combinations:

flowchart TD
    A[Define candidate conditions] --> B[Combine into dict]
    B --> C{Use sim_conditions}
    C --> D[Generate all combinations<br/>2^n - 1 total]
    D --> E[Backtest each combination]
    E --> F[Collect all Reports]
    F --> G[Build ReportCollection]
    G --> H{Visualization choice}
    H -->|Heatmap| I[plot_stats heatmap]
    H -->|Bar chart| J[plot_stats bar]
    H -->|Line chart| K[plot_creturns]
    I --> L[Identify best combination]
    J --> L
    K --> L
    L --> M[Rerun backtest with best combination]

    style C fill:#fff4e1
    style G fill:#e8f5e9
    style L fill:#fce4ec

Workflow 3: Machine Learning Strategy

The complete ML strategy development workflow:

flowchart TD
    A[Load raw data<br/>finlab.data] --> B[Feature engineering<br/>finlab.ml.feature]
    B --> C[Label generation<br/>finlab.ml.label]
    C --> D[Model training<br/>finlab.ml.qlib]
    D --> E[Predict position weights]
    E --> F[Backtest<br/>finlab.backtest.sim]
    F --> G{Performance acceptable?}
    G -->|No| H[Adjust features or model]
    H --> B
    G -->|Yes| I[Run in-depth analysis]
    I --> J[Out-of-sample testing]
    J --> K{Validation passed?}
    K -->|No| H
    K -->|Yes| L[Deploy to live trading]

    style B fill:#e1f5ff
    style D fill:#fff4e1
    style I fill:#e8f5e9
    style L fill:#fce4ec

Workflow 4: Live Trading

The complete flow from backtesting to live trading:

sequenceDiagram
    participant User as User
    participant Report as Report object
    participant Account as Account object
    participant Executor as OrderExecutor
    participant Broker as Broker API

    User->>Report: report.upload(name)
    Report-->>User: Upload succeeded
    User->>Report: report.position_info()
    Report-->>User: Holdings list
    User->>Account: Configure broker account
    Account-->>User: Connected
    User->>Executor: OrderExecutor(report, account)
    Executor->>Executor: Compute buy/sell list
    User->>Executor: executor.execute()
    Executor->>Broker: Submit buy orders
    Broker-->>Executor: Buy order acks
    Executor->>Broker: Submit sell orders
    Broker-->>Executor: Sell order acks
    Executor-->>User: Execution complete
    User->>Account: Query positions
    Account->>Broker: Fetch position info
    Broker-->>Account: Position data
    Account-->>User: Show live holdings

Module Function Reference Table

Module Main Classes/Functions Purpose Dependencies
finlab.data get() Fetch financial data from the database (none)
finlab.dataframe FinlabDataFrame Enhanced DataFrame with financial computation support pandas
finlab.backtest sim() Backtest engine for simulating trades data, dataframe, market
finlab.report Report Backtest report and performance analysis backtest
finlab.optimize sim_conditions(), ReportCollection Test multiple condition combinations backtest
finlab.analysis LiquidityAnalysis, MaeMfeAnalysis, etc. In-depth strategy analysis report
finlab.market Market, TWMarket, USMarket Market data and characteristics data
finlab.ml.feature add_ta(), add_fundamental() ML feature engineering data
finlab.ml.label cls_label(), reg_label() ML label generation data
finlab.ml.qlib train() Model training ml.feature, ml.label
finlab.portfolio Portfolio, PortfolioSyncManager Multi-strategy portfolio management report
finlab.online OrderExecutor, SinopacAccount Live trading execution report, portfolio
finlab.plot plot_creturns(), plot_heatmap() Data visualization dataframe

Data Flow Diagram

The complete data flow from data sources to live trading:

flowchart LR
    A[Data sources<br/>GCS/BigQuery] --> B[finlab.data.get]
    B --> C[DataFrame]
    C --> D{Purpose?}

    D -->|Technical indicators| E[FinlabDataFrame<br/>Compute methods]
    D -->|Machine learning| F[ml.feature<br/>Feature engineering]

    E --> G[Generate signals<br/>bool DataFrame]
    F --> H[ml.label<br/>Label generation]
    H --> I[ml.qlib<br/>Model training]
    I --> J[Predict weights<br/>float DataFrame]

    G --> K[finlab.backtest.sim]
    J --> K

    K --> L[Report object]
    L --> M{Analysis type?}

    M -->|Parameter optimization| N[optimize.sim_conditions]
    M -->|In-depth analysis| O[analysis module]
    M -->|Visualization| P[plot module]
    M -->|Live trading| Q[online.OrderExecutor]

    N --> L
    O --> R[Analysis report]
    P --> S[Interactive chart]
    Q --> T[Broker API]
    T --> U[Live positions]

    style B fill:#e1f5ff
    style K fill:#fff4e1
    style O fill:#e8f5e9
    style Q fill:#fce4ec

Class Inheritance Diagrams

Market Inheritance Tree

classDiagram
    class Market {
        <<abstract>>
        +get_name() str
        +get_freq() str
        +get_price() DataFrame
        +get_benchmark() Series
        +get_asset_id_to_name() dict
    }

    class TWMarket {
        +get_name() "tw_stock"
        +get_benchmark() TAIEX
        +get_industry() dict
        +market_close_at_timestamp()
    }

    class USMarket {
        +get_name() "us_stock"
        +get_benchmark() S&P 500
        +market_close_at_timestamp()
    }

    class ROTCMarket {
        +get_name() "rotc_stock"
    }

    class CustomMarket {
        +get_name() "custom"
        +custom_data_source
    }

    Market <|-- TWMarket
    Market <|-- USMarket
    Market <|-- ROTCMarket
    Market <|-- CustomMarket

Analysis Inheritance Tree

classDiagram
    class Analysis {
        <<abstract>>
        +calculate_trade_info() list
        +analyze() dict
        +display() Figure
        +is_market_supported() bool
    }

    class LiquidityAnalysis {
        +Liquidity risk detection
        +Limit-up/down and volume
    }

    class MaeMfeAnalysis {
        +Volatility analysis
        +12 MAE/MFE subplots
    }

    class PeriodStatsAnalysis {
        +Period stability
        +Annual and monthly performance
    }

    class InequalityAnalysis {
        +Inequality factors
        +Gini coefficient
    }

    class AlphaBetaAnalysis {
        +Alpha/Beta analysis
    }

    class DrawdownAnalysis {
        +Drawdown analysis
    }

    class CustomAnalysis {
        +Custom analysis logic
    }

    Analysis <|-- LiquidityAnalysis
    Analysis <|-- MaeMfeAnalysis
    Analysis <|-- PeriodStatsAnalysis
    Analysis <|-- InequalityAnalysis
    Analysis <|-- AlphaBetaAnalysis
    Analysis <|-- DrawdownAnalysis
    Analysis <|-- CustomAnalysis

FinlabDataFrame Inheritance

classDiagram
    class DataFrame {
        <<pandas>>
        +Basic DataFrame functionality
    }

    class FinlabDataFrame {
        +average() moving average
        +is_largest() ranking
        +is_smallest() ranking
        +quantile() quantiles
        +hold_until() entry/exit sugar
        +and_then() combine logic
    }

    DataFrame <|-- FinlabDataFrame

Beginner Learning Path

flowchart TD
    A[Start] --> B[Learn finlab.data<br/>Data access]
    B --> C[Learn FinlabDataFrame<br/>Indicator calculation]
    C --> D[Learn finlab.backtest<br/>Basic backtesting]
    D --> E[Learn Report object<br/>Performance analysis]
    E --> F{Want to go deeper?}
    F -->|Yes| G[Learn finlab.analysis<br/>In-depth analysis]
    F -->|No| K[Start live trading<br/>finlab.online]
    G --> H[Learn finlab.optimize<br/>Parameter optimization]
    H --> I[Learn the full workflow]
    I --> J{Use machine learning?}
    J -->|Yes| L[Learn finlab.ml<br/>ML modules]
    J -->|No| K
    L --> K
    K --> M[Mastery!]

    style A fill:#e1f5ff
    style D fill:#fff4e1
    style G fill:#e8f5e9
    style K fill:#fce4ec
    style M fill:#c8e6c9

Advanced Development Path

flowchart TD
    A[Comfortable with basic backtesting] --> B{Development need?}
    B -->|Multi-strategy portfolio| C[Learn finlab.portfolio<br/>Portfolio management]
    B -->|Custom market| D[Subclass Market<br/>Build a custom market]
    B -->|Custom analysis| E[Subclass Analysis<br/>Build analysis modules]
    B -->|Machine learning| F[Learn finlab.ml<br/>Features, labels, models]
    C --> G[Multi-strategy live trading]
    D --> H[Backtest overseas markets / crypto]
    E --> I[Customized analysis reports]
    F --> J[ML-driven strategies]
    G --> K[Advanced applications]
    H --> K
    I --> K
    J --> K

    style A fill:#e1f5ff
    style B fill:#fff4e1
    style K fill:#c8e6c9

Data Exchange Formats Between Modules

Primary Data Formats

Data Type Format Description Example
Price Data pd.DataFrame index: dates, columns: stock symbols data.get('price:收盤價')
Selection Signal pd.DataFrame(bool) True = buy, False = not holding close > ma20
Position Weight pd.DataFrame(float) Values between 0-1, representing position ratio ML model predictions
Trade Records pd.DataFrame Detailed per-trade information report.get_trades()
Performance Metrics dict Various statistical indicators of the strategy report.get_stats()
Benchmark Index pd.Series Time series of price data TWMarket.get_benchmark()

Signal Conversion Flow

flowchart LR
    A[Condition check<br/>close > ma20] --> B[bool DataFrame<br/>True/False]
    B --> C{Need exit logic?}
    C -->|Yes| D[hold_until<br/>Add exit condition]
    C -->|No| E[Use directly as position]
    D --> E
    E --> F[finlab.backtest.sim<br/>Run backtest]
    F --> G[Report object<br/>Contains performance and trades]

    style B fill:#e1f5ff
    style D fill:#fff4e1
    style F fill:#e8f5e9

Common Usage Patterns

Pattern 1: Quick Strategy Validation

# Minimal flow
close = data.get('price:收盤價')
position = close > close.average(20)
report = sim(position, resample='M')
report.display()

Pattern 2: Complete Strategy Development

# Complete flow
data → indicator calc → signal generation → parameter optimization → in-depth analysis → out-of-sample test → live trading

Pattern 3: Machine Learning Strategy

# ML flow
data → feature engineering → label generation → model training → prediction → backtest → analysis

Pattern 4: Multi-Strategy Portfolio

# Portfolio flow
Strategy A report →
Strategy B report →  Portfolio → PortfolioSyncManager → multi-strategy live trading
Strategy C report →

Performance Optimization Tips

Data Access

flowchart TD
    A[Need data] --> B{Data size?}
    B -->|Small<br/>Single stock| C[Use data.get directly]
    B -->|Medium<br/>Dozens of indicators| D[Load once<br/>Avoid repeated calls]
    B -->|Large<br/>Many backtests| E[Consider<br/>local caching]

    C --> F[Run strategy logic]
    D --> F
    E --> F

    style C fill:#e8f5e9
    style E fill:#fff4e1

Backtest Optimization

Bottleneck Optimization Method
Too many condition combinations Pre-filter with individual conditions, remove poor ones
Data range too large Shorten backtest period or increase resample frequency
High computational complexity Use vectorized operations, avoid loops
Insufficient memory Process in batches or use longer resample periods

Reference Resources

Detailed Documentation

API Reference

Practical Examples