Walk Forward Analysis: Simulating Real-World Trading

Walk forward analysis (WFA) is a powerful technique for validating your trading algorithm. It simulates real-world trading by training parameters on in-sample data and then testing them on unseen, out-of-sample data. This process is rolled forward over time, creating a performance report that recreates how your algorithm would have performed if you had gone live with it.

Understanding the Benefits of WFA

• Out-of-Sample Performance: WFA provides a realistic assessment of your algorithm's potential by evaluating it on data it hasn't seen before.
• Parameter Validation: It helps you understand if the trained parameter values truly capture short-term market predictability.
• Overfitting Prevention: WFA helps mitigate the risk of overfitting by testing the algorithm on unseen data.

Cycle Settings

A walk forward cycle is defined by:

• Out-of-Sample (OOS) Bars: The number of bars used in each out-of-sample test period.
• Training Ratio: The ratio of training bars to test bars.

Recommended Starting Point:

A good starting point is an OOS ratio of 15% with 500 bars as the duration. This means that for every 500 bars tested, the first ~2900 will be used for training.

Controlling the Analysis

• Toggles: Control which types of logic (Signal, Filter, Stops) and parameters are included in the analysis. This allows you to validate your algorithm step-by-step, avoiding unnecessary complexity.
• Market Modifiers: Apply market modifiers (e.g., gap fixing, oversampling) to create additional datasets and further validate performance.

Running the Analysis

1. Starting Point: Select the starting point for your data. The lookback and training periods are calculated from this point, so the first test cycle will begin later in the dataset.
2. Data Validation: The engine verifies that sufficient market data exists for the specified period.
3. Cycle Execution:
   • Parameter Selection: The engine selects optimal parameters based on the training options defined in the Model Builder.
   • Out-of-Sample Backtest: It runs an out-of-sample backtest using the selected parameters, simulating live trading from the chosen start date.

Adding Cycles:

You can add more cycles to extend the analysis forward or backward in time. The chart automatically updates with each new cycle tested.

Training Results

You can examine the parameters chosen for each cycle in the parameters report. This provides insights into how the algorithm adapts to different market conditions.

Performance Analysis

The bottom panel displays the out-of-sample performance of your algorithm.

Metrics

• Monte Carlo Simulation: The engine automatically runs a Monte Carlo simulation to estimate potential drawdowns. This provides a more realistic assessment of risk than simply relying on the historical drawdown.
• Adjusted Metrics: All subsequent metrics (e.g., Sortino ratio) are calculated using the drawdown estimate from the Monte Carlo simulation.

Expectancy: Equity Monte Carlo Analysis

This analysis runs 200 Monte Carlo simulations to generate a distribution of potential equity curves and drawdowns. This helps visualize the range of possible outcomes and understand the most likely performance scenario.

Validation: Testing on Randomized Data

To further validate your algorithm and ensure it's not simply exploiting random noise or overfitting to historical data, you can test it on randomized data.

• Randomization Process: The engine shuffles the data without replacement and detrends it, removing potential market inefficiencies.
• Performance Comparison: Your algorithm's performance is compared to the results on the randomized data. Ideally, your algorithm should perform better than 90% of the randomized trials.

Trade Analysis

You can explore all the out-of-sample trades taken, highlight them on the chart, and view detailed information about each trade.

Visualizations

• Signal Forecasting Ability: Similar to the signal research functionality, this chart plots the average price action following each trade, helping you assess the signal's predictive power.
• PnL Distribution: Shows the distribution of profits and losses for long and short trades. This helps identify the skewness of your strategy (positive skew: many small losses with a few large winners; negative skew: many small wins with a few large losses).
• MAE/MFE: Plots the final trade PnL against the Maximum Adverse Excursion (MAE) and Maximum Favorable Excursion (MFE). This helps identify potential issues with profit-taking or stop-loss settings.

Tips for Effective WFA

• Varying Start Dates: Test your algorithm with different starting points to assess its performance across various market conditions.
• Adjusting OOS Bars: Experiment with different numbers of out-of-sample bars to see how it affects the results.
• Modifying Training Splits: Try different training-to-test ratios to find the optimal balance.
• Using Market Modifiers: Apply market modifiers like inverting, detrending, oversampling, and jitter to create diverse datasets and rigorously validate your algorithm.