Step-by-Step Guide

While backtesting is useful for verifying your algorithm's logic and execution, it's not enough to guarantee profitability in live trading. Walk forward analysis (WFA) provides a more robust validation method by simulating real-world trading on unseen data.

Backtesting vs. Walk Forward Analysis

• Backtesting: Confirms that your algorithm executes trades accurately according to your defined logic.
• Walk Forward Analysis: Tests your algorithm's ability to generate profits on unseen data, providing a more realistic assessment of its potential performance in live trading.

Walk Forward Analysis

Opening the Engine

You can access the walk forward engine in two ways:

1. From the Model Builder: Navigate to the "Validation" section and click "Validate Now."
2. From the Models Main Page: Click the "Validate" button on the card for the model you want to analyze.

Creating a New Analysis

1. Click "Add Analysis": This opens a new analysis view where you'll configure your WFA.
2. Configure WFA Settings:
   • Analysis Name: Provide a descriptive name for your analysis.
   • Test/Training Split: Define the ratio of data used for training and testing (e.g., 85% training, 15% testing).
   • OOS Bars: Specify the number of bars to use in each out-of-sample test period.
   • Parameter and Logic Type: Select the parameters and logic types to include in the analysis.
   • Market Data Modification: Choose whether to apply market modifiers for robustness testing.
3. Create the Analysis: Click the button to create the walk forward analysis.

Walk Forward Cycles

A walk forward analysis consists of multiple cycles that roll through the data, training parameters on one portion and testing them on another.

Creating the First Cycle

1. Set the Starting Point:
   • Manual Selection: Choose the year, month, and window (which third of the month) to start your analysis.
   • Copying from Existing Analysis: Copy cycles from a previous WFA. This is useful for testing different algorithm settings on the same data.
2. Data Validation: The engine checks if there's enough data for the lookback, training, and testing periods.

Running a Cycle

1. Run the Cycle: Click the "Run" button to train the parameters and execute the out-of-sample test.
2. View Results: The chart updates with the out-of-sample performance, and the performance and trades reports are generated. You can also view the chosen parameters and their performance sensitivity.

Adding More Cycles:

Continue adding cycles (around 10 is usually sufficient) until you have enough trades or have used up your testing dataset.

Important Note: Avoiding Overfitting in WFA

Even with WFA, you can still overfit your algorithm by repeatedly reusing the same out-of-sample data while making adjustments. To prevent this:

• Use Earlier Data for Iteration: Perform initial WFA runs on earlier market data.
• Hold Out Recent Data: Reserve a portion of the most recent data for a final, truly out-of-sample test once you've finalized your algorithm.

Further Analysis

Validating Performance

Utilize the performance validation tools to gain confidence in your algorithm's robustness.

• Monte Carlo Simulation: Assess potential drawdowns.
• Randomized Data Testing: Compare performance on randomized data to ensure your algorithm isn't exploiting noise or overfitting.

Testing with Modified Market Data

Re-run the WFA with modified market data (oversampled, inverted, detrended, jittered) to further validate your algorithm's performance under different conditions. If it remains profitable across these variations, you likely have a robust and reliable strategy.