Algo trading system: the 4 parts and how to test each one

Contents

1. What components does an algo trading system need?
2. How are the four parts built and connected?
3. How do you backtest the whole system with realistic costs?
4. How do you validate it with out-of-sample and walk-forward tests?
5. What is the difference between backtest and live results?
6. How do you monitor an automated system live?
7. Frequently asked questions
8. References

What components does an algo trading system need?

An algo trading system has four parts: data, signals, execution and risk. Data supplies correct bars on time, signals decide when to enter and exit, execution turns a signal into an order at a known price on a known bar, and risk sets the contract count and the stopping point. Test each part alone, then the chain together.

An algo trading system is a set of written rules and the software that runs them, which reads price data, decides trades, sends orders and limits risk without a person choosing each trade. Like a chain, the system is as strong as its weakest part. A good signal rule fed by bars that arrive late trades on prices that no longer exist. A correct order module with no risk rule sends the right order in the wrong size. For the background on what the rules cost to run and why most of them lose after costs, start with what algo trading is and what it costs.

Part	Question it answers	Test on its own	A bad result looks like
Data	Are the bars correct and on time?	Compare a sample of bars with the exchange record; check gaps, time zone and roll dates	Missing sessions, bars stamped in the wrong hour, a roll gap traded as a real move
Signals	When do we enter and exit?	Backtest the rule alone with costs; check that neighbouring settings behave alike	One setting is good and its neighbours are poor
Execution	What order, at what price, on which bar?	Compare backtest fills with paper-trading fills on the same bars	Backtest fills on the signal bar close; live fills come one bar later and worse
Risk	How many contracts, and when do we stop?	Check contract count against account size and the validated drawdown	A drawdown the backtest called normal forces a margin call live

The signal part gets most of the attention, but it is one of four. It can be any rule a computer can evaluate on a bar: trend following, mean reversion, breakout or a calendar rule. The four quant trading strategy types and how each is tested explains what changes in the test from one type to the next. The other three parts rarely appear in a strategy description, and they are where most live failures begin.

How are the four parts built and connected?

The four parts form one loop that runs on every bar, and the order of the steps inside the loop is what makes the backtest honest. The loop starts when a bar closes and ends when the position record is updated after the fill. Every step reads only what the steps before it produced.

1. Bar N closes. The data part receives the open, high, low, close and volume of the finished bar.
2. Data check. The system confirms the bar arrived on time, has no missing field and follows the previous bar in sequence.
3. Signal computed on bar N. The rule reads bar N and earlier bars only, and returns one of three states: go long, go short, or stay as you are.
4. Risk check sets the contract count. The risk part reads the account size, the current drawdown and the open position, and returns the number of contracts, which may be zero.
5. Order sent. The execution part sends a market or stop order of that size.
6. Filled at the open of bar N+1, position record updated. The fill price and time are written to the position record, which the risk check reads on the next pass.

Next-bar execution is the practice of deciding on the close of one bar and filling at the open of the next, so that the system never uses a price it could not have traded at. A backtest that decides on the close of bar N and also fills on that close is reading the future by one bar. Step 6 in the loop above is where this is enforced. A day trading system adds one more risk step before step 5: flatten the position before the session close, with the session end taken from the data part, not from a clock on the trader's computer. The article on how day trading is defined and tested after costs covers what that forced exit does to the trade count.

The position record is the part most beginners leave out. It is the one place that knows what the system actually holds, and both the risk check and the monitoring in the last section read from it. Without it, a restart after a disconnect leaves the system with a position in the account and none in its memory.

How do you backtest the whole system with realistic costs?

The backtest replays the full loop on historical bars with commission, slippage and next-bar fills charged in every single evaluation, not added at the end. Slippage is the difference between the price the signal used and the price the order actually fills at. If costs are added only to the finalists, the search ranks candidates by a number no account could earn, and the strategies that trade most often look best for exactly the wrong reason. AlphaPrime (Valynox) charges fees and slippage and fills on the next bar inside every candidate's backtesting run, so the ranking the search produces is already net of those costs.

The run below shows what a full-system report looks like once those settings are in place. It tested NQ daily bars from 2019-01-02 to 2025-06-30, long and short with one contract, charging $20 commission and $10 slippage on every trade. It used a Bipower Jump Ratio rule as its only entry and exit, with no stop, target or filter: a stand-in rule built from standard indicators, not a test of the four-part structure itself. What you see is the one candidate kept from 1,024 tried, chosen on return against drawdown.

Two things in this report are worth reading the way a system builder reads them. First, the cost setting is realistic for NQ: CME lists the minimum tick of the E-mini Nasdaq-100 at 0.25 index points, worth $5.00, so $10 of slippage is two ticks per trade. Exchanges can change tick sizes and values, so check the current contract page before setting costs; the free futures contract calculator shows tick size, tick value and point value for the main contracts. Second, the $30 of cost charged per trade is small next to the average trade of $1,866.32, which is the average trade test from the first section passed with room to spare. A rule with the same gross result and ten times the trades would not pass it.

What the report does not show is the other candidates. The kept one is the best of many tries on the same history, and the best of many always looks better than a typical one, because part of what made it best is noise that happened to line up with those bars. Bailey, Borwein, López de Prado and Zhu (2014) show that this selection bonus grows with the number of candidates compared, and that the in-sample number of the winner is inflated even when no single candidate was tuned by hand. Picking this candidate after the fact flatters it. The numbers above describe what one rule did on one history; they are not what the rule earns and not what a reader should expect from it.

How do you validate it with out-of-sample and walk-forward tests?

A single backtest cannot separate a pattern from luck, so the system is graded on data the search never touched, first a held-out block and then rolling walk-forward windows, with the pass rule written before you look. Out-of-sample data is the part of the history set aside before the search and used only to run the kept candidate unchanged. Walk-forward analysis is a test that tunes settings on one window, runs them unchanged on the window that follows, moves forward and repeats, then joins the test segments into one out-of-sample curve. Pardo (2008) describes the method as the standard way to judge whether an optimized system was ever tradable rather than merely fitted.

1. Write the pass rule first. Decide the minimum number of out-of-sample trades, the share of the in-sample average trade that must survive, and how many walk-forward windows must be positive. Write the three numbers down before any out-of-sample run.
2. Run the held-out block once. The kept candidate runs unchanged on the block the search never saw, with the same costs and next-bar fills. Read the average trade after costs and the trade count against the rule.
3. Run the walk-forward windows. Each window re-tunes on its own in-sample segment and tests on the segment after it. Count the positive test segments and read the joined curve.
4. Accept or reject, without changing the rule. A candidate that fails is a new idea if you change it, and a new idea is graded on data it has not seen.

Each look at an out-of-sample result spends some of it. Harvey and Liu (2015) show that a result read after many candidates were compared on the same data needs to be discounted for the number of tests behind it, which is why the held-out block is run once and the pass rule is fixed in advance. How to size the windows, how many you need and how to read the joined curve are covered in walk-forward analysis and the one backtest number you could have traded. The validation steps in AlphaPrime stop the later checks as soon as a candidate fails an earlier one, so few candidates ever reach the later data.

What is the difference between backtest and live results?

Live results differ from the backtest for named, measurable reasons, so write down the expected size of each gap before going live instead of discovering it afterwards. The backtest made assumptions about fills, data and uptime that the live system has to earn. Each assumption has a measurement that tells you how far reality is from it.

Source of the gap	What the backtest assumed	What happens live	How to measure the gap
Fill price	Open of bar N+1 plus a fixed slippage	Fills vary with order size, time of day and news	Average live fill minus backtest fill, per trade, in ticks
Data feed differences	One clean historical series	Live feed differs from the history vendor in bar close times or in ticks included	Count bars where the live close differs from the historical close for the same minute
Missed or late bars	Every bar present and on time	A bar arrives late and the signal fires one bar later or not at all	Count signals the backtest shows that the live system did not take
Disconnects and restarts	The system is always running	Broker or network outage; position record and account disagree after restart	Count restarts per month and the trades affected
Commission and slippage drift	$20 and two ticks, fixed	Broker changes fees; volatility widens the spread	Live cost per trade from statements, compared with the setting, each quarter

The fill price row is usually the largest gap and the easiest to measure. Paper trading the system beside the backtest for the same bars gives a live fill list that can be subtracted from the backtest fill list trade by trade. If the average difference is one tick per side when the backtest assumed two, the cost setting was conservative and the gap is in your favour; if it is three, raise the slippage setting and re-run the validation before the first live contract. The five checks in the checks that show whether an algo strategy works consistently are built around the same idea: every difference between the test and the account is something to measure, not to explain away.

How do you monitor an automated system live?

Monitoring compares the live trade list with the backtest trade list for the same bars, and the live drawdown with a limit set from the validation, and a mismatch in fills or a drawdown past that limit is a stop signal, not a tuning signal. The backtest is still running in parallel on the live bars; it is the reference. The live system either matches it, trade for trade, on every bar, within the fill gap you measured, or something in the four parts has changed.

Three checks run on a fixed schedule:

1. Every day: trade list match. For each live trade, find the backtest trade on the same bar and compare direction, size and fill price. A trade in one list and not in the other points at the data or execution part. A size difference points at the risk part or the position record.
2. Every week: drawdown against the limit. Compute the live drawdown from the equity peak and compare it with the limit. The free drawdown calculator computes maximum drawdown and recovery from any equity series you paste in.
3. Every quarter: cost drift. Compare commission and slippage per trade from the broker statements with the settings in the backtest.

The next thing to do is on your own system, today: open the trade list of your last backtest and the trade list from your paper or live account for the same two weeks, line them up bar by bar, and write down the average fill difference in ticks. That one number tells you whether the slippage setting in your backtest is honest, and it is the first row of the monitoring you will run for as long as the system trades.

Frequently asked questions

How do you test an algo trading system?

Test the four parts alone first: compare bars with the exchange record, backtest the signal rule with costs, compare backtest fills with paper fills, and check the contract count against the account. Then backtest the whole chain with commission, slippage and next-bar fills in every evaluation, grade the kept candidate on a held-out block and on walk-forward windows with a pass rule written in advance, and paper trade it beside the backtest before the first live contract.

What is the difference between an algo trading platform and an algo trading system?

An algo trading platform is the software environment that runs strategies, charts data and sends orders to a broker. An algo trading system is the complete set of rules and components you run on it: the data checks, the signal rule, the execution logic and the risk limits. The platform supplies the tools; the system is what you build and test with them, and the same system can be written for more than one platform.

Do I need to code to build an algo trading system?

No. Strategy generators can turn rule ideas and parameter ranges into testable strategies and export the source code to a trading platform, so the signal part can be built without writing code. The data, execution and risk parts still need to be set up and tested on the platform you trade from, and the testing steps in this article apply whether the rule was written by hand or generated.

How long should I paper trade before going live?

Long enough to collect a fill list you can compare with the backtest trade by trade, which depends on how often the system trades rather than on calendar time. A system that trades daily gives a usable fill sample in a few weeks; a system that trades a few times a month needs several months. The paper period ends when the average fill difference is measured and the slippage setting in the backtest is adjusted to it.

References

• Bailey, D. H., Borwein, J., López de Prado, M., & Zhu, Q. J. (2014). Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the American Mathematical Society, 61(5), 458–471.
• Harvey, C. R., & Liu, Y. (2015). Backtesting. The Journal of Portfolio Management, 42(1), 13–28.
• Pardo, R. (2008). The Evaluation and Optimization of Trading Strategies (2nd ed.). John Wiley & Sons.
• CME Group. E-mini Nasdaq-100 Futures Contract Specs. https://www.cmegroup.com/markets/equities/nasdaq/e-mini-nasdaq-100.contractSpecs.html