Most trading strategies do not fail because the idea was completely worthless. They fail because the strategy only worked in one neat, overly cooperative version of history. That is why I keep coming back to a guy most traders have probably never heard of: Genichi Taguchi. Genichi Taguchi Taguchi became a big deal in quality engineering because he helped popularize a deceptively simple idea: Taguchi A good design is not just one that performs well on average. It is one that still performs when noise and variation show up. A good design is not just one that performs well on average. It is one that still performs when noise and variation show up. His ideas were adopted by many large players such as NASA, 3M, Ford and Nissan. And frankly, this is a much better lens for trading system design than “best backtest wins.” The Problem With How Many Traders Evaluate Strategies A lot of strategy research still quietly assumes the following: if the backtest looks good, the system is good
if one out-of-sample period holds up, the system is robust
if the equity curve is smooth, the edge must be real
if the optimization found a good pocket, the parameters are valid if the backtest looks good, the system is good if one out-of-sample period holds up, the system is robust if the equity curve is smooth, the edge must be real if the optimization found a good pocket, the parameters are valid That is how traders end up with strategies that look amazing until they meet: a different volatility regime
a slightly worse fill
a different symbol
a small parameter shift
or a less flattering start date a different volatility regime a slightly worse fill a different symbol a small parameter shift or a less flattering start date In other words, they mistake historical fit for robust design. historical fit robust design Taguchi’s framework is useful because it attacks exactly that mistake. Taguchi’s Core Idea At a high level, Taguchi’s worldview is this: You do not judge quality by whether something barely passes a threshold and pray you have a good threshold. You judge it by how much loss is created when the output drifts away from the target. Translated into plain English: A design is not truly good if it only works in a narrow setting. It should still behave acceptably when conditions get messy. It should still behave acceptably when conditions get messy. That should sound very familiar to anyone who has ever watched a beautiful backtest fall apart in live trading. If you haven’t, take a look at this lying backtest case study. lying backtest case study The Most Important Distinction: Controllable Factors vs Noise Factors This is where Taguchi gets especially useful especially for trading system developers. He separates the world into two buckets: Controllable factors -  things you choose In trading, that means: entry logic
exit logic
stop/target structure
holding period
timeframe
filters
position sizing
portfolio construction rules entry logic exit logic stop/target structure holding period timeframe filters position sizing portfolio construction rules Noise factors -  things you do not control but still have to survive In trading, that means: regime shifts


volatility expansion or contraction


slippage/spread changes


different date windows


small parameter shifts


execution imperfections


symbol-specific weirdness


perturbed or synthetic paths
Most bad strategies are not bad because they had zero edge. They are bad because the design required the noise factors to cooperate. That is a fragile design destined to lose in live markets that traders have found out the hard way since the first online broker came around. regime shifts regime shifts volatility expansion or contraction volatility expansion or contraction slippage/spread changes slippage/spread changes different date windows different date windows small parameter shifts small parameter shifts execution imperfections execution imperfections symbol-specific weirdness symbol-specific weirdness perturbed or synthetic paths
Most bad strategies are not bad because they had zero edge. They are bad because the design required the noise factors to cooperate. That is a fragile design destined to lose in live markets that traders have found out the hard way since the first online broker came around. perturbed or synthetic paths Most bad strategies are not bad because they had zero edge. They are bad because the design required the noise factors to cooperate. That is a fragile design destined to lose in live markets that traders have found out the hard way since the first online broker came around. The Part Traders Should Really Pay Attention To Taguchi does not just say “noise exists.” He effectively says: bring the noise into the experiment. bring the noise into the experiment. That is the real breakthrough for strategy development. Do not just test the strategy in one clean historical path and congratulate yourself. Disturb it on purpose. Change assumptions. Shift the dates.Perturb the data.Stress the fills.Try nearby parameters.Force it to deal with uglier conditions. Then see what survives. That is much closer to real robustness than the usual “highest net profit over one window” game. What This Looks Like in Trading In my world, this translates very directly. The strategy logic itself is the thing you are designing - control factors. The market environment is the thing trying to break that design - noise factors. So the question is not: Which strategy backtested best? The better question is: Which strategy still behaves well after repeated exposure to noise? Which strategy still behaves well after repeated exposure to noise? That is a different mindset. It also changes what you optimize for. Instead of just chasing: net profit,
Sharpe,
CAGR,
profit factor, net profit, Sharpe, CAGR, profit factor, you start caring more about: stability across perturbations
downside under stress
parameter sensitivity
date sensitivity
degradation under noisy conditions stability across perturbations downside under stress parameter sensitivity date sensitivity degradation under noisy conditions These are much healthier objectives. Why This Matters More Than Ever in Systematic Trading Modern traders can test more ideas than ever. That sounds like an advantage. It is, but it also massively increases the chance of finding things that look special only because you searched hard enough. That is, the more you will find things that work by pure luck (Vs random benchmarking helps here). Vs random benchmarking The more search power you have, the more you need robustness discipline. That is part of why I think Taguchi’s philosophy matters so much for trading today. He gives you a way to think about strategy research that is not centered on “What tested best?” It is centered on: What is least sensitive to the nonsense? What is least sensitive to the nonsense? A Trading Example Imagine two strategies with almost identical headline backtests. Both made about the same money.Both traded often enough.Both have a nice-looking equity curve. A trader who stops there says they are basically equivalent. A robustness-minded trader keeps going. Now perturb the data.Run parameter noise.Try different date windows.Simulate rougher conditions. Here is what happens: Strategy A Strategy A median result stays solid
lower tail holds up reasonably well
nearby settings behave similarly
noise outcomes cluster relatively tightly median result stays solid lower tail holds up reasonably well nearby settings behave similarly noise outcomes cluster relatively tightly Strategy B Strategy B median falls harder
lower tail gets ugly fast
nearby settings diverge
noise outcomes scatter all over the place median falls harder lower tail gets ugly fast nearby settings diverge noise outcomes scatter all over the place Same headline backtest. Totally different design quality. Both strategies “hit the target” in the clean test. Only one was insensitive to noise, one is not. Which would you take live? A Few Practical Trading Metrics in That Spirit These are not Taguchi’s original formulas, but just trading-friendly ways to score the same idea. 1) Normalized percentile spread (P90−P10)/(∣Median∣+0.0001) This asks: How wide is the distribution of outcomes relative to the typical result? Lower is usually better. 2) Downside robustness loss (Median−P10)/(∣Median∣+0.0001) This focuses more on how bad weaker scenarios get. For traders, this is often more useful than total spread because downside matters more than unusually good outcomes (typically). 3) Survival ratio under stress P10/(∣Median∣+0.0001) This asks: How much of the median survives when things get less cooperative? Higher is better, assuming the median is positive. Again, these are not “Taguchi formulas.” They are just modern trading translations of a very Taguchi-like objective: reward central performance, penalize sensitivity to noise. reward central performance, penalize sensitivity to noise. The Mistake I See Traders Make A lot of traders say they care about robustness. What they often mean is: “I ran one extra test after the backtest still looked good.” “I ran one extra test after the backtest still looked good.” That is not really robustness. Robustness is a design philosophy. Robustness It means the strategy does not need the world to be perfectly cooperative in order to remain useful. Where This Gets Really Interesting Once you think this way, a platform should not just surface every strategy that looked good in-sample. It should be able to: generate candidates
stress them
score them on central tendency and fragility
filter out the weak survivors
return the designs that degrade the least generate candidates stress them score them on central tendency and fragility filter out the weak survivors return the designs that degrade the least That is a much better workflow than “optimize until the leaderboard looks pretty.” It is also much closer to how engineers in other fields think about design quality. Build Alpha is a trading platform that can automate this entire process. Here’s the Taguchi inspired filters with arbitrary thresholds set as an automated workflow. One Important Caveat Taguchi is useful, but I would not treat him as scripture. Real trading systems have messy interactions everywhere. A filter may help only with one type of exit. A mean revesion system may need tested differently than a trend system.A stop may work on daily bars and fail on intraday.  A parameter may look stable until a different component changes. But this is an argument for stealing the right principle: Do not optimize for beauty. Optimize for resilience to disturbance. Do not optimize for beauty. Optimize for resilience to disturbance. That is the lesson here and big takeaway from my 15+ years in automated strategy development. Final Thought Most traders do not blow up because they had no idea. They blow up because they trusted a design that only worked when the historical path was unusually cooperative. That is why Taguchi is still useful. He reminds us that the goal is not to find the strategy that looks best in ideal conditions. The goal is to find the one that still behaves when the conditions stop being ideal - because that is what Robustness is. Read more at Taguchi Robustness Methods here. Taguchi Robustness Methods Thanks for reading, Dave Founder Build Alpha

This story contains new, firsthand information uncovered by the writer.

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

What Traders Can Steal From Genichi Taguchi About Robustness

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Using Monte Carlo Simulation for Algorithmic Trading

Predicting Flight Risks: How Artificial Intelligence is Helping SMBs to Combat High Turnover Rates

Nikhil Dodda: Pioneering AI Innovation Through Visionary Leadership

174 AI Tools To Try

3 Trends of the Neural Network Usage for Algorithmic Trading

48 Stories To Learn About Algorithmic Trading

Using Monte Carlo Simulation for Algorithmic Trading

Predicting Flight Risks: How Artificial Intelligence is Helping SMBs to Combat High Turnover Rates

Nikhil Dodda: Pioneering AI Innovation Through Visionary Leadership

174 AI Tools To Try

3 Trends of the Neural Network Usage for Algorithmic Trading

48 Stories To Learn About Algorithmic Trading

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps