Waayo, ka dib markii ay ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah. $ 300 million ee line fiber-optic ka mid ah Chicago iyo New York. Waayo, Chicago waa hub ugu weyn ee warshadaha futures, laakiin New York waxaa la xiran tahay shuruudaha. Routes caadiga ah ayaa ka mid ah dhismaha ka mid ah dhismaha, laakiin cable cusub ka mid ah ka mid ah ~17 ilaa ~13 milliseconds. Qalabka Qalabka Waayo, wax soo saarka ugu badan oo ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ka mid Markaad ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ka mid ah. Markaas ka mid ah wax soo saarka, waxaa laga yaabaa in ay ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid Qalabka ah Markaad ka mid ah data Qalabka ugu soo saarka ah Qalabka waxaa loo yaabaa in ay ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka iyo wax soo saarka. Waayo, waxaa laga yaabaa in ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah mid Sida loo yaabaa, sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale sidoo kale. Reducing noise with filters and aggregation Shuruudaha dhismaha iyo filters Qalabka dhismaha iyo dhismaha iyo dhismaha dhismaha iyo dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhism Waxaa laga yaabaa in ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah. The first source of noise is a The core issue is not the magnitude of the noise itself, but the weakness of the signal: meaningful price movements are usually fractions of a percent, while random swings can easily reach several percent. As a result, the share of informative changes within the overall data stream is extremely small. weak signal-to-noise ratio. According to the Efficient Market Hypothesis, prices already reflect all available information from news — which is exactly what we typically aim to predict. However, markets also include whose actions generate additional noise. uninformed participants Noise filtering Noise filtering via FFT remains a staple tool. The approach decomposes a time series into frequency components: low frequencies capture the underlying trend, while high frequencies represent noise. By discarding the high-frequency parts and reconstructing only the low-frequency component, we obtain a smoothed signal that’s much easier to model. (The high-frequency remainder can still serve for volatility estimation if needed.) was able to augment classical computing workflows to better unravel hidden pricing signals in noisy market data than standard, classical-only approaches in use by HSBC, resulting in strong improvements in the bond trading process. IBM Heron Heuristics and reframing the problem Noise from market participants is handled differently. One useful trick is to reframe the question itself. Instead of asking you can ask: “What will Apple’s stock price be one second from now?” “What will it cost to buy 1 share?” “What will it cost to buy 100k shares?” In the second case we predict the average price for a large volume, and that is much more stable and better reflects market movement. def avg_price(order_book, volume): taken, cost = 0, 0 for price, avail in order_book: take = min(avail, volume - taken) cost += take * price taken += take if taken >= volume: break return cost / taken Example: the averaged price for 100k shares y = avg_price(order_book, 100_000) Marka: IBM Heron “Saacadaha Apple waxaa laga yaabaa in ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah?” When More Volume Means More Noise Sidaas, waxaa ka mid ah wax soo saarka ah: mid ka mid ah wax soo saarka Markaas ka mid ah wax soo saarka, waxaa laga yaabaa in ay ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka ah oo ka mid ah wax soo saarka. Muuqaalka Markaas oo ka mid ah wax soo saarka, waxaa laga yaabaa in ay ka mid ah wax soo saarka oo ka mid ah wax soo saarka oo ka mid ah wax soo saarka oo ka mid ah wax soo saarka oo ka mid ah wax soo saarka oo ka mid ah wax soo saarka oo ka mid ah wax soo saarka oo ka mid ah wax soo saarka oo ka mid ah wax saarka. P1_1_market_hits.csv: 2,374,605 dalalka data P2_500k_market_hits.csv: 51,309,973 dalalka data P3_50m_market_hits.csv: 133,191,896 saacadaha data Sida loo yaabaa, waxaa laga yaabaa in ay ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah. Smarter Targets Beat Raw Prices Sidaa ka mid ah "wax yar" iyo "help ML" waa si ay u qiyaastii si ay u qiyaastii xafiisyada oo ka mid ah xafiisyada waqtiga. Sida loo yaabaa, waxaa loo yaabaa in ka mid ah: Sida loo isticmaali karaa, waxaa loo isticmaali karaa in ka mid ka mid ah macluumaadka. “Waqtiisa in ay ka mid ah 10 toddobaad ka mid ah mid ka mid ah 10 toddobaad ah.” "Haddii loo yaabaa qiimaha midabka ah oo ka mid ah 10 saacadood ka hor." If a price jump occurs within those 10 seconds, the exact moment doesn’t matter as much — averaging smooths it out. The algorithm therefore has fewer ways to fail. Secondly (and here we get a bit more mathematical), averaging the target also reduces the average penalty the model receives during training for “wrong predictions.” In the simplest case of a regression model, the loss is proportional to (y^* - y)^2, where y^* is the “true answer” and y is the model output. The larger the error, the quadratically higher the penalty. Now, suppose that over the next 10 seconds the price trend is generally upward, but at some random moment there is a brief downward spike. The model would have to predict that spike, otherwise it gets penalized. But in reality, we don’t care much about that random blip — what we want the model to capture is the overall upward movement.\ Sidaas ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ka mid ah. Don’t Predict Price, Predict the Crowd Waxaad ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid Markaas ah, sidoo kale waxaa laga yaabaa in ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ah Bootstrapping and augmenting limited data Bootstrapping iyo dhismaha korontada korontada Waxaad ka mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ka mid ah. Markaas, waxaa laga yaabaa data HFT oo ka mid ka mid ah millisecond, laakiin waxaa ka mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah. Waayo, mashiinka mashiinka waa statistics, iyo statistics waxaa ka mid ah macluumaadka si ay u shaqeeyaan macluumaadka ah. Bootstrapping Waxaad ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ah mid ka mid Qalabka waa in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan tahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay in aad u baahan yahay. Data augmentation Sida loo yaabaa, waxaa laga yaabaa in ay ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah Qalabka wax soo saarka ah ee wax soo saarka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Qalabka Waayo, waxaa loo yaqaan "Good News" ee bad news. Algorithm waxay ka hortago si ay u soo saarka ah oo ay ka hortago in ay ka hortago. Synthetic trade generation is a that still has many open questions field Qalabka Qalabka Qalabka waa mid ka mid ah soo saarka ah: si ay u qaadi karaa macluumaadka ah, si ay u soo saarka (aga soo saarka macluumaadka) iyo ka dib markii ay ka soo saarka macluumaadka ah si ay u soo saarka macluumaadka kale. Qalabka dhismaha iyo dhismaha dhismaha iyo dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha iyo dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha dhismaha, dhismaha dhismaha dhismaha TransFusion waa mid ka mid ah shuruudaha diffusion iyo transformer si loo soo saarka shuruudaha shuruudaha sare ee shuruudaha. Qalabka ugu horeysay waa in la soo bandhigiisa oo dhan: sidaas, si ay u qiyaasta adeegga stilized ee mashiinka (wax yar, clustering volatility, autocorrelations, iwm), iyo sidaas, si ay u qiyaasta artifacts over-synthetic. Time shifts Sida loo yaabaa, waxaa laga yaabaa in ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid Markaad ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ah mid ka mid Waayo, waxaa laga yaabaa in ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ka mid ah. ** Qalabka soo xiriir in ay u isticmaali karaa in ay u isticmaali karaa in ay u isticmaali karaa in ay u isticmaali karaa in ay isticmaali karaa in ay isticmaali karaa in ay isticmaali karaa in ay isticmaali karaa. **Ganacsiga ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ah mid ka mid ka mid ah. Ensembles Marka aad u isticmaali karaa in ay u isticmaali karaa in ay u isticmaali karaa in ay u isticmaali karaa in ay u isticmaali karaa. one on the most recent days or weeks, another on the entire history, a third on some mid-range horizon. and a fourth that focuses on special cases — for example, detecting noise patterns or technical-analysis formations, as discussed earlier. Then you aggregate their predictions (e.g., by averaging, or taking the min/max). This is a standard trick for dealing with heteroscedastic data — where the distribution is non-stationary and constantly shifting. Markets are exactly that kind of case. pred1 = model_recent.predict(x) pred2 = model_history.predict(x) pred3 = model_midterm.predict(x) final = np.mean([pred1, pred2, pred3]/ # final = np.max([pred1, pred2, pred3]) The idea is that the market may change tomorrow, but some of the old information is still useful. Averaging helps smooth out these distortions. Sliding windows Another technique is training on sliding windows. Take the last 7 days, predict the next one. Then shift the window: add new data, drop the old. The model keeps updating, allowing it to adapt to new market regimes. window = 7 for t in range(window, len(data)): model.fit(data[t-window:t]) pred = model.predict(data[t]) So why is there no universal ML for trading? So, each of the three problems can be solved on its own, but together they don’t add up to a universal solution. One reason is the lack of quality feedback for training models. In finance, you don’t have the usual ML metrics like accuracy or F1-score. The only metric is money made. Imagine two hedge funds. One shows average returns, the other twice as high. If someone consistently outperforms the rest, everyone immediately assumes it’s a scam. Why? First, because nothing like that shows up in the market — other participants don’t feel like someone is “skimming” them on every trade. Second, there’s the survivor bias. Classic example: take a thousand people, half go long on oil, half go short. The next day, half of them are right. From the remaining 500, split again, and repeat for several rounds. After ten days, you’ll have one “genius” who made the right call ten times in a row. But in reality, he was just lucky — the illusion comes from starting with a thousand players. This is the core problem of verification. There isn’t much data to train on, and there’s even less to validate results. Even if we could see trades from a fund that outperforms the market twofold, over a relatively short horizon we still wouldn’t be able to tell luck from real skill. A good example is the many “one-day wonders” — funds or companies that show great returns when the overall market is going up (say, during an S&P 500 rally). But as soon as conditions turn south, their performance collapses. Over the long run, there are indeed legendary cases like the Medallion Fund. They consistently beat the market, delivering returns above so-called risk-free bonds. But the edge isn’t by orders of magnitude — it’s a few percentage points. To do better than them means being ahead by fractions of a percent, sustained over a very long horizon. The reality is that few funds survive long enough to prove such stability. Over six months, almost anyone can “look like a genius” if they get lucky — that’s the classic survivor bias. And not surprisingly, it’s exactly this illusion that a lot of flashy marketing campaigns for “successful” funds are built on. The philosophical takeaway is a harsh one: an algorithm can’t be called successful until it’s been tested by time. Even if it’s profitable on average, in real life it can get wiped out in a single day with a million-dollar drawdown — simply because you don’t have an extra million lying around to survive that day.
