What a backtest is, and why you need one
Watching investment videos, you always stall in the same place: the moment someone says "this is the bottom." The reasoning sounds plausible and the chart on screen is persuasive, but there is no way to check it, so you end up either believing it or not.
A backtest opens a third path. You turn a trading judgment into a rule, run that rule against historical data exactly as written, and count whether it actually made money. It is not a tool for predicting the future — it is a tool for checking whether the claim you are hearing right now used to work.
It is easy to assume this requires professional software and paid data. It does not. One free tool that extracts subtitles plus two APIs that serve prices publicly will get you through in half a day. What follows is the exact order used to verify one actual video.
Step 1 — Pull the video's subtitles into text with yt-dlp
First, get the content as text. Instead of rewinding a 17-minute video over and over, download the subtitles as a text file and you can search for where each claim appears.
yt-dlp is the free command-line tool for this. The name marks it as the successor to youtube-dl, and it downloads video and subtitles from a very large number of sites, YouTube included. On a Mac, run brew install yt-dlp in the terminal; if you have Python, pip install yt-dlp.
Once installed, skip the video and take only the subtitles. --skip-download means don't fetch the video file, and --write-auto-sub means take YouTube's machine-generated captions when no human-written ones exist. What lands is a .vtt subtitle file, which mixes in timestamps and duplicated lines, so one cleanup pass is needed to leave only the text.
After cleanup, a 17-minute video becomes roughly 10,000 characters of prose. From that point you never open the video again.
# Install (either one)
brew install yt-dlp # macOS
pip install yt-dlp # Python environment
# Skip the video, take Korean and English subtitles
yt-dlp --skip-download \
--write-auto-sub --write-sub \
--sub-lang "ko,en" --sub-format vtt \
-o "vid.%(ext)s" "<video URL>"
# Result: vid.ko.vttimport re
lines, prev = [], None
for line in open("vid.ko.vtt", encoding="utf-8"):
# drop timestamp lines and headers
if "-->" in line or line.startswith(("WEBVTT", "Kind:", "Language:")):
continue
text = re.sub(r"<[^>]+>", "", line).strip() # strip tags
if text and text != prev: # drop consecutive duplicates
lines.append(text)
prev = text
open("transcript.txt", "w", encoding="utf-8").write("\n".join(lines))
print(len(lines), "lines")Step 2 — Translate the claim into a numeric rule
This step is the whole test. Find the core claim in the transcript and convert a sentence written in human language into a condition a computer can evaluate.
Take "it fell a lot and then bounced hard, so the bottom is confirmed." You cannot test that as written. How far down counts as a lot? How much of a rise, over how many days, counts as a hard bounce? And when exactly are you supposed to buy? Rewrite it as "if a stock is down 35% or more from its highest close in the last 60 trading days, and today's close is up 20% or more from yesterday's, buy at today's close" — and now the computer can find every date in history that qualifies.
One principle governs the translation: only use information you would actually have at the moment of purchase. If a phrase like "after it bottomed out" slips in — something you can only know later — you have built a rule that peeks at the future, and the result will always look good. This is called look-ahead bias, and it is the single most common reason a beginner's backtest fails to work in practice.
Half the claims get filtered out right here, because some cannot be written as numeric conditions at all. A sentence like "someone deliberately took it down" offers no condition that could be judged true or false. That is not a failed verification — it is the discovery that the claim was never verifiable, and that verdict is itself a result. Simply separating the testable claims from the untestable ones already tells you a great deal about a video's reliability.
Step 3 — Get historical prices from free public APIs
This step is surprisingly the easiest. You do not need to buy data. Nasdaq serves daily prices for US stocks and Naver Finance serves them for Korean ones, both at public URLs. No signup, no API key, no payment.
Nasdaq's historical path under api.nasdaq.com takes a ticker and a date range and returns JSON with date, open, close, and volume. Two caveats: the prices arrive as strings with dollar signs and commas, so you need one line to convert them to numbers, and the retrievable window generally reaches back about ten years.
For Korean names, give Naver Finance's price URL a six-digit ticker and start and end dates — 005930 for Samsung Electronics, 000660 for SK Hynix. The response is not proper JSON but an array using single quotes, so parse it by swapping the quotes or pulling the values with a regular expression.
Both are built on the assumption that a person is calling them from a browser, so when you call from a program it is safer to send a request header (User-Agent) that looks like an ordinary browser. Calling too many times too quickly can also get you blocked temporarily, so pause about half a second between tickers, and save what you fetch to a file so you can reuse it.
import json, urllib.request
UA = {"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126 Safari/537.36"}
def daily(symbol, asset="stocks", frm="2016-01-01", to="2026-07-31"):
url = (f"https://api.nasdaq.com/api/quote/{symbol}/historical"
f"?assetclass={asset}&fromdate={frm}&todate={to}&limit=99999")
req = urllib.request.Request(url, headers=UA)
rows = json.load(urllib.request.urlopen(req, timeout=30))
rows = rows["data"]["tradesTable"]["rows"]
out = []
for r in rows:
close = float(r["close"].replace("$", "").replace(",", "")) # "$207.12" → 207.12
mm, dd, yy = r["date"].split("/") # "07/30/2026"
out.append({"d": f"{yy}-{mm}-{dd}", "c": close})
out.sort(key=lambda x: x["d"])
return out
bars = daily("MU")
print(len(bars), bars[-1])import re, urllib.request
UA = {"User-Agent": "Mozilla/5.0"}
def daily_kr(code, start="20160101", end="20260731"):
url = ("https://api.finance.naver.com/siseJson.naver"
f"?symbol={code}&requestType=1&startTime={start}&endTime={end}&timeframe=day")
req = urllib.request.Request(url, headers=UA)
text = urllib.request.urlopen(req, timeout=30).read().decode("utf-8")
# not proper JSON — a single-quoted array, so pull values with a regex
# ['20260730', open, high, low, close, volume, foreign ownership]
rows = re.findall(r"\['(\d{8})',\s*[\d.]+,\s*[\d.]+,\s*[\d.]+,\s*([\d.]+)", text)
out = [{"d": f"{d[:4]}-{d[4:6]}-{d[6:]}", "c": float(c)} for d, c in rows]
out.sort(key=lambda x: x["d"])
return out
print(daily_kr("005930")[-1]) # Samsung ElectronicsStep 4 — Run the rule, but always against a baseline
Once you have the data, find every date that satisfies the condition from step 2 and compute the return one week, one month, and three months later. That part is mechanical.
What matters comes next: always compute a baseline alongside it. What would you have earned over the same horizon by simply holding the same names regardless of the signal, and how much did the index (say, the S&P 500) rise? Place them side by side. Skip this and every rule looks good in a rising market, because any day you bought was a day that made money.
Run this on a basket of AI-related names and the signal's three-month return comes out at 12.6% on average, which looks respectable. But buying the same names on any random day and holding three months also averages 12.6%. The signal's contribution is essentially zero. Without the baseline, this rule would have looked like a fine discovery.
Record the sample size too. The tighter the condition, the fewer signals it produces, and the average return of a rule with three or four occurrences is coincidence, not statistics. Sweep the condition across several parameter combinations, and if every combination that looks good has three or four samples, that is not a discovery — it is overfitting.
import statistics as st
def backtest(bars, dd_thresh=-0.35, pop=0.20, horizon=63):
"""Down dd_thresh from the 60-day high, then up pop in one day → buy at that close"""
signals, baseline = [], []
for i in range(60, len(bars) - horizon):
c, prev = bars[i]["c"], bars[i - 1]["c"]
high60 = max(b["c"] for b in bars[i - 60:i + 1])
fwd = bars[i + horizon]["c"] / c - 1 # return horizon days later
baseline.append(fwd * 100) # baseline: every day
if c / prev - 1 >= pop and c / high60 - 1 <= dd_thresh:
signals.append(fwd * 100) # only days the signal fired
return signals, baseline
sig, base = backtest(bars)
print(f"signal n={len(sig):4d} mean {st.mean(sig):6.1f}% median {st.median(sig):6.1f}%")
print(f"baseline n={len(base):4d} mean {st.mean(base):6.1f}% median {st.median(base):6.1f}%")
# If the signal's mean cannot beat the baseline, the rule has no skillReading the result — the mean alone will tell you the opposite
When the numbers land, the mean is what you look at first, and in a backtest reading only the mean almost always leads you to the wrong conclusion. Put the mean and the median side by side.
A concrete case makes it obvious. Run the condition "down more than 35% from the 60-day high, then up 20% or more in one day" across ten years of data and you get 24 signals. Their average return three months later is 38.2%. That looks remarkable. But the median is -1.0%, and only 11 of the 24 were profitable — under half.
What happened is that a handful of outliers (one returned 258% over three months) carried the average by themselves. The other half kept falling from that same point. Reading only the mean, you would have concluded that a bounce after a crash is a powerful buy signal. Reading the median alongside it, you get the opposite conclusion: a 46%-win-rate gamble whose expected value rests on a few jackpots.
The other thing to watch for is survivorship bias. Build a basket out of the names that are doing well today and run it back ten years, and of course the result is good — the names that disappeared over those ten years are not on the list. If a stock that listed only a year ago is in the basket, you are computing returns for a period in which it did not exist. Include only the names that actually existed at each point in time, and if you excluded any for insufficient history, say so in the result.
Where people get stuck
If a price API suddenly returns 429 or Too Many Requests, you called it too often in too short a window. Pause half a second between tickers, and always save what you fetch to a file so a rerun does not fetch it again.
Some videos have no downloadable subtitles at all — no human captions and auto-captions disabled. That requires a separate speech-to-text step. Note also that auto-generated captions frequently mangle numbers and proper nouns, so never take a figure lifted from a transcript as fact; confirm it against the original source.
US tickers disappear when a company merges or renames. If a historical query comes back with zero rows, first check whether the ticker still exists. Quietly dropping such names makes the result look better than it was.
Finally, do not let signals with insufficient elapsed time into the average. If you are measuring three-month performance and a signal fired two weeks ago, that case has no result yet. Forcing it in lets recent market conditions contaminate the whole set. Exclude cases without a completed horizon, and state how many you excluded.