But Is That Good?
A post crossed my feed claiming that buying the market every time the Total Put/Call Ratio closes above 1.12 produces a high win rate and strong average returns. In fact, over the following two and three months, the win rate is 100%. Here, have a look:
Source: @SubuTrade on X
The post looks credible. It has the standard table that many of these posts contain, complete with green cells for positive numbers and red cells for negative numbers.
We will take it at face value that the data is correct. We will ignore, for now, the small sample size. We will also ignore, for now, that clustered signals are treated independently.
A.I. Has Democratized Research
With the simplest of A.I. tools, a study like this is very easy to produce.
You don't even have to be good at Excel anymore.
Advocates will tell you that this is the "democratization of quant" or something like that. But I have learned that when you hear democratization or democratize in this business, you should be skeptical.
It has become far too easy to produce studies and push them out on places like LinkedIn and X. Get a big enough following and you can write a weekly newsletter and charge $25 a month for your "research."
There is a very low barrier to production.
What used to take a week of research can now be done in an hour. But is that really a good thing?
I will argue that it is not.
The Key Problems
Anyone can produce a credible-looking study with the right data feed and the right prompt.
I can give instructions to my 15-year-old daughter, and she can have a table for you in an hour. And, let me tell you, it will be aesthetic!
The problem is you do not know what kind of rigor was put into the study. In the case above, why were only the last 20 readings used? Why was the 1.12 level selected?
Were these simply the best parameters to prove the author's point?
In other words, how much was this study optimized? How much was it curve-fit?
A.I. tools have the potential to give the illusion of certainty.
"Let's Test This"
I want to be fair here; the studies are not completely useless. They can be a good starting point for deeper, more rigorous research.
So that's what we did.
Honestly, we do this all the time. We have even created new indicators based on studies like this that we have come across. We're not closed-minded. We're skeptics.
I gave the post to a junior research associate with one instruction: find out if we can trade this in the real world and make money.
My analyst wrote the code that was needed to conduct a real test. The code included an entry rule, a holding period, commission, and slippage.
Perhaps more importantly, our test was conducted over a 24-year period and included nearly 150 individual trades.
Did we find that this strategy makes money when subjected to a realistic test?
The answer was a resounding NO. The statistics that you see above proved to be about as realistic as Santa Claus.
These studies appear to be far better at generating clicks than managing money.
What This Means for You
Treat every study you see on your feed as a hypothesis, not a strategy.
Before you act on someone's table, ask four questions:
How many trades? Not signals, but trades.
Do the signals overlap? If three of the readings fire in the same two-week window, you did not get three independent tests. You got one.
Where did the threshold come from? If the author cannot tell you how the strategy performs at 1.05 and 1.20, there is a good chance that the published level was cherry-picked.
What are the exit and cost assumptions? A study without a holding period, exit rules, commissions, and slippage is not the same as a backtest. These datapoints matter in the real world.
And one more, the one that matters: does the person publishing it have money on the line? Their own money or their clients'?
There is a wide gap between people who publish studies and people who must live with the drawdown.
None of this means you should ignore these studies. Intellectual curiosity is an asset; in fact, it is one of the core values that we look for in employees.
But there is a world of difference between "that's interesting, let's test it" and "that's interesting, let's trade it."
The edge is no longer access to data or the ability to run a study. Everyone has that now.
The edge is the discipline to do the real work with intellectual honesty and rigor. If you do, you will find that most ideas fail, and that's fine.
If everything worked, nothing would work.
