Back to all posts

Idea to Algorithm: Turning a Predictive Model into Trades

Quantitative finance is, at the core, a process which is attempting to turn thought into money. In that light, it is no different from many other jobs, the main question is what path quants take in order to turn their thinking in cash. We’ll lay out some brief thoughts here on very high level workflows quants use, and link to supporting materials for those that would like to learn more. Please click through to linked lectures for hands on examples with real code.

 

The flow from idea to realized returns.

The flow from idea to realized returns.

Here is a webinar version of this post if you'd like to watch/listen instead.

Step 1: Hypothesize

“In general, we look for a new law by the following process: First we guess it. Then we – now don't laugh, that's really true. Then we compute the consequences of the guess to see what, if this is right, if this law that we guessed is right, to see what it would imply. And then we compare the computation results to nature, or we say compare to experiment or experience, compare it directly with observations to see if it works. If it disagrees with experiment, it's wrong. In that simple statement is the key to science. It doesn't make any difference how beautiful your guess is, it doesn't make any difference how smart you are, who made the guess, or what his name is. If it disagrees with experiment, it's wrong. That's all there is to it.” - Richard Feynman

Quantitative finance applies the scientific method and automation to the process of predicting market behavior and using that knowledge to make money. The first step is always to come up with a model of the world. A good model allows you to predict what will happen in the future, a bad model gives false confidence. Knowing whether a model is good or bad often boils down to being brutally specific about what the model is and what assumptions are being made. For instance, a sloppy model would be “Stocks I read about in the newspaper are likely to go up tomorrow.”. This model is sloppy for many reasons. There is no definition of “read”, nor is the set of stocks you read about well defined. What if you tend to read only certain pages or sections? Why do you think they should ‘go up’ tomorrow? What is the mechanism underlying this behavior? In general, science forces us to be explicit about all of this, and by being explicit, methods of testing our models become obvious. For example, let’s break down the above model into a more scientific statement. “If a stock’s company name or ticker appears printed in the Wall Street Journal’s website today, its returns 5 days forward will have increased volatility.” This is by no means a perfect hypothesis, but it unwraps the statement a bit, makes things more specific, and defines a framework for testing. We can start devising a program to count all mentions of a ticker in the WSJ. We can measure volatility via variance of the returns.

Implicitly, your hypothesis must come from your current model of how the world works. That is the reasoning underlying the hypothesis. As such, your work should be directed by where your understanding of the world breaks -- this is a core principle in the natural sciences. If your model disagrees with real observations in the real world it means that the model is wrong in some way. You should spend time examining how your model is wrong so that you can improve it, not examining where it’s right.

 

Step 2: Your Predictions Must be Correlated with Future Outcomes

Now that you have a model, you want to check if those predictions do indeed correlate with future returns. (https://www.quantopian.com/lectures/factor-analysis) If they do, then that satisfies our first check. In industry, this is sometimes referred to as the ‘information coefficient’ of a model. What it means to correlated is also discussed in the linked lecture.

It’s important to note that this is not nearly enough. Your predictions have to survive the rest of the way through the process to realized returns. There are many steps ahead which can contort your predictions and lessen their correlations with future outcomes. This image is an example analysis from the linked lecture:

Example analysis of a predictive model.

Step 3: Your Predictions Must Survive Portfolio Optimization

Let’s say you have a model which you believe fits well, and you’ve tested it rigorously and scientifically (https://www.quantopian.com/lectures/the-dangers-of-overfitting). That model will yield predictions of how the market should behave. Based on these predictions you can come up with a way to place trades that should take advantage of these predictions. This process is known as portfolio design, and is an often overlooked part of the workflow. Your portfolio should be designed to be risk aware/constrained, and try to take maximum advantage of your predictions. This lecture describes a canonical example of what quants refer to as a cross sectional portfolio (https://www.quantopian.com/lectures/long-short-equity).

One example I like to give is that of playing tennis against Federer. If you went up against Federer in a match, you’d want the match to last one serve (sample the process once). That way the volatility is maximized and you have the highest chance of beating him in a fluke. Meanwhile he wants a very long match with many events that sample both of your skills and decrease the volatility. As such you generally want to make as many independent predictions as possible if you truly think you can beat the market (https://www.quantopian.com/lectures/position-concentration-risk).

Basically, you want to take advantage of your predictions without taking on too much risk. As such you can imagine a function R(P) that for each portfolio P, tells you whether or not P takes on too much risk. You want to get your portfolio weights as close as possible to your predictions without violating R(P). This process is known as constrained optimization and is discussed at length here (https://www.quantopian.com/lectures/risk-constrained-portfolio-optimization).

The portfolio weights that you end up with will not be exactly as you would have wanted due to these constraints. As such some of the correlation with future outcomes will be lost.

Step 4: Your Portfolio Weights Must Survive Portfolio Execution

Now that you have risk-constrained portfolio weights, we need to actually trade them. This is a very complicated process and the subject of a large amount of research. Your trades, if not placed carefully, can incur a large amount of trading costs (https://www.quantopian.com/lectures/introduction-to-volume-slippage-and-liquidity). This will effectively be a discount on your final realized returns. Also, sometimes market conditions do not allow you to place precisely the trades you wanted. That means that the conversion of portfolio weights to real positions will contort your weights, and we now have a set of positions which are twice contorted from our original predictions.

Step 5: Realized Returns

Finally, only if your realized predictions have sufficient correlation with future returns will you make money. There have been two major steps which have contorted your original predictions at this point, so your original predictions must have been pretty good to survive this far. At this point you can still have issues converting your gains to cash as liquidating positions can be difficult for the same reasons trading is difficult.

 

Resources to Learn More

What I’ve tried to convey here is a very high level understanding of what a quant workflow looks like, and some of the broad principles necessary to work in the space. I linked many lectures which convey much deeper information, and in general there is always room for improvement. Quantitative finance, like any science, is an iterative process in which your knowledge can always be more complete. If you’d like to learn more I encourage you to:

 

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.