No. of Recommendations: 33
There's a new toy at gtr1.net/2013/hacktest.cgi .

This was created for a friend who does a lot of trading system development and backtesting in Excel, but who finds the GTR1 backtester too complicated for most purposes. With the "Hacktester", he'll get the best of both worlds: The ease and familiarity of calculating historical signals in Excel, but with all of the performance metrics that the GTR1 backtester has to offer.

The page allows you to enter any sequence of trades of Yahoo!-recognized symbols and build a GTR1 backtester link by pressing the button at the bottom of the form. When you follow the backtester link and run it, the backtester simply executes exactly the trades you entered and generates the usual reports.

I call it a "Hacktester" simply to convey its inferiority to what proper backtesting does for the following reasons:

1. Unlike a proper backtest, which calculates all signals and stock picks on the fly, the Hacktester does none of this, and instead, you do it yourself.

2. All data for specified ticker symbols comes from Yahoo! Finance via Volition's perl script (a copy of the version in use can always be found at http://gtr1.net/2013/csv/yahoo_quote2.pl), which is quite literally a hack. It's a relatively simple and legal hack, but it's a hack. Consequently, there are no guarantees that the Hacktester will work at any given time. But that's OK, since you don't use the Hacktester to generate signals or stock picks.

3. As a consequence of (2), data on delisted symbols is not available--only symbls that are active according to Yahoo! Finance.

There is an exception to the latter two drawbacks: You can create "symbol-screen references" that allow you to assign virtual Yahoo! ticker symbols to any regular GTR1 screen backtests and trade in and out of the screens like stocks. If a referenced GTR1 backtest is in the root universe of stocks, then that component of the "hacktest" will of course be based on the root universe, not Yahoo! Finance.

Unlike the GTR1 Backtester page, the Hacktester page was designed to be self-explanatory enough that anyone could start using it immediately without any further explanation. If it fails at this, please let me know how the user interface or instructions can be improved (aside from it looking like a relic from the 1990s, an issue I don't foresee ever making the top of my priority list).


Symbol-Screen References and Lag/Trade Price Settings

There is one area that may never be easy to grasp, and that concerns the "symbol-screen references" and how lags are handled with the various trade price options, which include the following:

o Same-day open.
o Same-day close (default).
o Next-day open.
o Next-day close.

If your hacktest does not reference any other backtests via symbol-screen references (i.e., the only thing you trade are real Yahoo! Finance ticker symbols), then you can thoughtlessly toggle between these modes and everything works as you'd expect. There is no need to read any further, and in fact, I advise against doing so, lest what follows needlessly intimidates or confuses you.

If you've decided to keep reading, then I'll say that it really is simple, though it may never seem simple. If you use symbol-screen references, then always keep the following three points in mind:

1. The Hacktester does not make any changes to the lag settings of your referenced screens as you toggle between the four trade price settings. It may seem like it should, but actually, it shouldn't.

2. The Hacktester (really, the Universe Builder) uses the referenced screens in backtesting mode, not screener mode. (If you aren't familiar with what these "modes" are, they simply describe the two different cases in how lag adjustments are applied when the screen's URL leaves them unspecified.)

3. The Hacktester does modify the trade price settings (open or close) of your referenced screens.

(3) isn't done so much as a favor to you (I'd rather be able to say that the Hacktester doesn't touch your referenced screen ULRs, period)--it's done because the Hacktester has no choice. The generated backtest link simply won't run without these modifications. This is because in a custom universe involving root universe backtests, whether trading at open or close is supported is determined at universe building time based on the trade price settings of the constituent screens, which all must be the same. That is, you cannot build a custom universe from some screen URLs that specify trading at the close and others that specify trading at the open. So the Hacktester must ensure that all of the screens in the universe URL that it creates are configured to trade at the same prices you specify (open or close).

By contrast, Yahoo! ticker symbols require no such configuration. A universe consisting entirely of Yahoo! ticker symbols will always support trading at both open and close.


Example: The "Nas100Momentum" Purchase

The otherwise empty Hacktester page comes with a sample of trades that include a symbol-screen reference: On 10/3/2011, a purchase of the symbol "Nas100Momentum" is entered. Nas100Momentum, as defined in the sample data, refers to a simple screen that buys the top 5 NASDAQ 100 stocks by one-year total return and holds for 252 market days. I'll explain how the hacktest handles this purchase in each of the four cases for trade prices, starting with the default:


Same-day close

When "Same-day close" is selected, all trades happen at close prices on the same day (10/3/2011), no matter what lag settings exist in the referenced screen. Remember, GTR1 "lag" settings refer to data lag, not trading lag. So five stocks are bought at 10/3/2011 closing prices.

But which five stocks, exactly? If you open your web browser at the referenced screen's URL (as of the writing of this post, it is http://gtr1.net/2013/?nas100.a:et1:dspo%281%29al252:trp%281,... ), you'll see that the lag adjustment settings are blank, as I strongly recommend for a top-level backtest (as opposed to lower-level field calculation exports). This means that in backtest mode, the lag adjustment settings default to 0. The Hacktester uses the screen in backtest mode. Thus, the effective lag adjustments (Price Lag Adjustment and Field File Lag Adjustment) are both zero. Since the screen's fields are defined with 1-day lag ("nas100.a = 1" is synonymous with "nas100.a:1 = 1", and trp(1,252) calculates 252-day total return with a one-day lag), effective lags in backtesting remains 1 for all fields. So the proper backtest of the screen is always buying one-day-old stock picks at all times. This is not changed by the Hacktester. So at the close of 10/3/2011, the hacktest buys the five stocks in the NASDAQ 100 on 9/30/2011 that had the highest 252-day total returns through 9/30/2011.


Next-day close

When "Next-day close" is selected, all trades happen at close prices of the next market day (10/4/2011), no matter what lag settings exist in the referenced screen. Remember, GTR1 "lag" settings refer to data lag, not trading lag. So five stocks are bought at 10/4/2011 closing prices.

But which five stocks, exactly? Once again, lag settings in the URL for Nas100Momentum are blank. Thus the default lag adjustments of zero are applied in backtesting mode. Since all of the screen's fields use a lag of 1 market day, there is effectively a lag of one market day in Nas100Momentum backtests. So at the close of 10/4/2011, the hacktest buys the five stocks in the NASDAQ 100 on 10/3/2011 that had the highest 252-day total returns through 10/3/2011.


Same-day open

When "Same-day open" is selected, all trades happen at open prices on the same day (10/3/2011), no matter what lag settings exist in the referenced screen. Remember, GTR1 "lag" settings refer to data lag, not trading lag. So five stocks are bought at 10/3/2011 opening prices.

But which five stocks, exactly? This time, the Hacktester does make a change to the Nas100Momentum URL, converting it to http://gtr1.net/2013/?o::nas100.a:et1:dspo%281%29al252:trp%2... , which trades at open prices.

But as before, lag settings in the URL for Nas100Momentum are blank. Thus the default lag adjustments of zero are applied in backtesting mode. Since all of the screen's fields use a lag of 1 market day, there is effectively a lag of one market day (which would be better described as an "overnight" lag, since trading is at the open) in Nas100Momentum backtests. So at the open of 10/3/2011, the hacktest buys the five stocks in the NASDAQ 100 on 9/30/2011 that had the highest 252-day total returns through 9/30/2011.


Next-day open

When "Next-day open" is selected, all trades happen at open prices of the next market day (10/4/2011), no matter what lag settings exist in the referenced screen. Remember, GTR1 "lag" settings refer to data lag, not trading lag. So five stocks are bought at 10/4/2011 opening prices.

But which five stocks, exactly? This time, the Hacktester does make a change to the Nas100Momentum URL, converting it to http://gtr1.net/2013/?o::nas100.a:et1:dspo%281%29al252:trp%2... , which trades at open prices.

But as before, lag settings in the URL for Nas100Momentum are blank. Thus the default lag adjustments of zero are applied in backtesting mode. Since all of the screen's fields use a lag of 1 market day, there is effectively a lag of one market day (or "overnight" lag, when trading at the open) in Nas100Momentum backtests. So at the open of 10/4/2011, the hacktest buys the five stocks in the NASDAQ 100 on 10/4/2011 that had the highest 252-day total returns through 10/4/2011.


How long are the Nas100Momentum stocks held after 10/3/2011 or 10/4/2011?

In all four of the above cases, the generated backtest URL is designed to pick the correct trading cycle (among the 252 possible due to the 252-day holding period) for whichever day Nas100Momentum is actually entered. The sample hacktest holds Nas100Momentum through the end of 2017. But under the hood, the stocks are refreshed every 252 market days, approximately on the anniversary of the original 10/3/2011 or 10/4/2011 purchase of Nas100Momentum.


What could go wrong?

In all four examples, the results were what one would hope from a naive application of the trade price settings. However, what may be counter-intuitive is that this happened without the Hacktester ever changing any lag settings in the referenced screen--it just worked anyway. But this won't be the case if a referenced screen is already effectively using zero lags in any of its fields. In particular, combining a zero-lag screen with "Same-day open" will lead to crystal-ball trading in the hacktest. The Hacktester does not attempt to read your mind and infer that what you really meant is for the zero-lag screen to be effectively used with an "overnight" lag with trading at the next open. Whether a screen is hazardous in this manner or not has to be determined on a case-by-case basis. But in general, a screen should be as safe to use under all four trade price settings as the Nas100Momentum screen provided:

1. Price Lag Adjustment and Field File Lag Adjustment are both blank.
2. Lag is unspecified (or at least 1) for all field files.
3. Lag is unspecified (or at least 1) for all imported signals and fields.
4. Price lag is at least 1 in all price-derived field functions (e.g., trp(1,252), rrs(1,252), etc).

If you have doubts about whether a hacktest is doing what you meant for it to do, you're welcome to ask me about it by email (there's no point in contacting me by fool.com private message, because any GTR1 links you include are guaranteed to get corrupted; in fact, even fool.com links get corrupted by fool.com's private messaging) at my foo.com username at gmail dot com.


Robbie Geary
Print the post  

Announcements

What was Your Dumbest Investment?
Share it with us -- and learn from others' stories of flubs.
When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and Glassdoor #1 Company to Work For 2015! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.