Message Font: Serif | Sans-Serif
No. of Recommendations: 51
This summer I looked at backtesting BI. What ended up happening is that I got a bird's eye view of its effectiveness, but no hard numbers that I can use, or for that matter, disseminate. Here's what I did:

Compustat has figures for the relevant parameters since 1961. Thus, I collected all the ROE, BV and price information for the entire NYSE/NASDAQ/AMEX stock market universe from 1961-2000.

Next, Compustat also has a fiscal year end (FYR) for every company. Most FYRs are 12 (December). This means that a 1970 BV or ROE # was published in December, 1970 and released to the public in April 1971 (4 months after the stated publication date of the Annual report. Thus, I lagged all the figures by an appropriate amount, somewhere between 12 and 20 months - most companies were lagged 16 months as most companies had December FYRs.

With all this data, I calculated 1-10 year BI #s for over 2 million company / months. A company / month is a single company on a single day. Thus, for example, I have IBM BI #s for 1 year BI, 2 year BI, ..., 9 year BI, 10 year BI starting in April 1963 (1 year only), May 1963, ..., April 1972 (1-10 year BI), May 1972, June 1972, ..., November 2000, December 2000.

Then, I started chunking this data. Every company / month got assigned a Market Cap # relating to which group of 100 it fell under in terms of market cap. Thus, Microsoft in the late 1990s was a 1, while Walmart (WMT) was an 8 in its debut in Jan 1974, moved up slowly to 7 and 6, hit 5 in 1978, 4 in late 1979, 3 in late 1980, 2 in late 1981, and has been part of the 100 largest companies by market cap since late 1982. I hope this idea of Market Cap # in terms of hundreds makes sense.

At this point in time, Ken told me that I should be using next year's VL estimates for ROE and BV and not historical numbers. We checked the difference on the 30 Dow stocks that he has followed real time since 1990 or so and it turns out it makes a HUGE difference to use the estimates - both in terms of results and optimal holding period.

At this point in time, I gave up on the backtest for the first time because I figured that the numbers weren't going to be any good.

However, I then decided to at least examine the numbers to see if any conclusions could be made.

Thus, I took the 5 year benchmark data and the 10 year benchmark data and organized it by % below downside target. I grouped this data by A) Market Cap #, and B) chunks of 5% in terms of "% below downside target." Thus, I had one row for:

o top 100 market cap, 0-5% below downside
o top 100 market cap, 5-10% below downside
o top 100 market cap, 10-15% below downside
o top 100 market cap, 15-20% below downside
o 2nd 100 market cap, 0-5% below downside
o 2nd 100 market cap, 5-10% below downside
o 2nd 100 market cap, 10-15% below downside
o 2nd 100 market cap, 15-20% below downside
I then aggregated my calculations for returns for holding periods of 1, 2, 3, 4, 6, 12, 18 and 24 months for all the companies in each group.

The results looked something like this:
%ile	MCAP	# COs	1mth	2mth	3mth	4mth	6mth	12mth	18mth	24mth
45 2 242 39.8% 32.2% 31.8% 34.9% 32.5% 23.6% 20.6% 15.7%
50 2 265 34.0% 27.1% 24.3% 28.9% 28.2% 20.7% 18.9% 16.1%
This shows that from 1961 to 2000, in the group of the 2nd biggest hundred market cap companies (MCAP), there were 242 company / months (#COs) where the company in question at that time was trading between 45 and 49.99 (%ile) OF its downside target (e.g. downside = $100, current price $45). These 242 companies had an average ARITHMETIC 1 month return of 39.8%, an average ARITHMETIC 2 month return of 32.2%, etc.

I then used conditional formatting to highlight the average returns that were above 20%, with the colors turning brighter and brighter as the returns went up. Subsequently, I visually inspected the grid of every group of top, 2nd, 3rd, etc. 100 market cap companies and drew a box around what I saw as the sweet spot.

The result of this exercise was that I discovered, and shared with Ken, a best investment area of 40-70% below downside target (e.g. downside = $100, Current Stock price = $30 - $60. This worked for all 15 baskets for the top 100, 2nd 100, 3rd 100, ..., 15th largest group of 100 companies. In other words, more or less for the entire Value Line 1700 universe, not including the bottom 20% of companies. There was no real difference between segments, except for the fact that the returns gradually went up 3-4% per year going down to the 5th largest group of 100 companies. Following this gradual decline, the returns stayed pretty much constant through to the 15th largest group of 100 companies.

This is how the 45% - 70% below downside target came about.

Here is what's wrong with my efforts:

First of all, you can THROW OUT the holding period information.

When you go from historical numbers (e.g. 2000 BV and ROE for a 2001 investment) to forward looking estimated (e.g. 2001 Value Line estimated BV and ROE for a 2001 investment), the returns are better but take longer to come.

I don't know how much longer and I can't make any concrete statements because I haven't done the backtest.

Second of all, you can't come up with any version of a "backtested" CAGR because I haven't done that yet, nor do I plan to, yet. All my numbers are ARITHMETIC averages and do not take into account when the company / months happened. Thus, all 242 occurrences described above may have happened in two or three years, leaving 35+ years without any company / month in that small category.

Why haven't I done this?

Well, I want to backtest Ken's version of BI, not my version. For that I need estimates, and those are harder to come by. If someone would like to crunch this data for me, let me know.

What's next for me?

I have 4 CD-ROMs worth of VL company sheets from 1997 to the present in PDF format and scanning software which should be able to extract the relevant estimates. The software is very good (OmniPage Pro 11) but tricky to use, so it's taking a while to get to do all the tricks I want it to.

Once I do this, I will be able to do a Jan 1997 - Present backtest of the VL 1700 for all of the above holding periods. This will give me some sense of what's the best holding period and/or market cap range.

Beyond that, the only thing that will be left is to scan in volumes of print VLs and let the scanning software go to work. My 1997 - Present project will probably take a few hundred hours of CPU time as accurately scanning PDFs with a 2 1/2 year old computer is a time consuming process relative to the # of PDFs I'm going to be looking at (thousands of PDFs per year for 4+ years). To extend this prior to 1997 will involve massive amounts time and energy to turn the print VL data into an electronic version.

That's a summary of what I've been up to.

Peter Kuperman
Print the post  


When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and Glassdoor #1 Company to Work For 2015! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.