No. of Recommendations: 4
PEG26 performance variations

This is an update on the performance of the PEG26 screen going forward from 10/30/98 (just after the PEG screen was finalized). These portfolios track the performance of the PEG screen as published in TMFElan's weekly Rankings post. Note that until July 1999 everyone was using an incorrect filter in the screening process. Kuperman discovered this in the process of verifying Moe's research and expanding the PEG backtest to all months.

I am running this study by assuming a $2000 buy on each stock. The number of shares is determined such that the allocation is not exceeded. Weekly price data is downloaded from Yahoo. Splits and dividends are included. Buys are made at the closing price of the Monday following the Friday posting of the Screens. Sales are made 1 year later.

Data for a 1 year holds:

Date Published 5 Stock port 4 Stock port 3 Stock port
10/30/98 86.17% 79.49% 20.83%
11/06/98 16.91% 25.56% 25.56%
11/13/98 37.74% 51.04% 57.45%
11/20/98 39.68% 54.87% 41.28%
11/27/98 67.08% 52.94% 63.00%
12/04/99 29.03% 39.64% 21.45%
12/11/98 26.33% 16.26% 24.04%
12/18/99 29.88% 10.88% 19.48%
12/25/99 86.38% 35.58% 12.81%
01/01/99 19.97% 3.72% -16.89%
01/08/99 13.41% -5.88% -19.72%
01/15/99 106.91% 78.72% 24.22%
01/22/99 51.73% 50.33% -6.50%
01/29/99 19.70% -6.22% -10.79%
02/05/99 229.01% 64.02% 40.36%
02/12/99 59.20% 60.31% 43.78%

Averages and standard deviations for each set:

stdev 53.74% 28.06% 24.99%
average 57.45% 38.20% 21.27%

As you can see, it was a good quarter to start 5-stock portfolios.

I am currently looking into the positional returns for this data.

George

Ps: In case anyone was wondering the returns for 2/4/99 were driven by UNPH/JDSU with a greater than 800% return.


Print the post Back To Top
No. of Recommendations: 2
Warning, newbie post, but....

Good grief, at times (e.g., January to early February), these numbers seem darn near random! So a 5 stock port started 2/05/99 returned 229%, while a 4 stock port started one week earlier returned -6% ?! Ouch!

Don't numbers like these make others feel less confident about this strategy? Or is my inexperienced eye playing tricks on me?

Do the other screens show similarly dramatic variations?

Thanks for any feedback and encouragement (for a new investor who has recently invested in PEG stocks!),
TGO

P.S. Thanks to schamfool for this data.

Print the post Back To Top
No. of Recommendations: 1
Good grief, at times (e.g., January to early February), these numbers seem darn near random! So a 5 stock port started 2/05/99 returned 229%, while a 4 stock port started one week earlier returned -6% ?! Ouch!

Don't numbers like these make others feel less confident about this strategy? Or is my inexperienced eye playing tricks on me?

Do the other screens show similarly dramatic variations?

Thanks for any feedback and encouragement (for a new investor who has recently invested in PEG stocks!),
TGO



TGO,
To my eye all of these returns are within the expected bounds, with the possible exception of the 229% for the week of 2/5/99. Our sample size is just too small to pu much faith in the GSD's as true and absolute numbers. In the course of my real job I've done too many mulit-variable Monte-Carlo simulations that had the mean targeted early (less than 100 cases) but the distribution was not well defined until upwards of 700 cases. I am not expert in statistics ut I have had close encounters with small sample theories, and it doesn't seem to me that we can say with 50% confidence what the 1,2, or 3 sigma levels for these screens are.

This is why I am tracking these results on each weekly screen selection. Why within 3-4 years we might know what the 1 sigma level is ;-} .

George
Print the post Back To Top
No. of Recommendations: 0
Our sample size is just too small to put much faith in the GSD's as true and absolute numbers...I have had close encounters with small sample theories, and it doesn't seem to me that we can say with 50% confidence what the 1,2, or 3 sigma levels for these screens are.

George-

Thanks for bringing this up. It's very important to recognize the folly of comparing minor differences in CAGR and GSD, lest we end up with egg on our faces like the Foolish Four. This real-life example shows the danger of backtesting leading us down false paths, possibly steering us toward bad screens and away from good ones by the luck of the draw in gathering the historic monthly picks and their prices. I also suspect that the practical results from MI will be more random than the backtests show. I'm still giving MI a try, but I'm also willing to acknowledge that it's not unlike walking through a mine field to get to the pot o' gold.

-another George
Print the post Back To Top
No. of Recommendations: 1
This information just reinforces the notion that there is an element of luck in your personal returns depending on which week you happen to invest, especially in a screen such as PEG in which the rankings will vary considerably from week to week.

H2OBURY
Print the post Back To Top
No. of Recommendations: 0
This information just reinforces the notion that there is an element of luck in your personal returns depending on which week you happen to invest

At least in the short term. In the long term, I would guess it would average out.

Brian
Print the post Back To Top
No. of Recommendations: 0
<<...there is an element of luck in your personal returns...>>

At least in the short term. In the long term, I would guess it would average out.


It's highly possible to have a few "lucky" or "unlucky" years in a row when your strategy is to just buy a small batch of stocks for a year and forget about them. That's why frequent rebalancing can add stability, just like a broad portfolio. The more rebalances, the less likely one is to have "bad luck". It's unfortunate that the IRS can't appreciate that point of view; so for annual portfolios, our only hedge against bad luck is owning more than just a few stocks.
Print the post Back To Top
No. of Recommendations: 6
TGO writes:
 Good grief, at times (e.g., January to early February), these numbers
seem darn near random! So a 5 stock port started 2/05/99 returned 229%,
while a 4 stock port started one week earlier returned -6% ?! Ouch!

Don't numbers like these make others feel less confident about this
strategy? Or is my inexperienced eye playing tricks on me?

Do the other screens show similarly dramatic variations?



After seeing similar numbers on the Foolish Four:
http://www.fool.com/ddow/2000/ddow000210.htm

I also wondered how various screens perform week-to-week as well as day-to-day? 

If a screen has such large variations as the FF on a daily basis, this leads me
to wonder about the backtest accuracy (is it luck, datamining, or a good sample
of expected results?)

Anyhow, here are the numbers for Spark, RS-26, PEG starting in Dec 98
(various dates)and holding for 1 year. The thing I noticed right off is that
Spark does not change the underlying stock selections often, and these are not
volatile stocks. This gives Spark a more stable performance than the others


Start Date   Spark1-5 Spark2-5 PEG1-3 PEG1-4  PEG1-5 RS26 1-3 RS26 1-4 RS26 1-5   SPY    QQQ
12/4/98         88.4%	97.8%	34.8%	53.5%	40.7%	188.2%	168.5%	152.8%	22.7%	94.6%
12/7/98	        88.1%	99.4%	30.6%	48.0%	36.2%	194.6%	171.0%	151.1%	21.2%	90.5%
12/8/98	        87.5%	98.9%	27.2%	44.9%	33.0%	211.1%	182.9%	161.5%	20.8%	91.5%
12/9/98	        85.9%	98.0%	26.0%	43.6%	32.3%	202.0%	175.6%	154.6%	19.8%	87.2%
12/10/98	88.0%	99.8%	23.6%	38.7%	27.9%	224.7%	189.5%	166.8%	22.2%	91.8%
12/11/98	83.4%	94.3%	36.3%	23.7%	34.5%	213.3%	155.1%	139.6%	22.3%	90.8%
12/14/98	90.9%	100.9%	37.4%	25.1%	34.3%	219.8%	160.4%	142.6%	26.2%	100.7%
12/15/98	80.1%	87.7%	28.9%	19.3%	29.2%	196.2%	142.6%	127.9%	21.8%	91.2%
12/16/98	84.9%	90.5%	27.2%	18.7%	28.5%	186.5%	135.8%	122.1%	22.6%	95.8%
12/17/98	88.6%	93.4%	23.8%	16.0%	26.3%	182.3%	133.8%	120.5%	21.2%	96.6%
12/18/98	83.9%	88.1%	24.0%	16.2%	26.3%	174.3%	196.6%	184.9%	21.6%	93.2%
12/21/98	80.1%	84.9%	21.2%	12.7%	22.3%	151.4%	174.3%	165.1%	19.1%	90.0%
12/22/98	83.1%	87.0%	24.4%	14.1%	25.9%	150.1%	177.5%	169.7%	20.3%	102.4%
12/23/98	78.0%	81.5%	20.9%	11.4%	29.2%	158.5%	183.1%	174.6%	18.2%	96.1%
12/24/98	85.7%	90.7%	19.5%	10.9%	29.9%	156.9%	183.6%	176.4%	20.5%	99.8%
12/31/98	93.8%	100.2%	-6.5%	11.1%	29.6%	132.5%	109.3%	98.7%	20.2%	99.1%
1/3/99	        89.2%	95.2%	-5.2%	16.6%	36.1%	147.0%	120.7%	108.6%	19.3%	104.4%
1/4/99	        70.9%	75.2%	-15.2%	7.2%	22.0%	129.4%	106.6%	101.9%	13.4%	85.5%
1/5/99	        67.7%	72.0%	-14.2%	3.4%	19.5%	102.0%	84.5%	86.1%	11.0%	75.2%
1/6/99	        57.6%	60.4%	-18.9%	0.4%	18.2%	81.1%	68.5%	72.7%	9.7%	62.9%
										
										
Mean:	        82.8%	89.8%	17.3%	21.8%	29.1%	170.1%	151.0%	138.9%	19.7%	92.0%
Std Dev:	8.7%	10.8%	18.2%	15.6%	5.8%	39.3%	36.8%	32.5%	4.0%	9.5%
Mean -2std	65.4%	68.3%	-19.1%	-9.3%	17.4%	91.6%	77.3%	73.8%	11.7%	73.0%
										
Dec Backtest:	62%	69%	36%	65%	68%	168%	164%	146%	23%	95%
Jan Backtest:	80%	84%	-6%	16%	36%	147%	121%	117%	20%	99%
Print the post Back To Top
No. of Recommendations: 0
This information just reinforces the notion that there is an element of luck in your personal returns depending on which week you happen to invest

At least in the short term. In the long term, I would guess it would average out.


Presumably yes. A similar kind of averaging occurs across screens. The hypothetical PEG ports that did poorly because, say, they missed QCOM or VISX, would not have felt so bad if the owner was also in one of the RS monthlies. All the screens will turn up their share of dogs. That's just part of the deal. PEG is unique in allowing you to miss stocks that are appreciating rapidly.

Baltassar
Print the post Back To Top
No. of Recommendations: 0
Geocorona writes:
for annual portfolios, our only hedge against bad luck is owning more than just a few stocks.

Actually, looking at the daily data for a screen, one way to hedge is to spread your purchases out over a few weeks or months. This approximates numerous rebalances within a year (all run in parallel) and has the effect of averaging out the daily/weekly variances.

-D
Print the post Back To Top
No. of Recommendations: 0
DNeumann,

I thought your chart was fascinating!!! Left me hungering for more...!

I noticed a couple of things that I wanted to comment on/ask about. At the bottom of your chart, you list returns for "Dec Backtest" and "Jan Backtest". These numbers seem to have come from Gritton's backtester. In many cases, the "Jan Backtest" figures match your picks from 1/3/99, in some cases they match your picks from 12/31/98, and in some cases they don't quite match either of those dates. Leaves me wondering a bit about the discrepancies, but also seems to further drive home the point about not taking backtested returns TOO literally. Any thoughts?

(Also, very minor observation, but you showed every date in the time frame except 12/28-12/30/98. Not that this info would have added much, but was the omission inadvertent?)

Thanks very much for your information,
TGO
Print the post Back To Top
No. of Recommendations: 0
TGO,

The numbers at the bottom of my chart do come from Gritton's backtester. The only exception are SPY and QQQ, which I pulled the 12/31 numbers (as you point out). For various reasons, my numbers do not always exactly match Grittons backtester. For example, in the Spark screen for January, Gritton's tester includes MSFT, HD, DELL, NOK, VOD, while the rankings archive indicates MSFT, DELL, NOK, EMC, GDT. I am not sure which is correct, I went with the ranking archives provided in:
http://www.fool.com/Workshop/1998/WorkshopRankings981231.htm

One of the hazards of backtesting is the inconsistency of data.

Regarding the gap from 12/28/99 - 12/30/99, I inadvertently left this out when I created my spreadsheet, and caught it after everything was put together. Just too lazy to go back and put it in. It met my goal without these dates, so I left it out...Oh well.

I suspect the numbers still convey the same message: if you are putting a lot into PEG or RS-26 I would seriously consider spreading out the purchases over a few weeks to 'blend' start dates. Your results should approach the average expected from the screen for that timeframe.

-D

Print the post Back To Top
No. of Recommendations: 0
For those of you who looked at the stocks that make up this different weeks, did you notice anything out of the ordinary for those week's picks that had poor returns? For example, was it full of freshman picks? Did all the old picks fall away during the next week? (meaning the bad picks were going off the list), Was it cause by one really poor stock?


Thanks for this intriguing information,
Richard
Print the post Back To Top
No. of Recommendations: 0
At least in the short term. In the long term, I would guess it would average out.

It also would be a good idea to run multiple screens at the same time to increase your chances of getting a good selection.

George
Print the post Back To Top
No. of Recommendations: 0
For those of you who looked at the stocks that make up this different weeks, did you notice anything out of the ordinary for those
week's picks that had poor returns? For example, was it full of freshman picks? Did all the old picks fall away during the next week?
(meaning the bad picks were going off the list), Was it cause by one really poor stock?


Thanks for this intriguing information,
Richard


I'll post the stock table when I get home (late).

George
Print the post Back To Top
No. of Recommendations: 3
Richard asks:
For those of you who looked at the stocks that make up this different weeks, did you notice anything out of the ordinary for those week's picks that had poor returns? For example, was it full of freshman picks? Did all the old picks fall away during the next week? (meaning the bad picks were going off the list), Was it cause by one really poor stock?



First, I want to point that this type of day-to-day variation is probably normal for any screen. Some are more, some are less. We do not really have any good measures of what happens in between the buy-sell points on the screens.

Regarding Spark: This is the most stable. The entire month only saw 1 change in underlying stocks for the 4 weekly listings. The drop off in performance is almost entirely due to the general market drop in Jan 00. (This happened in the last few days of the annual holding period, yet for some stocks wiped out almost half the gains.

Regarding PEG: This screen seems to have more turnover week-to-week in the underlying stocks chosen. Over the 4 weekly listings, only 2 stocks were held the entire time. The decline from Dec start to Dec end is due partly to changes in the underlying stocks. (In the last week, we picked up OMI which dropped 41%) In addition 1 stock which was on the screen all month, ANN started out as a +23% contributor, and by the end of the month was a -41% detractor. The purchase price was increasing through December of the buy month, and the sell price was declining through December of the sell month a year later. This is normal, and can be expected. Throughout the year, this stock did not do much. It suffered heavily the very last day of holding the last day of the year (drop from 34 to 24 in 1 week). (I believe this is why Moe recommends PEG as a semi, it should be a bit more stable)

Regarding RS-26: Similar to PEG, seems to change underlying as often as PEG. Throughout the month only picked 2 stocks repeatedly. In this case, the drops in performance come all in the last week, and are entirely an effect of the general market correction in Jan 00. Some of the stocks (YHOO, AOL) were on a very steep price increase at the time of purchase, and were also on an increase at the time of sale a year later. When the correction hit, it took a lot out of these stocks relative to their price gains across a 1 year hold. Still a normal effect of the screens we use, particularly those with higher growth rates.

Summary: The screens we use have high growth rates. They will not sit still and wait for us to buy them, but rather are likely moving up when we buy them, and moving up when we sell them. Sometimes a market slides backwards right at the sell point, and you take a bit of a loss on the sold stocks, but that means you are buying next years stocks that much cheaper. Market timing is a REAL impact that can be measured in hindsight. But you cannot use this information in real-time. (Like driving your car while looking in the rear-view mirror) In actual application, all we can do so far is treat it as a random variable in our screens and deal with it mechanically. The traditional way to deal with it mechanically is to make one rebalance per year (or month/quarter/semi), then repeat mechanically at the net rebalance time exactly the same. This will average out over the long term, but long term may be many years.

What I have seen here is that another good risk reduction technique is to simply spread out the rebalance times over a few weeks or months. This adds costs in transaction fees, but has a great impact to reduce screen variation, and gives more consistent performance. This minimizes variation in the near term, and has no impact on performance long term. More backtest data (weekly rankings) would confirm this, but the concept is simple enough that it seems intuitive. I will run some numbers on the monthly
backtests, and post if I find anything interesting.

Comments?

-D
Print the post Back To Top
No. of Recommendations: 0
What I have seen here is that another good risk reduction technique is to simply spread out the rebalance times over a few weeks or months. This adds costs in transaction fees, but has a great impact to reduce screen variation, and gives more consistent performance.

It would be nice to have the same backtest available for the past decade or more, but without weekly picks & prices I guess it's out of the question. Assuming that this huge performance variation is more of a total market function than an individual screen function, your strategy would probably work best.

But whether it's spreading out the rebalance period, buying more stocks at one time, or frequent rebalancing that works best, it probably all comes down to the old saying "it takes money to make money"; and small investors will have to take bigger risks to get the same gains as large investors, or pay more fees (relatively speaking) and/or taxes to get stability.
Print the post Back To Top
No. of Recommendations: 0
What I have seen here is that another good risk reduction technique is to simply spread out the rebalance times over a few weeks or months. This adds costs in transaction fees, but has a great impact to reduce screen variation, and gives more consistent performance. This minimizes variation in the near term, and has no impact on performance long term. More backtest data (weekly rankings) would confirm this, but the concept is simple enough that it seems intuitive. I will run some numbers on the monthly
backtests, and post if I find anything interesting.

Comments?

-D


D-

Found your post to be an excellent summary of this discusions and your conclusions insightful. Makes people wonder if they should be starting all their screens on the highest performance day from the past or spreading them around a little more. It also shows why if you have an AMEX or very low commission account running more then one of the same screen with slightly different start dates versus one screen with all the money. Of course this is what D- just said.

Moe
Print the post Back To Top
No. of Recommendations: 0
For those of you who looked at the stocks that make up this different weeks, did you notice anything out of the ordinary for those week's picks that had poor returns? For example, was it full of freshman picks? Did all the old picks fall away during the next week? (meaning the bad picks were going off the list), Was it cause by one really poor stock?


Thanks for this intriguing information,
Richard


Here are the stocks:

10/30/98 ADAC ANN BGEN SUNW EMC
11/06/98 DY ADAC BGEN ANN UFS
11/13/98 DY ALO BGEN ANN GENZ
11/20/98 ANN BBY ALO BGEN UFS
11/28/98 BBY ALO BGEN APCC VTSS
12/04/98 AVI AEOS ANN BBY ALO
12/11/98 AEOS ANN DY ALO BGEN
12/18/98 AEOS ANN DY ALO BGEN
12/25/98 AEOS ANN DY BGEN ORCL
01/01/99 OMI AEOS ANN BBY BGEN
01/08/99 ANN CHTT BBY DY SLR
01/15/99 ANN VISX BGEN ORCL UVSGA
01/22/99 ANN VISX GNTX UVSGA SLR
01/29/99 ANN VISX BBY APCC SLR
02/05/99 VISX BGEN BBY MACR UNPH
02/12/99 AFWY CTS VISX BGEN SLR

and the returns by position:

Peg1 Peg2 Peg3 Peg4 Peg5
10/30/98 -50.98% 34.12% 98.37% 323.49% 158.82%
11/06/98 88.67% -53.70% 88.08% 25.93% -20.84%
11/13/98 79.36% 7.12% 86.21% 32.04% -15.12%
11/20/98 29.53% 100.87% -6.82% 95.69% -22.40%
11/28/98 109.26% -10.04% 90.21% 23.08% 135.21%
12/04/98 -3.86% 61.45% 6.75% 94.20% -13.41%
12/11/98 49.20% 1.73% 21.13% -6.90% 67.00%
12/18/98 47.47% -5.69% 16.21% -8.11% 109.14%
12/25/98 35.08% -12.68% 15.70% 103.61% 289.78%
01/01/99 -35.19% 27.13% -42.71% 66.88% 85.15%
01/08/99 -39.14% -60.16% 40.54% 34.91% 92.29%
01/15/99 -35.42% 10.38% 99.19% 239.99% 218.58%
01/22/99 -42.58% -7.79% 31.01% 218.58% 57.33%
01/29/99 -55.93% -9.08% 33.04% 7.33% 122.11%
02/05/00 -10.67% 109.74% 22.71% 133.75% 876.94%
02/12/00 -15.00% 163.18% -15.45% 109.74% 54.75%

I haven't looked through these in detail and I haven't double checked to be sure the spreadsheet references are correct. So take these with a grain of salt.

George
Print the post Back To Top
No. of Recommendations: 0
I suspect you will find that these are volatility reduction techniques, not necessarily risk reduction techniques. Of course, what do you mean by risk? If it turns out you get a reduced return over x years with reduced volatility, that sounds "risky" since the return was lower.
Print the post Back To Top
No. of Recommendations: 1
I suspect you will find that these are volatility reduction techniques, not necessarily risk reduction techniques. Of course, what do you mean by risk? If it turns out you get a reduced return over x years with reduced volatility, that sounds "risky" since the
return was lower.


Actually, the widely different returns of PEG26 depending on the starting week have brought home to me the significance of volatility as risk. I had it imprinted on my mind that short term volatility doesn't matter much. If I'm going to invest for 10 years, I don't care about the up and down gyrations of my investment during the holding period. All that matters is that over 10 years it will go higher.

The weekly PEG returns show that random chance plays a big role in our ultimate returns. The greater the volatility, the greater the odds that we will miss the average return by a significant amount, to the upside or the downside. If possible, we should avoid the risk that buying this week vs. next week will make a big difference. By diversifying over time we can improve the chance that our return will be closer to the expected average, without sacrificing anything in expected return other than transaction costs.

Elan
Print the post Back To Top
No. of Recommendations: 1
Elan writes:
The weekly PEG returns show that random chance plays a big role in our ultimate returns. The greater the volatility, the greater the odds that we will miss the
average return by a significant amount, to the upside or the downside. If possible, we should avoid the risk that buying this week vs. next week will make a big
difference. By diversifying over time we can improve the chance that our return will be closer to the expected average, without sacrificing anything in expected
return other than transaction costs.


I am of a similar opinion: If I can get the same return with less volatility, then my perception of risk goes down. I guess I do not know if there is any mathematical difference over the very long term, but personally, I would rather pocket the gains sooner rather than later, and in a more consistent manner rather than random chance.

One additional item this brings up is the reliability of our screens. It is clear that the data set we are working off from is quite limited (14 yrs for annuals). What this shows me is that the size of the data set needed to establish confidence in a screen is a function of the daily/weekly variance in the screen. If you have the feeling that the returns from PEG are a bit random, consider that a lot of our backtests are based on 14 points from this sample set. After looking at the various screens, I have much more confidence that the reported CAGR figures in the more stable screens (Spark, Plow, Key) are actually close to realizable numbers, while some of the others I am taking with a larger grain of salt.

This is a VERY REAL aspect of RISK that we should not underestimate. I think that the comment by Bmargolis

I suspect you will find that these are volatility reduction techniques, not necessarily risk reduction techniques. Of course, what do you mean by risk? If it turns
out you get a reduced return over x years with reduced volatility, that sounds "risky" since the
return was lower.


has one big flaw. If you assume that our screens are perfect representations of future gains, then he is correct, and we are not really reducing risk by averaging buys over time.

If on the other hand, you are suspicious about the accuracy of the stated returns of the screens, then there may be a large risk that we are making purchases in something which is unstable.

Overall, I believe the screens choose better than average stocks. I do not know how much better than average, but anything we can do to bring the variance under control means that we likely have a more repeatable process. That IS important for estimating future gains, and guaging where to invest our money.

-D
Print the post Back To Top