No. of Recommendations: 1

Hi. Most retirement/investment calculators that I run across use either a "Monte Carlo" type system (Keep flipping the coin. Who knows what numbers they're using) or alternately will make you input what *you think* returns will be and what *you think* inflation will be, and things like that.

Are there any *other* calculators that work the same way FireCalc works? Straight-up, year-by-year success rates, using the numbers i.e. bonds, stocks, interest rates, inflation, as they actually happened, ripped from history's headlines?

No. of Recommendations: 7

You may not understand how Monte Carlo (at least the good ones) calculators work. They look at history and record the periodic changes. Next they rank order the changes by size. If the changes are daily for 100 years the number of changes is about 25,000.

At this point the Monte Carlo simulation for Trail #1 happens. The computer randomly picks a change for for the first time period (lets say a day). That random choice tells what happens for day #1 of trial #1. Next the computer picks a change for day #2. Note each pick to completely random. i.e. the change picked for the any day is not removed from the set of changes. This process continues until the first trial has been completed. Either the portfolio has money at the end or or does not.

Next another trial is run -- usually the number of trials is many thousands.

If portfolio plan "A" has 4% of its trials run out of money before 30 years some people will favor it over plan "B" which has a 10% incidence of running out of money. Clearly plans that don't run out of money have a residual and the residual size varies both based on the randomly selected daily changes and on the portfolio management plan.

Some individuals have taken their plans and run these against historical data. A well known example is the work of William Bengen who tested against various portfolios comprised of the S&P500 and the All Bond Index. Bengen's work showed the worst time to retire since the mid 1920s was not June 1929, but rather almost any time in 1968 or 1969. Unless you think the future market movements will mirror history, this approach may not be the best predictor.

No. of Recommendations: 0

*Are there any other calculators that work the same way FireCalc works? Straight-up, year-by-year success rates, using the numbers i.e. bonds, stocks, interest rates, inflation, as they actually happened, ripped from history's headlines?*

I recall Vanguard providing me with simulation results when I first started accounts with them several years ago. They didn't run full historical simulations, like fireCalc, but they did run historical simulations with start dates of several key years (starting years around 1929 and in the 70's). These are the years that establish worst case in historical simulators so the Vanguard simulations provided indications of whether your investment plan and spending would survive and for how long. It would be the equivalent of running FireCalc with only 8 to 10 start dates, but using only the start dates that tend to be worst case. When you run the entire historical simulation set like FireCalc, you get probability information (ie. historically 95% of the time this portfolio would survive 30 years) but if you think about it, that's not really a probability that matters to you at any given time. The fact that the US economy has had more years that would be good times to start retirement than bad is interesting, but that ratio of good years to total years is not really instructive or useful.

No. of Recommendations: 4

*Note each pick to completely random. i.e. the change picked for the any day is not removed from the set of changes.*

This is a critical weakness of using Monte Carlo analysis for investment performance.

First, if you look at the distribution of historical stock performance, bond performance and inflation, they are not nice gaussian distributions or easily described with analytical functions. This is important in how the random number generator converts its results to actual data.

It is not impossible to describe the actual historical distributions with a numerical function description and then convert the random number to an appropriate choice for rate, but I have yet to see a financial Monte Carlo simulator that actually does that.

Second, the performance of stocks this year is not a random event uncorrelated to bond performance, to inflation, or even to stock performance in previous years. Those correlations, however, are virtually impossible to quantify. I am aware of Monte Carlo simulators that attempt to include at least some of the correlations by only choosing stock, bond and inflation numbers that actually occurred in a single year sometime in history, yet randomly assemble the years. So the random number generator serves to identify which year to use, then selects another and another until 30 years worth have been listed. This still doesn't account for the correlations of performance this year to previous years. So you could simulate 1993 immediately following 1929, for example.

No. of Recommendations: 0

Who was it that said "Most of history is bunk"?

CNC

No. of Recommendations: 2

*Second, the performance of stocks this year is not a random event uncorrelated to bond performance, to inflation, or even to stock performance in previous years. Those correlations, however, are virtually impossible to quantify. I am aware of Monte Carlo simulators that attempt to include at least some of the correlations by only choosing stock, bond and inflation numbers that actually occurred in a single year sometime in history, yet randomly assemble the years. So the random number generator serves to identify which year to use, then selects another and another until 30 years worth have been listed. This still doesn't account for the correlations of performance this year to previous years. So you could simulate 1993 immediately following 1929, for example.*

This is what I meant about Monte Carlo calculators when I said "Keep flipping the coin. Who knows what numbers they're using...?" Yesterday's "random event" (if it really *was* random) is part of what makes today what it is and not one of an infinite number of other things it could have been. Monte Carlo calculators seem to just shake the bag and mix up the juxtapostioning. If they intervene with some tactic e.g. we've eliminated 1929 stock market performance from occurring at the same time as 1979's inflation" then it's not really random. And what about the 1,000 other "tweakings" that might have been better, more relevant, that they didn't think of?

Monte Carlo calcultors might be good as a sort of "stress test" beyond what previous reality has thus far whipped up

No. of Recommendations: 1

As regards FireCalc: I thought it was one of the more valuable tools (although certainly not the only one) when it came to answering, how close am I to having enough to retire reasonably safely?

(Anyone who’s asked that question knows that it’s a deceptively complex question, with data-driven opinions drawing almost every possible conclusion)(many of which rapidly approach There Is No Safety This Side Of The Grave)

But as regards FireCalc specifically, my most valuable takeaway was that with any reasonable set of starting parameters - those that gave the historical failure rates of say less than 5% - those 20- or 30-year failures were virtually always those portfolios with a miserable first three years or so.

This reinforces the importance that *sequence *of returns is one of the most valuable criteria, but from a pragmatic perspective it helps in this way: if, say, 36 months from your last paycheck your investable net worth is appreciably higher than it was at the beginning, most of the remaining uncertainty has gone away e.g. a hypothetical historical failure rate of 5% is now 1-2%.

That’s my only addition to this conversation.

But I do have a question: **aside **from the “history tells us nothing” argument: what are the criticisms of FireCalc as a tool? Methodology, math, calculation errors?

- sutton

loquacious this week

No. of Recommendations: 1

*But I do have a question: aside from the “history tells us nothing” argument: what are the criticisms of FireCalc as a tool? Methodology, math, calculation errors?*

- sutton

I have heard there are some glitches. Minor. Mostly display type problems. Nothing that affects actual survival rates.

Also, I have heard that the way it calculates bond values for rebalancing is seriously outta whack. There is a similar calculator, FireSim or something, that supposedly uses more realistic bond rates and the safe withdrawal rate plummets. I am not in a position to substantiate any of this. This is just parts of the discussion over the years. Also, the FireSim site seems to have been left to go to seed. Hmmm....? FireCalc has also been left in a quasi-zombie mode. It gets updated every year with CPI, market performance, etc but that was only after several years of non-updating and the retired owner was sort of hunted down and encouraged to update it.

No. of Recommendations: 1

*But I do have a question: aside from the “history tells us nothing” argument: what are the criticisms of FireCalc as a tool? Methodology, math, calculation errors?*

The problem as many have said is correlations. Stocks, bonds, and inflation are all correlated over time and are correlated with each other. The correlation is poorly understood. So if you want to avoid the problem of correlations you need to use **non-overlapping** time periods. cFIREsim uses data from 1871 to today. There are only 5 non-overlapping 30 year periods since 1871. If the correlations extend over several years then it gets even worse, you need to leave a gap between the 30 year periods for them to be completely independent. Yes, there are 118 start years but there is a lot of correlation between the 30 years from 1970-1999 and 1971-2000. It sounds a lot more comforting to learn that a plan was found safe over 118 30 year periods than to learn it was safe over only 5 30 year periods, but 5 independent 30 year periods is all you get.

You could play games, assume that the correlation goes away after 10 years, and then you get 15 periods, which is better. But now you're assuming something about the unknown correlations in an ad-hoc way.

The advantage of a Monte Carlo simulation is that the assumptions about correlations are explicit in the code, even though many websites don't tell what they've assumed. In principle, you can play around with the correlation assumptions and see how the results change.

No. of Recommendations: 0

* It is not impossible to describe the actual historical distributions with a numerical function description and then convert the random number to an appropriate choice for rate, *

Prove it. Find that function.

Second, the performance of stocks this year is not a random event uncorrelated to bond performance, to inflation, or even to stock performance in previous years. Those correlations, however, are virtually impossible to quantify.

Which means they are random, as far as we know.

I should qualify that things are random until you discover a pattern. If I gave you digits 1000...1999 in the currently-known decimal expansion of pi, you would be justified in thinking they are random, though I could predict the next digit with perfect accuracy. But, unless you know the pattern, claiming that there is one, is just so much hot air. As someone (Henry Ford?) said, "no one is famous for what they are going to do in future."

No. of Recommendations: 3

I question that 5 non-overlapping periods would be even vaguely sensible as a basis since it would mean 5 rather arbitrary starting years vs checking each of 118 starting years.

I would think that the biggest criticism of the FireCalc approach is that there has be gradual secular change in how the market behaves so that the early starting years may not be indicative of future behavior ... but they are better than nothing.

No. of Recommendations: 7

*Prove it. Find that function. *

You could use a piecewise function of your choice to match the historical distributions. That's not the hard part. You simply plot the historical distribution curve and fit a line to it. A linear fit, a spline fit. . . anything can work. I've just never seen a Monte Carlo retirement investment simulator actually go to the trouble.

*Which means they are random, as far as we know.*

No. It doesn't mean that at all. We know that stocks and bonds, for example, are generally inversely related. We know that inflation and stock earnings tend to be correlated. But obviously there are other factors. The fact that we cannot identify all factors that impact these rates in a quantifiable way does not mean the correlation doesn't exist. It only means we don't have all the variables identified and quantified. The rates are still correlated in some manner.

No. of Recommendations: 0

*You simply plot the historical distribution curve and fit a line to it. A linear fit, a spline fit. . . *

Do you really not see the logical next step of this argument? If different functions can fit the same data then extending them in the future will give different results, which means none of them are particularly predictive, so what is the point of using them?

Overfitting, aka data mining, is frowned upon for a reason. If your function does not perform well out of sample then it is completely useless in prediction.

No. of Recommendations: 8

*If different functions can fit the same data then extending them in the future will give different results, which means none of them are particularly predictive, so what is the point of using them?*

That's not correct. Different descriptions of the future - different curves fit to historical data - can definitely be useful even if they are not perfectly predictive. As a trivial example, the very long term trend of broad stock market returns is something like 6% plus the current inflation rate. *** Are the market returns for the next year going to be exactly that? I'd wager an awful lot of money that they won't be exactly that. But that doesn't make the prediction useless. Given no other information about the current situation other than the inflation rate, you could plan on the market going up by 6% plus inflation year after year. You'd be high about 1/2 the time and you'd be low about 1/2 of the time. And that becomes a useful way to think about the long-term future.

--Peter

*** - I'm going from memory here. Please feel free to substitute the actual correct statistic for my poor recollection of the figure. The lack of accuracy in the figure does not affect my argument.

No. of Recommendations: 2

*Do you really not see the logical next step of this argument? If different functions can fit the same data then extending them in the future will give different results, which means none of them are particularly predictive, so what is the point of using them?*

Overfitting, aka data mining, is frowned upon for a reason. If your function does not perform well out of sample then it is completely useless in prediction.

You really don't know much about math . . . or investment/retirement simulators. If you are hoping for analytic predictive capability it doesn't matter how well you match the historical distribution, you won't be able to predict the future. But that's not the point. If you are able to approximate the historical distributions of the past, then you should at least be able to gain general understanding of how a portfolio might perform under conditions consistent with history.

The specific numerical function approximation you choose will have insignificant impact on the final answer provided your approximation is kept to low delta error. For example, if you are trying to compute the area under an arbitrary curve, you can get arbitrarily close to the exact answer with rectangular or trapezoidal summations - or with spline fits. You just need to pay attention to the details.

No. of Recommendations: 0

* You really don't know much about math . . . or investment/retirement simulators. *

That's very true.

* If you are hoping for analytic predictive capability it doesn't matter how well you match the historical distribution, you won't be able to predict the future. But that's not the point. If you are able to approximate the historical distributions of the past, then you should at least be able to gain general understanding of how a portfolio might perform under conditions consistent with history. *

Now this "general understanding" - what is that? It's great if you get an idea of how the portfolio has performed historically; but then what? Are not assuming it will do the same in future (which is the point of Monte Carlo trials - to predict several future paths)? To do so, you do have to predict, say, next year's returns (along with possible dispersion), do you not? And then for the year after that, and so on.

* The specific numerical function approximation you choose will have insignificant impact on the final answer provided your approximation is kept to low delta error. For example, if you are trying to compute the area under an arbitrary curve, you can get arbitrarily close to the exact answer with rectangular or trapezoidal summations - or with spline fits. You just need to pay attention to the details. *

Oh, is that what they call "integral"?

Back to your original assumption that this year's (or take any other period) returns are somehow a function of something - returns and dispersions from recent past for various asset classes as you assume - do you have any actual data to back it up?

The little math I know says that, despite the large dispersion (and corresponding low confidence), a simple average of past yearly returns is still the best (though not a good) predictor of the next year's returns. Let me know how more complexity helps.

No. of Recommendations: 9

*The little math I know says that, despite the large dispersion (and corresponding low confidence), a simple average of past yearly returns is still the best (though not a good) predictor of the next year's returns.*

I was reading recently that the reason we have gotten so much better at forecasting the path a hurricane will take is that they are doing a significant number of forecasts and plotting out the envelope of possibilities. Chances are that any specific hurricane won't follow exactly the path of the center of that envelope, but that it is highly likely that its path will be within the envelope.

This seems to me to be the purpose of a tool like FireCalc ... not to predict the exact, specific future of what will happen, but rather to project a probability envelope for what might happen based on past history. The projection is not that one will have $X in 30 years, but that of the probable outcomes in 30 years, only N% of them correspond to all the money being gone.

No. of Recommendations: 6

*The little math I know says that, despite the large dispersion (and corresponding low confidence), a simple average of past yearly returns is still the best (though not a good) predictor of the next year's returns. Let me know how more complexity helps.*

The complexity lets you see the range of what could happen. If you believe that the average is the best predictor, then you would presumably agree that history can help to predict the future. But since the average is just that, an average, it doesn't show the variability (ups and downs) that may occur.

The more complex models, like FireCalc and Monte Carlo analysis, show you what results may occur when that variability is considered. They DO NOT claim to predict next year's returns. What DO they claim to do (and do a pretty good job of) is to predict the range of returns you may get over a particular timeframe. This is helpful if you are looking for an estimate of how long your portfolio will survive. It's not so helpful if you are looking for a prediction of next year's returns.

AJ

No. of Recommendations: 0

*Back to your original assumption that this year's (or take any other period) returns are somehow a function of something - returns and dispersions from recent past for various asset classes as you assume - do you have any actual data to back it up? *

Correlation is a mathematical measurement of the similarity of any two series. A correlation coefficient can be calculated between any two series. Look it up or google "correlation coefficient" if you want to see how it works.

No. of Recommendations: 4

There are a lot of issues floating around here, and I certainly don't want to scare anyone off. https://slate.com/news-and-politics/2013/07/warm-weather-hom...

* The New Orleans Times-Picayune ran a piece last Friday attempting to answer a question the entire world has been asking: Should ice cream be blamed for murders? “The correlation between homicides and ice cream sales—when ice cream sales increase, the rate of homicides also increases—has long been a topic in statistics and science classrooms,” writes John Harper, citing several recent cases of ice cream-related crime.*

In the second paragraph of his piece, Harper thankfully reminds readers that correlation is not causation, and that ice cream’s relationship to homicide is a mere statistical coincidence. The idea that frozen treats cause crime is obviously ridiculous, unless you’re talking about that addictive Cocaine Chip ice cream I’ve heard so much about. But it does stand to reason that ice cream sells better in warm weather, and there is in fact plenty of evidence to suggest that murder rates rise when temperatures rise.

Using a variety of statistical methods, including transformations (square root, logarithmic, etc); different regression/correlation methods (simple linear, multiple regression, principal component analysis); and things like careful selection of dependent and independent variables, I can force pretty gnarly data to fit some sort of predictive relationship. Anyone who doesn't believe that has never been around a statistic driven project in need of demonstrating a statistically meaningful outcome. So correlation coefficients don't really impress me.

The first question you should ask when viewing statistics is "Is the data normally distributed?". That is going to tell you a lot about appropriate methods. Another good question is, does the thing I'm trying to predict fit within the previously observed outcomes? Assuming a normal distribution, does it fit within 2 standard deviations of the mean? What happens to the confidence interval as I get outside the previously observed outcomes or away from the mean?

Predicting what the market will do in the next quarter or year is challenging enough, regardless of the statistical method used. If you had told me last July we would be looking at a rate cut this July, I would have laughed at you. If it wasn't hard, and subject to a lot of randomness, then there would be a lot more rich people, right? Monte Carlo analysis and simulators like FIRECalc are intended to help by handling longer term. Monte Carlo simulation is often used when simple statistics don't work well, or suffer from things like the previously described sequencing risk.

For those using something like FIRECalc, sequencing risk is somewhat mitigated if you follow the advice to have 2-3 years of cash needs in cash or a near cash equivalent.

For folks like me, further from retirement, the big number to keep in mind is the annual inflation adjusted stock market return/Compound Annual Growth Rate (CAGR). Right now, I don't care if the SWR is actually 3.8, 4.0 or 4.2% because I'm far enough from retirement it has little effect on what I need to do. I do care of the CAGR over the next ten years is 6%, 7%, or materially higher or lower. If its a lot lower than 6%, I'm either going to have to work longer, or come up with a different (riskier?) asset class that will do better.

No. of Recommendations: 3

*In the second paragraph of his piece, Harper thankfully reminds readers that correlation is not causation, and that ice cream’s relationship to homicide is a mere statistical coincidence. *

Sometimes knowing a little about math can mislead. Although correlation is not causation, when two series are closely correlated that is almost always an indication that the two series share at least some common causes. This can sometimes be revealed through rigorous application of principle component analysis although that still requires that someone first hypothesize the correct causal force be considered.

This is important when evaluating the strengths and weaknesses of the Monte Carlo investment simulations and how correlations between input variables is accounted for. In the case of ice cream and homicide rate, there are many possible shared causes that might be at play here. And while it would take a massive amount of research to investigate which causes are shared and how they influence each result, the important thing to remember is that the correlation does truly exist. The data shows that. If a simulation that requires both crime rate and ice cream sales as inputs assumes that these two can be modeled with independent random number generators, that correlation will not be captured in the results. And if those correlations are not captured in the results, then the results are not reflecting reality.