A Random Walk Down Wall Street, written by Burton Malkiel in 1973, has become a classic in investment literature. Random walk theory jibes with the semi-strong efficient hypothesis in its assertion that it is impossible to outperform the market on a consistent basis. Malkiel puts both technical analysis and fundamental analysis to the test and reasons that both are largely a waste of time. In fact, he goes to great lengths to show that there is no proof to suggest that either can consistently outperform the market. Any success outperforming the market with technical analysis or fundamental analysis can be attributed to lady luck. If enough people try, some are bound to outperform the market, but most are still likely to underperform.

The basic random walk premise is that price movements are totally random. Judging from the chart, the price movements of Newmont Mining over this 5-month period would appear to be quite random. Prices have no memory, therefore past and present prices cannot be used to predict future prices (as implied in technical analysis). Prices move at random and adjust to new information as it comes available. The adjustment to this new information is so fast that it is impossible to profit from it. Furthermore, news and events are also random and trying to predict these (fundamental analysis) is also a lesson in futility.

Malkiel maintains that a buy and hold strategy is best and individuals should not attempt to time (or beat) the market. Attempts based on technical, fundamental or any other analysis are futile. Admittedly, he does have a point. Statistics have shown that the majority of equity mutual funds fail to outperform the market, as measured by the S&P 500. Investors can easily buy index-based securities with very low transactions costs.

While there are some good points to be gleaned from the random walk theory, it appears to be a bit dated and does not accurately reflect the current investment climate.

Random walk theory was introduced over 25 years ago when institutions dominated the market. These institutions had superior access to resources and the individual was at the mercy of the large brokerage houses for quality research. With the advent of online trading, power and influence are shifting from institutions to the individual. Resources are now widely available to all at minimal cost, if not free. Not only can individuals access information, but the Internet ensures that everyone will receive it almost instantaneously. They also have access to real time data and can trade like the pros. With the availability of real time data and almost instant executions, individuals can act on information like never before.

As little as 5 years ago, transactions costs were high and figured into any investment or trading strategy. Again, with the advent of online trading, transactions costs have become minimal. This has increased the amount of trading volume and probably volatility. Higher volatility increases the possibility that anomalies will develop. With better trading resources and low commissions, more traders and investors than ever are able to capitalize on potential anomalies.

For obvious reasons, the Wall Street establishment is not thrilled about random walk theory. After all, Wall Street is in the business of analysis, strategy and money management. However, it is a fact that about 75% of equity mutual funds underperform the S&P 500 year after year. Some of this underperformance can be blamed on transaction costs and management fees. However, with the advent of index-linked securities, the onus will be on the money managers to figure out a way to outperform the market, or lose business.

In truth, 75% of equity mutual funds underperforming is not as bad as it sounds. When the Random Walk theory was introduced in 1973, or even 15 years ago, around 90% of equity mutual funds underperformed the market. Since this number seems to have risen, it would appear that either stock picking is getting better or fees are getting smaller, or both. 15 years ago, the stock market and mutual funds were much more homogeneous. Even though there were tech stocks, they did not exert nearly as much influence. With the explosion of the NASDAQ, tech stocks play a much larger role in today's market. Internet stocks, which have also come to the forefront, did not even exist 15 years ago. With an increase in specialty mutual funds catering to tech and Internet, the total number of mutual funds has proliferated over the last few years. With the increase in mutual funds has also come an increase in the diversity of such funds. There are funds for almost every sector, industry or index imaginable and investors have a wide array of choices. The more homogeneous mutual funds there are, the less chance there is to outperform. However, this specialization has created a hierarchy among mutual funds and helped to increase the percentage funds that outperform the S&P 500 from 10% to 25%.

History has proved that a buy and hold strategy outperforms most attempts to time the market in absolute returns. In risk-adjusted returns, the argument loses some of its credibility. Buy and hold may take the guesswork out of beating the market, but it does little to compensate for the risk associated with a continuous investment in the market. There is a direct correlation with risk and return: the higher the expected return, the higher the associated risk. A portfolio with a timing strategy that seeks to move into risk-free treasuries when a bear market is signaled (Dow Theory for example), significantly reduces the amount of risk associated with that portfolio.

There is another school of thought that considers the markets efficient yet predictable. One of the leading proponents is Andrew Lo. Lo earned his Ph.D in economics at the University of Chicago and is currently a Professor of Finance at the Sloan School of Management at MIT. Lo is a bit of an odd ball among academics because of his beliefs regarding the efficient market hypothesis and his attraction to technical analysis. Lo and Mackinlay's book A Non-Random Walk Down Wall Street debunks many of the theories put forth in the 1973 classic with a similar name. (Remember that most academics subscribe to the random walk theory.) Lo's research concluded the following:

It is not only plausible that markets are efficient, but participants can also profit from efficient markets. However, Lo asserts that even though it is possible to outperform the markets, it requires ongoing research, continuous improvement and constant innovation. Beating the market does not come easy, nor is it something that is easy to maintain. Lo likens the pursuit of above-average returns to that of a company trying to maintain its competitive advantage. After introducing a hot new product, a company cannot just sit back and wait for the money to roll in. In order to remain above the competition, management must be flexible and look for ways to continuously improve and innovate. Otherwise the competition will overtake them. Money managers, traders and investors who find ways to outperform the market must also remain flexible and innovative. Just because a method works today, does not mean it will work tomorrow. In an interview with Technical Analysis of Stocks and Commodities, Lo sums it up by stating:

"The more creativity you bring to the investment process, the more rewarding it will be. The only way to maintain ongoing success, however, is to constantly innovate. That's much the same in all endeavors. The only way to continue making money, to continue growing and keeping your profit margins healthy, is to constantly come up with new ideas."

These rebuttals to random walk theory are not meant to suggest that the vast majority of individuals are going to suddenly start outperforming the market. Even though this may be true over the past 3 years, history suggests that it is not likely to be the case 10 years from now. In other words, history suggests that this is an anomaly and there will be a reversion to the mean. Nonetheless, the investment and trading landscape has changed drastically over the last 20 years, even over the last 5 years. Individuals have access to more information and tools, transactions costs are negligible, trades are executed almost instantaneously, equity mutual funds have improved their performance and the buy-and-hold strategy does not appear to be a profit maximizing strategy. It should come as no surprise that analysis can make a difference. The only question is which type: fundamental analysis, technical analysis or both?