One of the factors weighing on markets during the past two weeks, surprisingly, has been optimism. That sounds a bit odd, but several measures of investor sentiment — including surveys of AAII and Investors Intelligence opinion, and measures of option and leveraged fund data — are showing that market participants are at levels of bullishness not seen since the 2007 peak.

The thinking goes that whenever everyone is already in the market, there are no marginal new buyers available to propel it higher. The market tends to move against the majority at risk, and in this environment the only surprise can be disappointment.

The table above, which comes from the ICI website shows that money has continued to flow out of U.S. equities during the past month, just as it has most of this year. The top highlighted number, 47, shows that only $47 million flowed into long-term equity mutual funds in the week ending Dec. 1, while $489 million flowed into bond funds.

And of that $47 million, all of the positive flow was in funds aimed at overseas markets, marked “Foreign” on the third line of the table. Money flowed out of U.S. funds, such as Fidelity Magellan, in each of the past five reported weeks: -$1.1 billion in the week of Nov. 3, -$660 million the next week, -$2.8 billion the next week, -$2.5 billion the next week and -$1.8 billion the last week.

So despite all the talk about how investors are bulled up, the fact remains that the public is just not heavily involved in the U.S. stock market.

The table above shows what the flows look like when there is a lot more optimism and people are actually doing something about it. The scale here switches to monthly (rather than weekly) flows. As you can see, at the start of the last year of the 2003 to 2007 bull market, monthly inflow into mutual funds hit a pace of $28 billion a month, with $7 billion to $10 billion of that going to domestic funds. Both far outpaced bond fund flow.

Puzzled about this divergence — ebullient survey data despite clearly negative fund flow data — I emailed Jason Goepfert, who runs the website SentimenTrader.com.

I asked him whether sentiment should be viewed differently in a bullish cycle, which we’re in now, vs. a bearish cycle. Here an excerpt of his response:

“You’re exactly correct: Overly bullish sentiment in a bull market is a much different thing than during a bear market. But that’s mostly true for long-term returns (3 months plus).

When looking at the short- to intermediate-term, there is quite a bit less difference between the two.  The sweet spot for most sentiment extremes is around 1-3 months, and even during roaring bull markets, we often see the markets pause for that length of time when sentiment gets overheated (e.g. July ’98, June ’03, January ’04, November ’04, December ’06, April ’10).

One character difference we sometimes see is that during bull trends, once we see a sentiment extreme, prices can continue to chop modestly higher for several weeks — but almost every time, those short-term gains are erased in quick, severe corrections. In contrast, during a bear phase, once we hit a sentiment extreme the markets tend to turn within days. So the end result is still weak performance, but during bull trends the short-term makes one wonder, ‘Is it different this time?’

Defining whether we’re actually at an extreme, of course, is another question.  I believe we are, based on a cross-section of different measures.  We never get 100% agreement among all of them, and certainly right now fund flows on a long-term basis aren’t showing excessive bullishness.

Also long-term economic confidence surveys are closer to ‘oversold’ than ‘overbought’ readings, margin debt has quite a ways to go before becoming too worrying and we’re not seeing much volume increase in speculative penny stocks.  All of those tend to be longer-term in nature.

So there are some holdouts from the ‘excessive bullishness’ argument. But overall, I’d say on a 1-3 month time frame, we’re at an optimistic extreme during a bull market.  That doesn’t make me want to sell short with a position trade, but it does make me not want to buy stocks.”

That’s a great answer, nicely nuanced. He’s basically saying that if this were a bear phase, the high level of optimism expressed in AAII and Investors Intelligence data would lead the market to tip over right away into a prolonged slide. But since we’re in a bull phase, it could take awhile for the excessive bullishness to lead to a setback. And either way, he’s saying, expect a one to three month decline to wipe out the complacency, and then a swift retrace and new high.

A great example of that type of environment is shown in the chart of the NASDAQ 100 in 1999, above. The big-cap tech index rose a stunning 100% that year, but it was hard to capture it all because there were four separate corrections of -10%! Just to give you an idea of how severe that is, if a 10% decline were to hit the S&P 500 now it would take the index back to the first week of September at 1,116. That would be incredibly painful.

To round out this study, I also checked in with Bert Dohmen, a veteran trader and advisor who has been writing about the markets for four decades. He says he sees excessive bullishness in the sentiment data too. But in his latest report to customers, he added the following, which I quote with permission: ”There is an exception to using contrary opinion: The start of a new cyclical bull market or a very strong bear market rally. If we are in a cyclical bull market, which could last until the next presidential election, then the stock market can stay technically ‘overbought’ with bullish sentiment going higher and higher.”

My own sentiment work comes to the same conclusion. Paraphrasing Mae West, when bull markets are getting started, “too much of a good thing” is just enough. I’m alert to the potential for excessive optimism to corrode the market, but recommend that you don’t jump the gun on worrying about it.

For more insights like this, check out my daily investment advisories, Strategic Advantage and Trader’s Advantage.