Earlier this year, I decided to follow Warren Buffett’s famous investing advice about being “greedy where others are fearful” and added cheap, beleaguered Coach (COH) stock to my portfolio.

Since then I, along with every other investor, have watched shares of the maker of handbags tank. I’m watching them tumble even further this week in the wake of the company’s latest earnings report. At $34 a share, COH stock down about 40% this year after losing 6% following its earnings announcement Tuesday.

Even so, I haven’t given up faith that Coach, which is heavily shorted and hated by Wall Street analysts, will become fashionable again.

An Ugly Slide in Sales

Let’s look at the latest numbers, ugly as they may be. Sales fell more than 10 percent in the recent quarter to $1.04 billion. Net income fell to $119.1 million, or 43 cents a share, from $217.9 million, or 77 cents, a year earlier.

Worst of all, comparable store sales fell by more than 20 percent. The company said on the conference call that it expects sales declines at about that same rate for the rest of the year. One reason for the precipitous drop in the closely watched retail metric is that COH is scaling back on promotions. It is also closing 70 underperforming stores and opening or remodeling stores in its 12 biggest markets.

I think Coach is managing all these major changes surprisingly well. During the fiscal first quarter, its cost of sales figure (Selling, General and Administrative Expenses) was basically flat at $536 million vs. $505 million the prior year’s quarter. Cash at the company is actually higher than a year ago at $907 million, up from $854 million. Profits fell, but actually exceeded expectations. Excluding restructuring costs, EPS was 53 cents, topping the 45 cents Wall Street had expected.

Patience Will Be Rewarded

Coach no doubt has its work cut out for it. Investors will have to be patient. The brand has gone from aspirational to the discount rack over the past few years. Now Coach has to win over new fans while not alienating the customers who have stood by the brand for years — perhaps decades.

The company seems to have found the right person to meet these challenges in Creative Director Stuart Vevers. Coach hired the 40-year-old last year on the strength of his work for Mulberry and Loewe. His first collection was released last month and reviews in the fashion press have been positive. Indications are that more fashionable customers are paying more attention to Coach and buying more goods.

Writing in Style.com, Nicole Phelps, notes, “Stuart Vevers isn’t holding anything back in his repositioning of Coach. On the runway, at least, this is not the classic, all-American brand that your mother and your grandmother knew.”

Fashionista’s Dhani Mau added, “the Ryder handbag, the Apollo sweater, shearling coats and urban hiking boots have been standouts.” Vogue.com gushed about Coach’s “more youthful take on fashion than the safe options we have seen before.”

Despite the positive reviews,the way COH stock has been reacting indicates that many on Wall Street are expecting Vevers’ collection to flop. Keep in mind that the new items only hit stores in September.

Even if the naysayers are right, it still make sense for investors to buy COH because the brand, though diminished, is still relatively healthy and the stock is cheap, with a price-to-earnings ratio of only about 12. At these levels, the company could be an interesting acquisition target. You can bet that either a private equity player or a larger rival  such as Ralph Lauren (RL) or LVMH, the parent of Louis Vuitton, would be interested in buying COH.

Here’s an added bonus for investors: COH shares yield a fat 4% dividend, well above the 1.99% average of the S&P 500.

That’s the main reason why I am not cashing out. In fact, I may buy more COH on the dip.

Jonathan Berr is long COH.