It’s time for Men’s Wearhouse (MW) founder George Zimmer to put up or shut up.

Zimmer, who was canned last week by the men’s apparel retailer’s board, has denied MW’s claims that he was unable to relinquish the reins of power to his hand-chosen successor.

He argued in an “open letter” released Wednesday that recent decisions by the board — such as unloading the underperforming K&G stores and not pursuing a sale of the entire company — were done in haste. Then Zimmer backpedaled, saying, “To be clear, at this point I have not concluded that taking The Men’s Wearhouse private is a better means of preserving the unique culture and values that have made the company so successful over the years.”

The 64-year-old Zimmer offered little in terms of specifics, which is puzzling for someone reportedly considering trying to get back in the game. Published reports indicate that he owns less than 5% of the company’s outstanding shares — nowhere near enough for him to mount a takeover bid for the Fremont, Calif., company by himself.

Zimmer would need a partner with deep pockets, like a private equity player — and he would need to convince such an investor that he has a better vision for Men’s Wearhouse than the board, which might be easier said than done.

Take K&G. Men’s Wearhouse has done what it can to boost the 100-store chain’s sales, including ratcheting up television advertising, but nothing seems to have worked. Indeed, the seller of off-price and discontinued apparel has been a mess for years. According to the company’s latest 10-K filing, K&G has reported declines in same-store sales in every year but one between 2008 and 2012. The declines continued in the first quarter, when the measure of the performance of stores open at least a year fell 5.3%.

Given its modest size, the road ahead for K&G will be a difficult one since it competes against a plethora of larger rivals such as TJX (TJX) and Kohl’s (KSS). The company could try to build the business up through acquisitions, but there aren’t many chains that are available to buy. Meanwhile, adding new stores itself would be prohibitively expensive.

Indeed, the outlook for K&G is so bleak that any partner who would want to work with Zimmer likely would insist that it be sold — something CEO Douglas Ewert has wanted to sell, and something Zimmer fought to keep.

Men’s Wearhouse’s existing management team seems to be doing a good job. Comparable sales at its flagship brand rose 7.1% in the latest quarter. They also gained 4.8% in full-year 2012 , 9.1% in 2011 and 4.7% in 2010. Under Ewert, the company also has announced plans to buy back as much as $200 million in shares, which no doubt pleases shareholders.

Although some customers have vowed to never shop in a Men’s Wearhouse again after Zimmer was fired, I imagine most of that is just emotional bluster — most people probably will return. Moreover, as the economy continues to rebound, the chain should continue to do just fine.

As Bloomberg News notes, Zimmer’s actions have put the company he founded in 1973 in play. There certainly is value there.

“Even after its shares rose 34 percent in the past year, Men’s Wearhouse trades at a lower valuation relative to profit than 95 percent of similar-sized specialty retailers, according to data compiled by Bloomberg,” the news service says, adding that one analyst estimates that the company could fetch $50 in a takeover.

About the only thing that investors should fault Men’s Wearhouse for is the extraordinary deference it showed to Zimmer.

While the baritone executive has been the company’s TV pitchman since 1985, he hasn’t been chief executive since 2011, when Ewert took over the job. Exactly what he did as executive chairman of the company isn’t clear (besides appear in television commercials). Whatever he did, Zimmer was paid well for doing it. His total compensation for 2012 was $1.99 million, almost equal to the $2.09 million pay package Ewert earned. And Zimmer’s base salary was $1.02 million, much higher than the $616,635 his successor earned. Zimmer also got loads of perks, including private use of the company’s aircraft.

In the event of a sale of the company, he could get $2.69 million, topping the $2.43 million that Ewert could get. And should the company want to use Zimmer’s likeness in its ads, it will need to pay him a license fee of $250,000 now that he isn’t an employee.

That’s extraordinary.

Bottom Line

In the end, Zimmer will have to convince investors that they would look good if he controlled Men’s Wearhouse. Unlike his famous catchphrase, that will be difficult to guarantee.

As of this writing, Jonathan Berr did not hold a position in any of the aforementioned securities. Contact him on Twitter at @jdberr.