(REUTERS/Steve Marcus)
Steve Cohen.

There are more hedge funds today than there have ever been in history, says billionaire Steve Cohen, and it's bringing about one of his worst fears.

"One of my biggest worries is that there are so many players out there trying to do the same strategies ... if one big one goes down, will we take collateral damage?" Cohen said at the Milken Institute Global Conference in Los Angeles on Monday. "We were down 8% in February and for us that's a lot ... my worst fears were realized."

Cohen is the founder of Point72, a family office formerly known as hedge fund SAC Capital. He was speaking at a panel with AQR Capital's Cliff Asness and Neil Chriss of Hutchin Hill Capital.

His fear is one unique to the evolution of the hedge fund industry because of its explosive growth over the last decade-plus.

It's about differentiation — or rather the lack of it.

In February, the market was going absolutely nuts. Everyone thought the end of days were near as China tailspinned, dragging the rest of global stock markets with it, and commodities prices were at historic lows. It happened so suddenly that big investors with their money parked here and there were asking "How much can we not lose?" instead of "How much money can we make?"

And because a lot of hedge fund money was parked everywhere, losses were magnified. As big investors sloshed their money around in these volatile waters, some funds got wiped out in the volatility.

"It happened in four days," Cohen added later, "and the markets were going up at that time."

Too many hedge funds isn't the only problem

There are reasons why hedge funds can clobber each other with size now, aside from the fact that too many of them are doing the same thing.

As Chriss pointed out during the panel, the hedge fund industry used to be worth a couple $100 million. Today, the industry is worth $3 trillion. And, according to Cohen, every player in the industry thinks they have to be big to survive.

To Cohen, Asness, and Chriss, it feels like a day of reckoning is coming. It was all they could talk about.

Since the financial crisis, the stock market has had a wonderful run, while hedge funds overall have not outperformed the indices that they are compared against. Investors are getting annoyed, but Asness had an answer for that.

"Our criticism of hedge funds is that they're not hedged enough ... They're not supposed to beat the market in a bull market," said Asness. "The comparison against all stocks makes you look really good when the stock market goes down and bad when it goes up, and it's hurt them in the last six to seven years."