Bets against Australia's Big Four banks have piled up amid rising concerns that the country's property prices could be about to fall off a cliff.

The value of shares in the Big Four - Australia & New Zealand Bank (: ) (ANZ), Commonwealth Bank of Australia (ASX: CBA-AU) (CBA), National Australia Bank (: ) (NAB) and Westpac (ASX: WBC-AU) - that were sold short totted up to 9.03 billion Australian dollars (Exchange: AUD=) ($6.52 billion) as of May 16, up 85 percent since the start of the year, according to data from the Australian Securities and Investments Commission (ASIC).

About 3 percent of Westpac's and ANZ's shares have been sold short, while about 2 percent of CBA and NAB shares were short sold, the data showed.

So far this year, NAB's shares are down nearly 7 percent and CBA's are off almost 9 percent, while Westpac and ANZ are down more than 10 percent each by Monday's close. That's a sharp underperformance against the broader S&P/ASX 200 (ASX: .AXJO) index, which was up 0.4 percent for the year by Monday's close, even though the banks are heavily weighted in the benchmark.

Concerns over the banks has being driven in part by Australia's long-bubbly property market.

"Our house price-to-income ratio now in Australia makes the American housing bubble in the lead up to 2007 look positively moderate," Jonathan Pain, publisher of investment newsletter, The Pain Report, told CNBC's "Street Signs."

The median housing price in Sydney was 12.2 times more than the median annual household income in the third quarter of 2015, up from 9.8 in all of 2014, according to data from the most recent Demographia International Housing Affordability Survey. In Melbourne, the median home price was 9.7 times median income in 2015's third quarter, up from 8.2 across 2014, the data show.

"Australian banks are heavily invested in probably one of the most inflated real-estate markets in the world and we now have an avalanche of residential housing supply hitting the market," Pain said.

He said that the Big Four banks had issued more than 80 percent of the country's residential property mortgages, adding that mortgages as a percentage of banks' assets were "extraordinary."

At the same time, Pain noted that retail and institutional investors Down Under were heavily invested in the banks, particularly through the country's self-managed retirement accounts, or superannuation funds.

The number of mortgages in arrears has risen somewhat. In the fiscal first half of 2016, ended March 31, the percentage of Westpac's mortgage book that was 90 days past due and impaired rose to 0.46 percent, from 0.38 percent in the previous half, according to data from Deutsche Bank.