The red hot housing market threatens to scorch summer vacation plans for COVID-19 pandemic weary Americans who want to rent a summer getaway or buy a second home.

"The houses that do list are still flying off the market," according to celebrity real estate agent Noel Roberts, who is the head of Nest Seekers Private Client, a boutique practice that focuses on serving high net-worth clients.

Roberts, who also stars in the Netflix hit show "Million Dollar Beach House," told Yahoo Finance Live the luxury market in high-end resort areas, like The Hamptons, on Long Island, remain big draws for the monied elite.

"You know, we brokers sit around telling tales of last summer, and you'd think we were trading war stories. It was pure chaos in The Hamptons," he said. "This year although we're not dealing with nearly as much frenzy as we saw in the midst of the pandemic, the overall market remains quite healthy."

A luxury estate built by Henry Ford's grandson sold last month for $145 million and another house last week was rented, for the summer, for $2 million dollars.

It's all part of a trend, that began when wealthy families fled cities at the start of the COVID-19 pandemic and bought second homes in their favorite vacation locations, according to Roberts. "The average days on market is around 101 days on average, which is 32% faster than last summer," Roberts pointed out.

'Uneven economic recovery'

A report last month from real estate brokerage firm Redfin called the intense demand for vacation homes,"a symbol of the ongoing uneven economic recovery in the U.S."

The report found home prices in seasonal towns are up 19% year-over-year and sales of second homes are closing at a record pace. The number of buyers who locked in mortgage rates for second homes shot up a record 128% year-over-year in March. That marks the 10th straight month of 80%-plus annual growth, according to the report.

Roberts predicts, as the number of vaccinated people increases, people will return to their pre-pandemic travel routine. "But, I think that there's been a change that's going to be far lasting," he said. "Especially out in these vacation homes, not just in the Hamptons"

Redfin's report concluded, "The surging interest in vacation homes is indicative of the uneven financial recovery taking place throughout the U.S. Wealthy Americans are likely to have held onto their jobs—many with the freedom to work remotely—and they’re earning money through a robust stock market and rising real-estate values."