The value of properties in Port Fairy has risen by as much as $200,000 in some cases in recent months.
Lockett Real Estate agent Garry Lockett said the demand for properties in the seaside town was unprecedented.
The 74-year-old has been an agent for about 45 years and said he had never seen anything like it.
"I reckon in the last three or four months the average house has gone up by about $200,000," Mr Lockett said.
He said a number of factors had contributed to the increased demand.
"Between the (COVID-19) pandemic and low interest rates, it was a goldrush to buy," Mr Lockett said.
Listen to the latest episode of our weekly podcast The Booletin and Beyond:
He said Port Fairy had great facilities and was being no longer a hidden gem.
"It's amazing," he said.
Mr Lockett said some people had holidayed in the town before and had decided to make a move, while others had found the town after seeking to exit the city and falling in love with it.
He said all agents in the region were struggling to meet demand from buyers.
"It's the lowest by a mile," he said.
Mr Lockett said listings were coming in each week but often they snapped up before they were advertised.
He said the high interest was driving up prices, with a block of land at South Beach recently selling for $500,000.
Mr Lockett has kept a lodger of sales in the town for about 20 years.
"There are 900 homes in Port Fairy and in this I'm up to sale 1590," he said.
Lockett agent Paula Dwyer said properties listed a few months ago were being revalued to reflect the current market.
She said it was a seller's market and due to the lack of listings available in Port Fairy, properties in Koroit were also getting snapped up fast.
Recently she had two listings valued in the town and before they were advertised she had buyers ready to sign a contract.
"We have a list a mile long of buyers," Ms Dwyer said.
Stockdale and Leggo agent Nicole Dwyer said demand for properties in the town was extremely high.
"A lack of supply is pushing up the prices," she said.
Ms Dwyer said there were a number of farmers who had experienced a bumper season looking to invest in property in Port Fairy.
Have you signed up to The Standard's daily newsletter and breaking news emails? You can register below and make sure you are up to date with everything that's happening in the south-west.