Warrnambool businessman Ben Blain questioned why the council was pouring more than $3 million into the project.

The council currently spends about $800,000 to operate the library but Mr Blain said that when the upgrade was completed, that cost would spiral to about $1.2m to $1.5m.

"We're meant to be in this dire financial situation, how can you justify spending millions on a library?" he said.

"That's the thing that's really scary. We can't afford it, we don't really need it but you're going to put millions into it.

"With the library, you've increased the costs by nearly $1 million, how are you going to fund that extra million, where is it going to come from?"

The new $20.3 million library, which will be located on the TAFE site, is expected to be completed by mid-2022.

The Department of Education and Training contributed $16.3 million, TAFE $830,000 and the council will tip in $3.12 million.

Mr Blain raised concerns about the ongoing costs of the new library after getting a response from council on questions he raised during the last council meeting on asset renewal.

He said that after comparing budget forecast figures and actual costs over three years, he believed that Warrnambool's asset renewal backlog could have been cleared in three years without seeking a rate rise above the cap, provided it did not add any new projects to the $16.4m backlog.

"You can cover your annual upkeep and address the backlog if we weren't spending all the money on new asset renewal projects," he said.

"My big question is, if we've got this asset renewal backlog which is so important now that there's a $16.4m backlog that needs to be addressed and we need to raise rates in order to address this, why is it that we're taking on other asset renewal projects and taking on other projects when we should be focusing on the assets that we already know are in disrepair?"

The council responded to Mr Blain.

"The amount that council spends in a year on asset renewal is dependent on a number of factors including the level of external government grant funding for particular projects and the nature of projects in the capital works program," an emailed response said.

It explained in some years the council may invest more on asset renewal due to large government grants such as the CBD renewal, however it may not address existing items within the asset renewal backlog.

"Asset renewal is a complex issue and we have an asset management team that monitors the condition of the assets and the levels of investment required to renew the assets over time," it said.