The second man to drown in "dangerous" waters off Phillip Island's coast in a week was swimming between the flags, lifesavers say.

Local surfers are now calling for multilingual warning signs at Woolamai Beach, which was closed on Tuesday afternoon due to rough conditions, to stop the "avoidable" deaths.

The Mitcham man, 46, was washed off a sandbank at Woolamai Beach and pulled into a rip just before 5pm on New Year's Day.

He was seen struggling in the water but attempts to rescue him were unsuccessful and he died at the scene.

"He entered in between the flags where he was washed off a sandbank and into a a rip," Life Saving Victoria's general Manager Paul Shannon said.

"Sadly the poor fellow ... has become another drowning statistic."

It comes after Indian-born student Hemant Govekar, 28, drowned off Cape Woolamai during a swim on Christmas day.

Surfers spotted Mr Govekar in trouble at about 5.30pm, but he quickly disappeared from sight.

The beach was closed on that day and there were no lifesavers on patrol at the time of the incident, with the conditions deemed unsafe for inexperienced swimmers.

Two years ago, two people died after a group of seven co-workers were dragged into a rip as they stood in knee-deep water at the beach.

Surfer Sandy Ryan was part of the mass rescue, pulling one from the water and giving CPR before they later died in hospital.

"It's just tragic because it's so avoidable," Mr Ryan said.

"Rips are like a chairlift to the ocean for surfers. People get caught in them and panic."

The deaths have prompted Mr Ryan's friend Suzan Borelli, also a surfer, to start a petition to put multilingual signs at the beach, which she called one of Australia's most dangerous.

"It has unpredictable sandbank movements and strong rips along the entire strip of beach," she wrote on the AVAAZ.org petition.

"[The drownings are] tragic for not only the victim and their families but the bystanders, rescue team and local residents, who all believe that if the surf beach had adequate multilingual signage these drownings may have been prevented."

Another local surfer, Simon Robertson, who rescued a boogie boarder last summer, said tourists didn't understand the dangers of rips and education was urgently needed.

"I don't understand why we can't have signs that say 'heed danger'," he said.

He said the water between the surf breaks looked calm to the untrained eye, but within seconds someone could be dragged 40 metres out to sea.

Mr Shannon said there were already signs with pictures indicating danger at the beach but research suggested most people didn't look them.

He said more education on reading warning signs was needed, and people needed to better understand their own swimming abilities.

However, Ms Borelli, who teaches surfing, told Fairfax Media there were multilingual signs on other popular beaches around the world and it was worth giving them a try when people's lives were at stake.

She suggested being creative and making signs that doubled as a photo spot, so that people took notice. She said it could include directions to a safe beach just around the corner.

Life Saving Victoria statistics show men made up 78 per cent of the 45 people who lost their lives to drowning during the 2016/17 financial year - a 20 per cent increase on the long-term average.

What to do if you get caught in a rip:

1. Raise your arm and call for help
2. Float with the current and hopefully return back to safety
3. Swim parallel to the beach
4. If your chosen option isn't working, try another