The town of Belfast, (changed to Port Fairy in 1886), was based on seafaring people. Early fishermen caught barracouta, mullet, bream and Australian salmon from sail powered wooden craft. The fish was hawked around the town in wooden barrows. Melbourne didn't become a major destination for fish until adequate transportation existed to get the fresh product to market in good condition. An explosion in fishing effort occurred when the railway came in January 1890 and the number of boats increased from six to thirty over the next ten years.

The Port Fairy Preserving Company had a go at canning fish for local markets during the 1890s but they had problems in getting a continual supply of fish. Freezing fish (from 1938) like Australian salmon for bait and then later shark flesh helped to provide continual supply.

Rock lobster has always been an important species, right from the very beginning in Port Fairy. Barracouta was an important species during the ports early years until around the 1950s. Shark rose to importance during the 1930s and was a major species during the Second World War. Abalone diving started in the 1960s.

Today rock lobster and abalone fishing are the main fisheries in terms of value. The fishing boats now all have engines and are made from aluminium, fibreglass or steel. The wharf area on the Moyne River sees fishing boats unloading their catch of fish, shark, abalone, rock lobster and squid.

Visitors can still see the round top shed where Walter Kelly had his freezer. The Norfolk Island pine trees are a distinguishing feature of Port Fairy's township they were planted on Griffiths Island over 150 years ago by whalers and in Sackville Street at the turn of the last century by Mr Holden in honour of his sons twenty-first birthday. Port Fairy's famous Folk Festival, held in March, attracts thousands of visitors from around the world.