Walberswick's name probably derives from "Walbert", who was the Saxon founder of the settlement, and "wic", which is the Saxon word for "village". Vikings also sailed up the river, now the Blyth, which then entered the sea at Dunwich. In 1590 the men of Southwold and Walberswick cut through allowing the river to discharge between their two towns as it does today.

The Anglo-Saxon settlement prospered on trade and received its first charter in 1262 with five more charters following, the last being from Charles I in 1625.

These charters gave Walberswick relief from tolls and taxes which, in turn, gave the inhabitants a great advantage over neighbouring settlements when importing goods such as coal, salt and timber and exporting agricultural products and fish.

Dunwich controlled the then mouth of the river and a charter of 1230 had given Dunwich the right to charge tolls on goods bound for Walberswick. Not surprisingly, the conflicting nature of these charters led to many battles between the people of the two communities and eventually, to restore peace in this part of his kingdom, an enquiry was ordered in 1408 by Henry IV. This enquiry held that Dunwich had no right to charge such tolls.

So, Walberswick prospered and by the mid-1400s the port had a fleet of ships trading with the European mainland, Iceland and the Faroe Islands.

As a result of this prosperity, Walberswick had two churches. The older, at the bottom of what is now Stocks Lane (so named because of the stocks or baulks of timber used to support ships in the process of construction) was taken down in 1473. The second church, St Andrew's, was rebuilt and with its 90 feet tower served then and now as a noticeable landmark for sailors.

Silt build-up in the river was continuously causing a problem for ships especially as they grew in size, with the fortunes of Blythburgh, Walberswick and Dunwich all adversely affected.

Eventually, with its population dwindling, Walberswick continued to decrease in size and gradually became the small fishing village that is has been for most of the last 300 years, still facing the constant challenges of the river and the sea that have forged its long history. Walberswick could now no longer afford the upkeep of its large church, so the huge nave was dimantled in 1695 and the much smaller nave built inside the shell of the original.