Built in 1838 for about $5,000, the lighthouse takes its name from Ned Dexter, a local landowner. Its rubble-stone tower, built from local granite, has stood 39 feet tall and largely unchanged for nearly two centuries — climbing to the top means 32 interior stone steps.

The light was automated in 1923, ending the need for a resident keeper. In one of the more memorable moments in local lighthouse lore, the original keeper's house was floated by barge across Buzzards Bay to become part of Wing's Neck Light in Bourne — legend has it the last keeper cooked himself one final breakfast inside the house as it made the crossing.

The Town of Mattapoisett acquired the surrounding land in 1958, creating Veterans Memorial Park, and the lighthouse itself was added to the National Register of Historic Places in 1987. Today it remains an active U.S. Coast Guard navigation light, with local Coast Guard Auxiliary volunteers offering seasonal tours to the top for visitors ten and older.