It consisted of three large and two small city gates and several round towers. Today, only the remains of the city wall in the Weierhohl with the Witches' Tower and the Angels' Tower still exist. The Hexenturm from the 14th century is the oldest preserved secular building in the town. The name may refer to the witch hunts, during which it may have served as a prison. However, there is no evidence for this. Olpe local historian Dr. Manfred Schöne assumed that the name derives from "Hessenturm" (Hessian Tower).
In addition, the memorial designed by Ewald Mataré in memory of the victims of the war and as a memorial for the living has been located below the city wall since 1963, as well as the Bracken memorial since 1996.