Above Grevenbrück the noble lords of Gevore built their ancestral seat, today called Peperburg. It was used from the first half of the 12th century and was inhabited even after the general of Gevore moved to Bilstein until the last decades of the 13th century. The main part was destroyed in the 19th century during quarry works. Today, only the foundation walls of the southern part of the castle are preserved.
The original size of the core castle is unknown, but it must have been at least three times the size of the area preserved today. The core castle was enclosed by a two meter wide ring wall. In front of it was a moat, 14 meters wide, sunk into the rock. The buildings attached to the ring wall on the inside are farm buildings and towers that were rebuilt several times and were not erected until the second half of the 12th century.
The excavated walls were fortified, partially supplemented and left in the state they were in, even if this means that walls appear side by side that did not originally exist at the same time.