The site has been a defensive position since the earliest inhabitants, the Samnites, constructed defensive terraces there. Later the Romans built a thermal plant and remains can be seen in the castle garden. The position overlooked two important Roman roads, Via Appia and Via Traiana. The Benedictines had a monastery here and finally it became the seat of the papal governors, Rettori. It's still the heart of the city of Benevento.
Inside the lower part of the Palazzo dei Prefettura is a museum with contemporary art.

