The castle was built in 1275 by Sir William de Caverswall who was a knight of Edward 2nd after he had recieved a licence to crenellate his manor house. It was rebuilt in 1625 in a Jacobean style by the Mayor of Staffordshire, it retained its original turrets Norman Tower, dungeons, bridge and parapet walls. The castle has had different owners and varying uses over the years, it as been privately owned as well as a convent. It is now a home and a B and B. It also opens for tours and is a venue for weddings.

The castle as a reputation of being haunted, one of the ghosts said to roam around is that of a nun, and it is said that the place takes on an entirely different feeling after dark.