The land where the castle once stood were first bestowed to Ralph de Pomeroy by William the Conqueror, for his support and loyalty during the Norman Invasion. The construction of the castle began in the late 13th century by Henry de Pomeroy (9th generation) and was completed before his death in 1305. The castle now stands in ruins but part of the Norman structure can still be seen in the gatehouse.

In the 1540 the castle was either brought or acquired by Sir Edward Seymour who was the Lord Protector. He was also the brother of Jane Seymour, Henry VIII third wife. Edward commissioned a house to be built within the confines of the castle and that resulted in the mixture of the building we see today.

Edward was ousted by the Earl of Warwick; he was imprisoned in The Tower of London in 1549, he was sentenced to death and was executed in 1522.

Despite this the castle still remained in the Seymour family, and continued to be their home up until 1688 when it was abandoned. The castle was not properly used again until the reign of George the IV. The castle though has still remained in the guardianship of the Seymour’s. English Heritage took over the maintenance of the castle in 1977 have continued to carry out the restorations that are needed.

There are a few ghosts that still roam the castle, including a ‘blue lady’, ‘white lady’ and a cavalier. There have also been reports of strange lights, the sound of voices, and shadows have been seen even though there has been nothing there to cast them.