The building not a true 'castle' was built in 1886 by Robert Givens, it is thought to have been a surprise for his bride. All the rooms were beautifully decorated and had stained glass windows one of which had the motto 'Dum Spiro Spero' which means 'While I Live I Hope'. A carriage house for the castle is just west of the building and as since been totally remodeled. They lived at the castle until 1894 then temporarily moved out, during this time it was used by The Chicago Female College, they returned in 1904 and left again in 1909 when they moved to San Francisco. Robert died in 1915. The castle was brought by John B Burdett a manufacturer who had it all wired for electricity and added radiators. He sold the property in 1920 to Dr Miroslaw Siemans, due to the Depression things fell on hard times and he filed for bankruptcy in 1935, the castle was put into a trust in 1937. The castle was then brought by the Beverly Unitarian Fellowship in 1942, which was an off shoot from the Peoples Liberal Church, they later merged in 1951 as the Beverly Unitarian Church. The interiors were modified and work carried out by menmbers of the congregation. A church school building was built in 1959.

The castle is said to be haunted by a female spirit thought to be Roberts wife, and many strange sounds have been heard.