It was originally called Camp Drum. Building began in 1862 and reportedly cost 1 million dollars.The camp was named after Lt. Colonel Richard Coulter Drum, assistant Adjutant General of the Department The Drum Barracks is the last remaining Civil War military facility in the Los Angeles area. It served as the Union Army Headquarters in the Southwest from 1861-1871 and was used for Civil War operations from 1861-1865. It was the main base for military operations in the area. The camp also had the most important medical facilities in the western states. Many thousands of men passed through the barracks on their way to camps in Nevada, New Mexico, Arizona and Utah. Troops from the barracks, protected much of the Southwest and the territory that is now Arizona and New Mexico. On the way to Santa Fe the troops from the California Column fought in the battle of Picacho Pass the westernmost battle of the Civil War. They also fought Geronimo and his Apache warriors in the Indian Wars in Arizona. The hospital continued for two more years after the Camp closed in 1871, the land then reverted back to the original owners after the hospital closed. The site was then used for a variety of uses. Wilson College began on the site in 1874, but it closed after only 2 years. Up until it was opened as a museum in 1987 it was used as a High School for the Wilmington Township. The Museum is housed in the last remaining wooden building of the barracks, which used to be the junior officer's quarters of the camp.
Many apparitions have been seen at the camp including a woman, a boy and apache scouts. Footsteps are also heard on a regular basis and pipe smoke can sometimes be smelt.