The Castrum Peregrini (pilgrims fortress) foundation was established in 1951 on the initiative of a group of individuals who had gone into hiding during the Second World War in the home of the artist Gisèle van Waterschoot van der Gracht. In January 1941, she rented the third floor at number 401 Herengracht, where, together with Wolfgang Frommel, a German poet, she hid various Jewish schoolchildren. She withstood the war years by focussing not on suffering, but rather on art and literature. All of the children survived and after the war Gisèle bought the entire building at number 401 Herengracht and converted it into a single dwelling. She later donated the building to the Castrum Peregrini foundation, which is still housed here. Gisèle is the patroness of the foundation and lives on the top two floors of the building.
The canal-side building on the corner of Beulingstraat was built in 1724 for Coenraad Christoffels. It originally comprised two private dwellings on Herengracht and a stable in Beulingstraat. The building is five windows wide and has a bevelled corner with blank niches, thereby solving the design problem of a corner building with sloping facades. After the war, the three units were joined together to form a single building, with the loss of one of the stairwells.