Built in 1556, it was one of Spain's first school-orphanages. In the 18th century it operated as Income Administration Office. In the 19th century it was known as the Casa de la Gargantada, because of its owners, the Counts of Urbina, and functioned as school and orphanage for a number of years. Between 1923 and 1928 it was the St Joseph's school, managed by the French Marists in Tui, and had as many as 60 students.