The Norman knight Odo de Barri was granted the lands of Manorbier, Penally and Begelly in gratitude for his military help in conquering Pembrokeshire after 1003.

Two of his sons aquired larger estates in Ireland, which became the main power base of the de Barris, known as the Barris of Olethan. His fourth son was Gerald de Barri. Known commonly as Gerald of Wales (the great twelth century scholar) who was born at the castle. Renowned today for his chronicles and descriptions of life in his time.