In 1906 Gibb was to join Matisse, André Derain and other leading Fauvists in the south of France where the artists exploited the unique light qualities of the south. It is documented that Gibb shared a studio with Matisse but it is unclear as to whether this was in the south of France or in Paris. However Gibb was sharing a Paris studio with Matisse and Braque in 1909.
Significantly Gibb also associated closely with one of the foremost proponents of modernism, the American Gertrude Stein. As well as being an avid art collector, Stein was a novelist, poet and playwright. After moving to Paris in 1903, Stein hosted a salon where leading members of avant-garde literature and art would regularly meet – Picasso and Matisse to name but two. Stein befriended, supported and patronized Gibb and even used him as the inspiration for a literary work – ‘A Portrait of One Harry Phelan Gibb’. It is obvious from the text that Phelan Gibb was a troubled soul.