Yoshimi Takayori served as the tenth head of the Iwami-Yoshimi clan of Iwami Province and as a retainer of Ōuchi Yoshitaka.  The Yoshimi were kokujin, or provincial landowners, in Iwami Province, originating from the Seiwa-Genji clan with roots in the Imperial family.  The Iwami-Yoshimi were a branch of the Yoshimi clan.

Takayori was born in 1500 as the second son of Yoshimi Yorioki.  In 1510, his elder brother, Okinari, died at an early age so Takayori became the designated heir to his father.  Takayori received one of the characters in his name from Ōuchi Yoshitaka.  He wed Yoshitaka’s daughter, Ōmiya-hime.  Takayori became head of the clan following the death of his father in 1532.  In 1540, he traveled to a shrine in Kiso in the village of Tsuwano to worship his ancestors.  Thereafter, while sojourning in Yamaguchi, he was killed by unknown bandits.  He was then succeeded by his younger brother, Masayori, who, in turn, married Ōmiya-hime.  Takayori was known as a man of culture, and made a contribution by revitalizing the Gion Festival in Tsuwano.