The shrine is located in The Reach, far to the northeast of Markarth, a short walk northwest of Karthwasten, a little southwest of the Dragontooth Crater landmark. He will point you to Kesh at the Shrine to Peryite for more information before continuing on his way to High Rock, adding an objective to your journal.
He will say, when asked about his unhealthy appearance, that he suffers from the plague and would have died years ago without Peryite's protection. You may encounter an Afflicted Refugee during your travels, once you have reached level 10, who asks you to stop 'ogling the grotesque'.
The Shrine to Peryite in the mountains of The Reach