To get to the hamlet of Potoka go to the town of Smolyan and then continue towards the villages of Smilyan, Mogilitsa and Arda. There are two hiking paths that can get you there. One of them starts from the village of Arda and is marked with white-red-white signs. The other starts from the village of Arda and the markings are white-green-white. Both paths are around 3.5-4 km long and you need about an hour to take the distance. Keep in mind that almost everywhere in the Rhodopes the tourist paths are a bit confusing (with great many intersections, smaller and bigger paths starting as it seems from nowhere and edning nowhere) so please take a look at the schema bellow. It will (hopefully) help you to navigate through the maze of the Rhodopes dirt roads and trails and get to the hamlet.
If you aren't quite sure you won't get lost, there is a easier more straight-forward way. It starts from the village of Borikovo. It is an old dirt path once probably used by the border troops that were guarding the area. To find it, after the village of Mogilitsa (in case you are coming from the north) turn right and drive for 1.5 km. Once inside the village look for the bridge that is in the center, after the village store. The dirt path is rocky and although the locals seem to go that way with ordinary cars I wouldn't recomment you trying that. There are one or two places that are a bit tricky and after all it's not worth breaking your car to avoid a short hike.