"The problems come when it rains -- water runs down hill and if anything is loose it will wash away."

This is why I advise against stone. Stone and rock that's been embedded
for millenia are not effected, but a freshly dug hole on a slope will
erode a bit with every drop of rain, and the stone laid in it will
shift and get out of level, and the step resting on it may teeter—the last
thing I'd want when I'm relying on a firm foorting.

~aj