Alot of Municipalities out here in "Drought Stricken" SoCal encourage the use of Gray Water, although they like to regulate the heck out of the setup. But thats what California is so good about... Regulating.
But alot of people do and the byproducts of their gardens were just fine, As some people have posted already the contents of the water is virtually sterile and the use is restricted to pretty much the sources you described, As long as they arent mixing in raw sewage you should be fine.

JC