Tomas, you could say I go to the river, or I walk along/by the river. It is not correct to say I go by the river, unless you mean "I go by river from "A" to "B" in a boat.

Sure. Less formal, but still fine... So I go by the river and I can see fish jumping. In that case "go" could mean you walked, you drove, you were on your bike, whatever, you were moving by the river.