Usually, I don't use that expression in the negative sense.
However, it is often used in the opposite way.
People say, "He will go along with your ideas", but there is often a proviso or condition attached.
For example, "I'll go along with that if you promise to pay half".
"To go along" with something indicates agreement but not always wholehearted support.
To me, in the negative sense, you would also need to state a reason.
For example, "He doesn't go along with my ideas because he thinks they cost too much".
Hope this helps