I'd say "that's a subject for a different discussion" or "that's a topic for a different discussion".
But usually when someone says something is besides the point, they mean that it is irrelevant and does not pertain to the current discussion. It does not usually mean that they want to discuss it later.