You can't directly substitute 'which' for 'where', as 'which' on its own cannot refer to a place. If you want to rewrite your sentence using 'which', it will have to be like this:
Let's have dinner at the same restaurant in which we had a party last time.
Let's have dinner at the same restaurant which we had a party in last time.
While the above sentences are grammatically correct, they are both rather awkward. The best option is to use 'where', as in your original sentence.
Either 'a party' or 'the party' are possible. You would use 'the party' if the listener knows which party you are referring to.
I hope that helps.