hi

It sounds a bit awkard - maybe rephrase it to say "If you had asked me back then, I would have said no"

Not quite. You can write this either as a "second conditional" or "third conditional" depending on the situation. Second conditional: "If you were to ask me, I would say no." (Or "If you asked me, I would say no".) Third conditional: "If you had asked me, I would have said no". Many native English speakers use conditional sentences incorrectly or "mixed conditionals", so you may hear examples that deviate from what I have written here.