It means 'You almost fooled me' or 'I almost believed what you were saying for a moment.' We use it when we discover that someone has been telling us something that isn't true. (usually as a joke)

it means you tricked me.

"You had me going." means "I really believed what you were telling me" (I didn't know it wasn't true).