Hahaha, what a hilarious excerpt from the book Hoon.
How are you? Great I hope!
There is a difference:
Spend time = Just to literally do something that needs time, no matter how short or long. EG. I am going to spend my time to day watching television.
Take your time = To deliberately do something slowly so that you "savour" the moment or to do it properly. EG. I am going to take my time getting ready for the party, because I am not in a rush and I want to look my best!
In your example, the writer wants to go to a "proper" restaurant and savour the ocassion, eating slowly, enjoying the atmosphere, food, service, etc etc.
When he eats at KFC, he can't do this because it is "fast food".
Does that make sense?