Hi Joanna,
Both are correct, but they have different meanings.
"Their house is next to ours" means that there are no other houses in between their house and ours. Our properties touch each other. We might even have a fence separating our properties. "Their house is near ours" means that their house is close to mine but not directly next to it. It "isn't far". We probably live in the same neighborhood and can walk from one house to the other.