Exactly, where it is followed by an object, you use "your". You need to add extra information so it makes sense.
Example:
She is "your" mom.
Ela é a sua mai.
But if you want to use "yours". It is mostly placed at the end of the sentence.
The cat is "yours".
O gato é seu.