Yes. This sentence implies that you only worked with the other person once, and you don't have plans to work with them again. If you work with the other person often, and you plan to work with them again in the future, say:
"It's been a week since we've worked together."