Good question! If you look up "to try" you'll likely find the two forms 〜てみる and 〜ようとする. Essentially, X〜てみる is saying "to do X and see what happens as a result" while X〜ようとする means "to make an effort to do X", so as you suggested, it's not appropriate for your example.
Aside from the nuance, the major difference is that the result of X〜ようとする is unclear - that is, if you actually succeed(ed) or not in doing X. In particularly, in the past tense, the implication is that you *did not* manage to do it. So your example sentence doesn't work, since when using this structure in the past tense it can't have ended up in success.
To express what you want to say there are a number of ways, depending on the level of literariness/formality you're going for. Here are a couple.
The structure 〜た末・の末(read すえ) expresses a (usually positive) result after a long concerted effort or process, so it would work here: 難しかったですが、一週間大変勉強した末に、やっと試験に合格出来ました。Though it has a very formal ring to it.
If you want to say it more informally, you could always just use 頑張る(がんばる): 一週間頑張って勉強して、試験に合格出来ました。The use of 出来る also emphasizes that you managed to do something as a result of your effort (you might even say 頑張って...Xが出来た is something of a structure meaning "I managed to do X after a great effort").