'Hold fast' means 'hold tight'. There are many expressions and collocations in English where 'fast' has the meaning of 'tight' or 'firm', such as 'fast asleep' or if we say that something is 'stuck fast'.

The word "fast" can also mean "securely".

I think if one is thinking about "let your heart hold fast" it is useful to remember the meanings of "fast friends" (meaning close and loyal) and "steadfast", meaning "dutifully and resolutely unwavering". We often see "steadfast" used with "loyalty" as in "steadfast loyalty".

It might help to remember the expression "hold fast" (and, more generally, the alternate meaning of "fast" that Peachey mentioned) in connection with the verb "fasten," meaning "secure, attach, tighten."