…during difficult times/ during tough times

"My trying times" is a bit old-fashioned, but is still used a good deal, especially in writing. A related noun is "trial" -- "Climbing stairs can be a trial for older people." Also common: referring to troubles as "trials and tribulations"

in my low moments when I’m feeling blue

There are about a million ways to say it. When times are tough. When times are hard. When things aren't going well. When I'm down. When I'm at a low point. Actually, 'setback times' and 'challenge times' do not sound natural. One could say Challenging times.

Hi! Your sentence is accurate and can be used properly! If you want other ways to phrase your sentence instead of “my difficult times,” you can say these as well: “my challenging days” “my rough patches” “my uphill battles” “my tough times”