I understand your confusion. Many words are often written in katakana, even if they have Japanese origin. Sometimes its because the kanji is difficult and can mislead when written in hiragana, and sometimes without proper reasons. イルカ is probably between. It is written in 海豚, which means 'sea pig', and for me it doesn't look good, because 'sea pig' is more suitable for seals, I think. And just look at it; いるかはどこにいるの？(where is a dolphin?) Maybe you find it a bit difficult to see where to set off the words. Anyways, these writing ways all come out of custom, and unfortunately you have to learn it one by one. 今朝、自転車の乗っている時に、イルカが泳いでいるのを見ました。 乗りながら gives off the nuance that you saw them during your bike-ride. 時に sounds right, because the moment must be temporal. I'm so jealous!! I need to have an excursion. All the best!

今朝、自転車に乗りながらイルカが泳いでいるのを見ました。or 今朝自転車に乗りながらイルカが泳ぐのを見ました。would be correct but your sentence makes sense enough. Concerning the katakana problem you can see in the italki article "gairaigo",but it is difficult to explain for me "why".

I totally agree with Ryoko. After all, there is no rule, I guess.

Since either I don't know why we write イルカ in katakana, but it happens so often when we write the kind of animals, plants, vegetables, fruits.... As Bleh-san said, maybe it's to distinguish from other words in the sentence, or sometimes, it depends on the writers liking. For example, "apple" is りんご in Japanese and the kanji is 林檎 but we rarely use kanji, some use hiragana and others use katakana. After all, there is no rule. I hope you understand a liitle more than before.