I was fiddling with my phone while waiting.
If you had the word with -- then yes you can say it.
Fiddling here means that you were doing something that was random in nature, for example looking at your music options and checking the time and then adjusting the brightness. You had no specific purpose in mind.
However if you spent the time, finally sorting out your phone contacts because you had the opportunity, that would not be fiddling.