Great question! Usually we say a fling is with a person or a hobby. A fling is always temporarary.
´´I had a fling with him last year´´. (it means the two people dated briefly)
´´I´m having a fling with learning French´´ (this is not as common, but I have heard similar expressions)