Definitely not 'close by'. That has a different meaning from your intended one. "Look at the painting close by" is equivalent to "Look at the painting nearby" - in other words, a painting which happens to be near to whatever your reference point is.
"At close range" makes sense, but only if you were going to shoot it! "Close range" refers to a target. If you want an expression with 'at close..', you could say 'at close quarters'.
Of the remaining two, 'Look at the painting close up' sounds most natural to me.