two of them are talking about a verb 'study'.
first sentence '공부를 해요' means 'I study'. and it has same meaning with '공부해요'.
So, I doesn't make sense '(person) 을/를 공부해요'. Actually this sentence's mean is 'I study a person'.
You can use 2nd sentence like this way. 수학을 공부해요(I study maths) or 영어를 공부해요(I study English)