Depends on the context.
Normally the form of the verb is determined by the phrase or construction you are using.
Usually in the language either "to study" of "studying" is used, just "study" is more rare.
Also studying is a gerund, i.e. noun derived from verb.
1 I hate studying. I enjoy studying.
2 I like to study a lot (both versions are possible with a slight difference in meaning).
3 Please help me study, asap! (help smb. to do sth. is also possible)