To refer to being the first one/thing among a group, or for the first time, we usually use اولor اولین. These words are used in both formal and informal forms, but they're more common in informal cases. نخست or نخستین have the same meaning, while they're literary and are used in written form. For example: I was the first one who wrote this essay.=من اولین نفری بودم که این مقاله را نوشتم This is my first travel to the USA= این اولین سفرمن به آمریکااست Pay attention to the point that اول and نخست are Persian ordinal numbers, while اولین and نخستین are superlatives. Like I mentioned in my comment, یکم is usually used to refer to the first day of month. For example: October 1= یکم اکتبر Tomorrow is December 1= فردا یکم دسامبر است.

Imagine, you have a few objects that are arranged in a row. To counting them, in the ordinal position, you say; Avval, Dovvom, Sevvom, Cha-harom, Panjom, … As you see they are called ordinal numbers or اعداد ترتیبی Avval and Yekom are the same. Nokhost/Nakhost is used in the formal documents. The first object in the series is called Av-va-leen. Nokhosteen/Nakhosteen is the formal word of Avvaleen. Examples: او (نفر) اول شد. او نفر یکم مسابقه شد. او اولین نفر بود که به خط پایان مسابقه رسید. اولین بار که او را دیدم شناختم. نخستین انسانهای اولیه اتش را کشف کردند. نخست وزیر کانادا وارد ایران شد.

اولین ، نخستین mean "the first" like It is the first device here : این اولین (نخستین) دستگاه اینجاست اول ، نخست simply mean first I was first there : من اول ( نخست) آنجا بودم نخست has a more literal and formal use than اول but اول is used almost everywhere in everyday conversations and formal writings whereas نخست and نخستین are not used in non-formal conversations. and یکم has a very rare usage in a usual applications and i think is rather a new word created by intellectuals

Interesting question! I'm going to add some points and explain it in a different simple way. Choosing between اول and اولین depends on the situation. If you are using it independently, اول is the right word : من اول آنجا بودم. از اول تا آخر مسابقه را تماشا کردم If you are using it with another word to describe it, (I was the first person) in this case that word is "person": - Use اول if that word is coming first : من نفر اول بودم - Use اولین if that word is coming second: من اولین نفر بودم Hope it helps