There is clearly difference. 初め = first 始め = beginning, start, launch Difference between those two is "FIRST" and "BEGINNING or START". Hence, "how do you do?" goes to the former but not the latter one. Now you know what comes next? When you want to greet someone you've FIRST met, you will say 初めまして = nice to me you, how do you do? When you will START or BEGIN a new business, you will say あたらしい　しごとを　始めます。

I use 初めまして if they have never met me before. Actually both are correct according to online dictionaries and my dictionary, but I think 始めまして is less common nowadays. Also I often write it in ひらがな.

初めまして → Nice to meet you. 始めまして → I'm starting.

始め　and 初め are just two different ways to write the same word, which of these 2 characters you have to use, depends on the context. In case of the greeting "hajimemashite" it's the second option - 初めまして. Hope this helps.

初めまして → Nice to meet you. 始めまして → I'm starting.