The first sentence is ambiguous. "I met him yesterday" could mean you met for the first time. Or it could be the first part of a more complex phrase. "I met him yesterday [at a coffee shop]. He said he wasn't coming." It's part of the verb phrase "to meet somebody somewhere". Example: I met my friend yesterday at the restaurant. We planned to meet at 8pm but she was late so we ended up meeting at 9pm. "I met with him yesterday" has a business-meeting connotation. You meet with somebody about something. You don't "meet with" friends, unless you have something serious to discuss. You "meet up with" friends.

The first sentence means it was the first time you saw the person.

Yes. The first is more as if you met him for the first time yesterday and the second would be a meeting to discuss or solve something.

The second sentence definitely means you had a meeting; the first one, as Kellie said, could mean that it was the first time yesterday, but some people leave the "with" out, when they have a meeting with people; this could be confusing, so it's good to know the difference!

I think your interpretation of these two expressions is correct