The latter point is the most important. There are two points, and you are referring to the second of them (latter.) The second point is the most important. You are referring to the second point, but we don't know how many points there are. There could be two, but equally there could be more.