This is not easy to get to me as well..
Generally speaking, people are likely to cast their eyes or head down, when they get anxious, hesitated or something guilty feeling. And this speaker feels isolated and gets negative. Then, someone else might have give a suggestion to keep chin up. That is what I read from this text.