Eye contact is the single most obvious indicator that a conversation is happening. You can be silent and still have a conversation with a person when you look each other in the eye. Sometimes, keeping this conversation is difficult enough even with a single person. But it is very difficult with multiple people. Why does the audience love eye contact? Because it's an indication of honesty.
It's not a big secret that public speakers lie. I catch myself lying in almost every presentation. Of course, I try to minimize the amount of lies, but I can't predict everything in advance. Especially when I'm improvising, the desire to cut corners and say things that are easy to say is huge. But it's difficult to lie when you're looking other people in the eye.
According to Dr. Gwyneth Doherty-Sneddon of University of Northumbia at Newcastle,
Looking at faces is quite mentally demanding. We get useful information from the face when listening to someone, but human faces are very stimulating and all this takes processing. So when we are trying to concentrate and process something else that's mentally demanding, it's unhelpful to look at faces.
Looking at human faces requires a tremendous amount of mental processing, and so does lying. It is hard to do both at the same time; that is why parents ask their kids to look them in the eye while they're being questioned. Your audience isn't much different. Most Westerners are suspicious of shifty-eyed people.
NOTE In the East, this ...