Why do some Americans get angry at other people for not speaking English?

I've hears stories of some Americans telling other people who are speaking a non-English language "This is America, speak English!" even if the conversation has nothing to do with them. Why do they do this?