Frustrated with the crowd's guesses of "Puerto Rico", "Guam" and "France" as the origin of the colors on his cap, the lead singer was losing his patience