I'm may be mistaken, but in most states if a law enforcement officers asks you for ID you may refuse but you still are required to identify yourself at least verbally. Now you can always say that that you don't have ID because you're not driving and don't have your driver's license.
Now that being said, if an officers has some reason to believe that there is some sort of mischief afoot then they can pat you down for weapons. This is what has come to be referred to as a "Terry Stop" as a result of the Supreme Court case Terry v. Ohio.
Police–citizen encounters
In the United States, interactions between police and citizens fall into three general categories: consensual (“contact” or “conversation”), detention (often called a Terry stop, after Terry v. Ohio, 392 U.S. 1 (1968)), or arrest. “Stop and identify” laws pertain to detentions.
Different obligations apply to drivers of automobiles, who generally are required by state vehicle codes to present a driver’s license to a police officer upon request.
[edit] Consensual
At any time, a police officer may approach a person and ask questions. The objective may simply be a friendly conversation; however, the officer also may suspect involvement in a crime, but lack “specific and articulable facts”[4] that would justify a detention or arrest, and hope to obtain these facts from the questioning.
The person approached is not required to identify herself or answer any other questions, and may leave at any time.[5] Police are not usually required to tell a person that she is free to decline to answer questions and go about her business;[6] however, a person can usually determine whether the interaction is consensual by asking, “Am I free to go?”[7][8]
[edit] Detention
A person is detained when circumstances are such that a reasonable person would believe he is not free to leave.[9]
Police may briefly detain a person if they have reasonable suspicion that the person has committed, is committing, or is about to commit a crime
http://en.wikipedia.org/wiki/Stop_and_Identify_statutes
Unless the OP was suspicious in some way or the police officer suspected that OP was armed....
All he had to do - was keep his mouth shut and tell the officer politely that he did not wish to continue the conversation. They can ask all they want. You (unless you are in a vehicle) and depending on local law - can also answer. Or not answer. You may also invoke your 4th and 5th amendment rights.
Yes sir my name is John Smith. Am I free to go? OK. Have a good evening.
