I agree that a .22LR is a good rifle to start with. I grew up shooting a 30.06, so to me that is what I think of when the word rifle comes to mind. If recoil is an issue, then a .22LR is good, cheap, and readily available. I would recommend either a Marlin 60 (semi-auto, tube fed, 15 shots.) a Marlin 25N (bolt action, 7 round clip, but 10 rounders are available.) or a Ruger 10/22 (semi-auto, box clip fed 10 round standard, higher cap mags available.) All depends on what you like. For accuracy, it is hard to beat a bolt action.
Now if recoil is not an issue, and you want something that will be accurate and inexpensive, then I would get the new Remington 710 in 30.06. It costs less than $350 with a 3x9 scope included and bore sighted.
If you are looking for something just to blow away targets but don’t really care how accurate it is long distances, you can get a Ruger Mini-14 Ranch for less than $600 in stainless and synthetic. Or if you want the AR-15 style gun, Bushmaster makes a nice XM15 or similar model for around $800 last I checked.
Hope this helps some.
In Liberty,
Chulain