Assuming this thread is true and the discussion here would have any impact:
I would forget building an accurate rifle, especially something of essentially an anti-tank caliber.
Where are you going to source ammunition for it? Do you know how to get primers for a 20mm shell? What about projectiles? Do you have a supply somewhere of projectiles for a 20mm cannon? Otherwise, you'd need a really good lathe, consistent turning material, and a really good micrometer. Without these things, your ammo would be very inconsistent at best and horribly dangerous at worst. Your gunpowder would have to be incredibly consistent if you're trying to outrange something capable of shooting 2,000 yards or more.
Also, do you know the peak pressure of any anti-materiel rounds? Whether .50 BMG, 14.5mm, or 20mm, how do you plan on building an action capable of withstanding such immense peak pressures? How are you going to mount a scope securely enough so that it doesn't come loose every shot and where are you going to find a scope that can withstand the tremendous recoil?
I think what's more feasible is more conventional small arms. Surplus battle rifles like Mosin Nagants and Arisakas might be a good first option. If those aren't available, there are several other forms of firearms that are more easily manufactured. The Sten submachine gun was created because it was very simple to make in very limited shops out of not much more than a few pipes and springs. 12 gauge shotgun designs can be made as simply as two pipes and a nail (though I doubt their safety) going on up from there. A 12 gauge slug out of an appropriate barrel can easily reach 100 yards. Not to mention, if you're loading ammunition for a shotgun, I believe it to be much less sensitive to the powder type used and it leaves open the possibility of much more improvised projectiles. 12 gauge slugs can be cast out of recycled lead fairly easily, and other common enough objects like large steel ball bearings could theoretically be used.
Open bolt submachine guns, 12 gauge slug guns, single shot pistols/derringers/revolvers, all seem to be closer to the direction I would think you should be looking in. Another thing that might be worth looking into is machining sub-caliber inserts to allow a single shot 12 gauge to fire a variety of different ammunition when a source of one type can't be relied upon. Of course, machining the inserts would probably require at least a lathe and probably some other specialty tools.
There are a couple of books that might be worth picking up for you. There's Ragnar Benson's "Guerilla Gunsmithing", or actually any number of books by Benson. He's also got one on Homemade Grenade Launchers. Looking around, I also see such books as "gunsmithing with simple hand tools", "Improvised modified firearms", "Home Workshop Guns for Defense and Resistance." These types of books, while controversial to even exist, do tend to exist for exactly this reason.
I also am reminded of seeing several interesting articles on the home made weaponry of the Syrian rebels. I remember reading one about them taking a .50 caliber machine gun from a fallen plane and rigging up a mount to set it on the back of a truck. Of course even those guys sourced most of their weapons from captured materials from the enemies - whether entire guns or the unused explosives from a dud bomb that failed to go off. One such article is here.
http://www.theatlantic.com/photo/2013/02/diy-weapons-of-the-syrian-rebels/100461/
I would note that even in that series of weapons, they seem to have access to a somewhat sophisticated machine shop. A powered lathe, a drill press, a milling machine would all be greatly helpful. If you could track down any of those along with good stock, you could probably make just about anything with the right knowledge of how to do it. Without any of those, you're going to find it very difficult. And even with all of those machines, for any kind of real accuracy, you would need a rifled barrel. A .416 Barett or 20mm Anzio out of a smoothbore barrel may go a long distance, but I'm sure it won't hit what you're aiming at.
All of the above is obviously under the understanding that you plan to do what is legal under your area's laws or in exigent wartime circumstances as a matter of survival, and that you're not doing it (illegally) presently under a currently peaceful and otherwise presumably standard country that you're living in now.