I'll post a more rational answer.
Yes, it's illegal. Here's the applicable section from the 2000 FFL regulations 922.(k) You may not ever run across anyone who'll call you on it, but that wasn't the question, right?
"It shall be unlawful for any person knowingly to transport, ship, or receive,
in interstate or foreign commerce, any firearm which has had the importer's or
manufacturer's serial number removed, obliterated, or altered or to possess or
receive any firearm which has had the importer's or manufacturer's serial number
removed, obliterated, or altered and has, at any time, been shipped or transported in
interstate or foreign commerce."