Is there some way you could make a fixture to keep your drill and barrel reasonably co-axial?
The design of the brake hone itself doesn't allow for drift from the axis unless it was in the extreme. Flexi Hones are much the same.
Doing it off axis would be easy with a lathe. Not that a competent operator would do this, but getting one of the drill driven hones off axis would actually be more difficult.