Since I don’t know of a reasonable way to work around this, I think your best options are either to have your own custom key handler that overrides arrow motion around the borders of such an element, or to find another way to get the effect you’re looking for here (for example by placing widgets).