Well that your main loop runs in multiple vbl's shouldn't affect the border removing unless a few cases:
a) Do you use the blitter anything ?
b) Do you use big movem or mul/div ?
These situations can delay the border removing interrupt hence making it flicker. Especially using big blitter ops in hog mode will more or less make it hopeless to remove the border reliably.