The current implementation displays the minimum amount of annotations, only including those rows which are within the currently visible part of the stream. Which changes layout when scrolling sideways, as annotations "enter" or "leave" the screen. Which makes annotation rows hard to track by eye. Screen layout would be more consistent across side scrolled views if those annotations are shown (or space remains reserved) which are within the complete PD output, even when off-screen.
*** This bug has been marked as a duplicate of bug 977 ***