To see if this is the case, go to View > Show Invisibles (Command I for Mac and Control I for Windows) If you see any unnecessary space, paragraph break, tab, or Next Page markers at the end of your story, delete them. If you don't need to have Auto Page Insertion on, go ahead and turn it off. Then delete the unwanted page(s).
Background:
Hidden characters at the end of the text string contained in the Automatic Text Boxes is forcing create of new pages. If Auto Page Insertion is enabled, QuarkXPress automatically replaces deleted pages as needed to contain any text overflow. A space, paragraph break, tab, or Next Page marker that follows the last character of a story can cause a page to be inserted into your layout