macOS: Don't include QSystemTrayIcon windows in Dock menu window list

As of 3fcdb6cb6e35a37f9b511ec2705336102c194d6b we now have a hidden
Window menu, which results in macOS also populating the Dock menu
with the application's windows.

But since the AppKit logic for populating the window menu happens when
the window is shown, and we create the menu after many windows have
been already shown, we had a workaround to add the shown windows
manually.

Unfortunately this workaround didn't take into account the NSWindow
excludedFromWindowsMenu property, nor did it include various other
checks that AppKit itself uses to decide if a window should be
included in the window list, resulting in adding the NSStatusBarWindow
that AppKit uses to manage the status bar items.

Instead of trying to replicate the AppKit logic, we toggle the
excludedFromWindowsMenu property back and forth, which triggers
AppKit to reevaluate the situation for each window, and add it
to the window menu if necessary.

Fixes: QTBUG-107008
Pick-to: 6.4
Change-Id: I6c7f61c1f4610fec9ce1f814fcea2b6140230602
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>

1 files changed, 9 insertions, 3 deletions

diff --git a/src/plugins/platforms/cocoa/qcocoamenubar.mm b/src/plugins/platforms/cocoa/qcocoamenubar.mm
index bd8a08f54a..87eb3e1450 100644
--- a/src/plugins/platforms/cocoa/qcocoamenubar.mm
+++ b/src/plugins/platforms/cocoa/qcocoamenubar.mm
@@ -339,10 +339,16 @@ void QCocoaMenuBar::insertWindowMenu()
     [mainMenu insertItem:winMenuItem atIndex:mainMenu.itemArray.count];
     app.windowsMenu = winMenuItem.submenu;
 
-    // Windows, created and 'ordered front' before, will not be in this menu:
+    // Windows that have already been ordered in at this point have already been
+    // evaluated by AppKit via _addToWindowsMenuIfNecessary and added to the menu,
+    // but since the menu didn't exist at that point the addition was a noop.
+    // Instead of trying to duplicate the logic AppKit uses for deciding if
+    // a window should be part of the Window menu we toggle one of the settings
+    // that definitely will affect this, which results in AppKit reevaluating the
+    // situation and adding the window to the menu if necessary.
     for (NSWindow *win in app.windows) {
-        if (win.title && ![win.title isEqualToString:@""])
-            [app addWindowsItem:win title:win.title filename:NO];
+        win.excludedFromWindowsMenu = !win.excludedFromWindowsMenu;
+        win.excludedFromWindowsMenu = !win.excludedFromWindowsMenu;
     }
 }