macOS: Handle window titlebar buttons independently from style mask

Style masks such as NSWindowStyleMask{Resizable,Miniaturizable}
affect whether the window has a title bar button for the action,
but also whether the window can be resized or minimized through
other means, for example if the window border can be dragged
to resize.

By decoupling the visibility and enablement of the title bar buttons
from the style mask we can individually control the buttons, and
leave the style mask set to enable behaviors we always want. We
were already doing this for the NSWindowZoomButton.

Unfortunately AppKit not only checks NSWindowStyleMaskMiniaturizable
during a call to miniaturize, but also whether the title bar button
is enabled. To allow minimizing windows without the titlebar button
we detect the situation and give AppKit a NSWindowMiniaturizeButton
that we haven't disabled. The alternative would be to temporarily
enable the NSWindowMiniaturizeButton during the minimize, but this
results in the button flashing yellow for the duration of the
animation.

Task-number: QTBUG-65637
Task-number: QTBUG-46882
Task-number: QTBUG-64994
Task-number: QTBUG-71485
Change-Id: I2c1a9564d8b7516476aa018b2820670199124bc2
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>

1 files changed, 28 insertions, 0 deletions

diff --git a/src/plugins/platforms/cocoa/qnswindow.mm b/src/plugins/platforms/cocoa/qnswindow.mm
index bdd2c07f02..7c9e0dce2d 100644
--- a/src/plugins/platforms/cocoa/qnswindow.mm
+++ b/src/plugins/platforms/cocoa/qnswindow.mm
@@ -249,6 +249,7 @@ OSStatus CGSClearWindowTags(const CGSConnectionID, const CGSWindowID, int *, int
 {
     // Member variables
     QPointer<QCocoaWindow> m_platformWindow;
+    bool m_isMinimizing;
 }
 
 - (instancetype)initWithContentRect:(NSRect)contentRect styleMask:(NSWindowStyleMask)style
@@ -260,6 +261,8 @@ OSStatus CGSClearWindowTags(const CGSConnectionID, const CGSWindowID, int *, int
     // we can properly reflect the window's state during initialization.
     m_platformWindow = window;
 
+    m_isMinimizing = false;
+
     return [super initWithContentRect:contentRect styleMask:style backing:backingStoreType defer:defer screen:screen];
 }
 
@@ -395,6 +398,31 @@ OSStatus CGSClearWindowTags(const CGSConnectionID, const CGSWindowID, int *, int
         [qnsview_cast(m_platformWindow->view()) handleFrameStrutMouseEvent:theEvent];
 }
 
+- (void)miniaturize:(id)sender
+{
+    QBoolBlocker miniaturizeTracker(m_isMinimizing, true);
+    [super miniaturize:sender];
+}
+
+- (NSButton *)standardWindowButton:(NSWindowButton)buttonType
+{
+    NSButton *button = [super standardWindowButton:buttonType];
+
+    // When an NSWindow is asked to minimize it will check the
+    // NSWindowMiniaturizeButton for enablement before continuing,
+    // even if the style mask includes NSWindowStyleMaskMiniaturizable.
+    // To ensure that a window can be minimized, even when the
+    // minimize button has been disabled in response to the user
+    // setting CustomizeWindowHint, we temporarily return a default
+    // minimize-button that we haven't modified in updateTitleBarButtons.
+    // This ensures the window can be minimized, without visually
+    // toggling the actual minimize-button on and off.
+    if (buttonType == NSWindowMiniaturizeButton && m_isMinimizing && !button.enabled)
+        return [NSWindow standardWindowButton:buttonType forStyleMask:self.styleMask];
+
+    return button;
+}
+
 - (void)closeAndRelease
 {
     qCDebug(lcQpaWindow) << "Closing and releasing" << self;