never create a console for command line tools launched from gui apps

commit 219b0d217 fixed forwarding of stdio to a console. unfortunately,
it also forced the creation of a console in the first place if there was
none, which is not part of the contract.
instead, give the child process a window (== console) only we we already
have one. this can be done irrespective of our channel mode, which has
the nice side effect that an explicit redirection to a console would now
also work.

Change-Id: Id25cab5da1ac1cc8ce452127ff95bac8d0a0fea8
Reviewed-by: Lars Knoll <lars.knoll@digia.com>
Reviewed-by: Friedemann Kleint <Friedemann.Kleint@digia.com>
Reviewed-by: Maurice Kalinowski <maurice.kalinowski@digia.com>

1 files changed, 6 insertions, 3 deletions

diff --git a/src/corelib/io/qprocess_win.cpp b/src/corelib/io/qprocess_win.cpp
index 49c699d6cc..cd5147c4a4 100644
--- a/src/corelib/io/qprocess_win.cpp
+++ b/src/corelib/io/qprocess_win.cpp
@@ -479,9 +479,12 @@ void QProcessPrivate::startProcess()
     qDebug("   pass environment : %s", environment.isEmpty() ? "no" : "yes");
 #endif
 
-    // Forwarded channels must not set the CREATE_NO_WINDOW flag because this
-    // will render the stdout/stderr handles we're passing useless.
-    DWORD dwCreationFlags = (processChannelMode == QProcess::ForwardedChannels ? 0 : CREATE_NO_WINDOW);
+    // We cannot unconditionally set the CREATE_NO_WINDOW flag, because this
+    // will render the stdout/stderr handles connected to a console useless
+    // (this typically affects ForwardedChannels mode).
+    // However, we also do not want console tools launched from a GUI app to
+    // create new console windows (behavior consistent with UNIX).
+    DWORD dwCreationFlags = (GetConsoleWindow() ? 0 : CREATE_NO_WINDOW);
     dwCreationFlags |= CREATE_UNICODE_ENVIRONMENT;
     STARTUPINFOW startupInfo = { sizeof( STARTUPINFO ), 0, 0, 0,
                                  (ulong)CW_USEDEFAULT, (ulong)CW_USEDEFAULT,