QSslSocket: OpenSSL 1.1 backend

This patch-set implements a new QSslSocket backend based on OpenSSL 1.1.

1. General.

The code in this patch was organized to achieve these (somewhat contradicting)
objectives:
- keep the new code free of #if-ery, as far as possible;
- make it easy to clean away dead code when we're eventually able to retire
  out-dated OpenSSL versions;
- reduce the amount of code duplication.

If changes in some file/component were insignificant (~5 one-liners per file),
we still use pp-checks like: #if QT_CONFIG(opensslv11) ... #else ... #endif -
the logic is simple and it's still easy to clean the code if we remove the legacy
back-end. Where it saved #if-ery, we also introduced 'forward-compatible'
macros implementing equivalents of 1.1 functions using older OpenSSL.

In case some class contains a lot of version-specific ifdefs (particularly where
nested #if-ery was complex) we choose to split code into: "pre11" h/cpp files,
"shared" h/cpp files (they preserve their original names, e.g qsslsocket_openssl.cpp)
and "11" h/cpp files. If in future we remove the legacy back-end, "pre11" should be
removed; "shared" and "11" parts - merged.

2. Configuration.

We introduced a new feature 'opensslv11' which complements the pre-existing
'openssl' and 'openssl-linked' features. The 'opensslv11' feature is enabled
by a simple test which either compiles successfully or ends in a compilation
error, depending on a value of the OPENSSL_VERSION_NUMBER constant. If the
feature was enabled, we also append an additional compilation flag
-DOPENSSL_API_COMPAT=0x10100000L to make sure our new code does not contain
deprecated structures, function calls, macro-invocations from OpenSSL < 1.1.

Change-Id: I2064efbe9685def5d2bb2233a66f7581954fb74a
Reviewed-by: André Klitzing <aklitzing@gmail.com>
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>

1 files changed, 30 insertions, 8 deletions

diff --git a/src/network/ssl/qssldiffiehellmanparameters_openssl.cpp b/src/network/ssl/qssldiffiehellmanparameters_openssl.cpp
index 90687b05c5..5ebad822f1 100644
--- a/src/network/ssl/qssldiffiehellmanparameters_openssl.cpp
+++ b/src/network/ssl/qssldiffiehellmanparameters_openssl.cpp
@@ -1,6 +1,7 @@
 /****************************************************************************
 **
 ** Copyright (C) 2015 Mikkel Krautz <mikkel@krautz.dk>
+** Copyright (C) 2016 Richard J. Moore <rich@kde.org>
 ** Contact: https://www.qt.io/licensing/
 **
 ** This file is part of the QtNetwork module of the Qt Toolkit.
@@ -50,8 +51,8 @@
 #include <QtCore/qdebug.h>
 #endif
 
-// For q_BN_is_word.
 #include <openssl/bn.h>
+#include <openssl/dh.h>
 
 QT_BEGIN_NAMESPACE
 
@@ -62,13 +63,6 @@ static bool isSafeDH(DH *dh)
 
     QSslSocketPrivate::ensureInitialized();
 
-    // Mark p < 1024 bits as unsafe.
-    if (q_BN_num_bits(dh->p) < 1024) {
-        return false;
-    }
-
-    if (q_DH_check(dh, &status) != 1)
-        return false;
 
     // From https://wiki.openssl.org/index.php/Diffie-Hellman_parameters:
     //
@@ -81,11 +75,39 @@ static bool isSafeDH(DH *dh)
     //     Without the test, the IETF parameters would
     //     fail validation. For details, see Diffie-Hellman
     //     Parameter Check (when g = 2, must p mod 24 == 11?).
+#if QT_CONFIG(opensslv11)
+    // Mark p < 1024 bits as unsafe.
+    if (q_DH_bits(dh) < 1024)
+        return false;
+
+    if (q_DH_check(dh, &status) != 1)
+        return false;
+
+    const BIGNUM *p = nullptr;
+    const BIGNUM *q = nullptr;
+    const BIGNUM *g = nullptr;
+    q_DH_get0_pqg(dh, &p, &q, &g);
+
+    if (q_BN_is_word(const_cast<BIGNUM *>(g), DH_GENERATOR_2)) {
+        long residue = q_BN_mod_word(p, 24);
+        if (residue == 11 || residue == 23)
+            status &= ~DH_NOT_SUITABLE_GENERATOR;
+    }
+
+#else
+    // Mark p < 1024 bits as unsafe.
+    if (q_BN_num_bits(dh->p) < 1024)
+        return false;
+
+    if (q_DH_check(dh, &status) != 1)
+        return false;
+
     if (q_BN_is_word(dh->g, DH_GENERATOR_2)) {
         long residue = q_BN_mod_word(dh->p, 24);
         if (residue == 11 || residue == 23)
             status &= ~DH_NOT_SUITABLE_GENERATOR;
     }
+#endif
 
     bad |= DH_CHECK_P_NOT_PRIME;
     bad |= DH_CHECK_P_NOT_SAFE_PRIME;