Port qCompress() to zstream/deflate()

The zlib convenience API we've been using so far has two problems:

- On Windows-64, where sizeof(long) == 4, the use of ulong for sizes
  meant that we could not compress data compressable on other 64-bit
  platforms (Unix). While zstream also uses ulong, being a stream API,
  it allows feeding data in chunks. The total_in and total_out members
  are only required for gzip compression and are otherwise just
  informational. They're unsigned, so their overflow does not cause
  UB. In summary, using zstream + deflate() allows us to compress more
  than 4GiB of data even on Windows-64.

- On all platforms, we always allocated the output buffer in such a
  way as to accommodate the pathological case of random, incompressible
  data, so the output buffer was larger than the input. Using zstream
  + deflate(), we can start with a smaller buffer, then let zlib pick
  up where it left off when it ran out of output buffer space, saving
  memory in the common case that compression meaningfully reduces the
  size. To avoid the first few rounds of reallocations, we continue to
  use zlib's compressBound() for input less than 256KiB.

This completely fixes the compression side of QTBUG-106542 and
QTBUG-104972.

Pick-to: 6.4 6.3 6.2
Fixes: QTBUG-104972
Fixes: QTBUG-106542
Change-Id: Ia7e6c38403906b35462480fd611b482f05a5c59c
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>

1 files changed, 13 insertions, 0 deletions

diff --git a/tests/auto/corelib/text/qbytearray/tst_qbytearray.cpp b/tests/auto/corelib/text/qbytearray/tst_qbytearray.cpp
index 72f66640cd..bf21cf4c9a 100644
--- a/tests/auto/corelib/text/qbytearray/tst_qbytearray.cpp
+++ b/tests/auto/corelib/text/qbytearray/tst_qbytearray.cpp
@@ -360,6 +360,19 @@ void tst_QByteArray::qUncompress4GiBPlus()
         QCOMPARE(c.size(), 4 * GiB + 1);
         QCOMPARE(std::string_view{c}.find_first_not_of('X'),
                  std::string_view::npos);
+
+        // re-compress once
+        // (produces 18MiB, we shouldn't use much more than that in allocated capacity)
+        c = ::qCompress(c);
+        QVERIFY(!c.isNull());
+
+        // and un-compress again, to make sure compression worked (we
+        // can't compare with compressed_3x, because zlib may change):
+        c = ::qUncompress(c);
+
+        QCOMPARE(c.size(), 4 * GiB + 1);
+        QCOMPARE(std::string_view{c}.find_first_not_of('X'),
+                 std::string_view::npos);
     }
 }