QMimeMagicRule: endianness fixes

* apply endianness to the mask as well
* do not apply endianness to "host" entries, they were wrongly behaving
exactly like "big endian" entries.

The issue with the mask was detected by the audio/aac magic
 <match type="big16" value="0xFFF0" mask="0xFFF6" offset="0"/>
which failed to identify the test file ct_faac-adts.aac since it was
applying the mask 0xFFF6 instead of 0xF6FF (on a little-endian machine).

Not yet detected by tst_qmimedatabase which is based on shared-mime-info 1.0,
will be covered by the upgrade to 1.8 in dev.

Change-Id: I4fb7af2d367099817e712b14f2a031066d0ac432
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>

1 files changed, 12 insertions, 2 deletions

diff --git a/src/corelib/mimetypes/qmimemagicrule.cpp b/src/corelib/mimetypes/qmimemagicrule.cpp
index 7a807dccdb..5bbf1bba9d 100644
--- a/src/corelib/mimetypes/qmimemagicrule.cpp
+++ b/src/corelib/mimetypes/qmimemagicrule.cpp
@@ -299,20 +299,30 @@ QMimeMagicRule::QMimeMagicRule(const QString &type,
         }
         break;
     case Big16:
-    case Host16:
     case Little16:
         if (m_number <= quint16(-1)) {
             m_number = m_type == Little16 ? qFromLittleEndian<quint16>(m_number) : qFromBigEndian<quint16>(m_number);
+            if (m_numberMask != 0)
+                m_numberMask = m_type == Little16 ? qFromLittleEndian<quint16>(m_numberMask) : qFromBigEndian<quint16>(m_numberMask);
+        }
+        Q_FALLTHROUGH();
+    case Host16:
+        if (m_number <= quint16(-1)) {
             if (m_numberMask == 0)
                 m_numberMask = quint16(-1);
             m_matchFunction = &QMimeMagicRule::matchNumber<quint16>;
         }
         break;
     case Big32:
-    case Host32:
     case Little32:
         if (m_number <= quint32(-1)) {
             m_number = m_type == Little32 ? qFromLittleEndian<quint32>(m_number) : qFromBigEndian<quint32>(m_number);
+            if (m_numberMask != 0)
+                m_numberMask = m_type == Little32 ? qFromLittleEndian<quint32>(m_numberMask) : qFromBigEndian<quint32>(m_numberMask);
+        }
+        Q_FALLTHROUGH();
+    case Host32:
+        if (m_number <= quint32(-1)) {
             if (m_numberMask == 0)
                 m_numberMask = quint32(-1);
             m_matchFunction = &QMimeMagicRule::matchNumber<quint32>;