Add a "signature" to AST files to verify that they haven't changed

Since the order of the IDs in the AST file (e.g. DeclIDs, SelectorIDs)
is not stable, it is not safe to load an AST file that depends on
another AST file that has been rebuilt since the importer was built,
even if "nothing changed". We previously used size and modtime to check
this, but I've seen cases where a module rebuilt quickly enough to foil
this check and caused very hard to debug build errors.

To save cycles when we're loading the AST, we just generate a random
nonce value and check that it hasn't changed when we load an imported
module, rather than actually hash the whole file.

This is slightly complicated by the fact that we need to verify the
signature inside addModule, since we might otherwise consider that a
mdoule is "OutOfDate" when really it is the importer that is out of
date. I didn't see any regressions in module load time after this
change.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@220493 91177308-0d34-0410-b5e6-96231b3b80d8

4 files changed, 20 insertions, 1 deletions

diff --git a/include/clang/Serialization/ASTBitCodes.h b/include/clang/Serialization/ASTBitCodes.h
index 26e0ecde15..241c62e4ce 100644
--- a/include/clang/Serialization/ASTBitCodes.h
+++ b/include/clang/Serialization/ASTBitCodes.h
@@ -288,7 +288,10 @@ namespace clang {
 
       /// \brief Record code for the module map file that was used to build this
       /// AST file.
-      MODULE_MAP_FILE = 14
+      MODULE_MAP_FILE = 14,
+
+      /// \brief Record code for the signature that identifiers this AST file.
+      SIGNATURE = 15
     };
 
     /// \brief Record types that occur within the input-files block
diff --git a/include/clang/Serialization/ASTReader.h b/include/clang/Serialization/ASTReader.h
index 730a257bca..d967ec8d21 100644
--- a/include/clang/Serialization/ASTReader.h
+++ b/include/clang/Serialization/ASTReader.h
@@ -1125,6 +1125,7 @@ private:
                             SourceLocation ImportLoc, ModuleFile *ImportedBy,
                             SmallVectorImpl<ImportedModule> &Loaded,
                             off_t ExpectedSize, time_t ExpectedModTime,
+                            serialization::ASTFileSignature ExpectedSignature,
                             unsigned ClientLoadCapabilities);
   ASTReadResult ReadControlBlock(ModuleFile &F,
                                  SmallVectorImpl<ImportedModule> &Loaded,
diff --git a/include/clang/Serialization/Module.h b/include/clang/Serialization/Module.h
index a889e8b654..f6889cfe8e 100644
--- a/include/clang/Serialization/Module.h
+++ b/include/clang/Serialization/Module.h
@@ -97,6 +97,8 @@ public:
   bool isNotFound() const { return Val.getInt() == NotFound; }
 };
 
+typedef unsigned ASTFileSignature;
+
 /// \brief Information about a module that has been loaded by the ASTReader.
 ///
 /// Each instance of the Module class corresponds to a single AST file, which
@@ -152,6 +154,10 @@ public:
   /// \brief The file entry for the module file.
   const FileEntry *File;
 
+  /// \brief The signature of the module file, which may be used along with size
+  /// and modification time to identify this particular file.
+  ASTFileSignature Signature;
+
   /// \brief Whether this module has been directly imported by the
   /// user.
   bool DirectlyImported;
diff --git a/include/clang/Serialization/ModuleManager.h b/include/clang/Serialization/ModuleManager.h
index 96c3619510..3d10fad0a1 100644
--- a/include/clang/Serialization/ModuleManager.h
+++ b/include/clang/Serialization/ModuleManager.h
@@ -179,6 +179,12 @@ public:
   /// \param ExpectedModTime The expected modification time of the module
   /// file, used for validation. This will be zero if unknown.
   ///
+  /// \param ExpectedSignature The expected signature of the module file, used
+  /// for validation. This will be zero if unknown.
+  ///
+  /// \param ReadSignature Reads the signature from an AST file without actually
+  /// loading it.
+  ///
   /// \param Module A pointer to the module file if the module was successfully
   /// loaded.
   ///
@@ -191,6 +197,9 @@ public:
                             SourceLocation ImportLoc,
                             ModuleFile *ImportedBy, unsigned Generation,
                             off_t ExpectedSize, time_t ExpectedModTime,
+                            ASTFileSignature ExpectedSignature,
+                            std::function<ASTFileSignature(llvm::BitstreamReader &)>
+                                ReadSignature,
                             ModuleFile *&Module,
                             std::string &ErrorStr);