Lines Matching refs:metadata

13 Almost all metadata on XFS is dynamically allocated. The only fixed location
14 metadata is the allocation group headers (SB, AGF, AGFL and AGI), while all
15 other metadata structures need to be discovered by walking the filesystem
28 However, if we scale the filesystem up to 1PB, we now have 10x as much metadata
39 One of the problems with the current metadata format is that apart from the
40 magic number in the metadata block, we have no other way of identifying what it
42 you can't look at a single metadata block in isolation and say "yes, it is
46 verification of metadata values, looking for values that are in range (and hence
53 Hence we need to record more information into the metadata to allow us to
54 quickly determine if the metadata is intact and can be ignored for the purpose
57 self describing metadata.
59 The first, fundamental requirement of self describing metadata is that the
60 metadata object contains some form of unique identifier in a well known
62 hence parse and verify the metadata object. IF we can't independently identify
63 the type of metadata in the object, then the metadata doesn't describe itself
66 Luckily, almost all XFS metadata has magic numbers embedded already - only the
70 numbers in the metadata objects. That is, if it has the current magic number,
71 the metadata isn't self identifying. If it contains a new magic number, it is
73 metadata object at runtime, during forensic analysis or repair.
75 As a primary concern, self describing metadata needs some form of overall
76 integrity checking. We cannot trust the metadata if we cannot verify that it has
78 integrity check, and this is done by adding CRC32c validation to the metadata
79 block. If we can verify the block contains the metadata it was intended to
82 CRC32c was selected as metadata cannot be more than 64k in length in XFS and
84 metadata blocks. CRC32c is also now hardware accelerated on common CPUs so it is
92 Self describing metadata needs to contain enough information so that the
93 metadata block can be verified as being in the correct place without needing to
94 look at any other metadata. This means it needs to contain location information.
95 Just adding a block number to the metadata is not sufficient to protect against
100 Another key information point in forensic analysis is knowing who the metadata
103 of the block is important as it allows us to find other related metadata to
108 in the metadata we have no idea of the scope of the corruption. If we have an
109 owner field in the metadata object, we can immediately do top down validation to
112 Different types of metadata have different owner identifiers. For example,
115 contents of the owner field are determined by the type of metadata object we are
119 Self describing metadata also needs to contain some indication of when it was
122 metadata blocks based on modification times is important as it can indicate
127 For example, we can determine whether a metadata object is supposed to be free
130 compared to when the metadata object itself was last written.  If the free space
134 To provide this "written timestamp", each metadata block gets the Log Sequence
138 the LSN we can tell if the corrupted metadata all belonged to the same log
140 the first and last instance of corrupt metadata on disk and, further, how much
147 Validation of self-describing metadata takes place at runtime in two places:
153 modification process, and seeks only to check that the metadata is what it says
154 it is and that the metadata fields are within bounds and internally consistent.
157 metadata, or there may be corruption of interblock relationships (e.g. corrupted
162 For read verification, the caller needs to specify the expected type of metadata
163 that it should see, and the IO completion process verifies that the metadata
175 object specific metadata validation. If any of these checks fail, then the
199 Depending on the metadata, this information may be part of a header structure
200 separate to the metadata contents, or may be distributed through an existing
201 structure. The latter occurs with metadata that already contains some of this
204 Other metadata may have different formats for the information, but the same
214 	  well. hence the additional metadata headers change the overall format
215 	  of the metadata.
320 This will verify the internal structure of the metadata before we go any
321 further, detecting corruptions that have occurred as the metadata has been
322 modified in memory. If the metadata verifies OK, and CRCs are enabled, we then
324 metadata. Once this is done, we can issue the IO.