Lines Matching refs:barrier
17 - Varieties of memory barrier.
21 - SMP barrier pairing.
22 - Examples of memory barrier sequences.
28 - Compiler barrier.
30 - MMIO write barrier.
39 (*) Inter-CPU locking barrier effects.
51 (*) Kernel I/O barrier effects.
327 ordering over the memory operations on either side of the barrier.
344 A write memory barrier gives a guarantee that all the STORE operations
345 specified before the barrier will appear to happen before all the STORE
346 operations specified after the barrier with respect to the other
349 A write barrier is a partial ordering on stores only; it is not required
353 memory system as time progresses. All stores before a write barrier will
354 occur in the sequence _before_ all the stores after the write barrier.
357 dependency barriers; see the "SMP barrier pairing" subsection.
362 A data dependency barrier is a weaker form of read barrier. In the case
365 load will be directed), a data dependency barrier would be required to
369 A data dependency barrier is a partial ordering on interdependent loads
375 considered can then perceive. A data dependency barrier issued by the CPU
378 time the barrier completes, the effects of all the stores prior to that
380 dependency barrier.
382 See the "Examples of memory barrier sequences" subsection for diagrams
389 a full read barrier or better is required. See the "Control dependencies"
393 write barriers; see the "SMP barrier pairing" subsection.
398 A read barrier is a data dependency barrier plus a guarantee that all the
399 LOAD operations specified before the barrier will appear to happen before
400 all the LOAD operations specified after the barrier with respect to the
403 A read barrier is a partial ordering on loads only; it is not required to
410 see the "SMP barrier pairing" subsection.
415 A general memory barrier gives a guarantee that all the LOAD and STORE
416 operations specified before the barrier will appear to happen before all
417 the LOAD and STORE operations specified after the barrier with respect to
420 A general memory barrier is a partial ordering over both loads and stores.
430 This acts as a one-way permeable barrier. It guarantees that all memory
445 This also acts as a one-way permeable barrier. It guarantees that all
455 for other sorts of memory barrier (but note the exceptions mentioned in
456 the subsection "MMIO write barrier"). In addition, a RELEASE+ACQUIRE
457 pair is -not- guaranteed to act as a full memory barrier. However, after
485 memory barrier will be _complete_ by the completion of a memory barrier
486 instruction; the barrier can be considered to draw a line in that CPU's
489 (*) There is no guarantee that issuing a memory barrier on one CPU will have
496 barrier, unless the first CPU _also_ uses a matching memory barrier (see
501 mechanisms should propagate the indirect effects of a memory barrier
522 <write barrier>
542 To deal with this, a data dependency barrier or better must be inserted
549 <write barrier>
552 <data dependency barrier>
576 <write barrier>
579 <data dependency barrier>
583 The data dependency barrier is very important to the RCU system,
595 A load-load control dependency requires a full read memory barrier, not
596 simply a data dependency barrier to make it work correctly. Consider the
601 <data dependency barrier> /* BUG: No data dependency!!! */
613 <read barrier>
646 barrier();
650 barrier();
659 barrier();
723 tempting to add a barrier(), but this does not help. The conditional
724 is gone, and the barrier won't bring it back. Therefore, if you are
804 to the same variable, a barrier() statement is required at the
828 When dealing with CPU-CPU interactions, certain types of memory barrier should
832 other types of barriers, albeit without transitivity. An acquire barrier
833 pairs with a release barrier, but both may also pair with other barriers,
834 including of course general barriers. A write barrier pairs with a data
835 dependency barrier, a control dependency, an acquire barrier, a release
836 barrier, a read barrier, or a general barrier. Similarly a read barrier,
837 control dependency, or a data dependency barrier pairs with a write
838 barrier, an acquire barrier, a release barrier, or a general barrier:
843 <write barrier>
845 <read barrier>
853 <write barrier>
855 <data dependency barrier>
863 <general barrier>
871 Basically, the read barrier always has to be there, even though it can be of
874 [!] Note that the stores before the write barrier would normally be expected to
875 match the loads after the read barrier or the data dependency barrier, and vice
882 <write barrier> \ <read barrier>
898 <write barrier>
915 | | wwwwwwwwwwwwwwww } <--- At this point the write barrier
917 | | : | E=5 | } barrier to be committed before
936 <write barrier>
942 effectively random order, despite the write barrier issued by CPU 1:
974 If, however, a data dependency barrier were to be placed between the load of C
982 <write barrier>
985 <data dependency barrier>
1015 And thirdly, a read barrier acts as a partial order on loads. Consider the
1022 <write barrier>
1028 some effectively random order, despite the write barrier issued by CPU 1:
1051 If, however, a read barrier were to be placed between the load of B and the
1058 <write barrier>
1061 <read barrier>
1081 barrier causes all effects \ +-------+ | |
1088 contained a load of A either side of the read barrier:
1094 <write barrier>
1098 <read barrier>
1121 barrier causes all effects \ +-------+ | |
1128 before the read barrier completes anyway:
1199 Placing a read barrier or a data dependency barrier just before the second
1207 <read barrier>
1211 dependent on the type of barrier used. If there was no change made to the
1266 <general barrier> <general barrier>
1287 For example, suppose that CPU 2's general barrier in the above example
1288 is changed to a read barrier as shown below:
1294 <read barrier> <general barrier>
1301 The key point is that although CPU 2's read barrier orders its pair
1319 (*) Compiler barrier.
1323 (*) MMIO write barrier.
1329 The Linux kernel has an explicit compiler barrier function that prevents the
1332 barrier();
1334 This is a general barrier -- there are no read-read or write-write
1335 variants of barrier(). However, READ_ONCE() and WRITE_ONCE() can be
1336 thought of as weak forms of barrier() that affect only the specific
1339 The barrier() function has the following effects:
1342 barrier() to precede any accesses preceding the barrier().
1531 barrier(), or similar primitives.
1533 This effect could also be achieved using barrier(), but READ_ONCE()
1536 indicated memory locations, while with barrier() the compiler must
1643 barrier. Data dependencies do not impose any additional compiler ordering.
1671 There are some more advanced barrier functions:
1676 barrier after it, depending on the function. It isn't guaranteed to
1677 insert anything more than a compiler barrier in a UP compilation.
1706 smp_read_barrier_depends() data-dependency barrier.
1760 The Linux kernel also has a special barrier for use with memory-mapped I/O
1765 This is a variation on the mandatory write barrier that causes writes to weakly
1832 locks do not imply any sort of barrier.
1838 An ACQUIRE followed by a RELEASE may not be assumed to be full memory barrier
1856 RELEASE may -not- be assumed to be a full memory barrier.
1859 not imply a full memory barrier. Therefore, the CPU's execution of the
1894 a memory barrier, which will force the earlier unlock operation
1904 See also the section on "Inter-CPU locking barrier effects".
1961 A general memory barrier is interpolated automatically by set_current_state()
1969 <general barrier>
1977 which therefore also imply a general memory barrier after setting the state.
1979 interpolate the memory barrier in the right place:
2001 A write memory barrier is implied by wake_up() and co. if and only if they wake
2002 something up. The barrier occurs before the task state is cleared, and so sits
2009 STORE current->state <write barrier>
2010 <general barrier> STORE current->state
2013 To repeat, this write memory barrier is present if and only if something
2022 wake_up(); load from Y sees 1, no memory barrier
2095 On SMP systems locking primitives give a more substantial form of barrier: one
2296 The way to deal with this is to insert a general SMP memory barrier:
2305 In this case, the barrier makes a guarantee that all memory accesses before the
2306 barrier will appear to happen before all the memory accesses after the barrier
2308 the memory accesses before the barrier will be complete by the time the barrier
2312 compiler barrier, thus making sure the compiler emits the instructions in the
2326 about the state (old or new) implies an SMP-conditional general memory barrier
2351 such the implicit memory barrier effects are necessary.
2363 With these the appropriate explicit memory barrier should be used if necessary
2382 sufficient references to make the lock, and thus a memory barrier unnecessary.
2401 [!] Note that special memory barrier primitives are available for these
2403 barriers, and so barrier instructions are superfluous in conjunction with them,
2404 and in such cases the special barrier primitives will be no-ops.
2537 Used with prefetchable I/O memory, an mmiowb() barrier may be required to
2549 required, an mmiowb() barrier can be used. Note that relaxed accesses to
2703 Imagine, then, that two writes are made on the first CPU, with a write barrier
2717 The write memory barrier forces the other CPUs in the system to perceive that
2753 To intervene, we need to interpolate a data dependency barrier or a read
2754 barrier between the loads. This will force the cache to commit its coherency
2777 Whilst most CPUs do imply a data dependency barrier on the read when a memory
2816 A memory barrier isn't sufficient in such a case, but rather the cache must be
2878 barrier. For instance with the following code:
2922 since, without either a write barrier or an WRITE_ONCE(), it can be
2928 may, without a memory barrier or an READ_ONCE() and WRITE_ONCE(), be
2943 the data dependency barrier really becomes necessary as this synchronises both
2947 The Alpha defines the Linux kernel's memory barrier model.