|
|
Log in / Subscribe / Register

Support for the TSO memory model on Arm CPUs

Support for the TSO memory model on Arm CPUs

Posted May 3, 2024 14:22 UTC (Fri) by anton (subscriber, #25547)
In reply to: Support for the TSO memory model on Arm CPUs by farnz
Parent article: Support for the TSO memory model on Arm CPUs

There are many unstated (and unsupported) and some stated (but also unsupported) assumptions in your postings, among them:

  • That the speed advantage of weak ordering on some hardware shows itself only on shared data.
  • That other "heavy compute scenarios" (which ones?) do not avoid data sharing, but HPC does.
  • That data sharing has historically (where, when?) been a really slow option, but that's no longer the case.
  • That the performance difference between TSO and a hypothetical weak ordering variant of the A64FX would be 5%-10%.
  • That supercomputer users are willing to throw away 5%-10% in performance for easier porting of code, while for the rest of the computing world performance is much more important, so it's entirely reasonable to let them suffer the problems that come from weak ordering.
  • That porting from Intel/AMD is of particular relevance for the users of A64FX-based supercomputers.
I think that most of these assumptions are not plausible or outright wrong.


to post comments

Support for the TSO memory model on Arm CPUs

Posted May 3, 2024 15:31 UTC (Fri) by farnz (subscriber, #17727) [Link]

In order:

  • If there's no data sharing, then there's no ordering between CPUs to begin with. In this situation, a high-performance implementation of TSO or even sequentially consistent memory models can delay the stores needed to meet the memory model rules until a read happens from another CPU. This, in turn, means that the speed advantage of weak ordering is only guaranteed to show up on shared data, or on low performance implementations.
  • All compute scenarios attempt to avoid data sharing; however, we were talking in the context of A64FX, which is a HPC processor, and the history of HPC matters here.
  • Data sharing in HPC has historically been things like MPI over a network, instead of shared memory. That biases HPC towards a "no shared data" model, meaning that the performance penalty of TSO is less significant than in something like a database server which uses shared state to manage the consistency part of ACID.
  • On the CPU cores we have that allow you to switch between TSO and weaker models, the weaker model has at least a 10% advantage over TSO on a basket of tasks, and considerably more than that in many cases. 5% is thus a reasonable estimate for the cost of TSO mode, assuming that cores are close to optimal today.
  • It's not justabout easier porting of code; weaker memory models result in silent errors, where you thought you had sufficient synchronization, but you did not. In HPC, that leads to unexpected wrong results, where the computation was incorrect.
  • HPC development is not normally done on the supercomputer itself; that's where you run your completed code. Development is done on "normal" laptops and desktops, just like everywhere else; a bug that only shows on the supercomputer but not on the developer workstations is challenging to debug, since the supercomputer is occupied by people running completed code. That makes porting from Intel/AMD relevant to the users of A64FX-based supercomputers, since the initial testing is going to be done on Intel/AMD for CPU-based code; the exception is GPU-based supercomputers, where the testing is done on a GPU from the same manufacturer.

On the other hand, your assertion, which is completely unbacked, is that if A64FX has chosen TSO, it must have done so because TSO and weak memory ordering has the same performance; and yet on every CPU out there that offers both options, including the latest Apple Silicon devices and previous Fujitsu SPARC64 CPUs, TSO has a performance penalty compared to the processor's relaxed memory ordering. However, given that A64FX is swimming against the stream in HPC (most HPC systems now use GPUs for the compute elements, with CPUs there to manage the GPUs, where Fujitsu are using CPUs without GPUs), it is entirely plausible that they chose to minimise risk of bad outputs in preference to extracting the last few percent of performance.

After all, being 4th instead of 1st in the Top500 list is preferable to being 4th but having to withdraw papers based on A64FX computations because they're demonstrably wrong; it's also possible that Fujitsu had actual data from their SPARC64 XII machines that showed that people ran in TSO mode rather than doing the work to get the last few % of performance in RMO mode.


Copyright © 2026, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds