Tags are stored in separate physical memory

Posted Oct 16, 2020 3:13 UTC (Fri) by CChittleborough (subscriber, #60775)
In reply to: The Arm64 memory tagging extension in Linux by mm7323
Parent article: The Arm64 memory tagging extension in Linux

I had the same question as mm7323. I happen to have the Armv8-A Architecture Reference Manual open, so I did a text search. I was surprised by what I found: you need separate physical memory!

“Tag load and store instructions to access Allocation tags in a tag physical address space, separate to the data physical address space accessed by data load and store instructions to access data in normal memory and devices.”
— §D6.1 on p2660 (of 7900!)

So chips which support MTE need to store a 4-bit tags for every 16-byte ‘granule’ of data. Moreover, tags applies to Logical Addresses, ie., virtual addresses, so you need enough tag storage to cover all the virtual addresses you will ever allocate.

If you require physical memory to be contiguous, you could just reserve some of it at boot time by setting a single implementation-specific register. But if you want to allow non-contiguous physical memory ranges, you might want multiple registers. Supporting hot-plugging of memory would be quite hairy. Maybe MTE and hot-pluggable memory would be mutally exclusive?

Are there any chips which support MTE on the market yet? Does anyone know how they handle tag storage?

Tags are stored in separate physical memory

Posted Oct 16, 2020 3:35 UTC (Fri) by mm7323 (subscriber, #87386) [Link] (1 responses)

Wowsers. I wonder what happens when you try and set more tags than there is memory to store the granule keys...

I also don't see an instruction to 'unset' a key either.

Tags are stored in separate physical memory

Posted Oct 16, 2020 5:23 UTC (Fri) by mm7323 (subscriber, #87386) [Link]

In answer to my own question, and having looked at the reference manual, it looks like the tags are translated from the virtual into physical tags, then keys looked up at that point.

Therefore there just needs to be $mem/32 bytes of extra RAM set aside for storing the Granule tags for each physical address, and it can never run out.

Other things of curiosity - the synchronous tag error mode has a significant runtime overhead, as does a tag value of all 1's, according to the manual.

Tags are stored in separate physical memory

Posted Oct 16, 2020 13:15 UTC (Fri) by anton (subscriber, #25547) [Link]

I expect it to be in physical memory: Why would you tag memory that is not physically backed? Well, actually there is quite a bit of virtual memory that is never used and is only ever backed by the same zero-filled page; do you need such unused memory to have tags already before use?

For physical RAM, I expect that it uses some bits that you get with ECC memory (you only need 8 SECDED ECC bits for 128bits (16 bytes) of payload, leaving 8 bits for other purposes.

For virtual memory backed by mass storage, some systems have mass storage with larger sectors to accomodate meta-data, but this mass storage is rare and therefore expensive. It may be cheaper to provide enough ECC RAM that you don't need swap space.

Tags are stored in separate physical memory

Posted Oct 16, 2020 16:30 UTC (Fri) by mwsealey (subscriber, #71282) [Link] (1 responses)

It does need a bus protocol that can transport it, so new interconnects and a memory controller that can receive and respond to it as the endpoint.

A "seperate PA space" isn't really "separate physical memory" in the sense that you don't need a *dedicated* memory controller or a particular SRAM block, for example, for MTE alone.

It's no different to, for example, the logical separation Secure vs Non-Secure memory. In theory, Secure 0x8000 and Non-Secure 0x8000 are two seperate PA spaces - the two numerical addresses aren't the same address. In practical reality, it's a "n+1th bit" of addressing, and the underlying memory technology (the cells or gates storing the information) are the same one for each address.

Most modern DRAM controllers have a TrustZone address space controller built in (or something broadly similar) which can effectively allow or deny access to particular regions of RAM based on the security state. It's just an address range, and the differentiator between being secure or non-secure read or write in the system being a single bit. So you can have 4GB of RAM and 2GB of it be Secure and 2GB of it be Non-Secure, but they're on the same 32Gbit DRAM die.. or separate 16Gbit ones, or striped across them, whatever you like.

Where that memory 'lives' is up to that system, maybe the top MBs will be partitioned off by the memory controller and interconnect for the tag space, and the tag management instructions will essentially be putting data in there. Your OS will be none the wiser except that it may be told that it only has 7.xGB available 'Normal' physical memory (which we already see since we can have software carve-outs for secure firmware or other purposes). It's not really an architectural question in the CPU sense, more of a thing for memory controller vendors to describe how they want to make it happen.

Tags are stored in separate physical memory

Posted Sep 15, 2022 4:47 UTC (Thu) by nikhildevshatwar (guest, #159628) [Link]

I am still wondering how it is sufficient to maintain tags for only that portion of virtual memory that is resident (i.e. not swapped out) and has valid mapping to physical addresses.

In reality, there will be swapping and the total virtual memory will be much more than the physical memory.
Is the kernel going to modify the swap in/out handler to also backup the tag memory corresponding to the data memory when swapping in/out.

If not done this way, the process which behaved nicely, will start seeing tag mismatch exceptions if it lost the tag memory while swapping the pages.