Commit cf5a501d authored by Muchun Song's avatar Muchun Song Committed by Will Deacon

arm64: avoid flushing icache multiple times on contiguous HugeTLB

When a contiguous HugeTLB page is mapped, set_pte_at() will be called
CONT_PTES/CONT_PMDS times.  Therefore, __sync_icache_dcache() will
flush cache multiple times if the page is executable (to ensure
the I-D cache coherency).  However, the first flushing cache already
covers subsequent cache flush operations.  So only flusing cache
for the head page if it is a HugeTLB page to avoid redundant cache
flushing.  In the next patch, it is also depends on this change
since the tail vmemmap pages of HugeTLB is mapped with read-only
meanning only head page struct can be modified.
Signed-off-by: default avatarMuchun Song <songmuchun@bytedance.com>
Reviewed-by: default avatarCatalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/20220302084624.33340-1-songmuchun@bytedance.comSigned-off-by: default avatarWill Deacon <will@kernel.org>
parent 2369f171
...@@ -52,6 +52,13 @@ void __sync_icache_dcache(pte_t pte) ...@@ -52,6 +52,13 @@ void __sync_icache_dcache(pte_t pte)
{ {
struct page *page = pte_page(pte); struct page *page = pte_page(pte);
/*
* HugeTLB pages are always fully mapped, so only setting head page's
* PG_dcache_clean flag is enough.
*/
if (PageHuge(page))
page = compound_head(page);
if (!test_bit(PG_dcache_clean, &page->flags)) { if (!test_bit(PG_dcache_clean, &page->flags)) {
sync_icache_aliases((unsigned long)page_address(page), sync_icache_aliases((unsigned long)page_address(page),
(unsigned long)page_address(page) + (unsigned long)page_address(page) +
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment