Skip to content

L
linux
Commit 6bfd2d44 authored by Linus Torvalds's avatar Linus Torvalds
Browse files
Options

Merge tag 'irq-core-2024-05-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 

Pull interrupt subsystem updates from Thomas Gleixner:
 "Core code:

   - Interrupt storm detection for the lockup watchdog:

     Lockups which are caused by interrupt storms are not easy to debug
     because there is no information about the events which make the
     lockup detector trigger.

     To make this more user friendly, provide an extenstion to interrupt
     statistics which allows to take snapshots and an interface to
     retrieve the delta to the snapshot. Use this new mechanism in the
     watchdog code to do a two stage lockup analysis by taking the
     snapshot and printing the deltas for the topmost active interrupts
     on the second trigger.

     Note: This contains both the interrupt and the watchdog changes as
     the latter depend on the former obviously.

   - Avoid summation loops in the /proc/interrupts output and use the
     global counter when possible

   - Skip suspended interrupts on CPU hotplug operations to ensure that
     they are not delivered before the system resumes the device drivers
     when coming out of suspend.

   - On CPU hot-unplug interrupts which are affine to the outgoing CPU
     are migrated to a different CPU in the affinity mask. This can fail
     when the CPUs have no vectors left. Instead of giving up try to
     migrate it to any online CPU and thereby breaking the affinity
     setting in order to prevent a stale device interrupt which targets
     an offline CPU

   - The usual small cleanups

  Driver code:

   - Support for the RISCV AIA MSI controller

   - Make the interrupt allocation for the Loongson PCH controller more
     flexible to prevent vector exhaustion

   - The usual set of cleanups and fixes all over the place"

* tag 'irq-core-2024-05-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (51 commits)
  irqchip/gic-v3-its: Remove BUG_ON in its_vpe_irq_domain_alloc
  cpuidle: Avoid explicit cpumask allocation on stack
  irqchip/sifive-plic: Avoid explicit cpumask allocation on stack
  irqchip/riscv-aplic-direct: Avoid explicit cpumask allocation on stack
  irqchip/loongson-eiointc: Avoid explicit cpumask allocation on stack
  irqchip/gic-v3-its: Avoid explicit cpumask allocation on stack
  irqchip/irq-bcm6345-l1: Avoid explicit cpumask allocation on stack
  cpumask: Introduce cpumask_first_and_and()
  irqchip/irq-brcmstb-l2: Avoid saving mask on shutdown
  genirq: Reuse irq_is_nmi()
  genirq/cpuhotplug: Retry with cpu_online_mask when migration fails
  genirq/cpuhotplug: Skip suspended interrupts when restoring affinity
  arm64: dts: st: Add interrupt parent to pinctrl on stm32mp251
  arm64: dts: st: Add exti1 and exti2 nodes on stm32mp251
  ARM: dts: stm32: List exti parent interrupts on stm32mp131
  ARM: dts: stm32: List exti parent interrupts on stm32mp151
  arm64: Kconfig.platforms: Enable STM32_EXTI for ARCH_STM32
  irqchip/stm32-exti: Mark events reserved with RIF configuration check
  irqchip/stm32-exti: Skip secure events
  irqchip/stm32-exti: Convert driver to standard PM
  ...
parents a9d9ce3f 382d2ffe
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 3977 additions and 182 deletions
Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml 0 → 100644
View file @ 6bfd2d44
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/interrupt-controller/riscv,aplic.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: RISC-V Advanced Platform Level Interrupt Controller (APLIC)
maintainers:
- Anup Patel <anup@brainfault.org>
description:
The RISC-V advanced interrupt architecture (AIA) defines an advanced
platform level interrupt controller (APLIC) for handling wired interrupts
in a RISC-V platform. The RISC-V AIA specification can be found at
https://github.com/riscv/riscv-aia.
The RISC-V APLIC is implemented as hierarchical APLIC domains where all
interrupt sources connect to the root APLIC domain and a parent APLIC
domain can delegate interrupt sources to it's child APLIC domains. There
is one device tree node for each APLIC domain.
allOf:
- $ref: /schemas/interrupt-controller.yaml#
properties:
compatible:
items:
- enum:
- qemu,aplic
- const: riscv,aplic
reg:
maxItems: 1
interrupt-controller: true
"#interrupt-cells":
const: 2
interrupts-extended:
minItems: 1
maxItems: 16384
description:
Given APLIC domain directly injects external interrupts to a set of
RISC-V HARTS (or CPUs). Each node pointed to should be a riscv,cpu-intc
node, which has a CPU node (i.e. RISC-V HART) as parent.
msi-parent:
description:
Given APLIC domain forwards wired interrupts as MSIs to a AIA incoming
message signaled interrupt controller (IMSIC). If both "msi-parent" and
"interrupts-extended" properties are present then it means the APLIC
domain supports both MSI mode and Direct mode in HW. In this case, the
APLIC driver has to choose between MSI mode or Direct mode.
riscv,num-sources:
$ref: /schemas/types.yaml#/definitions/uint32
minimum: 1
maximum: 1023
description:
Specifies the number of wired interrupt sources supported by this
APLIC domain.
riscv,children:
$ref: /schemas/types.yaml#/definitions/phandle-array
minItems: 1
maxItems: 1024
items:
maxItems: 1
description:
A list of child APLIC domains for the given APLIC domain. Each child
APLIC domain is assigned a child index in increasing order, with the
first child APLIC domain assigned child index 0. The APLIC domain child
index is used by firmware to delegate interrupts from the given APLIC
domain to a particular child APLIC domain.
riscv,delegation:
$ref: /schemas/types.yaml#/definitions/phandle-array
minItems: 1
maxItems: 1024
items:
items:
- description: child APLIC domain phandle
- description: first interrupt number of the parent APLIC domain (inclusive)
- description: last interrupt number of the parent APLIC domain (inclusive)
description:
A interrupt delegation list where each entry is a triple consisting
of child APLIC domain phandle, first interrupt number of the parent
APLIC domain, and last interrupt number of the parent APLIC domain.
Firmware must configure interrupt delegation registers based on
interrupt delegation list.
dependencies:
riscv,delegation: [ "riscv,children" ]
required:
- compatible
- reg
- interrupt-controller
- "#interrupt-cells"
- riscv,num-sources
anyOf:
- required:
- interrupts-extended
- required:
- msi-parent
unevaluatedProperties: false
examples:
- |
// Example 1 (APLIC domains directly injecting interrupt to HARTs):
interrupt-controller@c000000 {
compatible = "qemu,aplic", "riscv,aplic";
interrupts-extended = <&cpu1_intc 11>,
<&cpu2_intc 11>,
<&cpu3_intc 11>,
<&cpu4_intc 11>;
reg = <0xc000000 0x4080>;
interrupt-controller;
#interrupt-cells = <2>;
riscv,num-sources = <63>;
riscv,children = <&aplic1>, <&aplic2>;
riscv,delegation = <&aplic1 1 63>;
};
aplic1: interrupt-controller@d000000 {
compatible = "qemu,aplic", "riscv,aplic";
interrupts-extended = <&cpu1_intc 9>,
<&cpu2_intc 9>;
reg = <0xd000000 0x4080>;
interrupt-controller;
#interrupt-cells = <2>;
riscv,num-sources = <63>;
};
aplic2: interrupt-controller@e000000 {
compatible = "qemu,aplic", "riscv,aplic";
interrupts-extended = <&cpu3_intc 9>,
<&cpu4_intc 9>;
reg = <0xe000000 0x4080>;
interrupt-controller;
#interrupt-cells = <2>;
riscv,num-sources = <63>;
};
- |
// Example 2 (APLIC domains forwarding interrupts as MSIs):
interrupt-controller@c000000 {
compatible = "qemu,aplic", "riscv,aplic";
msi-parent = <&imsic_mlevel>;
reg = <0xc000000 0x4000>;
interrupt-controller;
#interrupt-cells = <2>;
riscv,num-sources = <63>;
riscv,children = <&aplic3>;
riscv,delegation = <&aplic3 1 63>;
};
aplic3: interrupt-controller@d000000 {
compatible = "qemu,aplic", "riscv,aplic";
msi-parent = <&imsic_slevel>;
reg = <0xd000000 0x4000>;
interrupt-controller;
#interrupt-cells = <2>;
riscv,num-sources = <63>;
};
...
Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml 0 → 100644
View file @ 6bfd2d44
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/interrupt-controller/riscv,imsics.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: RISC-V Incoming MSI Controller (IMSIC)
maintainers:
- Anup Patel <anup@brainfault.org>
description: |
The RISC-V advanced interrupt architecture (AIA) defines a per-CPU incoming
MSI controller (IMSIC) for handling MSIs in a RISC-V platform. The RISC-V
AIA specification can be found at https://github.com/riscv/riscv-aia.
The IMSIC is a per-CPU (or per-HART) device with separate interrupt file
for each privilege level (machine or supervisor). The configuration of
a IMSIC interrupt file is done using AIA CSRs and it also has a 4KB MMIO
space to receive MSIs from devices. Each IMSIC interrupt file supports a
fixed number of interrupt identities (to distinguish MSIs from devices)
which is same for given privilege level across CPUs (or HARTs).
The device tree of a RISC-V platform will have one IMSIC device tree node
for each privilege level (machine or supervisor) which collectively describe
IMSIC interrupt files at that privilege level across CPUs (or HARTs).
The arrangement of IMSIC interrupt files in MMIO space of a RISC-V platform
follows a particular scheme defined by the RISC-V AIA specification. A IMSIC
group is a set of IMSIC interrupt files co-located in MMIO space and we can
have multiple IMSIC groups (i.e. clusters, sockets, chiplets, etc) in a
RISC-V platform. The MSI target address of a IMSIC interrupt file at given
privilege level (machine or supervisor) encodes group index, HART index,
and guest index (shown below).
XLEN-1 > (HART Index MSB) 12 0
| | | |
-------------------------------------------------------------
|xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index| 0 |
-------------------------------------------------------------
allOf:
- $ref: /schemas/interrupt-controller.yaml#
- $ref: /schemas/interrupt-controller/msi-controller.yaml#
properties:
compatible:
items:
- enum:
- qemu,imsics
- const: riscv,imsics
reg:
minItems: 1
maxItems: 16384
description:
Base address of each IMSIC group.
interrupt-controller: true
"#interrupt-cells":
const: 0
msi-controller: true
"#msi-cells":
const: 0
interrupts-extended:
minItems: 1
maxItems: 16384
description:
This property represents the set of CPUs (or HARTs) for which given
device tree node describes the IMSIC interrupt files. Each node pointed
to should be a riscv,cpu-intc node, which has a CPU node (i.e. RISC-V
HART) as parent.
riscv,num-ids:
$ref: /schemas/types.yaml#/definitions/uint32
minimum: 63
maximum: 2047
description:
Number of interrupt identities supported by IMSIC interrupt file.
riscv,num-guest-ids:
$ref: /schemas/types.yaml#/definitions/uint32
minimum: 63
maximum: 2047
description:
Number of interrupt identities are supported by IMSIC guest interrupt
file. When not specified it is assumed to be same as specified by the
riscv,num-ids property.
riscv,guest-index-bits:
minimum: 0
maximum: 7
default: 0
description:
Number of guest index bits in the MSI target address.
riscv,hart-index-bits:
minimum: 0
maximum: 15
description:
Number of HART index bits in the MSI target address. When not
specified it is calculated based on the interrupts-extended property.
riscv,group-index-bits:
minimum: 0
maximum: 7
default: 0
description:
Number of group index bits in the MSI target address.
riscv,group-index-shift:
$ref: /schemas/types.yaml#/definitions/uint32
minimum: 0
maximum: 55
default: 24
description:
The least significant bit position of the group index bits in the
MSI target address.
required:
- compatible
- reg
- interrupt-controller
- msi-controller
- "#msi-cells"
- interrupts-extended
- riscv,num-ids
unevaluatedProperties: false
examples:
- |
// Example 1 (Machine-level IMSIC files with just one group):
interrupt-controller@24000000 {
compatible = "qemu,imsics", "riscv,imsics";
interrupts-extended = <&cpu1_intc 11>,
<&cpu2_intc 11>,
<&cpu3_intc 11>,
<&cpu4_intc 11>;
reg = <0x28000000 0x4000>;
interrupt-controller;
#interrupt-cells = <0>;
msi-controller;
#msi-cells = <0>;
riscv,num-ids = <127>;
};
- |
// Example 2 (Supervisor-level IMSIC files with two groups):
interrupt-controller@28000000 {
compatible = "qemu,imsics", "riscv,imsics";
interrupts-extended = <&cpu1_intc 9>,
<&cpu2_intc 9>,
<&cpu3_intc 9>,
<&cpu4_intc 9>;
reg = <0x28000000 0x2000>, /* Group0 IMSICs */
<0x29000000 0x2000>; /* Group1 IMSICs */
interrupt-controller;
#interrupt-cells = <0>;
msi-controller;
#msi-cells = <0>;
riscv,num-ids = <127>;
riscv,group-index-bits = <1>;
riscv,group-index-shift = <24>;
};
...
Documentation/devicetree/bindings/interrupt-controller/st,stm32-exti.yaml
View file @ 6bfd2d44
......@@ -89,8 +89,23 @@ examples:
reg = <0x5000d000 0x400>;
};
- |
//Example 2
exti2: interrupt-controller@40013c00 {
#include <dt-bindings/interrupt-controller/arm-gic.h>
exti2: interrupt-controller@5000d000 {
compatible = "st,stm32mp1-exti", "syscon";
interrupt-controller;
#interrupt-cells = <2>;
reg = <0x5000d000 0x400>;
interrupts-extended =
<&intc GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>;
};
- |
//Example 3
exti3: interrupt-controller@40013c00 {
compatible = "st,stm32-exti";
interrupt-controller;
#interrupt-cells = <2>;
......
MAINTAINERS
View file @ 6bfd2d44
......@@ -11454,6 +11454,7 @@ S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core
F: Documentation/core-api/irq/irq-domain.rst
F: include/linux/irqdomain.h
F: include/linux/irqdomain_defs.h
F: kernel/irq/irqdomain.c
F: kernel/irq/msi.c
......@@ -11463,6 +11464,10 @@ L: linux-kernel@vger.kernel.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core
F: include/linux/group_cpus.h
F: include/linux/irq.h
F: include/linux/irqhandler.h
F: include/linux/irqnr.h
F: include/linux/irqreturn.h
F: kernel/irq/
F: lib/group_cpus.c
......@@ -11473,6 +11478,7 @@ S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/core
F: Documentation/devicetree/bindings/interrupt-controller/
F: drivers/irqchip/
F: include/linux/irqchip.h
ISA
M: William Breathitt Gray <william.gray@linaro.org>
......@@ -18974,6 +18980,20 @@ S: Maintained
F: drivers/mtd/nand/raw/r852.c
F: drivers/mtd/nand/raw/r852.h
RISC-V AIA DRIVERS
M: Anup Patel <anup@brainfault.org>
L: linux-riscv@lists.infradead.org
S: Maintained
F: Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
F: Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
F: drivers/irqchip/irq-riscv-aplic-*.c
F: drivers/irqchip/irq-riscv-aplic-*.h
F: drivers/irqchip/irq-riscv-imsic-*.c
F: drivers/irqchip/irq-riscv-imsic-*.h
F: drivers/irqchip/irq-riscv-intc.c
F: include/linux/irqchip/riscv-aplic.h
F: include/linux/irqchip/riscv-imsic.h
RISC-V ARCHITECTURE
M: Paul Walmsley <paul.walmsley@sifive.com>
M: Palmer Dabbelt <palmer@dabbelt.com>
......
arch/arm/boot/dts/st/stm32mp131.dtsi
View file @ 6bfd2d44
......@@ -783,10 +783,82 @@ usb33: usb33 {
};
exti: interrupt-controller@5000d000 {
compatible = "st,stm32mp13-exti", "syscon";
compatible = "st,stm32mp1-exti", "syscon";
interrupt-controller;
#interrupt-cells = <2>;
reg = <0x5000d000 0x400>;
interrupts-extended =
<&intc GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_0 */
<&intc GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 24 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 66 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 41 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_10 */
<&intc GIC_SPI 43 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 78 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>,
<0>, /* EXTI_20 */
<&intc GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 34 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 93 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 114 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 39 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 40 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_30 */
<&intc GIC_SPI 54 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 84 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>, /* EXTI_40 */
<0>,
<0>,
<0>,
<&intc GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 92 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 117 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_50 */
<0>,
<&intc GIC_SPI 118 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>, /* EXTI_60 */
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<&intc GIC_SPI 63 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 98 IRQ_TYPE_LEVEL_HIGH>; /* EXTI_70 */
};
syscfg: syscon@50020000 {
......
arch/arm/boot/dts/st/stm32mp151.dtsi
View file @ 6bfd2d44
......@@ -176,6 +176,81 @@ exti: interrupt-controller@5000d000 {
interrupt-controller;
#interrupt-cells = <2>;
reg = <0x5000d000 0x400>;
interrupts-extended =
<&intc GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_0 */
<&intc GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 23 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 66 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 40 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_10 */
<&intc GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 121 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>,
<0>, /* EXTI_20 */
<&intc GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 33 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 107 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 37 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 39 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 71 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 52 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_30 */
<&intc GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 82 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>, /* EXTI_40 */
<0>,
<0>,
<0>,
<0>,
<0>,
<&intc GIC_SPI 151 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 93 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_50 */
<0>,
<&intc GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 135 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>, /* EXTI_60 */
<&intc GIC_SPI 100 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<0>,
<&intc GIC_SPI 144 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 62 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_70 */
<0>,
<0>,
<&intc GIC_SPI 129 IRQ_TYPE_LEVEL_HIGH>;
};
syscfg: syscon@50020000 {
......
arch/arm64/Kconfig.platforms
View file @ 6bfd2d44
......@@ -309,6 +309,7 @@ config ARCH_STM32
select GPIOLIB
select PINCTRL
select PINCTRL_STM32MP257
select STM32_EXTI
select ARM_SMC_MBOX
select ARM_SCMI_PROTOCOL
select COMMON_CLK_SCMI
......
arch/arm64/boot/dts/st/stm32mp251.dtsi
View file @ 6bfd2d44
......@@ -443,6 +443,99 @@ rcc: clock-controller@44200000 {
<&clk_dsi_txbyte>;
};
exti1: interrupt-controller@44220000 {
compatible = "st,stm32mp1-exti", "syscon";
interrupt-controller;
#interrupt-cells = <2>;
reg = <0x44220000 0x400>;
interrupts-extended =
<&intc GIC_SPI 268 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_0 */
<&intc GIC_SPI 269 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 270 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 271 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 272 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 273 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 274 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 275 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 276 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 277 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 278 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_10 */
<&intc GIC_SPI 279 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 280 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 281 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 282 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 283 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 0 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 260 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 259 IRQ_TYPE_LEVEL_HIGH>,
<0>, /* EXTI_20 */
<&intc GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 137 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 168 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 181 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 114 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 115 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 136 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 126 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_30 */
<&intc GIC_SPI 127 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 149 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 150 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 125 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 152 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 153 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_40 */
<&intc GIC_SPI 154 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 155 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 169 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 182 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 209 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 229 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 166 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 215 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 208 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 210 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_50 */
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<&intc GIC_SPI 171 IRQ_TYPE_LEVEL_HIGH>,
<0>, /* EXTI_60 */
<&intc GIC_SPI 173 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 220 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 134 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_70 */
<0>,
<&intc GIC_SPI 224 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 202 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 253 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 254 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 255 IRQ_TYPE_LEVEL_HIGH>,
<0>, /* EXTI_80 */
<0>,
<0>,
<&intc GIC_SPI 257 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 258 IRQ_TYPE_LEVEL_HIGH>;
};
syscfg: syscon@44230000 {
compatible = "st,stm32mp25-syscfg", "syscon";
reg = <0x44230000 0x10000>;
......@@ -453,6 +546,8 @@ pinctrl: pinctrl@44240000 {
#size-cells = <1>;
compatible = "st,stm32mp257-pinctrl";
ranges = <0 0x44240000 0xa0400>;
interrupt-parent = <&exti1>;
st,syscfg = <&exti1 0x60 0xff>;
pins-are-numbered;
gpioa: gpio@44240000 {
......@@ -582,6 +677,8 @@ pinctrl_z: pinctrl@46200000 {
#size-cells = <1>;
compatible = "st,stm32mp257-z-pinctrl";
ranges = <0 0x46200000 0x400>;
interrupt-parent = <&exti1>;
st,syscfg = <&exti1 0x60 0xff>;
pins-are-numbered;
gpioz: gpio@46200000 {
......@@ -597,5 +694,84 @@ gpioz: gpio@46200000 {
};
};
exti2: interrupt-controller@46230000 {
compatible = "st,stm32mp1-exti", "syscon";
interrupt-controller;
#interrupt-cells = <2>;
reg = <0x46230000 0x400>;
interrupts-extended =
<&intc GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_0 */
<&intc GIC_SPI 18 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 19 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 20 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 21 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 22 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 23 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 24 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 25 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 26 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 27 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_10 */
<&intc GIC_SPI 28 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 31 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<0>, /* EXTI_20 */
<&intc GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 15 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 212 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 151 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 156 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 216 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 217 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_30 */
<&intc GIC_SPI 218 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 207 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 175 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 177 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 199 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_40 */
<0>,
<0>,
<&intc GIC_SPI 200 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 11 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 5 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>, /* EXTI_50 */
<&intc GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 2 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>,
<0>, /* EXTI_60 */
<&intc GIC_SPI 221 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 246 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<&intc GIC_SPI 247 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 248 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 249 IRQ_TYPE_LEVEL_HIGH>,
<&intc GIC_SPI 256 IRQ_TYPE_LEVEL_HIGH>,
<0>,
<0>,
<&intc GIC_SPI 213 IRQ_TYPE_LEVEL_HIGH>; /* EXTI_70 */
};
};
};
arch/mips/dec/setup.c
View file @ 6bfd2d44
......@@ -756,7 +756,7 @@ void __init arch_init_irq(void)
NULL))
pr_err("Failed to register fpu interrupt\n");
desc_fpu = irq_to_desc(irq_fpu);
fpu_kstat_irq = this_cpu_ptr(desc_fpu->kstat_irqs);
fpu_kstat_irq = this_cpu_ptr(&desc_fpu->kstat_irqs->cnt);
}
if (dec_interrupt[DEC_IRQ_CASCADE] >= 0) {
if (request_irq(dec_interrupt[DEC_IRQ_CASCADE], no_action,
......
arch/parisc/kernel/smp.c
View file @ 6bfd2d44
......@@ -344,7 +344,7 @@ static int smp_boot_one_cpu(int cpuid, struct task_struct *idle)
struct irq_desc *desc = irq_to_desc(i);
if (desc && desc->kstat_irqs)
*per_cpu_ptr(desc->kstat_irqs, cpuid) = 0;
*per_cpu_ptr(desc->kstat_irqs, cpuid) = (struct irqstat) { };
}
#endif
......
arch/powerpc/kvm/book3s_hv_rm_xics.c
View file @ 6bfd2d44
......@@ -837,7 +837,7 @@ static inline void this_cpu_inc_rm(unsigned int __percpu *addr)
*/
static void kvmppc_rm_handle_irq_desc(struct irq_desc *desc)
{
this_cpu_inc_rm(desc->kstat_irqs);
this_cpu_inc_rm(&desc->kstat_irqs->cnt);
__this_cpu_inc(kstat.irqs_sum);
}
......
arch/riscv/Kconfig
View file @ 6bfd2d44
......@@ -173,6 +173,8 @@ config RISCV
select PCI_DOMAINS_GENERIC if PCI
select PCI_MSI if PCI
select RISCV_ALTERNATIVE if !XIP_KERNEL
select RISCV_APLIC
select RISCV_IMSIC
select RISCV_INTC
select RISCV_TIMER if RISCV_SBI
select SIFIVE_PLIC
......
drivers/cpuidle/coupled.c
View file @ 6bfd2d44
......@@ -439,13 +439,8 @@ static int cpuidle_coupled_clear_pokes(int cpu)
static bool cpuidle_coupled_any_pokes_pending(struct cpuidle_coupled *coupled)
{
cpumask_t cpus;
int ret;
cpumask_and(&cpus, cpu_online_mask, &coupled->coupled_cpus);
ret = cpumask_and(&cpus, &cpuidle_coupled_poke_pending, &cpus);
return ret;
return cpumask_first_and_and(cpu_online_mask, &coupled->coupled_cpus,
&cpuidle_coupled_poke_pending) < nr_cpu_ids;
}
/**
......@@ -626,9 +621,7 @@ int cpuidle_enter_state_coupled(struct cpuidle_device *dev,
static void cpuidle_coupled_update_online_cpus(struct cpuidle_coupled *coupled)
{
cpumask_t cpus;
cpumask_and(&cpus, cpu_online_mask, &coupled->coupled_cpus);
coupled->online_count = cpumask_weight(&cpus);
coupled->online_count = cpumask_weight_and(cpu_online_mask, &coupled->coupled_cpus);
}
/**
......
drivers/irqchip/Kconfig
View file @ 6bfd2d44
......@@ -540,6 +540,31 @@ config RISCV_INTC
depends on RISCV
select IRQ_DOMAIN_HIERARCHY
config RISCV_APLIC
bool
depends on RISCV
select IRQ_DOMAIN_HIERARCHY
config RISCV_APLIC_MSI
bool
depends on RISCV_APLIC
select GENERIC_MSI_IRQ
default RISCV_APLIC
config RISCV_IMSIC
bool
depends on RISCV
select IRQ_DOMAIN_HIERARCHY
select GENERIC_IRQ_MATRIX_ALLOCATOR
select GENERIC_MSI_IRQ
config RISCV_IMSIC_PCI
bool
depends on RISCV_IMSIC
depends on PCI
depends on PCI_MSI
default RISCV_IMSIC
config SIFIVE_PLIC
bool
depends on RISCV
......@@ -568,7 +593,7 @@ config IRQ_LOONGARCH_CPU
bool
select GENERIC_IRQ_CHIP
select IRQ_DOMAIN
select GENERIC_IRQ_EFFECTIVE_AFF_MASK
select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP
select LOONGSON_HTVEC
select LOONGSON_LIOINTC
select LOONGSON_EIOINTC
......
drivers/irqchip/Makefile
View file @ 6bfd2d44
......@@ -95,6 +95,9 @@ obj-$(CONFIG_QCOM_MPM) += irq-qcom-mpm.o
obj-$(CONFIG_CSKY_MPINTC) += irq-csky-mpintc.o
obj-$(CONFIG_CSKY_APB_INTC) += irq-csky-apb-intc.o
obj-$(CONFIG_RISCV_INTC) += irq-riscv-intc.o
obj-$(CONFIG_RISCV_APLIC) += irq-riscv-aplic-main.o irq-riscv-aplic-direct.o
obj-$(CONFIG_RISCV_APLIC_MSI) += irq-riscv-aplic-msi.o
obj-$(CONFIG_RISCV_IMSIC) += irq-riscv-imsic-state.o irq-riscv-imsic-early.o irq-riscv-imsic-platform.o
obj-$(CONFIG_SIFIVE_PLIC) += irq-sifive-plic.o
obj-$(CONFIG_STARFIVE_JH8100_INTC) += irq-starfive-jh8100-intc.o
obj-$(CONFIG_IMX_IRQSTEER) += irq-imx-irqsteer.o
......
drivers/irqchip/irq-alpine-msi.c
View file @ 6bfd2d44
......@@ -165,7 +165,7 @@ static int alpine_msix_middle_domain_alloc(struct irq_domain *domain,
return 0;
err_sgi:
irq_domain_free_irqs_parent(domain, virq, i - 1);
irq_domain_free_irqs_parent(domain, virq, i);
alpine_msix_free_sgi(priv, sgi, nr_irqs);
return err;
}
......
drivers/irqchip/irq-bcm6345-l1.c
View file @ 6bfd2d44
......@@ -192,14 +192,10 @@ static int bcm6345_l1_set_affinity(struct irq_data *d,
u32 mask = BIT(d->hwirq % IRQS_PER_WORD);
unsigned int old_cpu = cpu_for_irq(intc, d);
unsigned int new_cpu;
struct cpumask valid;
unsigned long flags;
bool enabled;
if (!cpumask_and(&valid, &intc->cpumask, dest))
return -EINVAL;
new_cpu = cpumask_any_and(&valid, cpu_online_mask);
new_cpu = cpumask_first_and_and(&intc->cpumask, dest, cpu_online_mask);
if (new_cpu >= nr_cpu_ids)
return -EINVAL;
......
drivers/irqchip/irq-brcmstb-l2.c
View file @ 6bfd2d44
......@@ -118,7 +118,7 @@ static void brcmstb_l2_intc_irq_handle(struct irq_desc *desc)
chained_irq_exit(chip, desc);
}
static void brcmstb_l2_intc_suspend(struct irq_data *d)
static void __brcmstb_l2_intc_suspend(struct irq_data *d, bool save)
{
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
struct irq_chip_type *ct = irq_data_get_chip_type(d);
......@@ -127,7 +127,8 @@ static void brcmstb_l2_intc_suspend(struct irq_data *d)
irq_gc_lock_irqsave(gc, flags);
/* Save the current mask */
b->saved_mask = irq_reg_readl(gc, ct->regs.mask);
if (save)
b->saved_mask = irq_reg_readl(gc, ct->regs.mask);
if (b->can_wake) {
/* Program the wakeup mask */
......@@ -137,6 +138,16 @@ static void brcmstb_l2_intc_suspend(struct irq_data *d)
irq_gc_unlock_irqrestore(gc, flags);
}
static void brcmstb_l2_intc_shutdown(struct irq_data *d)
{
__brcmstb_l2_intc_suspend(d, false);
}
static void brcmstb_l2_intc_suspend(struct irq_data *d)
{
__brcmstb_l2_intc_suspend(d, true);
}
static void brcmstb_l2_intc_resume(struct irq_data *d)
{
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
......@@ -252,7 +263,7 @@ static int __init brcmstb_l2_intc_of_init(struct device_node *np,
ct->chip.irq_suspend = brcmstb_l2_intc_suspend;
ct->chip.irq_resume = brcmstb_l2_intc_resume;
ct->chip.irq_pm_shutdown = brcmstb_l2_intc_suspend;
ct->chip.irq_pm_shutdown = brcmstb_l2_intc_shutdown;
if (data->can_wake) {
/* This IRQ chip can wake the system, set all child interrupts
......
drivers/irqchip/irq-gic-v3-its.c
View file @ 6bfd2d44
......@@ -3832,9 +3832,9 @@ static int its_vpe_set_affinity(struct irq_data *d,
bool force)
{
struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
struct cpumask common, *table_mask;
unsigned int from, cpu = nr_cpu_ids;
struct cpumask *table_mask;
unsigned long flags;
int from, cpu;
/*
* Changing affinity is mega expensive, so let's be as lazy as
......@@ -3856,10 +3856,15 @@ static int its_vpe_set_affinity(struct irq_data *d,
* If we are offered another CPU in the same GICv4.1 ITS
* affinity, pick this one. Otherwise, any CPU will do.
*/
if (table_mask && cpumask_and(&common, mask_val, table_mask))
cpu = cpumask_test_cpu(from, &common) ? from : cpumask_first(&common);
else
if (table_mask)
cpu = cpumask_any_and(mask_val, table_mask);
if (cpu < nr_cpu_ids) {
if (cpumask_test_cpu(from, mask_val) &&
cpumask_test_cpu(from, table_mask))
cpu = from;
} else {
cpu = cpumask_first(mask_val);
}
if (from == cpu)
goto out;
......@@ -4527,8 +4532,6 @@ static int its_vpe_irq_domain_alloc(struct irq_domain *domain, unsigned int virq
struct page *vprop_page;
int base, nr_ids, i, err = 0;
BUG_ON(!vm);
bitmap = its_lpi_alloc(roundup_pow_of_two(nr_irqs), &base, &nr_ids);
if (!bitmap)
return -ENOMEM;
......
drivers/irqchip/irq-loongson-eiointc.c
View file @ 6bfd2d44
......@@ -59,6 +59,7 @@ static int cpu_to_eio_node(int cpu)
return cpu_logical_map(cpu) / CORES_PER_EIO_NODE;
}
#ifdef CONFIG_SMP
static void eiointc_set_irq_route(int pos, unsigned int cpu, unsigned int mnode, nodemask_t *node_map)
{
int i, node, cpu_node, route_node;
......@@ -92,19 +93,15 @@ static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *af
unsigned int cpu;
unsigned long flags;
uint32_t vector, regaddr;
struct cpumask intersect_affinity;
struct eiointc_priv *priv = d->domain->host_data;
raw_spin_lock_irqsave(&affinity_lock, flags);
cpumask_and(&intersect_affinity, affinity, cpu_online_mask);
cpumask_and(&intersect_affinity, &intersect_affinity, &priv->cpuspan_map);
if (cpumask_empty(&intersect_affinity)) {
cpu = cpumask_first_and_and(&priv->cpuspan_map, affinity, cpu_online_mask);
if (cpu >= nr_cpu_ids) {
raw_spin_unlock_irqrestore(&affinity_lock, flags);
return -EINVAL;
}
cpu = cpumask_first(&intersect_affinity);
vector = d->hwirq;
regaddr = EIOINTC_REG_ENABLE + ((vector >> 5) << 2);
......@@ -126,6 +123,7 @@ static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *af
return IRQ_SET_MASK_OK;
}
#endif
static int eiointc_index(int node)
{
......@@ -238,7 +236,9 @@ static struct irq_chip eiointc_irq_chip = {
.irq_ack = eiointc_ack_irq,
.irq_mask = eiointc_mask_irq,
.irq_unmask = eiointc_unmask_irq,
#ifdef CONFIG_SMP
.irq_set_affinity = eiointc_set_irq_affinity,
#endif
};
static int eiointc_domain_alloc(struct irq_domain *domain, unsigned int virq,
......
drivers/irqchip/irq-loongson-pch-msi.c
View file @ 6bfd2d44
......@@ -136,7 +136,7 @@ static int pch_msi_middle_domain_alloc(struct irq_domain *domain,
err_hwirq:
pch_msi_free_hwirq(priv, hwirq, nr_irqs);
irq_domain_free_irqs_parent(domain, virq, i - 1);
irq_domain_free_irqs_parent(domain, virq, i);
return err;
}
......
drivers/irqchip/irq-loongson-pch-pic.c
View file @ 6bfd2d44
......@@ -33,6 +33,7 @@
#define PIC_COUNT (PIC_COUNT_PER_REG * PIC_REG_COUNT)
#define PIC_REG_IDX(irq_id) ((irq_id) / PIC_COUNT_PER_REG)
#define PIC_REG_BIT(irq_id) ((irq_id) % PIC_COUNT_PER_REG)
#define PIC_UNDEF_VECTOR 255
static int nr_pics;
......@@ -46,12 +47,19 @@ struct pch_pic {
u32 saved_vec_en[PIC_REG_COUNT];
u32 saved_vec_pol[PIC_REG_COUNT];
u32 saved_vec_edge[PIC_REG_COUNT];
u8 table[PIC_COUNT];
int inuse;
};
static struct pch_pic *pch_pic_priv[MAX_IO_PICS];
struct fwnode_handle *pch_pic_handle[MAX_IO_PICS];
static inline u8 hwirq_to_bit(struct pch_pic *priv, int hirq)
{
return priv->table[hirq];
}
static void pch_pic_bitset(struct pch_pic *priv, int offset, int bit)
{
u32 reg;
......@@ -80,45 +88,47 @@ static void pch_pic_mask_irq(struct irq_data *d)
{
struct pch_pic *priv = irq_data_get_irq_chip_data(d);
pch_pic_bitset(priv, PCH_PIC_MASK, d->hwirq);
pch_pic_bitset(priv, PCH_PIC_MASK, hwirq_to_bit(priv, d->hwirq));
irq_chip_mask_parent(d);
}
static void pch_pic_unmask_irq(struct irq_data *d)
{
struct pch_pic *priv = irq_data_get_irq_chip_data(d);
int bit = hwirq_to_bit(priv, d->hwirq);
writel(BIT(PIC_REG_BIT(d->hwirq)),
priv->base + PCH_PIC_CLR + PIC_REG_IDX(d->hwirq) * 4);
writel(BIT(PIC_REG_BIT(bit)),
priv->base + PCH_PIC_CLR + PIC_REG_IDX(bit) * 4);
irq_chip_unmask_parent(d);
pch_pic_bitclr(priv, PCH_PIC_MASK, d->hwirq);
pch_pic_bitclr(priv, PCH_PIC_MASK, bit);
}
static int pch_pic_set_type(struct irq_data *d, unsigned int type)
{
struct pch_pic *priv = irq_data_get_irq_chip_data(d);
int bit = hwirq_to_bit(priv, d->hwirq);
int ret = 0;
switch (type) {
case IRQ_TYPE_EDGE_RISING:
pch_pic_bitset(priv, PCH_PIC_EDGE, d->hwirq);
pch_pic_bitclr(priv, PCH_PIC_POL, d->hwirq);
pch_pic_bitset(priv, PCH_PIC_EDGE, bit);
pch_pic_bitclr(priv, PCH_PIC_POL, bit);
irq_set_handler_locked(d, handle_edge_irq);
break;
case IRQ_TYPE_EDGE_FALLING:
pch_pic_bitset(priv, PCH_PIC_EDGE, d->hwirq);
pch_pic_bitset(priv, PCH_PIC_POL, d->hwirq);
pch_pic_bitset(priv, PCH_PIC_EDGE, bit);
pch_pic_bitset(priv, PCH_PIC_POL, bit);
irq_set_handler_locked(d, handle_edge_irq);
break;
case IRQ_TYPE_LEVEL_HIGH:
pch_pic_bitclr(priv, PCH_PIC_EDGE, d->hwirq);
pch_pic_bitclr(priv, PCH_PIC_POL, d->hwirq);
pch_pic_bitclr(priv, PCH_PIC_EDGE, bit);
pch_pic_bitclr(priv, PCH_PIC_POL, bit);
irq_set_handler_locked(d, handle_level_irq);
break;
case IRQ_TYPE_LEVEL_LOW:
pch_pic_bitclr(priv, PCH_PIC_EDGE, d->hwirq);
pch_pic_bitset(priv, PCH_PIC_POL, d->hwirq);
pch_pic_bitclr(priv, PCH_PIC_EDGE, bit);
pch_pic_bitset(priv, PCH_PIC_POL, bit);
irq_set_handler_locked(d, handle_level_irq);
break;
default:
......@@ -133,11 +143,12 @@ static void pch_pic_ack_irq(struct irq_data *d)
{
unsigned int reg;
struct pch_pic *priv = irq_data_get_irq_chip_data(d);
int bit = hwirq_to_bit(priv, d->hwirq);
reg = readl(priv->base + PCH_PIC_EDGE + PIC_REG_IDX(d->hwirq) * 4);
if (reg & BIT(PIC_REG_BIT(d->hwirq))) {
writel(BIT(PIC_REG_BIT(d->hwirq)),
priv->base + PCH_PIC_CLR + PIC_REG_IDX(d->hwirq) * 4);
reg = readl(priv->base + PCH_PIC_EDGE + PIC_REG_IDX(bit) * 4);
if (reg & BIT(PIC_REG_BIT(bit))) {
writel(BIT(PIC_REG_BIT(bit)),
priv->base + PCH_PIC_CLR + PIC_REG_IDX(bit) * 4);
}
irq_chip_ack_parent(d);
}
......@@ -159,6 +170,8 @@ static int pch_pic_domain_translate(struct irq_domain *d,
{
struct pch_pic *priv = d->host_data;
struct device_node *of_node = to_of_node(fwspec->fwnode);
unsigned long flags;
int i;
if (of_node) {
if (fwspec->param_count < 2)
......@@ -171,12 +184,33 @@ static int pch_pic_domain_translate(struct irq_domain *d,
return -EINVAL;
*hwirq = fwspec->param[0] - priv->gsi_base;
if (fwspec->param_count > 1)
*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
else
*type = IRQ_TYPE_NONE;
}
raw_spin_lock_irqsave(&priv->pic_lock, flags);
/* Check pic-table to confirm if the hwirq has been assigned */
for (i = 0; i < priv->inuse; i++) {
if (priv->table[i] == *hwirq) {
*hwirq = i;
break;
}
}
if (i == priv->inuse) {
/* Assign a new hwirq in pic-table */
if (priv->inuse >= PIC_COUNT) {
pr_err("pch-pic domain has no free vectors\n");
raw_spin_unlock_irqrestore(&priv->pic_lock, flags);
return -EINVAL;
}
priv->table[priv->inuse] = *hwirq;
*hwirq = priv->inuse++;
}
raw_spin_unlock_irqrestore(&priv->pic_lock, flags);
return 0;
}
......@@ -194,6 +228,9 @@ static int pch_pic_alloc(struct irq_domain *domain, unsigned int virq,
if (err)
return err;
/* Write vector ID */
writeb(priv->ht_vec_base + hwirq, priv->base + PCH_INT_HTVEC(hwirq_to_bit(priv, hwirq)));
parent_fwspec.fwnode = domain->parent->fwnode;
parent_fwspec.param_count = 1;
parent_fwspec.param[0] = hwirq + priv->ht_vec_base;
......@@ -222,7 +259,7 @@ static void pch_pic_reset(struct pch_pic *priv)
for (i = 0; i < PIC_COUNT; i++) {
/* Write vector ID */
writeb(priv->ht_vec_base + i, priv->base + PCH_INT_HTVEC(i));
writeb(priv->ht_vec_base + i, priv->base + PCH_INT_HTVEC(hwirq_to_bit(priv, i)));
/* Hardcode route to HT0 Lo */
writeb(1, priv->base + PCH_INT_ROUTE(i));
}
......@@ -284,6 +321,7 @@ static int pch_pic_init(phys_addr_t addr, unsigned long size, int vec_base,
u32 gsi_base)
{
struct pch_pic *priv;
int i;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (!priv)
......@@ -294,6 +332,10 @@ static int pch_pic_init(phys_addr_t addr, unsigned long size, int vec_base,
if (!priv->base)
goto free_priv;
priv->inuse = 0;
for (i = 0; i < PIC_COUNT; i++)
priv->table[i] = PIC_UNDEF_VECTOR;
priv->ht_vec_base = vec_base;
priv->vec_count = ((readq(priv->base) >> 48) & 0xff) + 1;
priv->gsi_base = gsi_base;
......
drivers/irqchip/irq-mxs.c
View file @ 6bfd2d44
......@@ -130,7 +130,7 @@ static struct irq_chip asm9260_icoll_chip = {
IRQCHIP_SKIP_SET_WAKE,
};
asmlinkage void __exception_irq_entry icoll_handle_irq(struct pt_regs *regs)
static void __exception_irq_entry icoll_handle_irq(struct pt_regs *regs)
{
u32 irqnr;
......
drivers/irqchip/irq-renesas-rzg2l.c
View file @ 6bfd2d44
......@@ -138,7 +138,7 @@ static void rzg2l_irqc_eoi(struct irq_data *d)
irq_chip_eoi_parent(d);
}
static void rzg2l_irqc_irq_disable(struct irq_data *d)
static void rzg2l_tint_irq_endisable(struct irq_data *d, bool enable)
{
unsigned int hw_irq = irqd_to_hwirq(d);
......@@ -151,30 +151,24 @@ static void rzg2l_irqc_irq_disable(struct irq_data *d)
raw_spin_lock(&priv->lock);
reg = readl_relaxed(priv->base + TSSR(tssr_index));
reg &= ~(TIEN << TSSEL_SHIFT(tssr_offset));
if (enable)
reg |= TIEN << TSSEL_SHIFT(tssr_offset);
else
reg &= ~(TIEN << TSSEL_SHIFT(tssr_offset));
writel_relaxed(reg, priv->base + TSSR(tssr_index));
raw_spin_unlock(&priv->lock);
}
}
static void rzg2l_irqc_irq_disable(struct irq_data *d)
{
rzg2l_tint_irq_endisable(d, false);
irq_chip_disable_parent(d);
}
static void rzg2l_irqc_irq_enable(struct irq_data *d)
{
unsigned int hw_irq = irqd_to_hwirq(d);
if (hw_irq >= IRQC_TINT_START && hw_irq < IRQC_NUM_IRQ) {
struct rzg2l_irqc_priv *priv = irq_data_to_priv(d);
u32 offset = hw_irq - IRQC_TINT_START;
u32 tssr_offset = TSSR_OFFSET(offset);
u8 tssr_index = TSSR_INDEX(offset);
u32 reg;
raw_spin_lock(&priv->lock);
reg = readl_relaxed(priv->base + TSSR(tssr_index));
reg |= TIEN << TSSEL_SHIFT(tssr_offset);
writel_relaxed(reg, priv->base + TSSR(tssr_index));
raw_spin_unlock(&priv->lock);
}
rzg2l_tint_irq_endisable(d, true);
irq_chip_enable_parent(d);
}
......
drivers/irqchip/irq-riscv-aplic-direct.c 0 → 100644
View file @ 6bfd2d44
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#include <linux/bitfield.h>
#include <linux/bitops.h>
#include <linux/cpu.h>
#include <linux/interrupt.h>
#include <linux/irqchip.h>
#include <linux/irqchip/chained_irq.h>
#include <linux/irqchip/riscv-aplic.h>
#include <linux/module.h>
#include <linux/of_address.h>
#include <linux/printk.h>
#include <linux/smp.h>
#include "irq-riscv-aplic-main.h"
#define APLIC_DISABLE_IDELIVERY 0
#define APLIC_ENABLE_IDELIVERY 1
#define APLIC_DISABLE_ITHRESHOLD 1
#define APLIC_ENABLE_ITHRESHOLD 0
struct aplic_direct {
struct aplic_priv priv;
struct irq_domain *irqdomain;
struct cpumask lmask;
};
struct aplic_idc {
unsigned int hart_index;
void __iomem *regs;
struct aplic_direct *direct;
};
static unsigned int aplic_direct_parent_irq;
static DEFINE_PER_CPU(struct aplic_idc, aplic_idcs);
static void aplic_direct_irq_eoi(struct irq_data *d)
{
/*
* The fasteoi_handler requires irq_eoi() callback hence
* provide a dummy handler.
*/
}
#ifdef CONFIG_SMP
static int aplic_direct_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
bool force)
{
struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
struct aplic_direct *direct = container_of(priv, struct aplic_direct, priv);
struct aplic_idc *idc;
unsigned int cpu, val;
void __iomem *target;
if (force)
cpu = cpumask_first_and(&direct->lmask, mask_val);
else
cpu = cpumask_first_and_and(&direct->lmask, mask_val, cpu_online_mask);
if (cpu >= nr_cpu_ids)
return -EINVAL;
idc = per_cpu_ptr(&aplic_idcs, cpu);
target = priv->regs + APLIC_TARGET_BASE + (d->hwirq - 1) * sizeof(u32);
val = FIELD_PREP(APLIC_TARGET_HART_IDX, idc->hart_index);
val |= FIELD_PREP(APLIC_TARGET_IPRIO, APLIC_DEFAULT_PRIORITY);
writel(val, target);
irq_data_update_effective_affinity(d, cpumask_of(cpu));
return IRQ_SET_MASK_OK_DONE;
}
#endif
static struct irq_chip aplic_direct_chip = {
.name = "APLIC-DIRECT",
.irq_mask = aplic_irq_mask,
.irq_unmask = aplic_irq_unmask,
.irq_set_type = aplic_irq_set_type,
.irq_eoi = aplic_direct_irq_eoi,
#ifdef CONFIG_SMP
.irq_set_affinity = aplic_direct_set_affinity,
#endif
.flags = IRQCHIP_SET_TYPE_MASKED |
IRQCHIP_SKIP_SET_WAKE |
IRQCHIP_MASK_ON_SUSPEND,
};
static int aplic_direct_irqdomain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
unsigned long *hwirq, unsigned int *type)
{
struct aplic_priv *priv = d->host_data;
return aplic_irqdomain_translate(fwspec, priv->gsi_base, hwirq, type);
}
static int aplic_direct_irqdomain_alloc(struct irq_domain *domain, unsigned int virq,
unsigned int nr_irqs, void *arg)
{
struct aplic_priv *priv = domain->host_data;
struct aplic_direct *direct = container_of(priv, struct aplic_direct, priv);
struct irq_fwspec *fwspec = arg;
irq_hw_number_t hwirq;
unsigned int type;
int i, ret;
ret = aplic_irqdomain_translate(fwspec, priv->gsi_base, &hwirq, &type);
if (ret)
return ret;
for (i = 0; i < nr_irqs; i++) {
irq_domain_set_info(domain, virq + i, hwirq + i, &aplic_direct_chip,
priv, handle_fasteoi_irq, NULL, NULL);
irq_set_affinity(virq + i, &direct->lmask);
}
return 0;
}
static const struct irq_domain_ops aplic_direct_irqdomain_ops = {
.translate = aplic_direct_irqdomain_translate,
.alloc = aplic_direct_irqdomain_alloc,
.free = irq_domain_free_irqs_top,
};
/*
* To handle an APLIC direct interrupts, we just read the CLAIMI register
* which will return highest priority pending interrupt and clear the
* pending bit of the interrupt. This process is repeated until CLAIMI
* register return zero value.
*/
static void aplic_direct_handle_irq(struct irq_desc *desc)
{
struct aplic_idc *idc = this_cpu_ptr(&aplic_idcs);
struct irq_domain *irqdomain = idc->direct->irqdomain;
struct irq_chip *chip = irq_desc_get_chip(desc);
irq_hw_number_t hw_irq;
int irq;
chained_irq_enter(chip, desc);
while ((hw_irq = readl(idc->regs + APLIC_IDC_CLAIMI))) {
hw_irq = hw_irq >> APLIC_IDC_TOPI_ID_SHIFT;
irq = irq_find_mapping(irqdomain, hw_irq);
if (unlikely(irq <= 0)) {
dev_warn_ratelimited(idc->direct->priv.dev,
"hw_irq %lu mapping not found\n", hw_irq);
} else {
generic_handle_irq(irq);
}
}
chained_irq_exit(chip, desc);
}
static void aplic_idc_set_delivery(struct aplic_idc *idc, bool en)
{
u32 de = (en) ? APLIC_ENABLE_IDELIVERY : APLIC_DISABLE_IDELIVERY;
u32 th = (en) ? APLIC_ENABLE_ITHRESHOLD : APLIC_DISABLE_ITHRESHOLD;
/* Priority must be less than threshold for interrupt triggering */
writel(th, idc->regs + APLIC_IDC_ITHRESHOLD);
/* Delivery must be set to 1 for interrupt triggering */
writel(de, idc->regs + APLIC_IDC_IDELIVERY);
}
static int aplic_direct_dying_cpu(unsigned int cpu)
{
if (aplic_direct_parent_irq)
disable_percpu_irq(aplic_direct_parent_irq);
return 0;
}
static int aplic_direct_starting_cpu(unsigned int cpu)
{
if (aplic_direct_parent_irq) {
enable_percpu_irq(aplic_direct_parent_irq,
irq_get_trigger_type(aplic_direct_parent_irq));
}
return 0;
}
static int aplic_direct_parse_parent_hwirq(struct device *dev, u32 index,
u32 *parent_hwirq, unsigned long *parent_hartid)
{
struct of_phandle_args parent;
int rc;
/*
* Currently, only OF fwnode is supported so extend this
* function for ACPI support.
*/
if (!is_of_node(dev->fwnode))
return -EINVAL;
rc = of_irq_parse_one(to_of_node(dev->fwnode), index, &parent);
if (rc)
return rc;
rc = riscv_of_parent_hartid(parent.np, parent_hartid);
if (rc)
return rc;
*parent_hwirq = parent.args[0];
return 0;
}
int aplic_direct_setup(struct device *dev, void __iomem *regs)
{
int i, j, rc, cpu, current_cpu, setup_count = 0;
struct aplic_direct *direct;
struct irq_domain *domain;
struct aplic_priv *priv;
struct aplic_idc *idc;
unsigned long hartid;
u32 v, hwirq;
direct = devm_kzalloc(dev, sizeof(*direct), GFP_KERNEL);
if (!direct)
return -ENOMEM;
priv = &direct->priv;
rc = aplic_setup_priv(priv, dev, regs);
if (rc) {
dev_err(dev, "failed to create APLIC context\n");
return rc;
}
/* Setup per-CPU IDC and target CPU mask */
current_cpu = get_cpu();
for (i = 0; i < priv->nr_idcs; i++) {
rc = aplic_direct_parse_parent_hwirq(dev, i, &hwirq, &hartid);
if (rc) {
dev_warn(dev, "parent irq for IDC%d not found\n", i);
continue;
}
/*
* Skip interrupts other than external interrupts for
* current privilege level.
*/
if (hwirq != RV_IRQ_EXT)
continue;
cpu = riscv_hartid_to_cpuid(hartid);
if (cpu < 0) {
dev_warn(dev, "invalid cpuid for IDC%d\n", i);
continue;
}
cpumask_set_cpu(cpu, &direct->lmask);
idc = per_cpu_ptr(&aplic_idcs, cpu);
idc->hart_index = i;
idc->regs = priv->regs + APLIC_IDC_BASE + i * APLIC_IDC_SIZE;
idc->direct = direct;
aplic_idc_set_delivery(idc, true);
/*
* Boot cpu might not have APLIC hart_index = 0 so check
* and update target registers of all interrupts.
*/
if (cpu == current_cpu && idc->hart_index) {
v = FIELD_PREP(APLIC_TARGET_HART_IDX, idc->hart_index);
v |= FIELD_PREP(APLIC_TARGET_IPRIO, APLIC_DEFAULT_PRIORITY);
for (j = 1; j <= priv->nr_irqs; j++)
writel(v, priv->regs + APLIC_TARGET_BASE + (j - 1) * sizeof(u32));
}
setup_count++;
}
put_cpu();
/* Find parent domain and register chained handler */
domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
DOMAIN_BUS_ANY);
if (!aplic_direct_parent_irq && domain) {
aplic_direct_parent_irq = irq_create_mapping(domain, RV_IRQ_EXT);
if (aplic_direct_parent_irq) {
irq_set_chained_handler(aplic_direct_parent_irq,
aplic_direct_handle_irq);
/*
* Setup CPUHP notifier to enable parent
* interrupt on all CPUs
*/
cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
"irqchip/riscv/aplic:starting",
aplic_direct_starting_cpu,
aplic_direct_dying_cpu);
}
}
/* Fail if we were not able to setup IDC for any CPU */
if (!setup_count)
return -ENODEV;
/* Setup global config and interrupt delivery */
aplic_init_hw_global(priv, false);
/* Create irq domain instance for the APLIC */
direct->irqdomain = irq_domain_create_linear(dev->fwnode, priv->nr_irqs + 1,
&aplic_direct_irqdomain_ops, priv);
if (!direct->irqdomain) {
dev_err(dev, "failed to create direct irq domain\n");
return -ENOMEM;
}
/* Advertise the interrupt controller */
dev_info(dev, "%d interrupts directly connected to %d CPUs\n",
priv->nr_irqs, priv->nr_idcs);
return 0;
}
drivers/irqchip/irq-riscv-aplic-main.c 0 → 100644
View file @ 6bfd2d44
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#include <linux/bitfield.h>
#include <linux/irqchip/riscv-aplic.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_irq.h>
#include <linux/platform_device.h>
#include <linux/printk.h>
#include "irq-riscv-aplic-main.h"
void aplic_irq_unmask(struct irq_data *d)
{
struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
writel(d->hwirq, priv->regs + APLIC_SETIENUM);
}
void aplic_irq_mask(struct irq_data *d)
{
struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
writel(d->hwirq, priv->regs + APLIC_CLRIENUM);
}
int aplic_irq_set_type(struct irq_data *d, unsigned int type)
{
struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
void __iomem *sourcecfg;
u32 val = 0;
switch (type) {
case IRQ_TYPE_NONE:
val = APLIC_SOURCECFG_SM_INACTIVE;
break;
case IRQ_TYPE_LEVEL_LOW:
val = APLIC_SOURCECFG_SM_LEVEL_LOW;
break;
case IRQ_TYPE_LEVEL_HIGH:
val = APLIC_SOURCECFG_SM_LEVEL_HIGH;
break;
case IRQ_TYPE_EDGE_FALLING:
val = APLIC_SOURCECFG_SM_EDGE_FALL;
break;
case IRQ_TYPE_EDGE_RISING:
val = APLIC_SOURCECFG_SM_EDGE_RISE;
break;
default:
return -EINVAL;
}
sourcecfg = priv->regs + APLIC_SOURCECFG_BASE;
sourcecfg += (d->hwirq - 1) * sizeof(u32);
writel(val, sourcecfg);
return 0;
}
int aplic_irqdomain_translate(struct irq_fwspec *fwspec, u32 gsi_base,
unsigned long *hwirq, unsigned int *type)
{
if (WARN_ON(fwspec->param_count < 2))
return -EINVAL;
if (WARN_ON(!fwspec->param[0]))
return -EINVAL;
/* For DT, gsi_base is always zero. */
*hwirq = fwspec->param[0] - gsi_base;
*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
WARN_ON(*type == IRQ_TYPE_NONE);
return 0;
}
void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode)
{
u32 val;
#ifdef CONFIG_RISCV_M_MODE
u32 valh;
if (msi_mode) {
val = lower_32_bits(priv->msicfg.base_ppn);
valh = FIELD_PREP(APLIC_xMSICFGADDRH_BAPPN, upper_32_bits(priv->msicfg.base_ppn));
valh |= FIELD_PREP(APLIC_xMSICFGADDRH_LHXW, priv->msicfg.lhxw);
valh |= FIELD_PREP(APLIC_xMSICFGADDRH_HHXW, priv->msicfg.hhxw);
valh |= FIELD_PREP(APLIC_xMSICFGADDRH_LHXS, priv->msicfg.lhxs);
valh |= FIELD_PREP(APLIC_xMSICFGADDRH_HHXS, priv->msicfg.hhxs);
writel(val, priv->regs + APLIC_xMSICFGADDR);
writel(valh, priv->regs + APLIC_xMSICFGADDRH);
}
#endif
/* Setup APLIC domaincfg register */
val = readl(priv->regs + APLIC_DOMAINCFG);
val |= APLIC_DOMAINCFG_IE;
if (msi_mode)
val |= APLIC_DOMAINCFG_DM;
writel(val, priv->regs + APLIC_DOMAINCFG);
if (readl(priv->regs + APLIC_DOMAINCFG) != val)
dev_warn(priv->dev, "unable to write 0x%x in domaincfg\n", val);
}
static void aplic_init_hw_irqs(struct aplic_priv *priv)
{
int i;
/* Disable all interrupts */
for (i = 0; i <= priv->nr_irqs; i += 32)
writel(-1U, priv->regs + APLIC_CLRIE_BASE + (i / 32) * sizeof(u32));
/* Set interrupt type and default priority for all interrupts */
for (i = 1; i <= priv->nr_irqs; i++) {
writel(0, priv->regs + APLIC_SOURCECFG_BASE + (i - 1) * sizeof(u32));
writel(APLIC_DEFAULT_PRIORITY,
priv->regs + APLIC_TARGET_BASE + (i - 1) * sizeof(u32));
}
/* Clear APLIC domaincfg */
writel(0, priv->regs + APLIC_DOMAINCFG);
}
int aplic_setup_priv(struct aplic_priv *priv, struct device *dev, void __iomem *regs)
{
struct of_phandle_args parent;
int rc;
/*
* Currently, only OF fwnode is supported so extend this
* function for ACPI support.
*/
if (!is_of_node(dev->fwnode))
return -EINVAL;
/* Save device pointer and register base */
priv->dev = dev;
priv->regs = regs;
/* Find out number of interrupt sources */
rc = of_property_read_u32(to_of_node(dev->fwnode), "riscv,num-sources",
&priv->nr_irqs);
if (rc) {
dev_err(dev, "failed to get number of interrupt sources\n");
return rc;
}
/*
* Find out number of IDCs based on parent interrupts
*
* If "msi-parent" property is present then we ignore the
* APLIC IDCs which forces the APLIC driver to use MSI mode.
*/
if (!of_property_present(to_of_node(dev->fwnode), "msi-parent")) {
while (!of_irq_parse_one(to_of_node(dev->fwnode), priv->nr_idcs, &parent))
priv->nr_idcs++;
}
/* Setup initial state APLIC interrupts */
aplic_init_hw_irqs(priv);
return 0;
}
static int aplic_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
bool msi_mode = false;
void __iomem *regs;
int rc;
/* Map the MMIO registers */
regs = devm_platform_ioremap_resource(pdev, 0);
if (!regs) {
dev_err(dev, "failed map MMIO registers\n");
return -ENOMEM;
}
/*
* If msi-parent property is present then setup APLIC MSI
* mode otherwise setup APLIC direct mode.
*/
if (is_of_node(dev->fwnode))
msi_mode = of_property_present(to_of_node(dev->fwnode), "msi-parent");
if (msi_mode)
rc = aplic_msi_setup(dev, regs);
else
rc = aplic_direct_setup(dev, regs);
if (rc)
dev_err(dev, "failed to setup APLIC in %s mode\n", msi_mode ? "MSI" : "direct");
return rc;
}
static const struct of_device_id aplic_match[] = {
{ .compatible = "riscv,aplic" },
{}
};
static struct platform_driver aplic_driver = {
.driver = {
.name = "riscv-aplic",
.of_match_table = aplic_match,
},
.probe = aplic_probe,
};
builtin_platform_driver(aplic_driver);
drivers/irqchip/irq-riscv-aplic-main.h 0 → 100644
View file @ 6bfd2d44
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#ifndef _IRQ_RISCV_APLIC_MAIN_H
#define _IRQ_RISCV_APLIC_MAIN_H
#include <linux/device.h>
#include <linux/io.h>
#include <linux/irq.h>
#include <linux/irqdomain.h>
#include <linux/fwnode.h>
#define APLIC_DEFAULT_PRIORITY 1
struct aplic_msicfg {
phys_addr_t base_ppn;
u32 hhxs;
u32 hhxw;
u32 lhxs;
u32 lhxw;
};
struct aplic_priv {
struct device *dev;
u32 gsi_base;
u32 nr_irqs;
u32 nr_idcs;
void __iomem *regs;
struct aplic_msicfg msicfg;
};
void aplic_irq_unmask(struct irq_data *d);
void aplic_irq_mask(struct irq_data *d);
int aplic_irq_set_type(struct irq_data *d, unsigned int type);
int aplic_irqdomain_translate(struct irq_fwspec *fwspec, u32 gsi_base,
unsigned long *hwirq, unsigned int *type);
void aplic_init_hw_global(struct aplic_priv *priv, bool msi_mode);
int aplic_setup_priv(struct aplic_priv *priv, struct device *dev, void __iomem *regs);
int aplic_direct_setup(struct device *dev, void __iomem *regs);
#ifdef CONFIG_RISCV_APLIC_MSI
int aplic_msi_setup(struct device *dev, void __iomem *regs);
#else
static inline int aplic_msi_setup(struct device *dev, void __iomem *regs)
{
return -ENODEV;
}
#endif
#endif
drivers/irqchip/irq-riscv-aplic-msi.c 0 → 100644
View file @ 6bfd2d44
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#include <linux/bitfield.h>
#include <linux/bitops.h>
#include <linux/cpu.h>
#include <linux/interrupt.h>
#include <linux/irqchip.h>
#include <linux/irqchip/riscv-aplic.h>
#include <linux/irqchip/riscv-imsic.h>
#include <linux/module.h>
#include <linux/msi.h>
#include <linux/of_irq.h>
#include <linux/platform_device.h>
#include <linux/printk.h>
#include <linux/smp.h>
#include "irq-riscv-aplic-main.h"
static void aplic_msi_irq_mask(struct irq_data *d)
{
aplic_irq_mask(d);
irq_chip_mask_parent(d);
}
static void aplic_msi_irq_unmask(struct irq_data *d)
{
irq_chip_unmask_parent(d);
aplic_irq_unmask(d);
}
static void aplic_msi_irq_eoi(struct irq_data *d)
{
struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
/*
* EOI handling is required only for level-triggered interrupts
* when APLIC is in MSI mode.
*/
switch (irqd_get_trigger_type(d)) {
case IRQ_TYPE_LEVEL_LOW:
case IRQ_TYPE_LEVEL_HIGH:
/*
* The section "4.9.2 Special consideration for level-sensitive interrupt
* sources" of the RISC-V AIA specification says:
*
* A second option is for the interrupt service routine to write the
* APLIC’s source identity number for the interrupt to the domain’s
* setipnum register just before exiting. This will cause the interrupt’s
* pending bit to be set to one again if the source is still asserting
* an interrupt, but not if the source is not asserting an interrupt.
*/
writel(d->hwirq, priv->regs + APLIC_SETIPNUM_LE);
break;
}
}
static void aplic_msi_write_msg(struct irq_data *d, struct msi_msg *msg)
{
unsigned int group_index, hart_index, guest_index, val;
struct aplic_priv *priv = irq_data_get_irq_chip_data(d);
struct aplic_msicfg *mc = &priv->msicfg;
phys_addr_t tppn, tbppn, msg_addr;
void __iomem *target;
/* For zeroed MSI, simply write zero into the target register */
if (!msg->address_hi && !msg->address_lo && !msg->data) {
target = priv->regs + APLIC_TARGET_BASE;
target += (d->hwirq - 1) * sizeof(u32);
writel(0, target);
return;
}
/* Sanity check on message data */
WARN_ON(msg->data > APLIC_TARGET_EIID_MASK);
/* Compute target MSI address */
msg_addr = (((u64)msg->address_hi) << 32) | msg->address_lo;
tppn = msg_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
/* Compute target HART Base PPN */
tbppn = tppn;
tbppn &= ~APLIC_xMSICFGADDR_PPN_HART(mc->lhxs);
tbppn &= ~APLIC_xMSICFGADDR_PPN_LHX(mc->lhxw, mc->lhxs);
tbppn &= ~APLIC_xMSICFGADDR_PPN_HHX(mc->hhxw, mc->hhxs);
WARN_ON(tbppn != mc->base_ppn);
/* Compute target group and hart indexes */
group_index = (tppn >> APLIC_xMSICFGADDR_PPN_HHX_SHIFT(mc->hhxs)) &
APLIC_xMSICFGADDR_PPN_HHX_MASK(mc->hhxw);
hart_index = (tppn >> APLIC_xMSICFGADDR_PPN_LHX_SHIFT(mc->lhxs)) &
APLIC_xMSICFGADDR_PPN_LHX_MASK(mc->lhxw);
hart_index |= (group_index << mc->lhxw);
WARN_ON(hart_index > APLIC_TARGET_HART_IDX_MASK);
/* Compute target guest index */
guest_index = tppn & APLIC_xMSICFGADDR_PPN_HART(mc->lhxs);
WARN_ON(guest_index > APLIC_TARGET_GUEST_IDX_MASK);
/* Update IRQ TARGET register */
target = priv->regs + APLIC_TARGET_BASE;
target += (d->hwirq - 1) * sizeof(u32);
val = FIELD_PREP(APLIC_TARGET_HART_IDX, hart_index);
val |= FIELD_PREP(APLIC_TARGET_GUEST_IDX, guest_index);
val |= FIELD_PREP(APLIC_TARGET_EIID, msg->data);
writel(val, target);
}
static void aplic_msi_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
{
arg->desc = desc;
arg->hwirq = (u32)desc->data.icookie.value;
}
static int aplic_msi_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
unsigned long *hwirq, unsigned int *type)
{
struct msi_domain_info *info = d->host_data;
struct aplic_priv *priv = info->data;
return aplic_irqdomain_translate(fwspec, priv->gsi_base, hwirq, type);
}
static const struct msi_domain_template aplic_msi_template = {
.chip = {
.name = "APLIC-MSI",
.irq_mask = aplic_msi_irq_mask,
.irq_unmask = aplic_msi_irq_unmask,
.irq_set_type = aplic_irq_set_type,
.irq_eoi = aplic_msi_irq_eoi,
#ifdef CONFIG_SMP
.irq_set_affinity = irq_chip_set_affinity_parent,
#endif
.irq_write_msi_msg = aplic_msi_write_msg,
.flags = IRQCHIP_SET_TYPE_MASKED |
IRQCHIP_SKIP_SET_WAKE |
IRQCHIP_MASK_ON_SUSPEND,
},
.ops = {
.set_desc = aplic_msi_set_desc,
.msi_translate = aplic_msi_translate,
},
.info = {
.bus_token = DOMAIN_BUS_WIRED_TO_MSI,
.flags = MSI_FLAG_USE_DEV_FWNODE,
.handler = handle_fasteoi_irq,
.handler_name = "fasteoi",
},
};
int aplic_msi_setup(struct device *dev, void __iomem *regs)
{
const struct imsic_global_config *imsic_global;
struct aplic_priv *priv;
struct aplic_msicfg *mc;
phys_addr_t pa;
int rc;
priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
rc = aplic_setup_priv(priv, dev, regs);
if (rc) {
dev_err(dev, "failed to create APLIC context\n");
return rc;
}
mc = &priv->msicfg;
/*
* The APLIC outgoing MSI config registers assume target MSI
* controller to be RISC-V AIA IMSIC controller.
*/
imsic_global = imsic_get_global_config();
if (!imsic_global) {
dev_err(dev, "IMSIC global config not found\n");
return -ENODEV;
}
/* Find number of guest index bits (LHXS) */
mc->lhxs = imsic_global->guest_index_bits;
if (APLIC_xMSICFGADDRH_LHXS_MASK < mc->lhxs) {
dev_err(dev, "IMSIC guest index bits big for APLIC LHXS\n");
return -EINVAL;
}
/* Find number of HART index bits (LHXW) */
mc->lhxw = imsic_global->hart_index_bits;
if (APLIC_xMSICFGADDRH_LHXW_MASK < mc->lhxw) {
dev_err(dev, "IMSIC hart index bits big for APLIC LHXW\n");
return -EINVAL;
}
/* Find number of group index bits (HHXW) */
mc->hhxw = imsic_global->group_index_bits;
if (APLIC_xMSICFGADDRH_HHXW_MASK < mc->hhxw) {
dev_err(dev, "IMSIC group index bits big for APLIC HHXW\n");
return -EINVAL;
}
/* Find first bit position of group index (HHXS) */
mc->hhxs = imsic_global->group_index_shift;
if (mc->hhxs < (2 * APLIC_xMSICFGADDR_PPN_SHIFT)) {
dev_err(dev, "IMSIC group index shift should be >= %d\n",
(2 * APLIC_xMSICFGADDR_PPN_SHIFT));
return -EINVAL;
}
mc->hhxs -= (2 * APLIC_xMSICFGADDR_PPN_SHIFT);
if (APLIC_xMSICFGADDRH_HHXS_MASK < mc->hhxs) {
dev_err(dev, "IMSIC group index shift big for APLIC HHXS\n");
return -EINVAL;
}
/* Compute PPN base */
mc->base_ppn = imsic_global->base_addr >> APLIC_xMSICFGADDR_PPN_SHIFT;
mc->base_ppn &= ~APLIC_xMSICFGADDR_PPN_HART(mc->lhxs);
mc->base_ppn &= ~APLIC_xMSICFGADDR_PPN_LHX(mc->lhxw, mc->lhxs);
mc->base_ppn &= ~APLIC_xMSICFGADDR_PPN_HHX(mc->hhxw, mc->hhxs);
/* Setup global config and interrupt delivery */
aplic_init_hw_global(priv, true);
/* Set the APLIC device MSI domain if not available */
if (!dev_get_msi_domain(dev)) {
/*
* The device MSI domain for OF devices is only set at the
* time of populating/creating OF device. If the device MSI
* domain is discovered later after the OF device is created
* then we need to set it explicitly before using any platform
* MSI functions.
*
* In case of APLIC device, the parent MSI domain is always
* IMSIC and the IMSIC MSI domains are created later through
* the platform driver probing so we set it explicitly here.
*/
if (is_of_node(dev->fwnode))
of_msi_configure(dev, to_of_node(dev->fwnode));
}
if (!msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN, &aplic_msi_template,
priv->nr_irqs + 1, priv, priv)) {
dev_err(dev, "failed to create MSI irq domain\n");
return -ENOMEM;
}
/* Advertise the interrupt controller */
pa = priv->msicfg.base_ppn << APLIC_xMSICFGADDR_PPN_SHIFT;
dev_info(dev, "%d interrupts forwarded to MSI base %pa\n", priv->nr_irqs, &pa);
return 0;
}
drivers/irqchip/irq-riscv-imsic-early.c 0 → 100644
View file @ 6bfd2d44
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#define pr_fmt(fmt) "riscv-imsic: " fmt
#include <linux/cpu.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/irq.h>
#include <linux/irqchip.h>
#include <linux/irqchip/chained_irq.h>
#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/smp.h>
#include "irq-riscv-imsic-state.h"
static int imsic_parent_irq;
#ifdef CONFIG_SMP
static void imsic_ipi_send(unsigned int cpu)
{
struct imsic_local_config *local = per_cpu_ptr(imsic->global.local, cpu);
writel_relaxed(IMSIC_IPI_ID, local->msi_va);
}
static void imsic_ipi_starting_cpu(void)
{
/* Enable IPIs for current CPU. */
__imsic_id_set_enable(IMSIC_IPI_ID);
}
static void imsic_ipi_dying_cpu(void)
{
/* Disable IPIs for current CPU. */
__imsic_id_clear_enable(IMSIC_IPI_ID);
}
static int __init imsic_ipi_domain_init(void)
{
int virq;
/* Create IMSIC IPI multiplexing */
virq = ipi_mux_create(IMSIC_NR_IPI, imsic_ipi_send);
if (virq <= 0)
return virq < 0 ? virq : -ENOMEM;
/* Set vIRQ range */
riscv_ipi_set_virq_range(virq, IMSIC_NR_IPI, true);
/* Announce that IMSIC is providing IPIs */
pr_info("%pfwP: providing IPIs using interrupt %d\n", imsic->fwnode, IMSIC_IPI_ID);
return 0;
}
#else
static void imsic_ipi_starting_cpu(void) { }
static void imsic_ipi_dying_cpu(void) { }
static int __init imsic_ipi_domain_init(void) { return 0; }
#endif
/*
* To handle an interrupt, we read the TOPEI CSR and write zero in one
* instruction. If TOPEI CSR is non-zero then we translate TOPEI.ID to
* Linux interrupt number and let Linux IRQ subsystem handle it.
*/
static void imsic_handle_irq(struct irq_desc *desc)
{
struct irq_chip *chip = irq_desc_get_chip(desc);
int err, cpu = smp_processor_id();
struct imsic_vector *vec;
unsigned long local_id;
chained_irq_enter(chip, desc);
while ((local_id = csr_swap(CSR_TOPEI, 0))) {
local_id >>= TOPEI_ID_SHIFT;
if (local_id == IMSIC_IPI_ID) {
if (IS_ENABLED(CONFIG_SMP))
ipi_mux_process();
continue;
}
if (unlikely(!imsic->base_domain))
continue;
vec = imsic_vector_from_local_id(cpu, local_id);
if (!vec) {
pr_warn_ratelimited("vector not found for local ID 0x%lx\n", local_id);
continue;
}
err = generic_handle_domain_irq(imsic->base_domain, vec->hwirq);
if (unlikely(err))
pr_warn_ratelimited("hwirq 0x%x mapping not found\n", vec->hwirq);
}
chained_irq_exit(chip, desc);
}
static int imsic_starting_cpu(unsigned int cpu)
{
/* Mark per-CPU IMSIC state as online */
imsic_state_online();
/* Enable per-CPU parent interrupt */
enable_percpu_irq(imsic_parent_irq, irq_get_trigger_type(imsic_parent_irq));
/* Setup IPIs */
imsic_ipi_starting_cpu();
/*
* Interrupts identities might have been enabled/disabled while
* this CPU was not running so sync-up local enable/disable state.
*/
imsic_local_sync_all();
/* Enable local interrupt delivery */
imsic_local_delivery(true);
return 0;
}
static int imsic_dying_cpu(unsigned int cpu)
{
/* Cleanup IPIs */
imsic_ipi_dying_cpu();
/* Mark per-CPU IMSIC state as offline */
imsic_state_offline();
return 0;
}
static int __init imsic_early_probe(struct fwnode_handle *fwnode)
{
struct irq_domain *domain;
int rc;
/* Find parent domain and register chained handler */
domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(), DOMAIN_BUS_ANY);
if (!domain) {
pr_err("%pfwP: Failed to find INTC domain\n", fwnode);
return -ENOENT;
}
imsic_parent_irq = irq_create_mapping(domain, RV_IRQ_EXT);
if (!imsic_parent_irq) {
pr_err("%pfwP: Failed to create INTC mapping\n", fwnode);
return -ENOENT;
}
/* Initialize IPI domain */
rc = imsic_ipi_domain_init();
if (rc) {
pr_err("%pfwP: Failed to initialize IPI domain\n", fwnode);
return rc;
}
/* Setup chained handler to the parent domain interrupt */
irq_set_chained_handler(imsic_parent_irq, imsic_handle_irq);
/*
* Setup cpuhp state (must be done after setting imsic_parent_irq)
*
* Don't disable per-CPU IMSIC file when CPU goes offline
* because this affects IPI and the masking/unmasking of
* virtual IPIs is done via generic IPI-Mux
*/
cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_IMSIC_STARTING, "irqchip/riscv/imsic:starting",
imsic_starting_cpu, imsic_dying_cpu);
return 0;
}
static int __init imsic_early_dt_init(struct device_node *node, struct device_node *parent)
{
struct fwnode_handle *fwnode = &node->fwnode;
int rc;
/* Setup IMSIC state */
rc = imsic_setup_state(fwnode);
if (rc) {
pr_err("%pfwP: failed to setup state (error %d)\n", fwnode, rc);
return rc;
}
/* Do early setup of IPIs */
rc = imsic_early_probe(fwnode);
if (rc)
return rc;
/* Ensure that OF platform device gets probed */
of_node_clear_flag(node, OF_POPULATED);
return 0;
}
IRQCHIP_DECLARE(riscv_imsic, "riscv,imsics", imsic_early_dt_init);
drivers/irqchip/irq-riscv-imsic-platform.c 0 → 100644
View file @ 6bfd2d44
// SPDX-License-Identifier: GPL-2.0
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#define pr_fmt(fmt) "riscv-imsic: " fmt
#include <linux/bitmap.h>
#include <linux/cpu.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/irq.h>
#include <linux/irqchip.h>
#include <linux/irqdomain.h>
#include <linux/module.h>
#include <linux/msi.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
#include <linux/spinlock.h>
#include <linux/smp.h>
#include "irq-riscv-imsic-state.h"
static bool imsic_cpu_page_phys(unsigned int cpu, unsigned int guest_index,
phys_addr_t *out_msi_pa)
{
struct imsic_global_config *global;
struct imsic_local_config *local;
global = &imsic->global;
local = per_cpu_ptr(global->local, cpu);
if (BIT(global->guest_index_bits) <= guest_index)
return false;
if (out_msi_pa)
*out_msi_pa = local->msi_pa + (guest_index * IMSIC_MMIO_PAGE_SZ);
return true;
}
static void imsic_irq_mask(struct irq_data *d)
{
imsic_vector_mask(irq_data_get_irq_chip_data(d));
}
static void imsic_irq_unmask(struct irq_data *d)
{
imsic_vector_unmask(irq_data_get_irq_chip_data(d));
}
static int imsic_irq_retrigger(struct irq_data *d)
{
struct imsic_vector *vec = irq_data_get_irq_chip_data(d);
struct imsic_local_config *local;
if (WARN_ON(!vec))
return -ENOENT;
local = per_cpu_ptr(imsic->global.local, vec->cpu);
writel_relaxed(vec->local_id, local->msi_va);
return 0;
}
static void imsic_irq_compose_vector_msg(struct imsic_vector *vec, struct msi_msg *msg)
{
phys_addr_t msi_addr;
if (WARN_ON(!vec))
return;
if (WARN_ON(!imsic_cpu_page_phys(vec->cpu, 0, &msi_addr)))
return;
msg->address_hi = upper_32_bits(msi_addr);
msg->address_lo = lower_32_bits(msi_addr);
msg->data = vec->local_id;
}
static void imsic_irq_compose_msg(struct irq_data *d, struct msi_msg *msg)
{
imsic_irq_compose_vector_msg(irq_data_get_irq_chip_data(d), msg);
}
#ifdef CONFIG_SMP
static void imsic_msi_update_msg(struct irq_data *d, struct imsic_vector *vec)
{
struct msi_msg msg = { };
imsic_irq_compose_vector_msg(vec, &msg);
irq_data_get_irq_chip(d)->irq_write_msi_msg(d, &msg);
}
static int imsic_irq_set_affinity(struct irq_data *d, const struct cpumask *mask_val,
bool force)
{
struct imsic_vector *old_vec, *new_vec;
struct irq_data *pd = d->parent_data;
old_vec = irq_data_get_irq_chip_data(pd);
if (WARN_ON(!old_vec))
return -ENOENT;
/* If old vector cpu belongs to the target cpumask then do nothing */
if (cpumask_test_cpu(old_vec->cpu, mask_val))
return IRQ_SET_MASK_OK_DONE;
/* If move is already in-flight then return failure */
if (imsic_vector_get_move(old_vec))
return -EBUSY;
/* Get a new vector on the desired set of CPUs */
new_vec = imsic_vector_alloc(old_vec->hwirq, mask_val);
if (!new_vec)
return -ENOSPC;
/* Point device to the new vector */
imsic_msi_update_msg(d, new_vec);
/* Update irq descriptors with the new vector */
pd->chip_data = new_vec;
/* Update effective affinity of parent irq data */
irq_data_update_effective_affinity(pd, cpumask_of(new_vec->cpu));
/* Move state of the old vector to the new vector */
imsic_vector_move(old_vec, new_vec);
return IRQ_SET_MASK_OK_DONE;
}
#endif
static struct irq_chip imsic_irq_base_chip = {
.name = "IMSIC",
.irq_mask = imsic_irq_mask,
.irq_unmask = imsic_irq_unmask,
.irq_retrigger = imsic_irq_retrigger,
.irq_compose_msi_msg = imsic_irq_compose_msg,
.flags = IRQCHIP_SKIP_SET_WAKE |
IRQCHIP_MASK_ON_SUSPEND,
};
static int imsic_irq_domain_alloc(struct irq_domain *domain, unsigned int virq,
unsigned int nr_irqs, void *args)
{
struct imsic_vector *vec;
/* Multi-MSI is not supported yet. */
if (nr_irqs > 1)
return -EOPNOTSUPP;
vec = imsic_vector_alloc(virq, cpu_online_mask);
if (!vec)
return -ENOSPC;
irq_domain_set_info(domain, virq, virq, &imsic_irq_base_chip, vec,
handle_simple_irq, NULL, NULL);
irq_set_noprobe(virq);
irq_set_affinity(virq, cpu_online_mask);
irq_data_update_effective_affinity(irq_get_irq_data(virq), cpumask_of(vec->cpu));
return 0;
}
static void imsic_irq_domain_free(struct irq_domain *domain, unsigned int virq,
unsigned int nr_irqs)
{
struct irq_data *d = irq_domain_get_irq_data(domain, virq);
imsic_vector_free(irq_data_get_irq_chip_data(d));
irq_domain_free_irqs_parent(domain, virq, nr_irqs);
}
static int imsic_irq_domain_select(struct irq_domain *domain, struct irq_fwspec *fwspec,
enum irq_domain_bus_token bus_token)
{
const struct msi_parent_ops *ops = domain->msi_parent_ops;
u32 busmask = BIT(bus_token);
if (fwspec->fwnode != domain->fwnode || fwspec->param_count != 0)
return 0;
/* Handle pure domain searches */
if (bus_token == ops->bus_select_token)
return 1;
return !!(ops->bus_select_mask & busmask);
}
#ifdef CONFIG_GENERIC_IRQ_DEBUGFS
static void imsic_irq_debug_show(struct seq_file *m, struct irq_domain *d,
struct irq_data *irqd, int ind)
{
if (!irqd) {
imsic_vector_debug_show_summary(m, ind);
return;
}
imsic_vector_debug_show(m, irq_data_get_irq_chip_data(irqd), ind);
}
#endif
static const struct irq_domain_ops imsic_base_domain_ops = {
.alloc = imsic_irq_domain_alloc,
.free = imsic_irq_domain_free,
.select = imsic_irq_domain_select,
#ifdef CONFIG_GENERIC_IRQ_DEBUGFS
.debug_show = imsic_irq_debug_show,
#endif
};
#ifdef CONFIG_RISCV_IMSIC_PCI
static void imsic_pci_mask_irq(struct irq_data *d)
{
pci_msi_mask_irq(d);
irq_chip_mask_parent(d);
}
static void imsic_pci_unmask_irq(struct irq_data *d)
{
irq_chip_unmask_parent(d);
pci_msi_unmask_irq(d);
}
#define MATCH_PCI_MSI BIT(DOMAIN_BUS_PCI_MSI)
#else
#define MATCH_PCI_MSI 0
#endif
static bool imsic_init_dev_msi_info(struct device *dev,
struct irq_domain *domain,
struct irq_domain *real_parent,
struct msi_domain_info *info)
{
const struct msi_parent_ops *pops = real_parent->msi_parent_ops;
/* MSI parent domain specific settings */
switch (real_parent->bus_token) {
case DOMAIN_BUS_NEXUS:
if (WARN_ON_ONCE(domain != real_parent))
return false;
#ifdef CONFIG_SMP
info->chip->irq_set_affinity = imsic_irq_set_affinity;
#endif
break;
default:
WARN_ON_ONCE(1);
return false;
}
/* Is the target supported? */
switch (info->bus_token) {
#ifdef CONFIG_RISCV_IMSIC_PCI
case DOMAIN_BUS_PCI_DEVICE_MSI:
case DOMAIN_BUS_PCI_DEVICE_MSIX:
info->chip->irq_mask = imsic_pci_mask_irq;
info->chip->irq_unmask = imsic_pci_unmask_irq;
break;
#endif
case DOMAIN_BUS_DEVICE_MSI:
/*
* Per-device MSI should never have any MSI feature bits
* set. It's sole purpose is to create a dumb interrupt
* chip which has a device specific irq_write_msi_msg()
* callback.
*/
if (WARN_ON_ONCE(info->flags))
return false;
/* Core managed MSI descriptors */
info->flags |= MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS |
MSI_FLAG_FREE_MSI_DESCS;
break;
case DOMAIN_BUS_WIRED_TO_MSI:
break;
default:
WARN_ON_ONCE(1);
return false;
}
/* Use hierarchial chip operations re-trigger */
info->chip->irq_retrigger = irq_chip_retrigger_hierarchy;
/*
* Mask out the domain specific MSI feature flags which are not
* supported by the real parent.
*/
info->flags &= pops->supported_flags;
/* Enforce the required flags */
info->flags |= pops->required_flags;
return true;
}
#define MATCH_PLATFORM_MSI BIT(DOMAIN_BUS_PLATFORM_MSI)
static const struct msi_parent_ops imsic_msi_parent_ops = {
.supported_flags = MSI_GENERIC_FLAGS_MASK |
MSI_FLAG_PCI_MSIX,
.required_flags = MSI_FLAG_USE_DEF_DOM_OPS |
MSI_FLAG_USE_DEF_CHIP_OPS,
.bus_select_token = DOMAIN_BUS_NEXUS,
.bus_select_mask = MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
.init_dev_msi_info = imsic_init_dev_msi_info,
};
int imsic_irqdomain_init(void)
{
struct imsic_global_config *global;
if (!imsic || !imsic->fwnode) {
pr_err("early driver not probed\n");
return -ENODEV;
}
if (imsic->base_domain) {
pr_err("%pfwP: irq domain already created\n", imsic->fwnode);
return -ENODEV;
}
/* Create Base IRQ domain */
imsic->base_domain = irq_domain_create_tree(imsic->fwnode,
&imsic_base_domain_ops, imsic);
if (!imsic->base_domain) {
pr_err("%pfwP: failed to create IMSIC base domain\n", imsic->fwnode);
return -ENOMEM;
}
imsic->base_domain->flags |= IRQ_DOMAIN_FLAG_MSI_PARENT;
imsic->base_domain->msi_parent_ops = &imsic_msi_parent_ops;
irq_domain_update_bus_token(imsic->base_domain, DOMAIN_BUS_NEXUS);
global = &imsic->global;
pr_info("%pfwP: hart-index-bits: %d, guest-index-bits: %d\n",
imsic->fwnode, global->hart_index_bits, global->guest_index_bits);
pr_info("%pfwP: group-index-bits: %d, group-index-shift: %d\n",
imsic->fwnode, global->group_index_bits, global->group_index_shift);
pr_info("%pfwP: per-CPU IDs %d at base PPN %pa\n",
imsic->fwnode, global->nr_ids, &global->base_addr);
pr_info("%pfwP: total %d interrupts available\n",
imsic->fwnode, num_possible_cpus() * (global->nr_ids - 1));
return 0;
}
static int imsic_platform_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
if (imsic && imsic->fwnode != dev->fwnode) {
dev_err(dev, "fwnode mismatch\n");
return -ENODEV;
}
return imsic_irqdomain_init();
}
static const struct of_device_id imsic_platform_match[] = {
{ .compatible = "riscv,imsics" },
{}
};
static struct platform_driver imsic_platform_driver = {
.driver = {
.name = "riscv-imsic",
.of_match_table = imsic_platform_match,
},
.probe = imsic_platform_probe,
};
builtin_platform_driver(imsic_platform_driver);
drivers/irqchip/irq-riscv-imsic-state.c 0 → 100644
View file @ 6bfd2d44
This diff is collapsed.
drivers/irqchip/irq-riscv-imsic-state.h 0 → 100644
View file @ 6bfd2d44
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#ifndef _IRQ_RISCV_IMSIC_STATE_H
#define _IRQ_RISCV_IMSIC_STATE_H
#include <linux/irqchip/riscv-imsic.h>
#include <linux/irqdomain.h>
#include <linux/fwnode.h>
#include <linux/timer.h>
#define IMSIC_IPI_ID 1
#define IMSIC_NR_IPI 8
struct imsic_vector {
/* Fixed details of the vector */
unsigned int cpu;
unsigned int local_id;
/* Details saved by driver in the vector */
unsigned int hwirq;
/* Details accessed using local lock held */
bool enable;
struct imsic_vector *move;
};
struct imsic_local_priv {
/* Local lock to protect vector enable/move variables and dirty bitmap */
raw_spinlock_t lock;
/* Local dirty bitmap for synchronization */
unsigned long *dirty_bitmap;
#ifdef CONFIG_SMP
/* Local timer for synchronization */
struct timer_list timer;
#endif
/* Local vector table */
struct imsic_vector *vectors;
};
struct imsic_priv {
/* Device details */
struct fwnode_handle *fwnode;
/* Global configuration common for all HARTs */
struct imsic_global_config global;
/* Per-CPU state */
struct imsic_local_priv __percpu *lpriv;
/* State of IRQ matrix allocator */
raw_spinlock_t matrix_lock;
struct irq_matrix *matrix;
/* IRQ domains (created by platform driver) */
struct irq_domain *base_domain;
};
extern struct imsic_priv *imsic;
void __imsic_eix_update(unsigned long base_id, unsigned long num_id, bool pend, bool val);
static inline void __imsic_id_set_enable(unsigned long id)
{
__imsic_eix_update(id, 1, false, true);
}
static inline void __imsic_id_clear_enable(unsigned long id)
{
__imsic_eix_update(id, 1, false, false);
}
void imsic_local_sync_all(void);
void imsic_local_delivery(bool enable);
void imsic_vector_mask(struct imsic_vector *vec);
void imsic_vector_unmask(struct imsic_vector *vec);
static inline bool imsic_vector_isenabled(struct imsic_vector *vec)
{
return READ_ONCE(vec->enable);
}
static inline struct imsic_vector *imsic_vector_get_move(struct imsic_vector *vec)
{
return READ_ONCE(vec->move);
}
void imsic_vector_move(struct imsic_vector *old_vec, struct imsic_vector *new_vec);
struct imsic_vector *imsic_vector_from_local_id(unsigned int cpu, unsigned int local_id);
struct imsic_vector *imsic_vector_alloc(unsigned int hwirq, const struct cpumask *mask);
void imsic_vector_free(struct imsic_vector *vector);
void imsic_vector_debug_show(struct seq_file *m, struct imsic_vector *vec, int ind);
void imsic_vector_debug_show_summary(struct seq_file *m, int ind);
void imsic_state_online(void);
void imsic_state_offline(void);
int imsic_setup_state(struct fwnode_handle *fwnode);
int imsic_irqdomain_init(void);
#endif
drivers/irqchip/irq-sifive-plic.c
View file @ 6bfd2d44
......@@ -164,15 +164,12 @@ static int plic_set_affinity(struct irq_data *d,
const struct cpumask *mask_val, bool force)
{
unsigned int cpu;
struct cpumask amask;
struct plic_priv *priv = irq_data_get_irq_chip_data(d);
cpumask_and(&amask, &priv->lmask, mask_val);
if (force)
cpu = cpumask_first(&amask);
cpu = cpumask_first_and(&priv->lmask, mask_val);
else
cpu = cpumask_any_and(&amask, cpu_online_mask);
cpu = cpumask_first_and_and(&priv->lmask, mask_val, cpu_online_mask);
if (cpu >= nr_cpu_ids)
return -EINVAL;
......
drivers/irqchip/irq-stm32-exti.c
View file @ 6bfd2d44
......@@ -19,13 +19,26 @@
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include <linux/platform_device.h>
#include <linux/syscore_ops.h>
#include <linux/pm.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
#define IRQS_PER_BANK 32
#define IRQS_PER_BANK 32
#define HWSPNLCK_TIMEOUT 1000 /* usec */
#define HWSPNLCK_TIMEOUT 1000 /* usec */
#define EXTI_EnCIDCFGR(n) (0x180 + (n) * 4)
#define EXTI_HWCFGR1 0x3f0
/* Register: EXTI_EnCIDCFGR(n) */
#define EXTI_CIDCFGR_CFEN_MASK BIT(0)
#define EXTI_CIDCFGR_CID_MASK GENMASK(6, 4)
#define EXTI_CIDCFGR_CID_SHIFT 4
/* Register: EXTI_HWCFGR1 */
#define EXTI_HWCFGR1_CIDWIDTH_MASK GENMASK(27, 24)
#define EXTI_CID1 1
struct stm32_exti_bank {
u32 imr_ofst;
......@@ -36,6 +49,7 @@ struct stm32_exti_bank {
u32 rpr_ofst;
u32 fpr_ofst;
u32 trg_ofst;
u32 seccfgr_ofst;
};
#define UNDEF_REG ~0
......@@ -54,17 +68,18 @@ struct stm32_exti_chip_data {
u32 mask_cache;
u32 rtsr_cache;
u32 ftsr_cache;
u32 event_reserved;
};
struct stm32_exti_host_data {
void __iomem *base;
struct device *dev;
struct stm32_exti_chip_data *chips_data;
const struct stm32_exti_drv_data *drv_data;
struct hwspinlock *hwlock;
bool dt_has_irqs_desc; /* skip internal desc_irqs array and get it from DT */
};
static struct stm32_exti_host_data *stm32_host_data;
static const struct stm32_exti_bank stm32f4xx_exti_b1 = {
.imr_ofst = 0x00,
.emr_ofst = 0x04,
......@@ -74,6 +89,7 @@ static const struct stm32_exti_bank stm32f4xx_exti_b1 = {
.rpr_ofst = 0x14,
.fpr_ofst = UNDEF_REG,
.trg_ofst = UNDEF_REG,
.seccfgr_ofst = UNDEF_REG,
};
static const struct stm32_exti_bank *stm32f4xx_exti_banks[] = {
......@@ -94,6 +110,7 @@ static const struct stm32_exti_bank stm32h7xx_exti_b1 = {
.rpr_ofst = 0x88,
.fpr_ofst = UNDEF_REG,
.trg_ofst = UNDEF_REG,
.seccfgr_ofst = UNDEF_REG,
};
static const struct stm32_exti_bank stm32h7xx_exti_b2 = {
......@@ -105,6 +122,7 @@ static const struct stm32_exti_bank stm32h7xx_exti_b2 = {
.rpr_ofst = 0x98,
.fpr_ofst = UNDEF_REG,
.trg_ofst = UNDEF_REG,
.seccfgr_ofst = UNDEF_REG,
};
static const struct stm32_exti_bank stm32h7xx_exti_b3 = {
......@@ -116,6 +134,7 @@ static const struct stm32_exti_bank stm32h7xx_exti_b3 = {
.rpr_ofst = 0xA8,
.fpr_ofst = UNDEF_REG,
.trg_ofst = UNDEF_REG,
.seccfgr_ofst = UNDEF_REG,
};
static const struct stm32_exti_bank *stm32h7xx_exti_banks[] = {
......@@ -138,6 +157,7 @@ static const struct stm32_exti_bank stm32mp1_exti_b1 = {
.rpr_ofst = 0x0C,
.fpr_ofst = 0x10,
.trg_ofst = 0x3EC,
.seccfgr_ofst = 0x14,
};
static const struct stm32_exti_bank stm32mp1_exti_b2 = {
......@@ -149,6 +169,7 @@ static const struct stm32_exti_bank stm32mp1_exti_b2 = {
.rpr_ofst = 0x2C,
.fpr_ofst = 0x30,
.trg_ofst = 0x3E8,
.seccfgr_ofst = 0x34,
};
static const struct stm32_exti_bank stm32mp1_exti_b3 = {
......@@ -160,6 +181,7 @@ static const struct stm32_exti_bank stm32mp1_exti_b3 = {
.rpr_ofst = 0x4C,
.fpr_ofst = 0x50,
.trg_ofst = 0x3E4,
.seccfgr_ofst = 0x54,
};
static const struct stm32_exti_bank *stm32mp1_exti_banks[] = {
......@@ -322,7 +344,7 @@ static void stm32_irq_handler(struct irq_desc *desc)
while ((pending = stm32_exti_pending(gc))) {
for_each_set_bit(n, &pending, IRQS_PER_BANK)
generic_handle_domain_irq(domain, irq_base + n);
}
}
}
chained_irq_exit(chip, desc);
......@@ -621,50 +643,32 @@ static int stm32_exti_h_set_affinity(struct irq_data *d,
return IRQ_SET_MASK_OK_DONE;
}
static int __maybe_unused stm32_exti_h_suspend(void)
static int stm32_exti_h_suspend(struct device *dev)
{
struct stm32_exti_host_data *host_data = dev_get_drvdata(dev);
struct stm32_exti_chip_data *chip_data;
int i;
for (i = 0; i < stm32_host_data->drv_data->bank_nr; i++) {
chip_data = &stm32_host_data->chips_data[i];
raw_spin_lock(&chip_data->rlock);
for (i = 0; i < host_data->drv_data->bank_nr; i++) {
chip_data = &host_data->chips_data[i];
stm32_chip_suspend(chip_data, chip_data->wake_active);
raw_spin_unlock(&chip_data->rlock);
}
return 0;
}
static void __maybe_unused stm32_exti_h_resume(void)
static int stm32_exti_h_resume(struct device *dev)
{
struct stm32_exti_host_data *host_data = dev_get_drvdata(dev);
struct stm32_exti_chip_data *chip_data;
int i;
for (i = 0; i < stm32_host_data->drv_data->bank_nr; i++) {
chip_data = &stm32_host_data->chips_data[i];
raw_spin_lock(&chip_data->rlock);
for (i = 0; i < host_data->drv_data->bank_nr; i++) {
chip_data = &host_data->chips_data[i];
stm32_chip_resume(chip_data, chip_data->mask_cache);
raw_spin_unlock(&chip_data->rlock);
}
}
static struct syscore_ops stm32_exti_h_syscore_ops = {
#ifdef CONFIG_PM_SLEEP
.suspend = stm32_exti_h_suspend,
.resume = stm32_exti_h_resume,
#endif
};
static void stm32_exti_h_syscore_init(struct stm32_exti_host_data *host_data)
{
stm32_host_data = host_data;
register_syscore_ops(&stm32_exti_h_syscore_ops);
}
static void stm32_exti_h_syscore_deinit(void)
{
unregister_syscore_ops(&stm32_exti_h_syscore_ops);
return 0;
}
static int stm32_exti_h_retrigger(struct irq_data *d)
......@@ -725,12 +729,35 @@ static int stm32_exti_h_domain_alloc(struct irq_domain *dm,
bank = hwirq / IRQS_PER_BANK;
chip_data = &host_data->chips_data[bank];
/* Check if event is reserved (Secure) */
if (chip_data->event_reserved & BIT(hwirq % IRQS_PER_BANK)) {
dev_err(host_data->dev, "event %lu is reserved, secure\n", hwirq);
return -EPERM;
}
event_trg = readl_relaxed(host_data->base + chip_data->reg_bank->trg_ofst);
chip = (event_trg & BIT(hwirq % IRQS_PER_BANK)) ?
&stm32_exti_h_chip : &stm32_exti_h_chip_direct;
irq_domain_set_hwirq_and_chip(dm, virq, hwirq, chip, chip_data);
if (host_data->dt_has_irqs_desc) {
struct of_phandle_args out_irq;
int ret;
ret = of_irq_parse_one(host_data->dev->of_node, hwirq, &out_irq);
if (ret)
return ret;
/* we only support one parent, so far */
if (of_node_to_fwnode(out_irq.np) != dm->parent->fwnode)
return -EINVAL;
of_phandle_args_to_fwspec(out_irq.np, out_irq.args,
out_irq.args_count, &p_fwspec);
return irq_domain_alloc_irqs_parent(dm, virq, 1, &p_fwspec);
}
if (!host_data->drv_data->desc_irqs)
return -EINVAL;
......@@ -771,8 +798,6 @@ stm32_exti_host_data *stm32_exti_host_init(const struct stm32_exti_drv_data *dd,
goto free_chips_data;
}
stm32_host_data = host_data;
return host_data;
free_chips_data:
......@@ -807,6 +832,10 @@ stm32_exti_chip_data *stm32_exti_chip_init(struct stm32_exti_host_data *h_data,
if (stm32_bank->emr_ofst != UNDEF_REG)
writel_relaxed(0, base + stm32_bank->emr_ofst);
/* reserve Secure events */
if (stm32_bank->seccfgr_ofst != UNDEF_REG)
chip_data->event_reserved = readl_relaxed(base + stm32_bank->seccfgr_ofst);
pr_info("%pOF: bank%d\n", node, bank_idx);
return chip_data;
......@@ -891,6 +920,27 @@ static const struct irq_domain_ops stm32_exti_h_domain_ops = {
.xlate = irq_domain_xlate_twocell,
};
static void stm32_exti_check_rif(struct stm32_exti_host_data *host_data)
{
unsigned int bank, i, event;
u32 cid, cidcfgr, hwcfgr1;
/* quit on CID not supported */
hwcfgr1 = readl_relaxed(host_data->base + EXTI_HWCFGR1);
if ((hwcfgr1 & EXTI_HWCFGR1_CIDWIDTH_MASK) == 0)
return;
for (bank = 0; bank < host_data->drv_data->bank_nr; bank++) {
for (i = 0; i < IRQS_PER_BANK; i++) {
event = bank * IRQS_PER_BANK + i;
cidcfgr = readl_relaxed(host_data->base + EXTI_EnCIDCFGR(event));
cid = (cidcfgr & EXTI_CIDCFGR_CID_MASK) >> EXTI_CIDCFGR_CID_SHIFT;
if ((cidcfgr & EXTI_CIDCFGR_CFEN_MASK) && cid != EXTI_CID1)
host_data->chips_data[bank].event_reserved |= BIT(i);
}
}
}
static void stm32_exti_remove_irq(void *data)
{
struct irq_domain *domain = data;
......@@ -898,11 +948,6 @@ static void stm32_exti_remove_irq(void *data)
irq_domain_remove(domain);
}
static void stm32_exti_remove(struct platform_device *pdev)
{
stm32_exti_h_syscore_deinit();
}
static int stm32_exti_probe(struct platform_device *pdev)
{
int ret, i;
......@@ -916,6 +961,9 @@ static int stm32_exti_probe(struct platform_device *pdev)
if (!host_data)
return -ENOMEM;
dev_set_drvdata(dev, host_data);
host_data->dev = dev;
/* check for optional hwspinlock which may be not available yet */
ret = of_hwspin_lock_get_id(np, 0);
if (ret == -EPROBE_DEFER)
......@@ -955,6 +1003,8 @@ static int stm32_exti_probe(struct platform_device *pdev)
for (i = 0; i < drv_data->bank_nr; i++)
stm32_exti_chip_init(host_data, i, np);
stm32_exti_check_rif(host_data);
parent_domain = irq_find_host(of_irq_find_parent(np));
if (!parent_domain) {
dev_err(dev, "GIC interrupt-parent not found\n");
......@@ -975,7 +1025,8 @@ static int stm32_exti_probe(struct platform_device *pdev)
if (ret)
return ret;
stm32_exti_h_syscore_init(host_data);
if (of_property_read_bool(np, "interrupts-extended"))
host_data->dt_has_irqs_desc = true;
return 0;
}
......@@ -988,12 +1039,16 @@ static const struct of_device_id stm32_exti_ids[] = {
};
MODULE_DEVICE_TABLE(of, stm32_exti_ids);
static const struct dev_pm_ops stm32_exti_dev_pm_ops = {
NOIRQ_SYSTEM_SLEEP_PM_OPS(stm32_exti_h_suspend, stm32_exti_h_resume)
};
static struct platform_driver stm32_exti_driver = {
.probe = stm32_exti_probe,
.remove_new = stm32_exti_remove,
.driver = {
.name = "stm32_exti",
.of_match_table = stm32_exti_ids,
.pm = &stm32_exti_dev_pm_ops,
},
};
......
drivers/irqchip/irq-sunxi-nmi.c
View file @ 6bfd2d44
......@@ -192,7 +192,6 @@ static int __init sunxi_sc_nmi_irq_init(struct device_node *node,
gc->chip_types[0].regs.type = reg_offs->ctrl;
gc->chip_types[1].type = IRQ_TYPE_EDGE_BOTH;
gc->chip_types[1].chip.name = gc->chip_types[0].chip.name;
gc->chip_types[1].chip.irq_ack = irq_gc_ack_set_bit;
gc->chip_types[1].chip.irq_mask = irq_gc_mask_clr_bit;
gc->chip_types[1].chip.irq_unmask = irq_gc_mask_set_bit;
......
drivers/irqchip/irq-tb10x.c
View file @ 6bfd2d44
......@@ -150,7 +150,6 @@ static int __init of_tb10x_init_irq(struct device_node *ictl,
gc->chip_types[0].regs.mask = AB_IRQCTL_INT_ENABLE;
gc->chip_types[1].type = IRQ_TYPE_EDGE_BOTH;
gc->chip_types[1].chip.name = gc->chip_types[0].chip.name;
gc->chip_types[1].chip.irq_ack = irq_gc_ack_set_bit;
gc->chip_types[1].chip.irq_mask = irq_gc_mask_clr_bit;
gc->chip_types[1].chip.irq_unmask = irq_gc_mask_set_bit;
......
include/linux/cpuhotplug.h
View file @ 6bfd2d44
......@@ -146,6 +146,7 @@ enum cpuhp_state {
CPUHP_AP_IRQ_MIPS_GIC_STARTING,
CPUHP_AP_IRQ_LOONGARCH_STARTING,
CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
CPUHP_AP_IRQ_RISCV_IMSIC_STARTING,
CPUHP_AP_ARM_MVEBU_COHERENCY,
CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING,
CPUHP_AP_PERF_X86_STARTING,
......
include/linux/cpumask.h
View file @ 6bfd2d44
......@@ -187,6 +187,23 @@ unsigned int cpumask_first_and(const struct cpumask *srcp1, const struct cpumask
return find_first_and_bit(cpumask_bits(srcp1), cpumask_bits(srcp2), small_cpumask_bits);
}
/**
* cpumask_first_and_and - return the first cpu from *srcp1 & *srcp2 & *srcp3
* @srcp1: the first input
* @srcp2: the second input
* @srcp3: the third input
*
* Return: >= nr_cpu_ids if no cpus set in all.
*/
static inline
unsigned int cpumask_first_and_and(const struct cpumask *srcp1,
const struct cpumask *srcp2,
const struct cpumask *srcp3)
{
return find_first_and_and_bit(cpumask_bits(srcp1), cpumask_bits(srcp2),
cpumask_bits(srcp3), small_cpumask_bits);
}
/**
* cpumask_last - get the last CPU in a cpumask
* @srcp: - the cpumask pointer
......
include/linux/find.h
View file @ 6bfd2d44
......@@ -29,6 +29,8 @@ unsigned long __find_nth_and_andnot_bit(const unsigned long *addr1, const unsign
unsigned long n);
extern unsigned long _find_first_and_bit(const unsigned long *addr1,
const unsigned long *addr2, unsigned long size);
unsigned long _find_first_and_and_bit(const unsigned long *addr1, const unsigned long *addr2,
const unsigned long *addr3, unsigned long size);
extern unsigned long _find_first_zero_bit(const unsigned long *addr, unsigned long size);
extern unsigned long _find_last_bit(const unsigned long *addr, unsigned long size);
......@@ -345,6 +347,31 @@ unsigned long find_first_and_bit(const unsigned long *addr1,
}
#endif
/**
* find_first_and_and_bit - find the first set bit in 3 memory regions
* @addr1: The first address to base the search on
* @addr2: The second address to base the search on
* @addr3: The third address to base the search on
* @size: The bitmap size in bits
*
* Returns the bit number for the first set bit
* If no bits are set, returns @size.
*/
static inline
unsigned long find_first_and_and_bit(const unsigned long *addr1,
const unsigned long *addr2,
const unsigned long *addr3,
unsigned long size)
{
if (small_const_nbits(size)) {
unsigned long val = *addr1 & *addr2 & *addr3 & GENMASK(size - 1, 0);
return val ? __ffs(val) : size;
}
return _find_first_and_and_bit(addr1, addr2, addr3, size);
}
#ifndef find_first_zero_bit
/**
* find_first_zero_bit - find the first cleared bit in a memory region
......
include/linux/irq.h
View file @ 6bfd2d44
......@@ -115,7 +115,7 @@ enum {
* Return value for chip->irq_set_affinity()
*
* IRQ_SET_MASK_OK - OK, core updates irq_common_data.affinity
* IRQ_SET_MASK_NOCPY - OK, chip did update irq_common_data.affinity
* IRQ_SET_MASK_NOCOPY - OK, chip did update irq_common_data.affinity
* IRQ_SET_MASK_OK_DONE - Same as IRQ_SET_MASK_OK for core. Special code to
* support stacked irqchips, which indicates skipping
* all descendant irqchips.
......
include/linux/irqchip/riscv-aplic.h 0 → 100644
View file @ 6bfd2d44
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#ifndef __LINUX_IRQCHIP_RISCV_APLIC_H
#define __LINUX_IRQCHIP_RISCV_APLIC_H
#include <linux/bitops.h>
#define APLIC_MAX_IDC BIT(14)
#define APLIC_MAX_SOURCE 1024
#define APLIC_DOMAINCFG 0x0000
#define APLIC_DOMAINCFG_RDONLY 0x80000000
#define APLIC_DOMAINCFG_IE BIT(8)
#define APLIC_DOMAINCFG_DM BIT(2)
#define APLIC_DOMAINCFG_BE BIT(0)
#define APLIC_SOURCECFG_BASE 0x0004
#define APLIC_SOURCECFG_D BIT(10)
#define APLIC_SOURCECFG_CHILDIDX_MASK 0x000003ff
#define APLIC_SOURCECFG_SM_MASK 0x00000007
#define APLIC_SOURCECFG_SM_INACTIVE 0x0
#define APLIC_SOURCECFG_SM_DETACH 0x1
#define APLIC_SOURCECFG_SM_EDGE_RISE 0x4
#define APLIC_SOURCECFG_SM_EDGE_FALL 0x5
#define APLIC_SOURCECFG_SM_LEVEL_HIGH 0x6
#define APLIC_SOURCECFG_SM_LEVEL_LOW 0x7
#define APLIC_MMSICFGADDR 0x1bc0
#define APLIC_MMSICFGADDRH 0x1bc4
#define APLIC_SMSICFGADDR 0x1bc8
#define APLIC_SMSICFGADDRH 0x1bcc
#ifdef CONFIG_RISCV_M_MODE
#define APLIC_xMSICFGADDR APLIC_MMSICFGADDR
#define APLIC_xMSICFGADDRH APLIC_MMSICFGADDRH
#else
#define APLIC_xMSICFGADDR APLIC_SMSICFGADDR
#define APLIC_xMSICFGADDRH APLIC_SMSICFGADDRH
#endif
#define APLIC_xMSICFGADDRH_L BIT(31)
#define APLIC_xMSICFGADDRH_HHXS_MASK 0x1f
#define APLIC_xMSICFGADDRH_HHXS_SHIFT 24
#define APLIC_xMSICFGADDRH_HHXS (APLIC_xMSICFGADDRH_HHXS_MASK << \
APLIC_xMSICFGADDRH_HHXS_SHIFT)
#define APLIC_xMSICFGADDRH_LHXS_MASK 0x7
#define APLIC_xMSICFGADDRH_LHXS_SHIFT 20
#define APLIC_xMSICFGADDRH_LHXS (APLIC_xMSICFGADDRH_LHXS_MASK << \
APLIC_xMSICFGADDRH_LHXS_SHIFT)
#define APLIC_xMSICFGADDRH_HHXW_MASK 0x7
#define APLIC_xMSICFGADDRH_HHXW_SHIFT 16
#define APLIC_xMSICFGADDRH_HHXW (APLIC_xMSICFGADDRH_HHXW_MASK << \
APLIC_xMSICFGADDRH_HHXW_SHIFT)
#define APLIC_xMSICFGADDRH_LHXW_MASK 0xf
#define APLIC_xMSICFGADDRH_LHXW_SHIFT 12
#define APLIC_xMSICFGADDRH_LHXW (APLIC_xMSICFGADDRH_LHXW_MASK << \
APLIC_xMSICFGADDRH_LHXW_SHIFT)
#define APLIC_xMSICFGADDRH_BAPPN_MASK 0xfff
#define APLIC_xMSICFGADDRH_BAPPN_SHIFT 0
#define APLIC_xMSICFGADDRH_BAPPN (APLIC_xMSICFGADDRH_BAPPN_MASK << \
APLIC_xMSICFGADDRH_BAPPN_SHIFT)
#define APLIC_xMSICFGADDR_PPN_SHIFT 12
#define APLIC_xMSICFGADDR_PPN_HART(__lhxs) \
(BIT(__lhxs) - 1)
#define APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) \
(BIT(__lhxw) - 1)
#define APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs) \
((__lhxs))
#define APLIC_xMSICFGADDR_PPN_LHX(__lhxw, __lhxs) \
(APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) << \
APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs))
#define APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) \
(BIT(__hhxw) - 1)
#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs) \
((__hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
#define APLIC_xMSICFGADDR_PPN_HHX(__hhxw, __hhxs) \
(APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) << \
APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs))
#define APLIC_IRQBITS_PER_REG 32
#define APLIC_SETIP_BASE 0x1c00
#define APLIC_SETIPNUM 0x1cdc
#define APLIC_CLRIP_BASE 0x1d00
#define APLIC_CLRIPNUM 0x1ddc
#define APLIC_SETIE_BASE 0x1e00
#define APLIC_SETIENUM 0x1edc
#define APLIC_CLRIE_BASE 0x1f00
#define APLIC_CLRIENUM 0x1fdc
#define APLIC_SETIPNUM_LE 0x2000
#define APLIC_SETIPNUM_BE 0x2004
#define APLIC_GENMSI 0x3000
#define APLIC_TARGET_BASE 0x3004
#define APLIC_TARGET_HART_IDX_SHIFT 18
#define APLIC_TARGET_HART_IDX_MASK 0x3fff
#define APLIC_TARGET_HART_IDX (APLIC_TARGET_HART_IDX_MASK << \
APLIC_TARGET_HART_IDX_SHIFT)
#define APLIC_TARGET_GUEST_IDX_SHIFT 12
#define APLIC_TARGET_GUEST_IDX_MASK 0x3f
#define APLIC_TARGET_GUEST_IDX (APLIC_TARGET_GUEST_IDX_MASK << \
APLIC_TARGET_GUEST_IDX_SHIFT)
#define APLIC_TARGET_IPRIO_SHIFT 0
#define APLIC_TARGET_IPRIO_MASK 0xff
#define APLIC_TARGET_IPRIO (APLIC_TARGET_IPRIO_MASK << \
APLIC_TARGET_IPRIO_SHIFT)
#define APLIC_TARGET_EIID_SHIFT 0
#define APLIC_TARGET_EIID_MASK 0x7ff
#define APLIC_TARGET_EIID (APLIC_TARGET_EIID_MASK << \
APLIC_TARGET_EIID_SHIFT)
#define APLIC_IDC_BASE 0x4000
#define APLIC_IDC_SIZE 32
#define APLIC_IDC_IDELIVERY 0x00
#define APLIC_IDC_IFORCE 0x04
#define APLIC_IDC_ITHRESHOLD 0x08
#define APLIC_IDC_TOPI 0x18
#define APLIC_IDC_TOPI_ID_SHIFT 16
#define APLIC_IDC_TOPI_ID_MASK 0x3ff
#define APLIC_IDC_TOPI_ID (APLIC_IDC_TOPI_ID_MASK << \
APLIC_IDC_TOPI_ID_SHIFT)
#define APLIC_IDC_TOPI_PRIO_SHIFT 0
#define APLIC_IDC_TOPI_PRIO_MASK 0xff
#define APLIC_IDC_TOPI_PRIO (APLIC_IDC_TOPI_PRIO_MASK << \
APLIC_IDC_TOPI_PRIO_SHIFT)
#define APLIC_IDC_CLAIMI 0x1c
#endif
include/linux/irqchip/riscv-imsic.h 0 → 100644
View file @ 6bfd2d44
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2021 Western Digital Corporation or its affiliates.
* Copyright (C) 2022 Ventana Micro Systems Inc.
*/
#ifndef __LINUX_IRQCHIP_RISCV_IMSIC_H
#define __LINUX_IRQCHIP_RISCV_IMSIC_H
#include <linux/types.h>
#include <linux/bitops.h>
#include <asm/csr.h>
#define IMSIC_MMIO_PAGE_SHIFT 12
#define IMSIC_MMIO_PAGE_SZ BIT(IMSIC_MMIO_PAGE_SHIFT)
#define IMSIC_MMIO_PAGE_LE 0x00
#define IMSIC_MMIO_PAGE_BE 0x04
#define IMSIC_MIN_ID 63
#define IMSIC_MAX_ID 2048
#define IMSIC_EIDELIVERY 0x70
#define IMSIC_EITHRESHOLD 0x72
#define IMSIC_EIP0 0x80
#define IMSIC_EIP63 0xbf
#define IMSIC_EIPx_BITS 32
#define IMSIC_EIE0 0xc0
#define IMSIC_EIE63 0xff
#define IMSIC_EIEx_BITS 32
#define IMSIC_FIRST IMSIC_EIDELIVERY
#define IMSIC_LAST IMSIC_EIE63
#define IMSIC_MMIO_SETIPNUM_LE 0x00
#define IMSIC_MMIO_SETIPNUM_BE 0x04
struct imsic_local_config {
phys_addr_t msi_pa;
void __iomem *msi_va;
};
struct imsic_global_config {
/*
* MSI Target Address Scheme
*
* XLEN-1 12 0
* | | |
* -------------------------------------------------------------
* |xxxxxx|Group Index|xxxxxxxxxxx|HART Index|Guest Index| 0 |
* -------------------------------------------------------------
*/
/* Bits representing Guest index, HART index, and Group index */
u32 guest_index_bits;
u32 hart_index_bits;
u32 group_index_bits;
u32 group_index_shift;
/* Global base address matching all target MSI addresses */
phys_addr_t base_addr;
/* Number of interrupt identities */
u32 nr_ids;
/* Number of guest interrupt identities */
u32 nr_guest_ids;
/* Per-CPU IMSIC addresses */
struct imsic_local_config __percpu *local;
};
#ifdef CONFIG_RISCV_IMSIC
const struct imsic_global_config *imsic_get_global_config(void);
#else
static inline const struct imsic_global_config *imsic_get_global_config(void)
{
return NULL;
}
#endif
#endif
include/linux/irqdesc.h
View file @ 6bfd2d44
......@@ -17,6 +17,18 @@ struct irq_desc;
struct irq_domain;
struct pt_regs;
/**
* struct irqstat - interrupt statistics
* @cnt: real-time interrupt count
* @ref: snapshot of interrupt count
*/
struct irqstat {
unsigned int cnt;
#ifdef CONFIG_GENERIC_IRQ_STAT_SNAPSHOT
unsigned int ref;
#endif
};
/**
* struct irq_desc - interrupt descriptor
* @irq_common_data: per irq and chip data passed down to chip functions
......@@ -55,7 +67,7 @@ struct pt_regs;
struct irq_desc {
struct irq_common_data irq_common_data;
struct irq_data irq_data;
unsigned int __percpu *kstat_irqs;
struct irqstat __percpu *kstat_irqs;
irq_flow_handler_t handle_irq;
struct irqaction *action; /* IRQ action list */
unsigned int status_use_accessors;
......@@ -119,7 +131,7 @@ extern struct irq_desc irq_desc[NR_IRQS];
static inline unsigned int irq_desc_kstat_cpu(struct irq_desc *desc,
unsigned int cpu)
{
return desc->kstat_irqs ? *per_cpu_ptr(desc->kstat_irqs, cpu) : 0;
return desc->kstat_irqs ? per_cpu(desc->kstat_irqs->cnt, cpu) : 0;
}
static inline struct irq_desc *irq_data_to_desc(struct irq_data *data)
......
include/linux/kernel_stat.h
View file @ 6bfd2d44
......@@ -79,6 +79,14 @@ static inline unsigned int kstat_cpu_softirqs_sum(int cpu)
return sum;
}
#ifdef CONFIG_GENERIC_IRQ_STAT_SNAPSHOT
extern void kstat_snapshot_irqs(void);
extern unsigned int kstat_get_irq_since_snapshot(unsigned int irq);
#else
static inline void kstat_snapshot_irqs(void) { }
static inline unsigned int kstat_get_irq_since_snapshot(unsigned int irq) { return 0; }
#endif
/*
* Number of interrupts per specific IRQ source, since bootup
*/
......
kernel/irq/Kconfig
View file @ 6bfd2d44
......@@ -108,6 +108,10 @@ config GENERIC_IRQ_MATRIX_ALLOCATOR
config GENERIC_IRQ_RESERVATION_MODE
bool
# Snapshot for interrupt statistics
config GENERIC_IRQ_STAT_SNAPSHOT
bool
# Support forced irq threading
config IRQ_FORCED_THREADING
bool
......
kernel/irq/cpuhotplug.c
View file @ 6bfd2d44
......@@ -130,6 +130,22 @@ static bool migrate_one_irq(struct irq_desc *desc)
* CPU.
*/
err = irq_do_set_affinity(d, affinity, false);
/*
* If there are online CPUs in the affinity mask, but they have no
* vectors left to make the migration work, try to break the
* affinity by migrating to any online CPU.
*/
if (err == -ENOSPC && !irqd_affinity_is_managed(d) && affinity != cpu_online_mask) {
pr_debug("IRQ%u: set affinity failed for %*pbl, re-try with online CPUs\n",
d->irq, cpumask_pr_args(affinity));
affinity = cpu_online_mask;
brokeaff = true;
err = irq_do_set_affinity(d, affinity, false);
}
if (err) {
pr_warn_ratelimited("IRQ%u: set affinity failed(%d).\n",
d->irq, err);
......@@ -195,10 +211,15 @@ static void irq_restore_affinity_of_irq(struct irq_desc *desc, unsigned int cpu)
!irq_data_get_irq_chip(data) || !cpumask_test_cpu(cpu, affinity))
return;
if (irqd_is_managed_and_shutdown(data)) {
irq_startup(desc, IRQ_RESEND, IRQ_START_COND);
/*
* Don't restore suspended interrupts here when a system comes back
* from S3. They are reenabled via resume_device_irqs().
*/
if (desc->istate & IRQS_SUSPENDED)
return;
}
if (irqd_is_managed_and_shutdown(data))
irq_startup(desc, IRQ_RESEND, IRQ_START_COND);
/*
* If the interrupt can only be directed to a single target
......
kernel/irq/internals.h
View file @ 6bfd2d44
......@@ -98,6 +98,8 @@ extern void mask_irq(struct irq_desc *desc);
extern void unmask_irq(struct irq_desc *desc);
extern void unmask_threaded_irq(struct irq_desc *desc);
extern unsigned int kstat_irqs_desc(struct irq_desc *desc, const struct cpumask *cpumask);
#ifdef CONFIG_SPARSE_IRQ
static inline void irq_mark_irq(unsigned int irq) { }
#else
......@@ -258,7 +260,7 @@ static inline void irq_state_set_masked(struct irq_desc *desc)
static inline void __kstat_incr_irqs_this_cpu(struct irq_desc *desc)
{
__this_cpu_inc(*desc->kstat_irqs);
__this_cpu_inc(desc->kstat_irqs->cnt);
__this_cpu_inc(kstat.irqs_sum);
}
......@@ -278,6 +280,11 @@ static inline int irq_desc_is_chained(struct irq_desc *desc)
return (desc->action && desc->action == &chained_action);
}
static inline bool irq_is_nmi(struct irq_desc *desc)
{
return desc->istate & IRQS_NMI;
}
#ifdef CONFIG_PM_SLEEP
bool irq_pm_check_wakeup(struct irq_desc *desc);
void irq_pm_install_action(struct irq_desc *desc, struct irqaction *action);
......
kernel/irq/irqdesc.c
View file @ 6bfd2d44
......@@ -134,7 +134,7 @@ static void desc_set_defaults(unsigned int irq, struct irq_desc *desc, int node,
desc->name = NULL;
desc->owner = owner;
for_each_possible_cpu(cpu)
*per_cpu_ptr(desc->kstat_irqs, cpu) = 0;
*per_cpu_ptr(desc->kstat_irqs, cpu) = (struct irqstat) { };
desc_smp_init(desc, node, affinity);
}
......@@ -186,7 +186,7 @@ static int init_desc(struct irq_desc *desc, int irq, int node,
const struct cpumask *affinity,
struct module *owner)
{
desc->kstat_irqs = alloc_percpu(unsigned int);
desc->kstat_irqs = alloc_percpu(struct irqstat);
if (!desc->kstat_irqs)
return -ENOMEM;
......@@ -911,10 +911,7 @@ int irq_set_percpu_devid_partition(unsigned int irq,
{
struct irq_desc *desc = irq_to_desc(irq);
if (!desc)
return -EINVAL;
if (desc->percpu_enabled)
if (!desc || desc->percpu_enabled)
return -EINVAL;
desc->percpu_enabled = kzalloc(sizeof(*desc->percpu_enabled), GFP_KERNEL);
......@@ -922,10 +919,7 @@ int irq_set_percpu_devid_partition(unsigned int irq,
if (!desc->percpu_enabled)
return -ENOMEM;
if (affinity)
desc->percpu_affinity = affinity;
else
desc->percpu_affinity = cpu_possible_mask;
desc->percpu_affinity = affinity ? : cpu_possible_mask;
irq_set_percpu_devid_flags(irq);
return 0;
......@@ -968,33 +962,58 @@ unsigned int kstat_irqs_cpu(unsigned int irq, int cpu)
{
struct irq_desc *desc = irq_to_desc(irq);
return desc && desc->kstat_irqs ?
*per_cpu_ptr(desc->kstat_irqs, cpu) : 0;
return desc && desc->kstat_irqs ? per_cpu(desc->kstat_irqs->cnt, cpu) : 0;
}
static bool irq_is_nmi(struct irq_desc *desc)
unsigned int kstat_irqs_desc(struct irq_desc *desc, const struct cpumask *cpumask)
{
return desc->istate & IRQS_NMI;
}
static unsigned int kstat_irqs(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
unsigned int sum = 0;
int cpu;
if (!desc || !desc->kstat_irqs)
return 0;
if (!irq_settings_is_per_cpu_devid(desc) &&
!irq_settings_is_per_cpu(desc) &&
!irq_is_nmi(desc))
return data_race(desc->tot_count);
for_each_possible_cpu(cpu)
sum += data_race(*per_cpu_ptr(desc->kstat_irqs, cpu));
for_each_cpu(cpu, cpumask)
sum += data_race(per_cpu(desc->kstat_irqs->cnt, cpu));
return sum;
}
static unsigned int kstat_irqs(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
if (!desc || !desc->kstat_irqs)
return 0;
return kstat_irqs_desc(desc, cpu_possible_mask);
}
#ifdef CONFIG_GENERIC_IRQ_STAT_SNAPSHOT
void kstat_snapshot_irqs(void)
{
struct irq_desc *desc;
unsigned int irq;
for_each_irq_desc(irq, desc) {
if (!desc->kstat_irqs)
continue;
this_cpu_write(desc->kstat_irqs->ref, this_cpu_read(desc->kstat_irqs->cnt));
}
}
unsigned int kstat_get_irq_since_snapshot(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
if (!desc || !desc->kstat_irqs)
return 0;
return this_cpu_read(desc->kstat_irqs->cnt) - this_cpu_read(desc->kstat_irqs->ref);
}
#endif
/**
* kstat_irqs_usr - Get the statistics for an interrupt from thread context
* @irq: The interrupt number
......
kernel/irq/irqdomain.c
View file @ 6bfd2d44
......@@ -909,10 +909,11 @@ EXPORT_SYMBOL_GPL(irq_create_of_mapping);
*/
void irq_dispose_mapping(unsigned int virq)
{
struct irq_data *irq_data = irq_get_irq_data(virq);
struct irq_data *irq_data;
struct irq_domain *domain;
if (!virq || !irq_data)
irq_data = virq ? irq_get_irq_data(virq) : NULL;
if (!irq_data)
return;
domain = irq_data->domain;
......
kernel/irq/manage.c
View file @ 6bfd2d44
......@@ -564,7 +564,7 @@ irq_set_affinity_notifier(unsigned int irq, struct irq_affinity_notify *notify)
/* The release function is promised process context */
might_sleep();
if (!desc || desc->istate & IRQS_NMI)
if (!desc || irq_is_nmi(desc))
return -EINVAL;
/* Complete initialisation of *notify */
......@@ -800,10 +800,14 @@ void __enable_irq(struct irq_desc *desc)
irq_settings_set_noprobe(desc);
/*
* Call irq_startup() not irq_enable() here because the
* interrupt might be marked NOAUTOEN. So irq_startup()
* needs to be invoked when it gets enabled the first
* time. If it was already started up, then irq_startup()
* will invoke irq_enable() under the hood.
* interrupt might be marked NOAUTOEN so irq_startup()
* needs to be invoked when it gets enabled the first time.
* This is also required when __enable_irq() is invoked for
* a managed and shutdown interrupt from the S3 resume
* path.
*
* If it was already started up, then irq_startup() will
* invoke irq_enable() under the hood.
*/
irq_startup(desc, IRQ_RESEND, IRQ_START_FORCE);
break;
......@@ -898,7 +902,7 @@ int irq_set_irq_wake(unsigned int irq, unsigned int on)
return -EINVAL;
/* Don't use NMIs as wake up interrupts please */
if (desc->istate & IRQS_NMI) {
if (irq_is_nmi(desc)) {
ret = -EINVAL;
goto out_unlock;
}
......@@ -1624,7 +1628,7 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new)
*/
unsigned int oldtype;
if (desc->istate & IRQS_NMI) {
if (irq_is_nmi(desc)) {
pr_err("Invalid attempt to share NMI for %s (irq %d) on irqchip %s.\n",
new->name, irq, desc->irq_data.chip->name);
ret = -EINVAL;
......@@ -2082,7 +2086,7 @@ const void *free_nmi(unsigned int irq, void *dev_id)
unsigned long flags;
const void *devname;
if (!desc || WARN_ON(!(desc->istate & IRQS_NMI)))
if (!desc || WARN_ON(!irq_is_nmi(desc)))
return NULL;
if (WARN_ON(irq_settings_is_per_cpu_devid(desc)))
......@@ -2548,7 +2552,7 @@ void free_percpu_nmi(unsigned int irq, void __percpu *dev_id)
if (!desc || !irq_settings_is_per_cpu_devid(desc))
return;
if (WARN_ON(!(desc->istate & IRQS_NMI)))
if (WARN_ON(!irq_is_nmi(desc)))
return;
kfree(__free_percpu_irq(irq, dev_id));
......@@ -2684,7 +2688,7 @@ int request_percpu_nmi(unsigned int irq, irq_handler_t handler,
return -EINVAL;
/* The line cannot already be NMI */
if (desc->istate & IRQS_NMI)
if (irq_is_nmi(desc))
return -EINVAL;
action = kzalloc(sizeof(struct irqaction), GFP_KERNEL);
......@@ -2745,7 +2749,7 @@ int prepare_percpu_nmi(unsigned int irq)
if (!desc)
return -EINVAL;
if (WARN(!(desc->istate & IRQS_NMI),
if (WARN(!irq_is_nmi(desc),
KERN_ERR "prepare_percpu_nmi called for a non-NMI interrupt: irq %u\n",
irq)) {
ret = -EINVAL;
......@@ -2787,7 +2791,7 @@ void teardown_percpu_nmi(unsigned int irq)
if (!desc)
return;
if (WARN_ON(!(desc->istate & IRQS_NMI)))
if (WARN_ON(!irq_is_nmi(desc)))
goto out;
irq_nmi_teardown(desc);
......
kernel/irq/proc.c
View file @ 6bfd2d44
......@@ -488,18 +488,15 @@ int show_interrupts(struct seq_file *p, void *v)
if (!desc || irq_settings_is_hidden(desc))
goto outsparse;
if (desc->kstat_irqs) {
for_each_online_cpu(j)
any_count |= data_race(*per_cpu_ptr(desc->kstat_irqs, j));
}
if (desc->kstat_irqs)
any_count = kstat_irqs_desc(desc, cpu_online_mask);
if ((!desc->action || irq_desc_is_chained(desc)) && !any_count)
goto outsparse;
seq_printf(p, "%*d: ", prec, i);
for_each_online_cpu(j)
seq_printf(p, "%10u ", desc->kstat_irqs ?
*per_cpu_ptr(desc->kstat_irqs, j) : 0);
seq_printf(p, "%10u ", desc->kstat_irqs ? per_cpu(desc->kstat_irqs->cnt, j) : 0);
raw_spin_lock_irqsave(&desc->lock, flags);
if (desc->irq_data.chip) {
......
kernel/irq/resend.c
View file @ 6bfd2d44
......@@ -190,7 +190,7 @@ int irq_inject_interrupt(unsigned int irq)
* - not NMI type
* - activated
*/
if ((desc->istate & IRQS_NMI) || !irqd_is_activated(&desc->irq_data))
if (irq_is_nmi(desc) || !irqd_is_activated(&desc->irq_data))
err = -EINVAL;
else
err = check_irq_resend(desc, true);
......
kernel/watchdog.c
View file @ 6bfd2d44
......@@ -12,20 +12,25 @@
#define pr_fmt(fmt) "watchdog: " fmt
#include <linux/mm.h>
#include <linux/cpu.h>
#include <linux/nmi.h>
#include <linux/init.h>
#include <linux/irq.h>
#include <linux/irqdesc.h>
#include <linux/kernel_stat.h>
#include <linux/kvm_para.h>
#include <linux/math64.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/nmi.h>
#include <linux/stop_machine.h>
#include <linux/sysctl.h>
#include <linux/tick.h>
#include <linux/sched/clock.h>
#include <linux/sched/debug.h>
#include <linux/sched/isolation.h>
#include <linux/stop_machine.h>
#include <asm/irq_regs.h>
#include <linux/kvm_para.h>
static DEFINE_MUTEX(watchdog_mutex);
......@@ -35,6 +40,8 @@ static DEFINE_MUTEX(watchdog_mutex);
# define WATCHDOG_HARDLOCKUP_DEFAULT 0
#endif
#define NUM_SAMPLE_PERIODS 5
unsigned long __read_mostly watchdog_enabled;
int __read_mostly watchdog_user_enabled = 1;
static int __read_mostly watchdog_hardlockup_user_enabled = WATCHDOG_HARDLOCKUP_DEFAULT;
......@@ -333,6 +340,188 @@ __setup("watchdog_thresh=", watchdog_thresh_setup);
static void __lockup_detector_cleanup(void);
#ifdef CONFIG_SOFTLOCKUP_DETECTOR_INTR_STORM
enum stats_per_group {
STATS_SYSTEM,
STATS_SOFTIRQ,
STATS_HARDIRQ,
STATS_IDLE,
NUM_STATS_PER_GROUP,
};
static const enum cpu_usage_stat tracked_stats[NUM_STATS_PER_GROUP] = {
CPUTIME_SYSTEM,
CPUTIME_SOFTIRQ,
CPUTIME_IRQ,
CPUTIME_IDLE,
};
static DEFINE_PER_CPU(u16, cpustat_old[NUM_STATS_PER_GROUP]);
static DEFINE_PER_CPU(u8, cpustat_util[NUM_SAMPLE_PERIODS][NUM_STATS_PER_GROUP]);
static DEFINE_PER_CPU(u8, cpustat_tail);
/*
* We don't need nanosecond resolution. A granularity of 16ms is
* sufficient for our precision, allowing us to use u16 to store
* cpustats, which will roll over roughly every ~1000 seconds.
* 2^24 ~= 16 * 10^6
*/
static u16 get_16bit_precision(u64 data_ns)
{
return data_ns >> 24LL; /* 2^24ns ~= 16.8ms */
}
static void update_cpustat(void)
{
int i;
u8 util;
u16 old_stat, new_stat;
struct kernel_cpustat kcpustat;
u64 *cpustat = kcpustat.cpustat;
u8 tail = __this_cpu_read(cpustat_tail);
u16 sample_period_16 = get_16bit_precision(sample_period);
kcpustat_cpu_fetch(&kcpustat, smp_processor_id());
for (i = 0; i < NUM_STATS_PER_GROUP; i++) {
old_stat = __this_cpu_read(cpustat_old[i]);
new_stat = get_16bit_precision(cpustat[tracked_stats[i]]);
util = DIV_ROUND_UP(100 * (new_stat - old_stat), sample_period_16);
__this_cpu_write(cpustat_util[tail][i], util);
__this_cpu_write(cpustat_old[i], new_stat);
}
__this_cpu_write(cpustat_tail, (tail + 1) % NUM_SAMPLE_PERIODS);
}
static void print_cpustat(void)
{
int i, group;
u8 tail = __this_cpu_read(cpustat_tail);
u64 sample_period_second = sample_period;
do_div(sample_period_second, NSEC_PER_SEC);
/*
* Outputting the "watchdog" prefix on every line is redundant and not
* concise, and the original alarm information is sufficient for
* positioning in logs, hence here printk() is used instead of pr_crit().
*/
printk(KERN_CRIT "CPU#%d Utilization every %llus during lockup:\n",
smp_processor_id(), sample_period_second);
for (i = 0; i < NUM_SAMPLE_PERIODS; i++) {
group = (tail + i) % NUM_SAMPLE_PERIODS;
printk(KERN_CRIT "\t#%d: %3u%% system,\t%3u%% softirq,\t"
"%3u%% hardirq,\t%3u%% idle\n", i + 1,
__this_cpu_read(cpustat_util[group][STATS_SYSTEM]),
__this_cpu_read(cpustat_util[group][STATS_SOFTIRQ]),
__this_cpu_read(cpustat_util[group][STATS_HARDIRQ]),
__this_cpu_read(cpustat_util[group][STATS_IDLE]));
}
}
#define HARDIRQ_PERCENT_THRESH 50
#define NUM_HARDIRQ_REPORT 5
struct irq_counts {
int irq;
u32 counts;
};
static DEFINE_PER_CPU(bool, snapshot_taken);
/* Tabulate the most frequent interrupts. */
static void tabulate_irq_count(struct irq_counts *irq_counts, int irq, u32 counts, int rank)
{
int i;
struct irq_counts new_count = {irq, counts};
for (i = 0; i < rank; i++) {
if (counts > irq_counts[i].counts)
swap(new_count, irq_counts[i]);
}
}
/*
* If the hardirq time exceeds HARDIRQ_PERCENT_THRESH% of the sample_period,
* then the cause of softlockup might be interrupt storm. In this case, it
* would be useful to start interrupt counting.
*/
static bool need_counting_irqs(void)
{
u8 util;
int tail = __this_cpu_read(cpustat_tail);
tail = (tail + NUM_HARDIRQ_REPORT - 1) % NUM_HARDIRQ_REPORT;
util = __this_cpu_read(cpustat_util[tail][STATS_HARDIRQ]);
return util > HARDIRQ_PERCENT_THRESH;
}
static void start_counting_irqs(void)
{
if (!__this_cpu_read(snapshot_taken)) {
kstat_snapshot_irqs();
__this_cpu_write(snapshot_taken, true);
}
}
static void stop_counting_irqs(void)
{
__this_cpu_write(snapshot_taken, false);
}
static void print_irq_counts(void)
{
unsigned int i, count;
struct irq_counts irq_counts_sorted[NUM_HARDIRQ_REPORT] = {
{-1, 0}, {-1, 0}, {-1, 0}, {-1, 0}, {-1, 0}
};
if (__this_cpu_read(snapshot_taken)) {
for_each_active_irq(i) {
count = kstat_get_irq_since_snapshot(i);
tabulate_irq_count(irq_counts_sorted, i, count, NUM_HARDIRQ_REPORT);
}
/*
* Outputting the "watchdog" prefix on every line is redundant and not
* concise, and the original alarm information is sufficient for
* positioning in logs, hence here printk() is used instead of pr_crit().
*/
printk(KERN_CRIT "CPU#%d Detect HardIRQ Time exceeds %d%%. Most frequent HardIRQs:\n",
smp_processor_id(), HARDIRQ_PERCENT_THRESH);
for (i = 0; i < NUM_HARDIRQ_REPORT; i++) {
if (irq_counts_sorted[i].irq == -1)
break;
printk(KERN_CRIT "\t#%u: %-10u\tirq#%d\n",
i + 1, irq_counts_sorted[i].counts,
irq_counts_sorted[i].irq);
}
/*
* If the hardirq time is less than HARDIRQ_PERCENT_THRESH% in the last
* sample_period, then we suspect the interrupt storm might be subsiding.
*/
if (!need_counting_irqs())
stop_counting_irqs();
}
}
static void report_cpu_status(void)
{
print_cpustat();
print_irq_counts();
}
#else
static inline void update_cpustat(void) { }
static inline void report_cpu_status(void) { }
static inline bool need_counting_irqs(void) { return false; }
static inline void start_counting_irqs(void) { }
static inline void stop_counting_irqs(void) { }
#endif
/*
* Hard-lockup warnings should be triggered after just a few seconds. Soft-
* lockups can have false positives under extreme conditions. So we generally
......@@ -364,7 +553,7 @@ static void set_sample_period(void)
* and hard thresholds) to increment before the
* hardlockup detector generates a warning
*/
sample_period = get_softlockup_thresh() * ((u64)NSEC_PER_SEC / 5);
sample_period = get_softlockup_thresh() * ((u64)NSEC_PER_SEC / NUM_SAMPLE_PERIODS);
watchdog_update_hrtimer_threshold(sample_period);
}
......@@ -434,6 +623,18 @@ static int is_softlockup(unsigned long touch_ts,
unsigned long now)
{
if ((watchdog_enabled & WATCHDOG_SOFTOCKUP_ENABLED) && watchdog_thresh) {
/*
* If period_ts has not been updated during a sample_period, then
* in the subsequent few sample_periods, period_ts might also not
* be updated, which could indicate a potential softlockup. In
* this case, if we suspect the cause of the potential softlockup
* might be interrupt storm, then we need to count the interrupts
* to find which interrupt is storming.
*/
if (time_after_eq(now, period_ts + get_softlockup_thresh() / NUM_SAMPLE_PERIODS) &&
need_counting_irqs())
start_counting_irqs();
/* Warn about unreasonable delays. */
if (time_after(now, period_ts + get_softlockup_thresh()))
return now - touch_ts;
......@@ -456,6 +657,7 @@ static DEFINE_PER_CPU(struct cpu_stop_work, softlockup_stop_work);
static int softlockup_fn(void *data)
{
update_touch_ts();
stop_counting_irqs();
complete(this_cpu_ptr(&softlockup_completion));
return 0;
......@@ -504,6 +706,8 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
*/
period_ts = READ_ONCE(*this_cpu_ptr(&watchdog_report_ts));
update_cpustat();
/* Reset the interval when touched by known problematic code. */
if (period_ts == SOFTLOCKUP_DELAY_REPORT) {
if (unlikely(__this_cpu_read(softlockup_touch_sync))) {
......@@ -539,6 +743,7 @@ static enum hrtimer_restart watchdog_timer_fn(struct hrtimer *hrtimer)
pr_emerg("BUG: soft lockup - CPU#%d stuck for %us! [%s:%d]\n",
smp_processor_id(), duration,
current->comm, task_pid_nr(current));
report_cpu_status();
print_modules();
print_irqtrace_events(current);
if (regs)
......
lib/Kconfig.debug
View file @ 6bfd2d44
......@@ -1030,6 +1030,20 @@ config SOFTLOCKUP_DETECTOR
chance to run. The current stack trace is displayed upon
detection and the system will stay locked up.
config SOFTLOCKUP_DETECTOR_INTR_STORM
bool "Detect Interrupt Storm in Soft Lockups"
depends on SOFTLOCKUP_DETECTOR && IRQ_TIME_ACCOUNTING
select GENERIC_IRQ_STAT_SNAPSHOT
default y if NR_CPUS <= 128
help
Say Y here to enable the kernel to detect interrupt storm
during "soft lockups".
"soft lockups" can be caused by a variety of reasons. If one is
caused by an interrupt storm, then the storming interrupts will not
be on the callstack. To detect this case, it is necessary to report
the CPU stats and the interrupt counts during the "soft lockups".
config BOOTPARAM_SOFTLOCKUP_PANIC
bool "Panic (Reboot) On Soft Lockups"
depends on SOFTLOCKUP_DETECTOR
......
lib/find_bit.c
View file @ 6bfd2d44
......@@ -116,6 +116,18 @@ unsigned long _find_first_and_bit(const unsigned long *addr1,
EXPORT_SYMBOL(_find_first_and_bit);
#endif
/*
* Find the first set bit in three memory regions.
*/
unsigned long _find_first_and_and_bit(const unsigned long *addr1,
const unsigned long *addr2,
const unsigned long *addr3,
unsigned long size)
{
return FIND_FIRST_BIT(addr1[idx] & addr2[idx] & addr3[idx], /* nop */, size);
}
EXPORT_SYMBOL(_find_first_and_and_bit);
#ifndef find_first_zero_bit
/*
* Find the first cleared bit in a memory region.
......
scripts/gdb/linux/interrupts.py
View file @ 6bfd2d44
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7