dummy/linux-stable-mirror
1
0
Fork 1
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2025-04-19 20:58:31 +09:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'topic/cxl' into next

Browse Source 
This merges in the removal of the IBM CAPI "cxl" driver.
This commit is contained in:
Michael Ellerman 2025-04-02 11:07:44 +11:00
commit 64f7efb0f5
48 changed files with 42 additions and 14311 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

55
Documentation/ABI/obsolete/sysfs-class-cxl → Documentation/ABI/removed/sysfs-class-cxl
View File

@ -1,5 +1,4 @@
The cxl driver is no longer maintained, and will be removed from the kernel in
the near future.
The cxl driver was removed in 6.14.
Please note that attributes that are shared between devices are stored in
the directory pointed to by the symlink device/.
@ -10,7 +9,7 @@ For example, the real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
Slave contexts (eg. /sys/class/cxl/afu0.0s):
What: /sys/class/cxl/<afu>/afu_err_buf
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
AFU Error Buffer contents. The contents of this file are
@ -21,7 +20,7 @@ Description: read only
What: /sys/class/cxl/<afu>/irqs_max
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read/write
Decimal value of maximum number of interrupts that can be
@ -32,7 +31,7 @@ Description: read/write
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/irqs_min
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Decimal value of the minimum number of interrupts that
@ -42,7 +41,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/mmio_size
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Decimal value of the size of the MMIO space that may be mmapped
@ -50,7 +49,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/modes_supported
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
List of the modes this AFU supports. One per line.
@ -58,7 +57,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/mode
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read/write
The current mode the AFU is using. Will be one of the modes
@ -68,7 +67,7 @@ Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/prefault_mode
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read/write
Set the mode for prefaulting in segments into the segment table
@ -88,7 +87,7 @@ Description: read/write
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/reset
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: write only
Writing 1 here will reset the AFU provided there are not
@ -96,14 +95,14 @@ Description: write only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/api_version
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Decimal value of the current version of the kernel/user API.
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/api_version_compatible
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Decimal value of the lowest version of the userspace API
@ -117,7 +116,7 @@ An AFU may optionally export one or more PCIe like configuration records, known
as AFU configuration records, which will show up here (if present).
What: /sys/class/cxl/<afu>/cr<config num>/vendor
Date: February 2015
Date: February 2015, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Hexadecimal value of the vendor ID found in this AFU
@ -125,7 +124,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/cr<config num>/device
Date: February 2015
Date: February 2015, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Hexadecimal value of the device ID found in this AFU
@ -133,7 +132,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/cr<config num>/class
Date: February 2015
Date: February 2015, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Hexadecimal value of the class code found in this AFU
@ -141,7 +140,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>/cr<config num>/config
Date: February 2015
Date: February 2015, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
This binary file provides raw access to the AFU configuration
@ -155,7 +154,7 @@ Users: https://github.com/ibm-capi/libcxl
Master contexts (eg. /sys/class/cxl/afu0.0m)
What: /sys/class/cxl/<afu>m/mmio_size
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Decimal value of the size of the MMIO space that may be mmapped
@ -163,14 +162,14 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>m/pp_mmio_len
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Decimal value of the Per Process MMIO space length.
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<afu>m/pp_mmio_off
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
(not in a guest)
@ -181,21 +180,21 @@ Users: https://github.com/ibm-capi/libcxl
Card info (eg. /sys/class/cxl/card0)
What: /sys/class/cxl/<card>/caia_version
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Identifies the CAIA Version the card implements.
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/psl_revision
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Identifies the revision level of the PSL.
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/base_image
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
(not in a guest)
@ -206,7 +205,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/image_loaded
Date: September 2014
Date: September 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
(not in a guest)
@ -215,7 +214,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/load_image_on_perst
Date: December 2014
Date: December 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read/write
(not in a guest)
@ -232,7 +231,7 @@ Description: read/write
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/reset
Date: October 2014
Date: October 2014, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: write only
Writing 1 will issue a PERST to card provided there are no
@ -243,7 +242,7 @@ Description: write only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/perst_reloads_same_image
Date: July 2015
Date: July 2015, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read/write
(not in a guest)
@ -257,7 +256,7 @@ Description: read/write
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/psl_timebase_synced
Date: March 2016
Date: March 2016, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Returns 1 if the psl timebase register is synchronized
@ -265,7 +264,7 @@ Description: read only
Users: https://github.com/ibm-capi/libcxl
What: /sys/class/cxl/<card>/tunneled_ops_supported
Date: May 2018
Date: May 2018, removed February 2025
Contact: linuxppc-dev@lists.ozlabs.org
Description: read only
Returns 1 if tunneled operations are supported in capi mode,

470
Documentation/arch/powerpc/cxl.rst
View File

@ -1,470 +0,0 @@
====================================
Coherent Accelerator Interface (CXL)
====================================
Introduction
============
The coherent accelerator interface is designed to allow the
coherent connection of accelerators (FPGAs and other devices) to a
POWER system. These devices need to adhere to the Coherent
Accelerator Interface Architecture (CAIA).
IBM refers to this as the Coherent Accelerator Processor Interface
or CAPI. In the kernel it's referred to by the name CXL to avoid
confusion with the ISDN CAPI subsystem.
Coherent in this context means that the accelerator and CPUs can
both access system memory directly and with the same effective
addresses.
**This driver is deprecated and will be removed in a future release.**
Hardware overview
=================
::
POWER8/9 FPGA
+----------+ +---------+
| | | |
| CPU | | AFU |
| | | |
| | | |
| | | |
+----------+ +---------+
| PHB | | |
| +------+ | PSL |
| | CAPP |<------>| |
+---+------+ PCIE +---------+
The POWER8/9 chip has a Coherently Attached Processor Proxy (CAPP)
unit which is part of the PCIe Host Bridge (PHB). This is managed
by Linux by calls into OPAL. Linux doesn't directly program the
CAPP.
The FPGA (or coherently attached device) consists of two parts.
The POWER Service Layer (PSL) and the Accelerator Function Unit
(AFU). The AFU is used to implement specific functionality behind
the PSL. The PSL, among other things, provides memory address
translation services to allow each AFU direct access to userspace
memory.
The AFU is the core part of the accelerator (eg. the compression,
crypto etc function). The kernel has no knowledge of the function
of the AFU. Only userspace interacts directly with the AFU.
The PSL provides the translation and interrupt services that the
AFU needs. This is what the kernel interacts with. For example, if
the AFU needs to read a particular effective address, it sends
that address to the PSL, the PSL then translates it, fetches the
data from memory and returns it to the AFU. If the PSL has a
translation miss, it interrupts the kernel and the kernel services
the fault. The context to which this fault is serviced is based on
who owns that acceleration function.
- POWER8 and PSL Version 8 are compliant to the CAIA Version 1.0.
- POWER9 and PSL Version 9 are compliant to the CAIA Version 2.0.
This PSL Version 9 provides new features such as:
* Interaction with the nest MMU on the P9 chip.
* Native DMA support.
* Supports sending ASB_Notify messages for host thread wakeup.
* Supports Atomic operations.
* etc.
Cards with a PSL9 won't work on a POWER8 system and cards with a
PSL8 won't work on a POWER9 system.
AFU Modes
=========
There are two programming modes supported by the AFU. Dedicated
and AFU directed. AFU may support one or both modes.
When using dedicated mode only one MMU context is supported. In
this mode, only one userspace process can use the accelerator at
time.
When using AFU directed mode, up to 16K simultaneous contexts can
be supported. This means up to 16K simultaneous userspace
applications may use the accelerator (although specific AFUs may
support fewer). In this mode, the AFU sends a 16 bit context ID
with each of its requests. This tells the PSL which context is
associated with each operation. If the PSL can't translate an
operation, the ID can also be accessed by the kernel so it can
determine the userspace context associated with an operation.
MMIO space
==========
A portion of the accelerator MMIO space can be directly mapped
from the AFU to userspace. Either the whole space can be mapped or
just a per context portion. The hardware is self describing, hence
the kernel can determine the offset and size of the per context
portion.
Interrupts
==========
AFUs may generate interrupts that are destined for userspace. These
are received by the kernel as hardware interrupts and passed onto
userspace by a read syscall documented below.
Data storage faults and error interrupts are handled by the kernel
driver.
Work Element Descriptor (WED)
=============================
The WED is a 64-bit parameter passed to the AFU when a context is
started. Its format is up to the AFU hence the kernel has no
knowledge of what it represents. Typically it will be the
effective address of a work queue or status block where the AFU
and userspace can share control and status information.
User API
========
1. AFU character devices
^^^^^^^^^^^^^^^^^^^^^^^^
For AFUs operating in AFU directed mode, two character device
files will be created. /dev/cxl/afu0.0m will correspond to a
master context and /dev/cxl/afu0.0s will correspond to a slave
context. Master contexts have access to the full MMIO space an
AFU provides. Slave contexts have access to only the per process
MMIO space an AFU provides.
For AFUs operating in dedicated process mode, the driver will
only create a single character device per AFU called
/dev/cxl/afu0.0d. This will have access to the entire MMIO space
that the AFU provides (like master contexts in AFU directed).
The types described below are defined in include/uapi/misc/cxl.h
The following file operations are supported on both slave and
master devices.
A userspace library libcxl is available here:
https://github.com/ibm-capi/libcxl
This provides a C interface to this kernel API.
open
----
Opens the device and allocates a file descriptor to be used with
the rest of the API.
A dedicated mode AFU only has one context and only allows the
device to be opened once.
An AFU directed mode AFU can have many contexts, the device can be
opened once for each context that is available.
When all available contexts are allocated the open call will fail
and return -ENOSPC.
Note:
IRQs need to be allocated for each context, which may limit
the number of contexts that can be created, and therefore
how many times the device can be opened. The POWER8 CAPP
supports 2040 IRQs and 3 are used by the kernel, so 2037 are
left. If 1 IRQ is needed per context, then only 2037
contexts can be allocated. If 4 IRQs are needed per context,
then only 2037/4 = 509 contexts can be allocated.
ioctl
-----
CXL_IOCTL_START_WORK:
Starts the AFU context and associates it with the current
process. Once this ioctl is successfully executed, all memory
mapped into this process is accessible to this AFU context
using the same effective addresses. No additional calls are
required to map/unmap memory. The AFU memory context will be
updated as userspace allocates and frees memory. This ioctl
returns once the AFU context is started.
Takes a pointer to a struct cxl_ioctl_start_work
::
struct cxl_ioctl_start_work {
__u64 flags;
__u64 work_element_descriptor;
__u64 amr;
__s16 num_interrupts;
__s16 reserved1;
__s32 reserved2;
__u64 reserved3;
__u64 reserved4;
__u64 reserved5;
__u64 reserved6;
};
flags:
Indicates which optional fields in the structure are
valid.
work_element_descriptor:
The Work Element Descriptor (WED) is a 64-bit argument
defined by the AFU. Typically this is an effective
address pointing to an AFU specific structure
describing what work to perform.
amr:
Authority Mask Register (AMR), same as the powerpc
AMR. This field is only used by the kernel when the
corresponding CXL_START_WORK_AMR value is specified in
flags. If not specified the kernel will use a default
value of 0.
num_interrupts:
Number of userspace interrupts to request. This field
is only used by the kernel when the corresponding
CXL_START_WORK_NUM_IRQS value is specified in flags.
If not specified the minimum number required by the
AFU will be allocated. The min and max number can be
obtained from sysfs.
reserved fields:
For ABI padding and future extensions
CXL_IOCTL_GET_PROCESS_ELEMENT:
Get the current context id, also known as the process element.
The value is returned from the kernel as a __u32.
mmap
----
An AFU may have an MMIO space to facilitate communication with the
AFU. If it does, the MMIO space can be accessed via mmap. The size
and contents of this area are specific to the particular AFU. The
size can be discovered via sysfs.
In AFU directed mode, master contexts are allowed to map all of
the MMIO space and slave contexts are allowed to only map the per
process MMIO space associated with the context. In dedicated
process mode the entire MMIO space can always be mapped.
This mmap call must be done after the START_WORK ioctl.
Care should be taken when accessing MMIO space. Only 32 and 64-bit
accesses are supported by POWER8. Also, the AFU will be designed
with a specific endianness, so all MMIO accesses should consider
endianness (recommend endian(3) variants like: le64toh(),
be64toh() etc). These endian issues equally apply to shared memory
queues the WED may describe.
read
----
Reads events from the AFU. Blocks if no events are pending
(unless O_NONBLOCK is supplied). Returns -EIO in the case of an
unrecoverable error or if the card is removed.
read() will always return an integral number of events.
The buffer passed to read() must be at least 4K bytes.
The result of the read will be a buffer of one or more events,
each event is of type struct cxl_event, of varying size::
struct cxl_event {
struct cxl_event_header header;
union {
struct cxl_event_afu_interrupt irq;
struct cxl_event_data_storage fault;
struct cxl_event_afu_error afu_error;
};
};
The struct cxl_event_header is defined as
::
struct cxl_event_header {
__u16 type;
__u16 size;
__u16 process_element;
__u16 reserved1;
};
type:
This defines the type of event. The type determines how
the rest of the event is structured. These types are
described below and defined by enum cxl_event_type.
size:
This is the size of the event in bytes including the
struct cxl_event_header. The start of the next event can
be found at this offset from the start of the current
event.
process_element:
Context ID of the event.
reserved field:
For future extensions and padding.
If the event type is CXL_EVENT_AFU_INTERRUPT then the event
structure is defined as
::
struct cxl_event_afu_interrupt {
__u16 flags;
__u16 irq; /* Raised AFU interrupt number */
__u32 reserved1;
};
flags:
These flags indicate which optional fields are present
in this struct. Currently all fields are mandatory.
irq:
The IRQ number sent by the AFU.
reserved field:
For future extensions and padding.
If the event type is CXL_EVENT_DATA_STORAGE then the event
structure is defined as
::
struct cxl_event_data_storage {
__u16 flags;
__u16 reserved1;
__u32 reserved2;
__u64 addr;
__u64 dsisr;
__u64 reserved3;
};
flags:
These flags indicate which optional fields are present in
this struct. Currently all fields are mandatory.
address:
The address that the AFU unsuccessfully attempted to
access. Valid accesses will be handled transparently by the
kernel but invalid accesses will generate this event.
dsisr:
This field gives information on the type of fault. It is a
copy of the DSISR from the PSL hardware when the address
fault occurred. The form of the DSISR is as defined in the
CAIA.
reserved fields:
For future extensions
If the event type is CXL_EVENT_AFU_ERROR then the event structure
is defined as
::
struct cxl_event_afu_error {
__u16 flags;
__u16 reserved1;
__u32 reserved2;
__u64 error;
};
flags:
These flags indicate which optional fields are present in
this struct. Currently all fields are Mandatory.
error:
Error status from the AFU. Defined by the AFU.
reserved fields:
For future extensions and padding
2. Card character device (powerVM guest only)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In a powerVM guest, an extra character device is created for the
card. The device is only used to write (flash) a new image on the
FPGA accelerator. Once the image is written and verified, the
device tree is updated and the card is reset to reload the updated
image.
open
----
Opens the device and allocates a file descriptor to be used with
the rest of the API. The device can only be opened once.
ioctl
-----
CXL_IOCTL_DOWNLOAD_IMAGE / CXL_IOCTL_VALIDATE_IMAGE:
Starts and controls flashing a new FPGA image. Partial
reconfiguration is not supported (yet), so the image must contain
a copy of the PSL and AFU(s). Since an image can be quite large,
the caller may have to iterate, splitting the image in smaller
chunks.
Takes a pointer to a struct cxl_adapter_image::
struct cxl_adapter_image {
__u64 flags;
__u64 data;
__u64 len_data;
__u64 len_image;
__u64 reserved1;
__u64 reserved2;
__u64 reserved3;
__u64 reserved4;
};
flags:
These flags indicate which optional fields are present in
this struct. Currently all fields are mandatory.
data:
Pointer to a buffer with part of the image to write to the
card.
len_data:
Size of the buffer pointed to by data.
len_image:
Full size of the image.
Sysfs Class
===========
A cxl sysfs class is added under /sys/class/cxl to facilitate
enumeration and tuning of the accelerators. Its layout is
described in Documentation/ABI/obsolete/sysfs-class-cxl
Udev rules
==========
The following udev rules could be used to create a symlink to the
most logical chardev to use in any programming mode (afuX.Yd for
dedicated, afuX.Ys for afu directed), since the API is virtually
identical for each::
SUBSYSTEM=="cxl", ATTRS{mode}=="dedicated_process", SYMLINK="cxl/%b"
SUBSYSTEM=="cxl", ATTRS{mode}=="afu_directed", \
KERNEL=="afu[0-9]*.[0-9]*s", SYMLINK="cxl/%b"

1
Documentation/arch/powerpc/index.rst
View File

@ -12,7 +12,6 @@ powerpc
bootwrapper
cpu_families
cpu_features
cxl
dawr-power9
dexcr
dscr

2
Documentation/userspace-api/ioctl/ioctl-number.rst
View File

@ -375,7 +375,7 @@ Code Seq# Include File Comments
0xB8 all uapi/linux/mshv.h Microsoft Hyper-V /dev/mshv driver
<mailto:linux-hyperv@vger.kernel.org>
0xC0 00-0F linux/usb/iowarrior.h
0xCA 00-0F uapi/misc/cxl.h
0xCA 00-0F uapi/misc/cxl.h Dead since 6.14
0xCA 10-2F uapi/misc/ocxl.h
0xCA 80-BF uapi/scsi/cxlflash_ioctl.h Dead since 6.14
0xCB 00-1F CBM serial IEC bus in development:

12
MAINTAINERS
View File

@ -6351,18 +6351,6 @@ S: Maintained
W: http://www.chelsio.com
F: drivers/net/ethernet/chelsio/cxgb4vf/
CXL (IBM Coherent Accelerator Processor Interface CAPI) DRIVER
M: Frederic Barrat <fbarrat@linux.ibm.com>
M: Andrew Donnellan <ajd@linux.ibm.com>
L: linuxppc-dev@lists.ozlabs.org
S: Obsolete
F: Documentation/ABI/obsolete/sysfs-class-cxl
F: Documentation/arch/powerpc/cxl.rst
F: arch/powerpc/platforms/powernv/pci-cxl.c
F: drivers/misc/cxl/
F: include/misc/cxl*
F: include/uapi/misc/cxl.h
CYBERPRO FB DRIVER
M: Russell King <linux@armlinux.org.uk>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)

1
arch/powerpc/configs/skiroot_defconfig
View File

@ -78,7 +78,6 @@ CONFIG_VIRTIO_BLK=m
CONFIG_BLK_DEV_NVME=m
CONFIG_NVME_MULTIPATH=y
CONFIG_EEPROM_AT24=m
# CONFIG_CXL is not set
# CONFIG_OCXL is not set
CONFIG_BLK_DEV_SD=m
CONFIG_BLK_DEV_SR=m

6
arch/powerpc/include/asm/copro.h
View File

@ -18,10 +18,4 @@ int copro_handle_mm_fault(struct mm_struct *mm, unsigned long ea,
int copro_calculate_slb(struct mm_struct *mm, u64 ea, struct copro_slb *slb);
#ifdef CONFIG_PPC_COPRO_BASE
void copro_flush_all_slbs(struct mm_struct *mm);
#else
static inline void copro_flush_all_slbs(struct mm_struct *mm) {}
#endif
#endif /* _ASM_POWERPC_COPRO_H */

3
arch/powerpc/include/asm/device.h
View File

@ -38,9 +38,6 @@ struct dev_archdata {
#ifdef CONFIG_FAIL_IOMMU
int fail_iommu;
#endif
#ifdef CONFIG_CXL_BASE
struct cxl_context *cxl_ctx;
#endif
#ifdef CONFIG_PCI_IOV
void *iov_data;
#endif

17
arch/powerpc/include/asm/pnv-pci.h
View File

@ -10,7 +10,6 @@
#include <linux/pci_hotplug.h>
#include <linux/irq.h>
#include <linux/of.h>
#include <misc/cxl-base.h>
#include <asm/opal-api.h>
#define PCI_SLOT_ID_PREFIX (1UL << 63)
@ -25,25 +24,9 @@ extern int pnv_pci_get_power_state(uint64_t id, uint8_t *state);
extern int pnv_pci_set_power_state(uint64_t id, uint8_t state,
struct opal_msg *msg);
extern int pnv_pci_set_tunnel_bar(struct pci_dev *dev, uint64_t addr,
int enable);
int pnv_phb_to_cxl_mode(struct pci_dev *dev, uint64_t mode);
int pnv_cxl_ioda_msi_setup(struct pci_dev *dev, unsigned int hwirq,
unsigned int virq);
int pnv_cxl_alloc_hwirqs(struct pci_dev *dev, int num);
void pnv_cxl_release_hwirqs(struct pci_dev *dev, int hwirq, int num);
int pnv_cxl_get_irq_count(struct pci_dev *dev);
struct device_node *pnv_pci_get_phb_node(struct pci_dev *dev);
int64_t pnv_opal_pci_msi_eoi(struct irq_data *d);
bool is_pnv_opal_msi(struct irq_chip *chip);
#ifdef CONFIG_CXL_BASE
int pnv_cxl_alloc_hwirq_ranges(struct cxl_irq_ranges *irqs,
struct pci_dev *dev, int num);
void pnv_cxl_release_hwirq_ranges(struct cxl_irq_ranges *irqs,
struct pci_dev *dev);
#endif
struct pnv_php_slot {
struct hotplug_slot slot;
uint64_t id;

13
arch/powerpc/mm/book3s64/hash_native.c
View File

@ -27,8 +27,6 @@
#include <asm/ppc-opcode.h>
#include <asm/feature-fixups.h>
#include <misc/cxl-base.h>
#ifdef DEBUG_LOW
#define DBG_LOW(fmt...) udbg_printf(fmt)
#else
@ -217,11 +215,9 @@ static inline void __tlbiel(unsigned long vpn, int psize, int apsize, int ssize)
static inline void tlbie(unsigned long vpn, int psize, int apsize,
int ssize, int local)
{
unsigned int use_local;
unsigned int use_local = local && mmu_has_feature(MMU_FTR_TLBIEL);
int lock_tlbie = !mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE);
use_local = local && mmu_has_feature(MMU_FTR_TLBIEL) && !cxl_ctx_in_use();
if (use_local)
use_local = mmu_psize_defs[psize].tlbiel;
if (lock_tlbie && !use_local)
@ -789,10 +785,6 @@ static void native_flush_hash_range(unsigned long number, int local)
unsigned long psize = batch->psize;
int ssize = batch->ssize;
int i;
unsigned int use_local;
use_local = local && mmu_has_feature(MMU_FTR_TLBIEL) &&
mmu_psize_defs[psize].tlbiel && !cxl_ctx_in_use();
local_irq_save(flags);
@ -827,7 +819,8 @@ static void native_flush_hash_range(unsigned long number, int local)
} pte_iterate_hashed_end();
}
if (use_local) {
if (mmu_has_feature(MMU_FTR_TLBIEL) &&
mmu_psize_defs[psize].tlbiel && local) {
asm volatile("ptesync":::"memory");
for (i = 0; i < number; i++) {
vpn = batch->vpn[i];

10
arch/powerpc/mm/book3s64/hash_utils.c
View File

@ -56,7 +56,7 @@
#include <asm/cacheflush.h>
#include <asm/cputable.h>
#include <asm/sections.h>
#include <asm/copro.h>
#include <asm/spu.h>
#include <asm/udbg.h>
#include <asm/text-patching.h>
#include <asm/fadump.h>
@ -1600,7 +1600,9 @@ void demote_segment_4k(struct mm_struct *mm, unsigned long addr)
if (get_slice_psize(mm, addr) == MMU_PAGE_4K)
return;
slice_set_range_psize(mm, addr, 1, MMU_PAGE_4K);
copro_flush_all_slbs(mm);
#ifdef CONFIG_SPU_BASE
spu_flush_all_slbs(mm);
#endif
if ((get_paca_psize(addr) != MMU_PAGE_4K) && (current->mm == mm)) {
copy_mm_to_paca(mm);
@ -1869,7 +1871,9 @@ int hash_page_mm(struct mm_struct *mm, unsigned long ea,
"to 4kB pages because of "
"non-cacheable mapping\n");
psize = mmu_vmalloc_psize = MMU_PAGE_4K;
copro_flush_all_slbs(mm);
#ifdef CONFIG_SPU_BASE
spu_flush_all_slbs(mm);
#endif
}
}

1
arch/powerpc/mm/book3s64/pgtable.c
View File

@ -10,7 +10,6 @@
#include <linux/pkeys.h>
#include <linux/debugfs.h>
#include <linux/proc_fs.h>
#include <misc/cxl-base.h>
#include <asm/pgalloc.h>
#include <asm/tlb.h>

6
arch/powerpc/mm/book3s64/slice.c
View File

@ -22,7 +22,7 @@
#include <linux/security.h>
#include <asm/mman.h>
#include <asm/mmu.h>
#include <asm/copro.h>
#include <asm/spu.h>
#include <asm/hugetlb.h>
#include <asm/mmu_context.h>
@ -248,7 +248,9 @@ static void slice_convert(struct mm_struct *mm,
spin_unlock_irqrestore(&slice_convert_lock, flags);
copro_flush_all_slbs(mm);
#ifdef CONFIG_SPU_BASE
spu_flush_all_slbs(mm);
#endif
}
/*

11
arch/powerpc/mm/copro_fault.c
View File

@ -12,8 +12,6 @@
#include <linux/export.h>
#include <asm/reg.h>
#include <asm/copro.h>
#include <asm/spu.h>
#include <misc/cxl-base.h>
/*
* This ought to be kept in sync with the powerpc specific do_page_fault
@ -135,13 +133,4 @@ int copro_calculate_slb(struct mm_struct *mm, u64 ea, struct copro_slb *slb)
return 0;
}
EXPORT_SYMBOL_GPL(copro_calculate_slb);
void copro_flush_all_slbs(struct mm_struct *mm)
{
#ifdef CONFIG_SPU_BASE
spu_flush_all_slbs(mm);
#endif
cxl_slbia(mm);
}
EXPORT_SYMBOL_GPL(copro_flush_all_slbs);
#endif

1
arch/powerpc/platforms/powernv/Makefile
View File

@ -21,7 +21,6 @@ obj-$(CONFIG_PRESERVE_FA_DUMP) += opal-fadump.o
obj-$(CONFIG_OPAL_CORE) += opal-core.o
obj-$(CONFIG_PCI) += pci.o pci-ioda.o pci-ioda-tce.o
obj-$(CONFIG_PCI_IOV) += pci-sriov.o
obj-$(CONFIG_CXL_BASE) += pci-cxl.o
obj-$(CONFIG_EEH) += eeh-powernv.o
obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o

153
arch/powerpc/platforms/powernv/pci-cxl.c
View File

@ -1,153 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014-2016 IBM Corp.
*/
#include <linux/module.h>
#include <misc/cxl-base.h>
#include <asm/pnv-pci.h>
#include <asm/opal.h>
#include "pci.h"
int pnv_phb_to_cxl_mode(struct pci_dev *dev, uint64_t mode)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
struct pnv_phb *phb = hose->private_data;
struct pnv_ioda_pe *pe;
int rc;
pe = pnv_ioda_get_pe(dev);
if (!pe)
return -ENODEV;
pe_info(pe, "Switching PHB to CXL\n");
rc = opal_pci_set_phb_cxl_mode(phb->opal_id, mode, pe->pe_number);
if (rc == OPAL_UNSUPPORTED)
dev_err(&dev->dev, "Required cxl mode not supported by firmware - update skiboot\n");
else if (rc)
dev_err(&dev->dev, "opal_pci_set_phb_cxl_mode failed: %i\n", rc);
return rc;
}
EXPORT_SYMBOL(pnv_phb_to_cxl_mode);
/* Find PHB for cxl dev and allocate MSI hwirqs?
* Returns the absolute hardware IRQ number
*/
int pnv_cxl_alloc_hwirqs(struct pci_dev *dev, int num)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
struct pnv_phb *phb = hose->private_data;
int hwirq = msi_bitmap_alloc_hwirqs(&phb->msi_bmp, num);
if (hwirq < 0) {
dev_warn(&dev->dev, "Failed to find a free MSI\n");
return -ENOSPC;
}
return phb->msi_base + hwirq;
}
EXPORT_SYMBOL(pnv_cxl_alloc_hwirqs);
void pnv_cxl_release_hwirqs(struct pci_dev *dev, int hwirq, int num)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
struct pnv_phb *phb = hose->private_data;
msi_bitmap_free_hwirqs(&phb->msi_bmp, hwirq - phb->msi_base, num);
}
EXPORT_SYMBOL(pnv_cxl_release_hwirqs);
void pnv_cxl_release_hwirq_ranges(struct cxl_irq_ranges *irqs,
struct pci_dev *dev)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
struct pnv_phb *phb = hose->private_data;
int i, hwirq;
for (i = 1; i < CXL_IRQ_RANGES; i++) {
if (!irqs->range[i])
continue;
pr_devel("cxl release irq range 0x%x: offset: 0x%lx limit: %ld\n",
i, irqs->offset[i],
irqs->range[i]);
hwirq = irqs->offset[i] - phb->msi_base;
msi_bitmap_free_hwirqs(&phb->msi_bmp, hwirq,
irqs->range[i]);
}
}
EXPORT_SYMBOL(pnv_cxl_release_hwirq_ranges);
int pnv_cxl_alloc_hwirq_ranges(struct cxl_irq_ranges *irqs,
struct pci_dev *dev, int num)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
struct pnv_phb *phb = hose->private_data;
int i, hwirq, try;
memset(irqs, 0, sizeof(struct cxl_irq_ranges));
/* 0 is reserved for the multiplexed PSL DSI interrupt */
for (i = 1; i < CXL_IRQ_RANGES && num; i++) {
try = num;
while (try) {
hwirq = msi_bitmap_alloc_hwirqs(&phb->msi_bmp, try);
if (hwirq >= 0)
break;
try /= 2;
}
if (!try)
goto fail;
irqs->offset[i] = phb->msi_base + hwirq;
irqs->range[i] = try;
pr_devel("cxl alloc irq range 0x%x: offset: 0x%lx limit: %li\n",
i, irqs->offset[i], irqs->range[i]);
num -= try;
}
if (num)
goto fail;
return 0;
fail:
pnv_cxl_release_hwirq_ranges(irqs, dev);
return -ENOSPC;
}
EXPORT_SYMBOL(pnv_cxl_alloc_hwirq_ranges);
int pnv_cxl_get_irq_count(struct pci_dev *dev)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
struct pnv_phb *phb = hose->private_data;
return phb->msi_bmp.irq_count;
}
EXPORT_SYMBOL(pnv_cxl_get_irq_count);
int pnv_cxl_ioda_msi_setup(struct pci_dev *dev, unsigned int hwirq,
unsigned int virq)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
struct pnv_phb *phb = hose->private_data;
unsigned int xive_num = hwirq - phb->msi_base;
struct pnv_ioda_pe *pe;
int rc;
if (!(pe = pnv_ioda_get_pe(dev)))
return -ENODEV;
/* Assign XIVE to PE */
rc = opal_pci_set_xive_pe(phb->opal_id, pe->pe_number, xive_num);
if (rc) {
pe_warn(pe, "%s: OPAL error %d setting msi_base 0x%x "
"hwirq 0x%x XIVE 0x%x PE\n",
pci_name(dev), rc, phb->msi_base, hwirq, xive_num);
return -EIO;
}
pnv_set_msi_irq_chip(phb, virq);
return 0;
}
EXPORT_SYMBOL(pnv_cxl_ioda_msi_setup);

43
arch/powerpc/platforms/powernv/pci-ioda.c
View File

@ -39,8 +39,6 @@
#include <asm/mmzone.h>
#include <asm/xive.h>
#include <misc/cxl-base.h>
#include "powernv.h"
#include "pci.h"
#include "../../../../drivers/pci/pci.h"
@ -1636,47 +1634,6 @@ int64_t pnv_opal_pci_msi_eoi(struct irq_data *d)
return opal_pci_msi_eoi(phb->opal_id, d->parent_data->hwirq);
}
/*
* The IRQ data is mapped in the XICS domain, with OPAL HW IRQ numbers
*/
static void pnv_ioda2_msi_eoi(struct irq_data *d)
{
int64_t rc;
unsigned int hw_irq = (unsigned int)irqd_to_hwirq(d);
struct pci_controller *hose = irq_data_get_irq_chip_data(d);
struct pnv_phb *phb = hose->private_data;
rc = opal_pci_msi_eoi(phb->opal_id, hw_irq);
WARN_ON_ONCE(rc);
icp_native_eoi(d);
}
/* P8/CXL only */
void pnv_set_msi_irq_chip(struct pnv_phb *phb, unsigned int virq)
{
struct irq_data *idata;
struct irq_chip *ichip;
/* The MSI EOI OPAL call is only needed on PHB3 */
if (phb->model != PNV_PHB_MODEL_PHB3)
return;
if (!phb->ioda.irq_chip_init) {
/*
* First time we setup an MSI IRQ, we need to setup the
* corresponding IRQ chip to route correctly.
*/
idata = irq_get_irq_data(virq);
ichip = irq_data_get_irq_chip(idata);
phb->ioda.irq_chip_init = 1;
phb->ioda.irq_chip = *ichip;
phb->ioda.irq_chip.irq_eoi = pnv_ioda2_msi_eoi;
}
irq_set_chip(virq, &phb->ioda.irq_chip);
irq_set_chip_data(virq, phb->hose);
}
static struct irq_chip pnv_pci_msi_irq_chip;
/*

61
arch/powerpc/platforms/powernv/pci.c
View File

@ -14,7 +14,6 @@
#include <linux/io.h>
#include <linux/msi.h>
#include <linux/iommu.h>
#include <linux/sched/mm.h>
#include <asm/sections.h>
#include <asm/io.h>
@ -33,8 +32,6 @@
#include "powernv.h"
#include "pci.h"
static DEFINE_MUTEX(tunnel_mutex);
int pnv_pci_get_slot_id(struct device_node *np, uint64_t *id)
{
struct device_node *node = np;
@ -744,64 +741,6 @@ struct iommu_table *pnv_pci_table_alloc(int nid)
return tbl;
}
struct device_node *pnv_pci_get_phb_node(struct pci_dev *dev)
{
struct pci_controller *hose = pci_bus_to_host(dev->bus);
return of_node_get(hose->dn);
}
EXPORT_SYMBOL(pnv_pci_get_phb_node);
int pnv_pci_set_tunnel_bar(struct pci_dev *dev, u64 addr, int enable)
{
struct pnv_phb *phb = pci_bus_to_pnvhb(dev->bus);
u64 tunnel_bar;
__be64 val;
int rc;
if (!opal_check_token(OPAL_PCI_GET_PBCQ_TUNNEL_BAR))
return -ENXIO;
if (!opal_check_token(OPAL_PCI_SET_PBCQ_TUNNEL_BAR))
return -ENXIO;
mutex_lock(&tunnel_mutex);
rc = opal_pci_get_pbcq_tunnel_bar(phb->opal_id, &val);
if (rc != OPAL_SUCCESS) {
rc = -EIO;
goto out;
}
tunnel_bar = be64_to_cpu(val);
if (enable) {
/*
* Only one device per PHB can use atomics.
* Our policy is first-come, first-served.
*/
if (tunnel_bar) {
if (tunnel_bar != addr)
rc = -EBUSY;
else
rc = 0; /* Setting same address twice is ok */
goto out;
}
} else {
/*
* The device that owns atomics and wants to release
* them must pass the same address with enable == 0.
*/
if (tunnel_bar != addr) {
rc = -EPERM;
goto out;
}
addr = 0x0ULL;
}
rc = opal_pci_set_pbcq_tunnel_bar(phb->opal_id, addr);
rc = opal_error_code(rc);
out:
mutex_unlock(&tunnel_mutex);
return rc;
}
EXPORT_SYMBOL_GPL(pnv_pci_set_tunnel_bar);
void pnv_pci_shutdown(void)
{
struct pci_controller *hose;

2
arch/powerpc/platforms/powernv/pci.h
View File

@ -163,7 +163,6 @@ struct pnv_phb {
unsigned int *io_segmap;
/* IRQ chip */
int irq_chip_init;
struct irq_chip irq_chip;
/* Sorted list of used PE's based
@ -281,7 +280,6 @@ extern int pnv_eeh_phb_reset(struct pci_controller *hose, int option);
extern struct pnv_ioda_pe *pnv_pci_bdfn_to_pe(struct pnv_phb *phb, u16 bdfn);
extern struct pnv_ioda_pe *pnv_ioda_get_pe(struct pci_dev *dev);
extern void pnv_set_msi_irq_chip(struct pnv_phb *phb, unsigned int virq);
extern unsigned long pnv_pci_ioda2_get_table_size(__u32 page_shift,
__u64 window_size, __u32 levels);
extern int pnv_eeh_post_init(void);

1
drivers/misc/Kconfig
View File

@ -641,7 +641,6 @@ source "drivers/misc/mei/Kconfig"
source "drivers/misc/vmw_vmci/Kconfig"
source "drivers/misc/genwqe/Kconfig"
source "drivers/misc/echo/Kconfig"
source "drivers/misc/cxl/Kconfig"
source "drivers/misc/ocxl/Kconfig"
source "drivers/misc/bcm-vk/Kconfig"
source "drivers/misc/cardreader/Kconfig"

1
drivers/misc/Makefile
View File

@ -50,7 +50,6 @@ obj-$(CONFIG_SRAM) += sram.o
obj-$(CONFIG_SRAM_EXEC) += sram-exec.o
obj-$(CONFIG_GENWQE) += genwqe/
obj-$(CONFIG_ECHO) += echo/
obj-$(CONFIG_CXL_BASE) += cxl/
obj-$(CONFIG_DW_XDATA_PCIE) += dw-xdata-pcie.o
obj-$(CONFIG_PCI_ENDPOINT_TEST) += pci_endpoint_test.o
obj-$(CONFIG_OCXL) += ocxl/

28
drivers/misc/cxl/Kconfig
View File

@ -1,28 +0,0 @@
# SPDX-License-Identifier: GPL-2.0-only
#
# IBM Coherent Accelerator (CXL) compatible devices
#
config CXL_BASE
bool
select PPC_COPRO_BASE
select PPC_64S_HASH_MMU
config CXL
tristate "Support for IBM Coherent Accelerators (CXL) (DEPRECATED)"
depends on PPC_POWERNV && PCI_MSI && EEH
select CXL_BASE
help
The cxl driver is deprecated and will be removed in a future
kernel release.
Select this option to enable driver support for IBM Coherent
Accelerators (CXL). CXL is otherwise known as Coherent Accelerator
Processor Interface (CAPI). CAPI allows accelerators in FPGAs to be
coherently attached to a CPU via an MMU. This driver enables
userspace programs to access these accelerators via /dev/cxl/afuM.N
devices.
CAPI adapters are found in POWER8 based systems.
If unsure, say N.

14
drivers/misc/cxl/Makefile
View File

@ -1,14 +0,0 @@
# SPDX-License-Identifier: GPL-2.0
ccflags-y := $(call cc-disable-warning, unused-const-variable)
ccflags-$(CONFIG_PPC_WERROR) += -Werror
cxl-y += main.o file.o irq.o fault.o native.o
cxl-y += context.o sysfs.o pci.o trace.o
cxl-y += vphb.o api.o cxllib.o
cxl-$(CONFIG_PPC_PSERIES) += flash.o guest.o of.o hcalls.o
cxl-$(CONFIG_DEBUG_FS) += debugfs.o
obj-$(CONFIG_CXL) += cxl.o
obj-$(CONFIG_CXL_BASE) += base.o
# For tracepoints to include our trace.h from tracepoint infrastructure:
CFLAGS_trace.o := -I$(src)

532
drivers/misc/cxl/api.c
View File

@ -1,532 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/pci.h>
#include <linux/slab.h>
#include <linux/file.h>
#include <misc/cxl.h>
#include <linux/module.h>
#include <linux/mount.h>
#include <linux/pseudo_fs.h>
#include <linux/sched/mm.h>
#include <linux/mmu_context.h>
#include <linux/irqdomain.h>
#include "cxl.h"
/*
* Since we want to track memory mappings to be able to force-unmap
* when the AFU is no longer reachable, we need an inode. For devices
* opened through the cxl user API, this is not a problem, but a
* userland process can also get a cxl fd through the cxl_get_fd()
* API, which is used by the cxlflash driver.
*
* Therefore we implement our own simple pseudo-filesystem and inode
* allocator. We don't use the anonymous inode, as we need the
* meta-data associated with it (address_space) and it is shared by
* other drivers/processes, so it could lead to cxl unmapping VMAs
* from random processes.
*/
#define CXL_PSEUDO_FS_MAGIC 0x1697697f
static int cxl_fs_cnt;
static struct vfsmount *cxl_vfs_mount;
static int cxl_fs_init_fs_context(struct fs_context *fc)
{
return init_pseudo(fc, CXL_PSEUDO_FS_MAGIC) ? 0 : -ENOMEM;
}
static struct file_system_type cxl_fs_type = {
.name = "cxl",
.owner = THIS_MODULE,
.init_fs_context = cxl_fs_init_fs_context,
.kill_sb = kill_anon_super,
};
void cxl_release_mapping(struct cxl_context *ctx)
{
if (ctx->kernelapi && ctx->mapping)
simple_release_fs(&cxl_vfs_mount, &cxl_fs_cnt);
}
static struct file *cxl_getfile(const char *name,
const struct file_operations *fops,
void *priv, int flags)
{
struct file *file;
struct inode *inode;
int rc;
/* strongly inspired by anon_inode_getfile() */
if (fops->owner && !try_module_get(fops->owner))
return ERR_PTR(-ENOENT);
rc = simple_pin_fs(&cxl_fs_type, &cxl_vfs_mount, &cxl_fs_cnt);
if (rc < 0) {
pr_err("Cannot mount cxl pseudo filesystem: %d\n", rc);
file = ERR_PTR(rc);
goto err_module;
}
inode = alloc_anon_inode(cxl_vfs_mount->mnt_sb);
if (IS_ERR(inode)) {
file = ERR_CAST(inode);
goto err_fs;
}
file = alloc_file_pseudo(inode, cxl_vfs_mount, name,
flags & (O_ACCMODE | O_NONBLOCK), fops);
if (IS_ERR(file))
goto err_inode;
file->private_data = priv;
return file;
err_inode:
iput(inode);
err_fs:
simple_release_fs(&cxl_vfs_mount, &cxl_fs_cnt);
err_module:
module_put(fops->owner);
return file;
}
struct cxl_context *cxl_dev_context_init(struct pci_dev *dev)
{
struct cxl_afu *afu;
struct cxl_context *ctx;
int rc;
afu = cxl_pci_to_afu(dev);
if (IS_ERR(afu))
return ERR_CAST(afu);
ctx = cxl_context_alloc();
if (!ctx)
return ERR_PTR(-ENOMEM);
ctx->kernelapi = true;
/* Make it a slave context. We can promote it later? */
rc = cxl_context_init(ctx, afu, false);
if (rc)
goto err_ctx;
return ctx;
err_ctx:
kfree(ctx);
return ERR_PTR(rc);
}
EXPORT_SYMBOL_GPL(cxl_dev_context_init);
struct cxl_context *cxl_get_context(struct pci_dev *dev)
{
return dev->dev.archdata.cxl_ctx;
}
EXPORT_SYMBOL_GPL(cxl_get_context);
int cxl_release_context(struct cxl_context *ctx)
{
if (ctx->status >= STARTED)
return -EBUSY;
cxl_context_free(ctx);
return 0;
}
EXPORT_SYMBOL_GPL(cxl_release_context);
static irq_hw_number_t cxl_find_afu_irq(struct cxl_context *ctx, int num)
{
__u16 range;
int r;
for (r = 0; r < CXL_IRQ_RANGES; r++) {
range = ctx->irqs.range[r];
if (num < range) {
return ctx->irqs.offset[r] + num;
}
num -= range;
}
return 0;
}
int cxl_set_priv(struct cxl_context *ctx, void *priv)
{
if (!ctx)
return -EINVAL;
ctx->priv = priv;
return 0;
}
EXPORT_SYMBOL_GPL(cxl_set_priv);
void *cxl_get_priv(struct cxl_context *ctx)
{
if (!ctx)
return ERR_PTR(-EINVAL);
return ctx->priv;
}
EXPORT_SYMBOL_GPL(cxl_get_priv);
int cxl_allocate_afu_irqs(struct cxl_context *ctx, int num)
{
int res;
irq_hw_number_t hwirq;
if (num == 0)
num = ctx->afu->pp_irqs;
res = afu_allocate_irqs(ctx, num);
if (res)
return res;
if (!cpu_has_feature(CPU_FTR_HVMODE)) {
/* In a guest, the PSL interrupt is not multiplexed. It was
* allocated above, and we need to set its handler
*/
hwirq = cxl_find_afu_irq(ctx, 0);
if (hwirq)
cxl_map_irq(ctx->afu->adapter, hwirq, cxl_ops->psl_interrupt, ctx, "psl");
}
if (ctx->status == STARTED) {
if (cxl_ops->update_ivtes)
cxl_ops->update_ivtes(ctx);
else WARN(1, "BUG: cxl_allocate_afu_irqs must be called prior to starting the context on this platform\n");
}
return res;
}
EXPORT_SYMBOL_GPL(cxl_allocate_afu_irqs);
void cxl_free_afu_irqs(struct cxl_context *ctx)
{
irq_hw_number_t hwirq;
unsigned int virq;
if (!cpu_has_feature(CPU_FTR_HVMODE)) {
hwirq = cxl_find_afu_irq(ctx, 0);
if (hwirq) {
virq = irq_find_mapping(NULL, hwirq);
if (virq)
cxl_unmap_irq(virq, ctx);
}
}
afu_irq_name_free(ctx);
cxl_ops->release_irq_ranges(&ctx->irqs, ctx->afu->adapter);
}
EXPORT_SYMBOL_GPL(cxl_free_afu_irqs);
int cxl_map_afu_irq(struct cxl_context *ctx, int num,
irq_handler_t handler, void *cookie, char *name)
{
irq_hw_number_t hwirq;
/*
* Find interrupt we are to register.
*/
hwirq = cxl_find_afu_irq(ctx, num);
if (!hwirq)
return -ENOENT;
return cxl_map_irq(ctx->afu->adapter, hwirq, handler, cookie, name);
}
EXPORT_SYMBOL_GPL(cxl_map_afu_irq);
void cxl_unmap_afu_irq(struct cxl_context *ctx, int num, void *cookie)
{
irq_hw_number_t hwirq;
unsigned int virq;
hwirq = cxl_find_afu_irq(ctx, num);
if (!hwirq)
return;
virq = irq_find_mapping(NULL, hwirq);
if (virq)
cxl_unmap_irq(virq, cookie);
}
EXPORT_SYMBOL_GPL(cxl_unmap_afu_irq);
/*
* Start a context
* Code here similar to afu_ioctl_start_work().
*/
int cxl_start_context(struct cxl_context *ctx, u64 wed,
struct task_struct *task)
{
int rc = 0;
bool kernel = true;
pr_devel("%s: pe: %i\n", __func__, ctx->pe);
mutex_lock(&ctx->status_mutex);
if (ctx->status == STARTED)
goto out; /* already started */
/*
* Increment the mapped context count for adapter. This also checks
* if adapter_context_lock is taken.
*/
rc = cxl_adapter_context_get(ctx->afu->adapter);
if (rc)
goto out;
if (task) {
ctx->pid = get_task_pid(task, PIDTYPE_PID);
kernel = false;
/* acquire a reference to the task's mm */
ctx->mm = get_task_mm(current);
/* ensure this mm_struct can't be freed */
cxl_context_mm_count_get(ctx);
if (ctx->mm) {
/* decrement the use count from above */
mmput(ctx->mm);
/* make TLBIs for this context global */
mm_context_add_copro(ctx->mm);
}
}
/*
* Increment driver use count. Enables global TLBIs for hash
* and callbacks to handle the segment table
*/
cxl_ctx_get();
/* See the comment in afu_ioctl_start_work() */
smp_mb();
if ((rc = cxl_ops->attach_process(ctx, kernel, wed, 0))) {
put_pid(ctx->pid);
ctx->pid = NULL;
cxl_adapter_context_put(ctx->afu->adapter);
cxl_ctx_put();
if (task) {
cxl_context_mm_count_put(ctx);
if (ctx->mm)
mm_context_remove_copro(ctx->mm);
}
goto out;
}
ctx->status = STARTED;
out:
mutex_unlock(&ctx->status_mutex);
return rc;
}
EXPORT_SYMBOL_GPL(cxl_start_context);
int cxl_process_element(struct cxl_context *ctx)
{
return ctx->external_pe;
}
EXPORT_SYMBOL_GPL(cxl_process_element);
/* Stop a context. Returns 0 on success, otherwise -Errno */
int cxl_stop_context(struct cxl_context *ctx)
{
return __detach_context(ctx);
}
EXPORT_SYMBOL_GPL(cxl_stop_context);
void cxl_set_master(struct cxl_context *ctx)
{
ctx->master = true;
}
EXPORT_SYMBOL_GPL(cxl_set_master);
/* wrappers around afu_* file ops which are EXPORTED */
int cxl_fd_open(struct inode *inode, struct file *file)
{
return afu_open(inode, file);
}
EXPORT_SYMBOL_GPL(cxl_fd_open);
int cxl_fd_release(struct inode *inode, struct file *file)
{
return afu_release(inode, file);
}
EXPORT_SYMBOL_GPL(cxl_fd_release);
long cxl_fd_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
return afu_ioctl(file, cmd, arg);
}
EXPORT_SYMBOL_GPL(cxl_fd_ioctl);
int cxl_fd_mmap(struct file *file, struct vm_area_struct *vm)
{
return afu_mmap(file, vm);
}
EXPORT_SYMBOL_GPL(cxl_fd_mmap);
__poll_t cxl_fd_poll(struct file *file, struct poll_table_struct *poll)
{
return afu_poll(file, poll);
}
EXPORT_SYMBOL_GPL(cxl_fd_poll);
ssize_t cxl_fd_read(struct file *file, char __user *buf, size_t count,
loff_t *off)
{
return afu_read(file, buf, count, off);
}
EXPORT_SYMBOL_GPL(cxl_fd_read);
#define PATCH_FOPS(NAME) if (!fops->NAME) fops->NAME = afu_fops.NAME
/* Get a struct file and fd for a context and attach the ops */
struct file *cxl_get_fd(struct cxl_context *ctx, struct file_operations *fops,
int *fd)
{
struct file *file;
int rc, flags, fdtmp;
char *name = NULL;
/* only allow one per context */
if (ctx->mapping)
return ERR_PTR(-EEXIST);
flags = O_RDWR | O_CLOEXEC;
/* This code is similar to anon_inode_getfd() */
rc = get_unused_fd_flags(flags);
if (rc < 0)
return ERR_PTR(rc);
fdtmp = rc;
/*
* Patch the file ops. Needs to be careful that this is rentrant safe.
*/
if (fops) {
PATCH_FOPS(open);
PATCH_FOPS(poll);
PATCH_FOPS(read);
PATCH_FOPS(release);
PATCH_FOPS(unlocked_ioctl);
PATCH_FOPS(compat_ioctl);
PATCH_FOPS(mmap);
} else /* use default ops */
fops = (struct file_operations *)&afu_fops;
name = kasprintf(GFP_KERNEL, "cxl:%d", ctx->pe);
file = cxl_getfile(name, fops, ctx, flags);
kfree(name);
if (IS_ERR(file))
goto err_fd;
cxl_context_set_mapping(ctx, file->f_mapping);
*fd = fdtmp;
return file;
err_fd:
put_unused_fd(fdtmp);
return NULL;
}
EXPORT_SYMBOL_GPL(cxl_get_fd);
struct cxl_context *cxl_fops_get_context(struct file *file)
{
return file->private_data;
}
EXPORT_SYMBOL_GPL(cxl_fops_get_context);
void cxl_set_driver_ops(struct cxl_context *ctx,
struct cxl_afu_driver_ops *ops)
{
WARN_ON(!ops->fetch_event || !ops->event_delivered);
atomic_set(&ctx->afu_driver_events, 0);
ctx->afu_driver_ops = ops;
}
EXPORT_SYMBOL_GPL(cxl_set_driver_ops);
void cxl_context_events_pending(struct cxl_context *ctx,
unsigned int new_events)
{
atomic_add(new_events, &ctx->afu_driver_events);
wake_up_all(&ctx->wq);
}
EXPORT_SYMBOL_GPL(cxl_context_events_pending);
int cxl_start_work(struct cxl_context *ctx,
struct cxl_ioctl_start_work *work)
{
int rc;
/* code taken from afu_ioctl_start_work */
if (!(work->flags & CXL_START_WORK_NUM_IRQS))
work->num_interrupts = ctx->afu->pp_irqs;
else if ((work->num_interrupts < ctx->afu->pp_irqs) ||
(work->num_interrupts > ctx->afu->irqs_max)) {
return -EINVAL;
}
rc = afu_register_irqs(ctx, work->num_interrupts);
if (rc)
return rc;
rc = cxl_start_context(ctx, work->work_element_descriptor, current);
if (rc < 0) {
afu_release_irqs(ctx, ctx);
return rc;
}
return 0;
}
EXPORT_SYMBOL_GPL(cxl_start_work);
void __iomem *cxl_psa_map(struct cxl_context *ctx)
{
if (ctx->status != STARTED)
return NULL;
pr_devel("%s: psn_phys%llx size:%llx\n",
__func__, ctx->psn_phys, ctx->psn_size);
return ioremap(ctx->psn_phys, ctx->psn_size);
}
EXPORT_SYMBOL_GPL(cxl_psa_map);
void cxl_psa_unmap(void __iomem *addr)
{
iounmap(addr);
}
EXPORT_SYMBOL_GPL(cxl_psa_unmap);
int cxl_afu_reset(struct cxl_context *ctx)
{
struct cxl_afu *afu = ctx->afu;
int rc;
rc = cxl_ops->afu_reset(afu);
if (rc)
return rc;
return cxl_ops->afu_check_and_enable(afu);
}
EXPORT_SYMBOL_GPL(cxl_afu_reset);
void cxl_perst_reloads_same_image(struct cxl_afu *afu,
bool perst_reloads_same_image)
{
afu->adapter->perst_same_image = perst_reloads_same_image;
}
EXPORT_SYMBOL_GPL(cxl_perst_reloads_same_image);
ssize_t cxl_read_adapter_vpd(struct pci_dev *dev, void *buf, size_t count)
{
struct cxl_afu *afu = cxl_pci_to_afu(dev);
if (IS_ERR(afu))
return -ENODEV;
return cxl_ops->read_adapter_vpd(afu->adapter, buf, count);
}
EXPORT_SYMBOL_GPL(cxl_read_adapter_vpd);

126
drivers/misc/cxl/base.c
View File

@ -1,126 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/module.h>
#include <linux/rcupdate.h>
#include <asm/errno.h>
#include <misc/cxl-base.h>
#include <linux/of.h>
#include <linux/of_platform.h>
#include "cxl.h"
/* protected by rcu */
static struct cxl_calls *cxl_calls;
atomic_t cxl_use_count = ATOMIC_INIT(0);
EXPORT_SYMBOL(cxl_use_count);
#ifdef CONFIG_CXL_MODULE
static inline struct cxl_calls *cxl_calls_get(void)
{
struct cxl_calls *calls = NULL;
rcu_read_lock();
calls = rcu_dereference(cxl_calls);
if (calls && !try_module_get(calls->owner))
calls = NULL;
rcu_read_unlock();
return calls;
}
static inline void cxl_calls_put(struct cxl_calls *calls)
{
BUG_ON(calls != cxl_calls);
/* we don't need to rcu this, as we hold a reference to the module */
module_put(cxl_calls->owner);
}
#else /* !defined CONFIG_CXL_MODULE */
static inline struct cxl_calls *cxl_calls_get(void)
{
return cxl_calls;
}
static inline void cxl_calls_put(struct cxl_calls *calls) { }
#endif /* CONFIG_CXL_MODULE */
/* AFU refcount management */
struct cxl_afu *cxl_afu_get(struct cxl_afu *afu)
{
return (get_device(&afu->dev) == NULL) ? NULL : afu;
}
EXPORT_SYMBOL_GPL(cxl_afu_get);
void cxl_afu_put(struct cxl_afu *afu)
{
put_device(&afu->dev);
}
EXPORT_SYMBOL_GPL(cxl_afu_put);
void cxl_slbia(struct mm_struct *mm)
{
struct cxl_calls *calls;
calls = cxl_calls_get();
if (!calls)
return;
if (cxl_ctx_in_use())
calls->cxl_slbia(mm);
cxl_calls_put(calls);
}
int register_cxl_calls(struct cxl_calls *calls)
{
if (cxl_calls)
return -EBUSY;
rcu_assign_pointer(cxl_calls, calls);
return 0;
}
EXPORT_SYMBOL_GPL(register_cxl_calls);
void unregister_cxl_calls(struct cxl_calls *calls)
{
BUG_ON(cxl_calls->owner != calls->owner);
RCU_INIT_POINTER(cxl_calls, NULL);
synchronize_rcu();
}
EXPORT_SYMBOL_GPL(unregister_cxl_calls);
int cxl_update_properties(struct device_node *dn,
struct property *new_prop)
{
return of_update_property(dn, new_prop);
}
EXPORT_SYMBOL_GPL(cxl_update_properties);
static int __init cxl_base_init(void)
{
struct device_node *np;
struct platform_device *dev;
int count = 0;
/*
* Scan for compatible devices in guest only
*/
if (cpu_has_feature(CPU_FTR_HVMODE))
return 0;
for_each_compatible_node(np, NULL, "ibm,coherent-platform-facility") {
dev = of_platform_device_create(np, NULL, NULL);
if (dev)
count++;
}
pr_devel("Found %d cxl device(s)\n", count);
return 0;
}
device_initcall(cxl_base_init);

362
drivers/misc/cxl/context.c
View File

@ -1,362 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/bitmap.h>
#include <linux/sched.h>
#include <linux/pid.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/debugfs.h>
#include <linux/slab.h>
#include <linux/idr.h>
#include <linux/sched/mm.h>
#include <linux/mmu_context.h>
#include <asm/cputable.h>
#include <asm/current.h>
#include <asm/copro.h>
#include "cxl.h"
/*
* Allocates space for a CXL context.
*/
struct cxl_context *cxl_context_alloc(void)
{
return kzalloc(sizeof(struct cxl_context), GFP_KERNEL);
}
/*
* Initialises a CXL context.
*/
int cxl_context_init(struct cxl_context *ctx, struct cxl_afu *afu, bool master)
{
int i;
ctx->afu = afu;
ctx->master = master;
ctx->pid = NULL; /* Set in start work ioctl */
mutex_init(&ctx->mapping_lock);
ctx->mapping = NULL;
ctx->tidr = 0;
ctx->assign_tidr = false;
if (cxl_is_power8()) {
spin_lock_init(&ctx->sste_lock);
/*
* Allocate the segment table before we put it in the IDR so that we
* can always access it when dereferenced from IDR. For the same
* reason, the segment table is only destroyed after the context is
* removed from the IDR. Access to this in the IOCTL is protected by
* Linux filesystem semantics (can't IOCTL until open is complete).
*/
i = cxl_alloc_sst(ctx);
if (i)
return i;
}
INIT_WORK(&ctx->fault_work, cxl_handle_fault);
init_waitqueue_head(&ctx->wq);
spin_lock_init(&ctx->lock);
ctx->irq_bitmap = NULL;
ctx->pending_irq = false;
ctx->pending_fault = false;
ctx->pending_afu_err = false;
INIT_LIST_HEAD(&ctx->irq_names);
/*
* When we have to destroy all contexts in cxl_context_detach_all() we
* end up with afu_release_irqs() called from inside a
* idr_for_each_entry(). Hence we need to make sure that anything
* dereferenced from this IDR is ok before we allocate the IDR here.
* This clears out the IRQ ranges to ensure this.
*/
for (i = 0; i < CXL_IRQ_RANGES; i++)
ctx->irqs.range[i] = 0;
mutex_init(&ctx->status_mutex);
ctx->status = OPENED;
/*
* Allocating IDR! We better make sure everything's setup that
* dereferences from it.
*/
mutex_lock(&afu->contexts_lock);
idr_preload(GFP_KERNEL);
i = idr_alloc(&ctx->afu->contexts_idr, ctx, 0,
ctx->afu->num_procs, GFP_NOWAIT);
idr_preload_end();
mutex_unlock(&afu->contexts_lock);
if (i < 0)
return i;
ctx->pe = i;
if (cpu_has_feature(CPU_FTR_HVMODE)) {
ctx->elem = &ctx->afu->native->spa[i];
ctx->external_pe = ctx->pe;
} else {
ctx->external_pe = -1; /* assigned when attaching */
}
ctx->pe_inserted = false;
/*
* take a ref on the afu so that it stays alive at-least till
* this context is reclaimed inside reclaim_ctx.
*/
cxl_afu_get(afu);
return 0;
}
void cxl_context_set_mapping(struct cxl_context *ctx,
struct address_space *mapping)
{
mutex_lock(&ctx->mapping_lock);
ctx->mapping = mapping;
mutex_unlock(&ctx->mapping_lock);
}
static vm_fault_t cxl_mmap_fault(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
struct cxl_context *ctx = vma->vm_file->private_data;
u64 area, offset;
vm_fault_t ret;
offset = vmf->pgoff << PAGE_SHIFT;
pr_devel("%s: pe: %i address: 0x%lx offset: 0x%llx\n",
__func__, ctx->pe, vmf->address, offset);
if (ctx->afu->current_mode == CXL_MODE_DEDICATED) {
area = ctx->afu->psn_phys;
if (offset >= ctx->afu->adapter->ps_size)
return VM_FAULT_SIGBUS;
} else {
area = ctx->psn_phys;
if (offset >= ctx->psn_size)
return VM_FAULT_SIGBUS;
}
mutex_lock(&ctx->status_mutex);
if (ctx->status != STARTED) {
mutex_unlock(&ctx->status_mutex);
pr_devel("%s: Context not started, failing problem state access\n", __func__);
if (ctx->mmio_err_ff) {
if (!ctx->ff_page) {
ctx->ff_page = alloc_page(GFP_USER);
if (!ctx->ff_page)
return VM_FAULT_OOM;
memset(page_address(ctx->ff_page), 0xff, PAGE_SIZE);
}
get_page(ctx->ff_page);
vmf->page = ctx->ff_page;
vma->vm_page_prot = pgprot_cached(vma->vm_page_prot);
return 0;
}
return VM_FAULT_SIGBUS;
}
ret = vmf_insert_pfn(vma, vmf->address, (area + offset) >> PAGE_SHIFT);
mutex_unlock(&ctx->status_mutex);
return ret;
}
static const struct vm_operations_struct cxl_mmap_vmops = {
.fault = cxl_mmap_fault,
};
/*
* Map a per-context mmio space into the given vma.
*/
int cxl_context_iomap(struct cxl_context *ctx, struct vm_area_struct *vma)
{
u64 start = vma->vm_pgoff << PAGE_SHIFT;
u64 len = vma->vm_end - vma->vm_start;
if (ctx->afu->current_mode == CXL_MODE_DEDICATED) {
if (start + len > ctx->afu->adapter->ps_size)
return -EINVAL;
if (cxl_is_power9()) {
/*
* Make sure there is a valid problem state
* area space for this AFU.
*/
if (ctx->master && !ctx->afu->psa) {
pr_devel("AFU doesn't support mmio space\n");
return -EINVAL;
}
/* Can't mmap until the AFU is enabled */
if (!ctx->afu->enabled)
return -EBUSY;
}
} else {
if (start + len > ctx->psn_size)
return -EINVAL;
/* Make sure there is a valid per process space for this AFU */
if ((ctx->master && !ctx->afu->psa) || (!ctx->afu->pp_psa)) {
pr_devel("AFU doesn't support mmio space\n");
return -EINVAL;
}
/* Can't mmap until the AFU is enabled */
if (!ctx->afu->enabled)
return -EBUSY;
}
pr_devel("%s: mmio physical: %llx pe: %i master:%i\n", __func__,
ctx->psn_phys, ctx->pe , ctx->master);
vm_flags_set(vma, VM_IO | VM_PFNMAP);
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
vma->vm_ops = &cxl_mmap_vmops;
return 0;
}
/*
* Detach a context from the hardware. This disables interrupts and doesn't
* return until all outstanding interrupts for this context have completed. The
* hardware should no longer access *ctx after this has returned.
*/
int __detach_context(struct cxl_context *ctx)
{
enum cxl_context_status status;
mutex_lock(&ctx->status_mutex);
status = ctx->status;
ctx->status = CLOSED;
mutex_unlock(&ctx->status_mutex);
if (status != STARTED)
return -EBUSY;
/* Only warn if we detached while the link was OK.
* If detach fails when hw is down, we don't care.
*/
WARN_ON(cxl_ops->detach_process(ctx) &&
cxl_ops->link_ok(ctx->afu->adapter, ctx->afu));
flush_work(&ctx->fault_work); /* Only needed for dedicated process */
/*
* Wait until no further interrupts are presented by the PSL
* for this context.
*/
if (cxl_ops->irq_wait)
cxl_ops->irq_wait(ctx);
/* release the reference to the group leader and mm handling pid */
put_pid(ctx->pid);
cxl_ctx_put();
/* Decrease the attached context count on the adapter */
cxl_adapter_context_put(ctx->afu->adapter);
/* Decrease the mm count on the context */
cxl_context_mm_count_put(ctx);
if (ctx->mm)
mm_context_remove_copro(ctx->mm);
ctx->mm = NULL;
return 0;
}
/*
* Detach the given context from the AFU. This doesn't actually
* free the context but it should stop the context running in hardware
* (ie. prevent this context from generating any further interrupts
* so that it can be freed).
*/
void cxl_context_detach(struct cxl_context *ctx)
{
int rc;
rc = __detach_context(ctx);
if (rc)
return;
afu_release_irqs(ctx, ctx);
wake_up_all(&ctx->wq);
}
/*
* Detach all contexts on the given AFU.
*/
void cxl_context_detach_all(struct cxl_afu *afu)
{
struct cxl_context *ctx;
int tmp;
mutex_lock(&afu->contexts_lock);
idr_for_each_entry(&afu->contexts_idr, ctx, tmp) {
/*
* Anything done in here needs to be setup before the IDR is
* created and torn down after the IDR removed
*/
cxl_context_detach(ctx);
/*
* We are force detaching - remove any active PSA mappings so
* userspace cannot interfere with the card if it comes back.
* Easiest way to exercise this is to unbind and rebind the
* driver via sysfs while it is in use.
*/
mutex_lock(&ctx->mapping_lock);
if (ctx->mapping)
unmap_mapping_range(ctx->mapping, 0, 0, 1);
mutex_unlock(&ctx->mapping_lock);
}
mutex_unlock(&afu->contexts_lock);
}
static void reclaim_ctx(struct rcu_head *rcu)
{
struct cxl_context *ctx = container_of(rcu, struct cxl_context, rcu);
if (cxl_is_power8())
free_page((u64)ctx->sstp);
if (ctx->ff_page)
__free_page(ctx->ff_page);
ctx->sstp = NULL;
bitmap_free(ctx->irq_bitmap);
/* Drop ref to the afu device taken during cxl_context_init */
cxl_afu_put(ctx->afu);
kfree(ctx);
}
void cxl_context_free(struct cxl_context *ctx)
{
if (ctx->kernelapi && ctx->mapping)
cxl_release_mapping(ctx);
mutex_lock(&ctx->afu->contexts_lock);
idr_remove(&ctx->afu->contexts_idr, ctx->pe);
mutex_unlock(&ctx->afu->contexts_lock);
call_rcu(&ctx->rcu, reclaim_ctx);
}
void cxl_context_mm_count_get(struct cxl_context *ctx)
{
if (ctx->mm)
mmgrab(ctx->mm);
}
void cxl_context_mm_count_put(struct cxl_context *ctx)
{
if (ctx->mm)
mmdrop(ctx->mm);
}

1135
drivers/misc/cxl/cxl.h
View File

File diff suppressed because it is too large Load Diff

271
drivers/misc/cxl/cxllib.c
View File

@ -1,271 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2017 IBM Corp.
*/
#include <linux/hugetlb.h>
#include <linux/sched/mm.h>
#include <asm/opal-api.h>
#include <asm/pnv-pci.h>
#include <misc/cxllib.h>
#include "cxl.h"
#define CXL_INVALID_DRA ~0ull
#define CXL_DUMMY_READ_SIZE 128
#define CXL_DUMMY_READ_ALIGN 8
#define CXL_CAPI_WINDOW_START 0x2000000000000ull
#define CXL_CAPI_WINDOW_LOG_SIZE 48
#define CXL_XSL_CONFIG_CURRENT_VERSION CXL_XSL_CONFIG_VERSION1
bool cxllib_slot_is_supported(struct pci_dev *dev, unsigned long flags)
{
int rc;
u32 phb_index;
u64 chip_id, capp_unit_id;
/* No flags currently supported */
if (flags)
return false;
if (!cpu_has_feature(CPU_FTR_HVMODE))
return false;
if (!cxl_is_power9())
return false;
if (cxl_slot_is_switched(dev))
return false;
/* on p9, some pci slots are not connected to a CAPP unit */
rc = cxl_calc_capp_routing(dev, &chip_id, &phb_index, &capp_unit_id);
if (rc)
return false;
return true;
}
EXPORT_SYMBOL_GPL(cxllib_slot_is_supported);
static DEFINE_MUTEX(dra_mutex);
static u64 dummy_read_addr = CXL_INVALID_DRA;
static int allocate_dummy_read_buf(void)
{
u64 buf, vaddr;
size_t buf_size;
/*
* Dummy read buffer is 128-byte long, aligned on a
* 256-byte boundary and we need the physical address.
*/
buf_size = CXL_DUMMY_READ_SIZE + (1ull << CXL_DUMMY_READ_ALIGN);
buf = (u64) kzalloc(buf_size, GFP_KERNEL);
if (!buf)
return -ENOMEM;
vaddr = (buf + (1ull << CXL_DUMMY_READ_ALIGN) - 1) &
(~0ull << CXL_DUMMY_READ_ALIGN);
WARN((vaddr + CXL_DUMMY_READ_SIZE) > (buf + buf_size),
"Dummy read buffer alignment issue");
dummy_read_addr = virt_to_phys((void *) vaddr);
return 0;
}
int cxllib_get_xsl_config(struct pci_dev *dev, struct cxllib_xsl_config *cfg)
{
int rc;
u32 phb_index;
u64 chip_id, capp_unit_id;
if (!cpu_has_feature(CPU_FTR_HVMODE))
return -EINVAL;
mutex_lock(&dra_mutex);
if (dummy_read_addr == CXL_INVALID_DRA) {
rc = allocate_dummy_read_buf();
if (rc) {
mutex_unlock(&dra_mutex);
return rc;
}
}
mutex_unlock(&dra_mutex);
rc = cxl_calc_capp_routing(dev, &chip_id, &phb_index, &capp_unit_id);
if (rc)
return rc;
rc = cxl_get_xsl9_dsnctl(dev, capp_unit_id, &cfg->dsnctl);
if (rc)
return rc;
cfg->version = CXL_XSL_CONFIG_CURRENT_VERSION;
cfg->log_bar_size = CXL_CAPI_WINDOW_LOG_SIZE;
cfg->bar_addr = CXL_CAPI_WINDOW_START;
cfg->dra = dummy_read_addr;
return 0;
}
EXPORT_SYMBOL_GPL(cxllib_get_xsl_config);
int cxllib_switch_phb_mode(struct pci_dev *dev, enum cxllib_mode mode,
unsigned long flags)
{
int rc = 0;
if (!cpu_has_feature(CPU_FTR_HVMODE))
return -EINVAL;
switch (mode) {
case CXL_MODE_PCI:
/*
* We currently don't support going back to PCI mode
* However, we'll turn the invalidations off, so that
* the firmware doesn't have to ack them and can do
* things like reset, etc.. with no worries.
* So always return EPERM (can't go back to PCI) or
* EBUSY if we couldn't even turn off snooping
*/
rc = pnv_phb_to_cxl_mode(dev, OPAL_PHB_CAPI_MODE_SNOOP_OFF);
if (rc)
rc = -EBUSY;
else
rc = -EPERM;
break;
case CXL_MODE_CXL:
/* DMA only supported on TVT1 for the time being */
if (flags != CXL_MODE_DMA_TVT1)
return -EINVAL;
rc = pnv_phb_to_cxl_mode(dev, OPAL_PHB_CAPI_MODE_DMA_TVT1);
if (rc)
return rc;
rc = pnv_phb_to_cxl_mode(dev, OPAL_PHB_CAPI_MODE_SNOOP_ON);
break;
default:
rc = -EINVAL;
}
return rc;
}
EXPORT_SYMBOL_GPL(cxllib_switch_phb_mode);
/*
* When switching the PHB to capi mode, the TVT#1 entry for
* the Partitionable Endpoint is set in bypass mode, like
* in PCI mode.
* Configure the device dma to use TVT#1, which is done
* by calling dma_set_mask() with a mask large enough.
*/
int cxllib_set_device_dma(struct pci_dev *dev, unsigned long flags)
{
int rc;
if (flags)
return -EINVAL;
rc = dma_set_mask(&dev->dev, DMA_BIT_MASK(64));
return rc;
}
EXPORT_SYMBOL_GPL(cxllib_set_device_dma);
int cxllib_get_PE_attributes(struct task_struct *task,
unsigned long translation_mode,
struct cxllib_pe_attributes *attr)
{
if (translation_mode != CXL_TRANSLATED_MODE &&
translation_mode != CXL_REAL_MODE)
return -EINVAL;
attr->sr = cxl_calculate_sr(false,
task == NULL,
translation_mode == CXL_REAL_MODE,
true);
attr->lpid = mfspr(SPRN_LPID);
if (task) {
struct mm_struct *mm = get_task_mm(task);
if (mm == NULL)
return -EINVAL;
/*
* Caller is keeping a reference on mm_users for as long
* as XSL uses the memory context
*/
attr->pid = mm->context.id;
mmput(mm);
attr->tid = task->thread.tidr;
} else {
attr->pid = 0;
attr->tid = 0;
}
return 0;
}
EXPORT_SYMBOL_GPL(cxllib_get_PE_attributes);
static int get_vma_info(struct mm_struct *mm, u64 addr,
u64 *vma_start, u64 *vma_end,
unsigned long *page_size)
{
struct vm_area_struct *vma = NULL;
int rc = 0;
mmap_read_lock(mm);
vma = find_vma(mm, addr);
if (!vma) {
rc = -EFAULT;
goto out;
}
*page_size = vma_kernel_pagesize(vma);
*vma_start = vma->vm_start;
*vma_end = vma->vm_end;
out:
mmap_read_unlock(mm);
return rc;
}
int cxllib_handle_fault(struct mm_struct *mm, u64 addr, u64 size, u64 flags)
{
int rc;
u64 dar, vma_start, vma_end;
unsigned long page_size;
if (mm == NULL)
return -EFAULT;
/*
* The buffer we have to process can extend over several pages
* and may also cover several VMAs.
* We iterate over all the pages. The page size could vary
* between VMAs.
*/
rc = get_vma_info(mm, addr, &vma_start, &vma_end, &page_size);
if (rc)
return rc;
for (dar = (addr & ~(page_size - 1)); dar < (addr + size);
dar += page_size) {
if (dar < vma_start || dar >= vma_end) {
/*
* We don't hold mm->mmap_lock while iterating, since
* the lock is required by one of the lower-level page
* fault processing functions and it could
* create a deadlock.
*
* It means the VMAs can be altered between 2
* loop iterations and we could theoretically
* miss a page (however unlikely). But that's
* not really a problem, as the driver will
* retry access, get another page fault on the
* missing page and call us again.
*/
rc = get_vma_info(mm, dar, &vma_start, &vma_end,
&page_size);
if (rc)
return rc;
}
rc = cxl_handle_mm_fault(mm, flags, dar);
if (rc)
return -EFAULT;
}
return 0;
}
EXPORT_SYMBOL_GPL(cxllib_handle_fault);

134
drivers/misc/cxl/debugfs.c
View File

@ -1,134 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/debugfs.h>
#include <linux/kernel.h>
#include <linux/slab.h>
#include "cxl.h"
static struct dentry *cxl_debugfs;
/* Helpers to export CXL mmaped IO registers via debugfs */
static int debugfs_io_u64_get(void *data, u64 *val)
{
*val = in_be64((u64 __iomem *)data);
return 0;
}
static int debugfs_io_u64_set(void *data, u64 val)
{
out_be64((u64 __iomem *)data, val);
return 0;
}
DEFINE_DEBUGFS_ATTRIBUTE(fops_io_x64, debugfs_io_u64_get, debugfs_io_u64_set,
"0x%016llx\n");
static void debugfs_create_io_x64(const char *name, umode_t mode,
struct dentry *parent, u64 __iomem *value)
{
debugfs_create_file_unsafe(name, mode, parent, (void __force *)value,
&fops_io_x64);
}
void cxl_debugfs_add_adapter_regs_psl9(struct cxl *adapter, struct dentry *dir)
{
debugfs_create_io_x64("fir1", S_IRUSR, dir, _cxl_p1_addr(adapter, CXL_PSL9_FIR1));
debugfs_create_io_x64("fir_mask", 0400, dir,
_cxl_p1_addr(adapter, CXL_PSL9_FIR_MASK));
debugfs_create_io_x64("fir_cntl", S_IRUSR, dir, _cxl_p1_addr(adapter, CXL_PSL9_FIR_CNTL));
debugfs_create_io_x64("trace", S_IRUSR | S_IWUSR, dir, _cxl_p1_addr(adapter, CXL_PSL9_TRACECFG));
debugfs_create_io_x64("debug", 0600, dir,
_cxl_p1_addr(adapter, CXL_PSL9_DEBUG));
debugfs_create_io_x64("xsl-debug", 0600, dir,
_cxl_p1_addr(adapter, CXL_XSL9_DBG));
}
void cxl_debugfs_add_adapter_regs_psl8(struct cxl *adapter, struct dentry *dir)
{
debugfs_create_io_x64("fir1", S_IRUSR, dir, _cxl_p1_addr(adapter, CXL_PSL_FIR1));
debugfs_create_io_x64("fir2", S_IRUSR, dir, _cxl_p1_addr(adapter, CXL_PSL_FIR2));
debugfs_create_io_x64("fir_cntl", S_IRUSR, dir, _cxl_p1_addr(adapter, CXL_PSL_FIR_CNTL));
debugfs_create_io_x64("trace", S_IRUSR | S_IWUSR, dir, _cxl_p1_addr(adapter, CXL_PSL_TRACE));
}
void cxl_debugfs_adapter_add(struct cxl *adapter)
{
struct dentry *dir;
char buf[32];
if (!cxl_debugfs)
return;
snprintf(buf, 32, "card%i", adapter->adapter_num);
dir = debugfs_create_dir(buf, cxl_debugfs);
adapter->debugfs = dir;
debugfs_create_io_x64("err_ivte", S_IRUSR, dir, _cxl_p1_addr(adapter, CXL_PSL_ErrIVTE));
if (adapter->native->sl_ops->debugfs_add_adapter_regs)
adapter->native->sl_ops->debugfs_add_adapter_regs(adapter, dir);
}
void cxl_debugfs_adapter_remove(struct cxl *adapter)
{
debugfs_remove_recursive(adapter->debugfs);
}
void cxl_debugfs_add_afu_regs_psl9(struct cxl_afu *afu, struct dentry *dir)
{
debugfs_create_io_x64("serr", S_IRUSR, dir, _cxl_p1n_addr(afu, CXL_PSL_SERR_An));
}
void cxl_debugfs_add_afu_regs_psl8(struct cxl_afu *afu, struct dentry *dir)
{
debugfs_create_io_x64("sstp0", S_IRUSR, dir, _cxl_p2n_addr(afu, CXL_SSTP0_An));
debugfs_create_io_x64("sstp1", S_IRUSR, dir, _cxl_p2n_addr(afu, CXL_SSTP1_An));
debugfs_create_io_x64("fir", S_IRUSR, dir, _cxl_p1n_addr(afu, CXL_PSL_FIR_SLICE_An));
debugfs_create_io_x64("serr", S_IRUSR, dir, _cxl_p1n_addr(afu, CXL_PSL_SERR_An));
debugfs_create_io_x64("afu_debug", S_IRUSR, dir, _cxl_p1n_addr(afu, CXL_AFU_DEBUG_An));
debugfs_create_io_x64("trace", S_IRUSR | S_IWUSR, dir, _cxl_p1n_addr(afu, CXL_PSL_SLICE_TRACE));
}
void cxl_debugfs_afu_add(struct cxl_afu *afu)
{
struct dentry *dir;
char buf[32];
if (!afu->adapter->debugfs)
return;
snprintf(buf, 32, "psl%i.%i", afu->adapter->adapter_num, afu->slice);
dir = debugfs_create_dir(buf, afu->adapter->debugfs);
afu->debugfs = dir;
debugfs_create_io_x64("sr", S_IRUSR, dir, _cxl_p1n_addr(afu, CXL_PSL_SR_An));
debugfs_create_io_x64("dsisr", S_IRUSR, dir, _cxl_p2n_addr(afu, CXL_PSL_DSISR_An));
debugfs_create_io_x64("dar", S_IRUSR, dir, _cxl_p2n_addr(afu, CXL_PSL_DAR_An));
debugfs_create_io_x64("err_status", S_IRUSR, dir, _cxl_p2n_addr(afu, CXL_PSL_ErrStat_An));
if (afu->adapter->native->sl_ops->debugfs_add_afu_regs)
afu->adapter->native->sl_ops->debugfs_add_afu_regs(afu, dir);
}
void cxl_debugfs_afu_remove(struct cxl_afu *afu)
{
debugfs_remove_recursive(afu->debugfs);
}
void __init cxl_debugfs_init(void)
{
if (!cpu_has_feature(CPU_FTR_HVMODE))
return;
cxl_debugfs = debugfs_create_dir("cxl", NULL);
}
void cxl_debugfs_exit(void)
{
debugfs_remove_recursive(cxl_debugfs);
}

341
drivers/misc/cxl/fault.c
View File

@ -1,341 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/workqueue.h>
#include <linux/sched/signal.h>
#include <linux/sched/mm.h>
#include <linux/pid.h>
#include <linux/mm.h>
#include <linux/moduleparam.h>
#undef MODULE_PARAM_PREFIX
#define MODULE_PARAM_PREFIX "cxl" "."
#include <asm/current.h>
#include <asm/copro.h>
#include <asm/mmu.h>
#include "cxl.h"
#include "trace.h"
static bool sste_matches(struct cxl_sste *sste, struct copro_slb *slb)
{
return ((sste->vsid_data == cpu_to_be64(slb->vsid)) &&
(sste->esid_data == cpu_to_be64(slb->esid)));
}
/*
* This finds a free SSTE for the given SLB, or returns NULL if it's already in
* the segment table.
*/
static struct cxl_sste *find_free_sste(struct cxl_context *ctx,
struct copro_slb *slb)
{
struct cxl_sste *primary, *sste, *ret = NULL;
unsigned int mask = (ctx->sst_size >> 7) - 1; /* SSTP0[SegTableSize] */
unsigned int entry;
unsigned int hash;
if (slb->vsid & SLB_VSID_B_1T)
hash = (slb->esid >> SID_SHIFT_1T) & mask;
else /* 256M */
hash = (slb->esid >> SID_SHIFT) & mask;
primary = ctx->sstp + (hash << 3);
for (entry = 0, sste = primary; entry < 8; entry++, sste++) {
if (!ret && !(be64_to_cpu(sste->esid_data) & SLB_ESID_V))
ret = sste;
if (sste_matches(sste, slb))
return NULL;
}
if (ret)
return ret;
/* Nothing free, select an entry to cast out */
ret = primary + ctx->sst_lru;
ctx->sst_lru = (ctx->sst_lru + 1) & 0x7;
return ret;
}
static void cxl_load_segment(struct cxl_context *ctx, struct copro_slb *slb)
{
/* mask is the group index, we search primary and secondary here. */
struct cxl_sste *sste;
unsigned long flags;
spin_lock_irqsave(&ctx->sste_lock, flags);
sste = find_free_sste(ctx, slb);
if (!sste)
goto out_unlock;
pr_devel("CXL Populating SST[%li]: %#llx %#llx\n",
sste - ctx->sstp, slb->vsid, slb->esid);
trace_cxl_ste_write(ctx, sste - ctx->sstp, slb->esid, slb->vsid);
sste->vsid_data = cpu_to_be64(slb->vsid);
sste->esid_data = cpu_to_be64(slb->esid);
out_unlock:
spin_unlock_irqrestore(&ctx->sste_lock, flags);
}
static int cxl_fault_segment(struct cxl_context *ctx, struct mm_struct *mm,
u64 ea)
{
struct copro_slb slb = {0,0};
int rc;
if (!(rc = copro_calculate_slb(mm, ea, &slb))) {
cxl_load_segment(ctx, &slb);
}
return rc;
}
static void cxl_ack_ae(struct cxl_context *ctx)
{
unsigned long flags;
cxl_ops->ack_irq(ctx, CXL_PSL_TFC_An_AE, 0);
spin_lock_irqsave(&ctx->lock, flags);
ctx->pending_fault = true;
ctx->fault_addr = ctx->dar;
ctx->fault_dsisr = ctx->dsisr;
spin_unlock_irqrestore(&ctx->lock, flags);
wake_up_all(&ctx->wq);
}
static int cxl_handle_segment_miss(struct cxl_context *ctx,
struct mm_struct *mm, u64 ea)
{
int rc;
pr_devel("CXL interrupt: Segment fault pe: %i ea: %#llx\n", ctx->pe, ea);
trace_cxl_ste_miss(ctx, ea);
if ((rc = cxl_fault_segment(ctx, mm, ea)))
cxl_ack_ae(ctx);
else {
mb(); /* Order seg table write to TFC MMIO write */
cxl_ops->ack_irq(ctx, CXL_PSL_TFC_An_R, 0);
}
return IRQ_HANDLED;
}
int cxl_handle_mm_fault(struct mm_struct *mm, u64 dsisr, u64 dar)
{
vm_fault_t flt = 0;
int result;
unsigned long access, flags, inv_flags = 0;
/*
* Add the fault handling cpu to task mm cpumask so that we
* can do a safe lockless page table walk when inserting the
* hash page table entry. This function get called with a
* valid mm for user space addresses. Hence using the if (mm)
* check is sufficient here.
*/
if (mm && !cpumask_test_cpu(smp_processor_id(), mm_cpumask(mm))) {
cpumask_set_cpu(smp_processor_id(), mm_cpumask(mm));
/*
* We need to make sure we walk the table only after
* we update the cpumask. The other side of the barrier
* is explained in serialize_against_pte_lookup()
*/
smp_mb();
}
if ((result = copro_handle_mm_fault(mm, dar, dsisr, &flt))) {
pr_devel("copro_handle_mm_fault failed: %#x\n", result);
return result;
}
if (!radix_enabled()) {
/*
* update_mmu_cache() will not have loaded the hash since current->trap
* is not a 0x400 or 0x300, so just call hash_page_mm() here.
*/
access = _PAGE_PRESENT | _PAGE_READ;
if (dsisr & CXL_PSL_DSISR_An_S)
access |= _PAGE_WRITE;
if (!mm && (get_region_id(dar) != USER_REGION_ID))
access |= _PAGE_PRIVILEGED;
if (dsisr & DSISR_NOHPTE)
inv_flags |= HPTE_NOHPTE_UPDATE;
local_irq_save(flags);
hash_page_mm(mm, dar, access, 0x300, inv_flags);
local_irq_restore(flags);
}
return 0;
}
static void cxl_handle_page_fault(struct cxl_context *ctx,
struct mm_struct *mm,
u64 dsisr, u64 dar)
{
trace_cxl_pte_miss(ctx, dsisr, dar);
if (cxl_handle_mm_fault(mm, dsisr, dar)) {
cxl_ack_ae(ctx);
} else {
pr_devel("Page fault successfully handled for pe: %i!\n", ctx->pe);
cxl_ops->ack_irq(ctx, CXL_PSL_TFC_An_R, 0);
}
}
/*
* Returns the mm_struct corresponding to the context ctx.
* mm_users == 0, the context may be in the process of being closed.
*/
static struct mm_struct *get_mem_context(struct cxl_context *ctx)
{
if (ctx->mm == NULL)
return NULL;
if (!mmget_not_zero(ctx->mm))
return NULL;
return ctx->mm;
}
static bool cxl_is_segment_miss(struct cxl_context *ctx, u64 dsisr)
{
if ((cxl_is_power8() && (dsisr & CXL_PSL_DSISR_An_DS)))
return true;
return false;
}
static bool cxl_is_page_fault(struct cxl_context *ctx, u64 dsisr)
{
if ((cxl_is_power8()) && (dsisr & CXL_PSL_DSISR_An_DM))
return true;
if (cxl_is_power9())
return true;
return false;
}
void cxl_handle_fault(struct work_struct *fault_work)
{
struct cxl_context *ctx =
container_of(fault_work, struct cxl_context, fault_work);
u64 dsisr = ctx->dsisr;
u64 dar = ctx->dar;
struct mm_struct *mm = NULL;
if (cpu_has_feature(CPU_FTR_HVMODE)) {
if (cxl_p2n_read(ctx->afu, CXL_PSL_DSISR_An) != dsisr ||
cxl_p2n_read(ctx->afu, CXL_PSL_DAR_An) != dar ||
cxl_p2n_read(ctx->afu, CXL_PSL_PEHandle_An) != ctx->pe) {
/* Most likely explanation is harmless - a dedicated
* process has detached and these were cleared by the
* PSL purge, but warn about it just in case
*/
dev_notice(&ctx->afu->dev, "cxl_handle_fault: Translation fault regs changed\n");
return;
}
}
/* Early return if the context is being / has been detached */
if (ctx->status == CLOSED) {
cxl_ack_ae(ctx);
return;
}
pr_devel("CXL BOTTOM HALF handling fault for afu pe: %i. "
"DSISR: %#llx DAR: %#llx\n", ctx->pe, dsisr, dar);
if (!ctx->kernel) {
mm = get_mem_context(ctx);
if (mm == NULL) {
pr_devel("%s: unable to get mm for pe=%d pid=%i\n",
__func__, ctx->pe, pid_nr(ctx->pid));
cxl_ack_ae(ctx);
return;
} else {
pr_devel("Handling page fault for pe=%d pid=%i\n",
ctx->pe, pid_nr(ctx->pid));
}
}
if (cxl_is_segment_miss(ctx, dsisr))
cxl_handle_segment_miss(ctx, mm, dar);
else if (cxl_is_page_fault(ctx, dsisr))
cxl_handle_page_fault(ctx, mm, dsisr, dar);
else
WARN(1, "cxl_handle_fault has nothing to handle\n");
if (mm)
mmput(mm);
}
static u64 next_segment(u64 ea, u64 vsid)
{
if (vsid & SLB_VSID_B_1T)
ea |= (1ULL << 40) - 1;
else
ea |= (1ULL << 28) - 1;
return ea + 1;
}
static void cxl_prefault_vma(struct cxl_context *ctx, struct mm_struct *mm)
{
u64 ea, last_esid = 0;
struct copro_slb slb;
VMA_ITERATOR(vmi, mm, 0);
struct vm_area_struct *vma;
int rc;
mmap_read_lock(mm);
for_each_vma(vmi, vma) {
for (ea = vma->vm_start; ea < vma->vm_end;
ea = next_segment(ea, slb.vsid)) {
rc = copro_calculate_slb(mm, ea, &slb);
if (rc)
continue;
if (last_esid == slb.esid)
continue;
cxl_load_segment(ctx, &slb);
last_esid = slb.esid;
}
}
mmap_read_unlock(mm);
}
void cxl_prefault(struct cxl_context *ctx, u64 wed)
{
struct mm_struct *mm = get_mem_context(ctx);
if (mm == NULL) {
pr_devel("cxl_prefault unable to get mm %i\n",
pid_nr(ctx->pid));
return;
}
switch (ctx->afu->prefault_mode) {
case CXL_PREFAULT_WED:
cxl_fault_segment(ctx, mm, wed);
break;
case CXL_PREFAULT_ALL:
cxl_prefault_vma(ctx, mm);
break;
default:
break;
}
mmput(mm);
}

699
drivers/misc/cxl/file.c
View File

@ -1,699 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/spinlock.h>
#include <linux/module.h>
#include <linux/export.h>
#include <linux/kernel.h>
#include <linux/bitmap.h>
#include <linux/sched/signal.h>
#include <linux/poll.h>
#include <linux/pid.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include <linux/sched/mm.h>
#include <linux/mmu_context.h>
#include <asm/cputable.h>
#include <asm/current.h>
#include <asm/copro.h>
#include "cxl.h"
#include "trace.h"
#define CXL_NUM_MINORS 256 /* Total to reserve */
#define CXL_AFU_MINOR_D(afu) (CXL_CARD_MINOR(afu->adapter) + 1 + (3 * afu->slice))
#define CXL_AFU_MINOR_M(afu) (CXL_AFU_MINOR_D(afu) + 1)
#define CXL_AFU_MINOR_S(afu) (CXL_AFU_MINOR_D(afu) + 2)
#define CXL_AFU_MKDEV_D(afu) MKDEV(MAJOR(cxl_dev), CXL_AFU_MINOR_D(afu))
#define CXL_AFU_MKDEV_M(afu) MKDEV(MAJOR(cxl_dev), CXL_AFU_MINOR_M(afu))
#define CXL_AFU_MKDEV_S(afu) MKDEV(MAJOR(cxl_dev), CXL_AFU_MINOR_S(afu))
#define CXL_DEVT_AFU(dev) ((MINOR(dev) % CXL_DEV_MINORS - 1) / 3)
#define CXL_DEVT_IS_CARD(dev) (MINOR(dev) % CXL_DEV_MINORS == 0)
static dev_t cxl_dev;
static int __afu_open(struct inode *inode, struct file *file, bool master)
{
struct cxl *adapter;
struct cxl_afu *afu;
struct cxl_context *ctx;
int adapter_num = CXL_DEVT_ADAPTER(inode->i_rdev);
int slice = CXL_DEVT_AFU(inode->i_rdev);
int rc = -ENODEV;
pr_devel("afu_open afu%i.%i\n", slice, adapter_num);
if (!(adapter = get_cxl_adapter(adapter_num)))
return -ENODEV;
if (slice > adapter->slices)
goto err_put_adapter;
spin_lock(&adapter->afu_list_lock);
if (!(afu = adapter->afu[slice])) {
spin_unlock(&adapter->afu_list_lock);
goto err_put_adapter;
}
/*
* taking a ref to the afu so that it doesn't go away
* for rest of the function. This ref is released before
* we return.
*/
cxl_afu_get(afu);
spin_unlock(&adapter->afu_list_lock);
if (!afu->current_mode)
goto err_put_afu;
if (!cxl_ops->link_ok(adapter, afu)) {
rc = -EIO;
goto err_put_afu;
}
if (!(ctx = cxl_context_alloc())) {
rc = -ENOMEM;
goto err_put_afu;
}
rc = cxl_context_init(ctx, afu, master);
if (rc)
goto err_put_afu;
cxl_context_set_mapping(ctx, inode->i_mapping);
pr_devel("afu_open pe: %i\n", ctx->pe);
file->private_data = ctx;
/* indicate success */
rc = 0;
err_put_afu:
/* release the ref taken earlier */
cxl_afu_put(afu);
err_put_adapter:
put_device(&adapter->dev);
return rc;
}
int afu_open(struct inode *inode, struct file *file)
{
return __afu_open(inode, file, false);
}
static int afu_master_open(struct inode *inode, struct file *file)
{
return __afu_open(inode, file, true);
}
int afu_release(struct inode *inode, struct file *file)
{
struct cxl_context *ctx = file->private_data;
pr_devel("%s: closing cxl file descriptor. pe: %i\n",
__func__, ctx->pe);
cxl_context_detach(ctx);
/*
* Delete the context's mapping pointer, unless it's created by the
* kernel API, in which case leave it so it can be freed by reclaim_ctx()
*/
if (!ctx->kernelapi) {
mutex_lock(&ctx->mapping_lock);
ctx->mapping = NULL;
mutex_unlock(&ctx->mapping_lock);
}
/*
* At this this point all bottom halfs have finished and we should be
* getting no more IRQs from the hardware for this context. Once it's
* removed from the IDR (and RCU synchronised) it's safe to free the
* sstp and context.
*/
cxl_context_free(ctx);
return 0;
}
static long afu_ioctl_start_work(struct cxl_context *ctx,
struct cxl_ioctl_start_work __user *uwork)
{
struct cxl_ioctl_start_work work;
u64 amr = 0;
int rc;
pr_devel("%s: pe: %i\n", __func__, ctx->pe);
/* Do this outside the status_mutex to avoid a circular dependency with
* the locking in cxl_mmap_fault() */
if (copy_from_user(&work, uwork, sizeof(work)))
return -EFAULT;
mutex_lock(&ctx->status_mutex);
if (ctx->status != OPENED) {
rc = -EIO;
goto out;
}
/*
* if any of the reserved fields are set or any of the unused
* flags are set it's invalid
*/
if (work.reserved1 || work.reserved2 || work.reserved3 ||
work.reserved4 || work.reserved5 ||
(work.flags & ~CXL_START_WORK_ALL)) {
rc = -EINVAL;
goto out;
}
if (!(work.flags & CXL_START_WORK_NUM_IRQS))
work.num_interrupts = ctx->afu->pp_irqs;
else if ((work.num_interrupts < ctx->afu->pp_irqs) ||
(work.num_interrupts > ctx->afu->irqs_max)) {
rc = -EINVAL;
goto out;
}
if ((rc = afu_register_irqs(ctx, work.num_interrupts)))
goto out;
if (work.flags & CXL_START_WORK_AMR)
amr = work.amr & mfspr(SPRN_UAMOR);
if (work.flags & CXL_START_WORK_TID)
ctx->assign_tidr = true;
ctx->mmio_err_ff = !!(work.flags & CXL_START_WORK_ERR_FF);
/*
* Increment the mapped context count for adapter. This also checks
* if adapter_context_lock is taken.
*/
rc = cxl_adapter_context_get(ctx->afu->adapter);
if (rc) {
afu_release_irqs(ctx, ctx);
goto out;
}
/*
* We grab the PID here and not in the file open to allow for the case
* where a process (master, some daemon, etc) has opened the chardev on
* behalf of another process, so the AFU's mm gets bound to the process
* that performs this ioctl and not the process that opened the file.
* Also we grab the PID of the group leader so that if the task that
* has performed the attach operation exits the mm context of the
* process is still accessible.
*/
ctx->pid = get_task_pid(current, PIDTYPE_PID);
/* acquire a reference to the task's mm */
ctx->mm = get_task_mm(current);
/* ensure this mm_struct can't be freed */
cxl_context_mm_count_get(ctx);
if (ctx->mm) {
/* decrement the use count from above */
mmput(ctx->mm);
/* make TLBIs for this context global */
mm_context_add_copro(ctx->mm);
}
/*
* Increment driver use count. Enables global TLBIs for hash
* and callbacks to handle the segment table
*/
cxl_ctx_get();
/*
* A barrier is needed to make sure all TLBIs are global
* before we attach and the context starts being used by the
* adapter.
*
* Needed after mm_context_add_copro() for radix and
* cxl_ctx_get() for hash/p8.
*
* The barrier should really be mb(), since it involves a
* device. However, it's only useful when we have local
* vs. global TLBIs, i.e SMP=y. So keep smp_mb().
*/
smp_mb();
trace_cxl_attach(ctx, work.work_element_descriptor, work.num_interrupts, amr);
if ((rc = cxl_ops->attach_process(ctx, false, work.work_element_descriptor,
amr))) {
afu_release_irqs(ctx, ctx);
cxl_adapter_context_put(ctx->afu->adapter);
put_pid(ctx->pid);
ctx->pid = NULL;
cxl_ctx_put();
cxl_context_mm_count_put(ctx);
if (ctx->mm)
mm_context_remove_copro(ctx->mm);
goto out;
}
rc = 0;
if (work.flags & CXL_START_WORK_TID) {
work.tid = ctx->tidr;
if (copy_to_user(uwork, &work, sizeof(work)))
rc = -EFAULT;
}
ctx->status = STARTED;
out:
mutex_unlock(&ctx->status_mutex);
return rc;
}
static long afu_ioctl_process_element(struct cxl_context *ctx,
int __user *upe)
{
pr_devel("%s: pe: %i\n", __func__, ctx->pe);
if (copy_to_user(upe, &ctx->external_pe, sizeof(__u32)))
return -EFAULT;
return 0;
}
static long afu_ioctl_get_afu_id(struct cxl_context *ctx,
struct cxl_afu_id __user *upafuid)
{
struct cxl_afu_id afuid = { 0 };
afuid.card_id = ctx->afu->adapter->adapter_num;
afuid.afu_offset = ctx->afu->slice;
afuid.afu_mode = ctx->afu->current_mode;
/* set the flag bit in case the afu is a slave */
if (ctx->afu->current_mode == CXL_MODE_DIRECTED && !ctx->master)
afuid.flags |= CXL_AFUID_FLAG_SLAVE;
if (copy_to_user(upafuid, &afuid, sizeof(afuid)))
return -EFAULT;
return 0;
}
long afu_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
struct cxl_context *ctx = file->private_data;
if (ctx->status == CLOSED)
return -EIO;
if (!cxl_ops->link_ok(ctx->afu->adapter, ctx->afu))
return -EIO;
pr_devel("afu_ioctl\n");
switch (cmd) {
case CXL_IOCTL_START_WORK:
return afu_ioctl_start_work(ctx, (struct cxl_ioctl_start_work __user *)arg);
case CXL_IOCTL_GET_PROCESS_ELEMENT:
return afu_ioctl_process_element(ctx, (__u32 __user *)arg);
case CXL_IOCTL_GET_AFU_ID:
return afu_ioctl_get_afu_id(ctx, (struct cxl_afu_id __user *)
arg);
}
return -EINVAL;
}
static long afu_compat_ioctl(struct file *file, unsigned int cmd,
unsigned long arg)
{
return afu_ioctl(file, cmd, arg);
}
int afu_mmap(struct file *file, struct vm_area_struct *vm)
{
struct cxl_context *ctx = file->private_data;
/* AFU must be started before we can MMIO */
if (ctx->status != STARTED)
return -EIO;
if (!cxl_ops->link_ok(ctx->afu->adapter, ctx->afu))
return -EIO;
return cxl_context_iomap(ctx, vm);
}
static inline bool ctx_event_pending(struct cxl_context *ctx)
{
if (ctx->pending_irq || ctx->pending_fault || ctx->pending_afu_err)
return true;
if (ctx->afu_driver_ops && atomic_read(&ctx->afu_driver_events))
return true;
return false;
}
__poll_t afu_poll(struct file *file, struct poll_table_struct *poll)
{
struct cxl_context *ctx = file->private_data;
__poll_t mask = 0;
unsigned long flags;
poll_wait(file, &ctx->wq, poll);
pr_devel("afu_poll wait done pe: %i\n", ctx->pe);
spin_lock_irqsave(&ctx->lock, flags);
if (ctx_event_pending(ctx))
mask |= EPOLLIN | EPOLLRDNORM;
else if (ctx->status == CLOSED)
/* Only error on closed when there are no futher events pending
*/
mask |= EPOLLERR;
spin_unlock_irqrestore(&ctx->lock, flags);
pr_devel("afu_poll pe: %i returning %#x\n", ctx->pe, mask);
return mask;
}
static ssize_t afu_driver_event_copy(struct cxl_context *ctx,
char __user *buf,
struct cxl_event *event,
struct cxl_event_afu_driver_reserved *pl)
{
/* Check event */
if (!pl) {
ctx->afu_driver_ops->event_delivered(ctx, pl, -EINVAL);
return -EFAULT;
}
/* Check event size */
event->header.size += pl->data_size;
if (event->header.size > CXL_READ_MIN_SIZE) {
ctx->afu_driver_ops->event_delivered(ctx, pl, -EINVAL);
return -EFAULT;
}
/* Copy event header */
if (copy_to_user(buf, event, sizeof(struct cxl_event_header))) {
ctx->afu_driver_ops->event_delivered(ctx, pl, -EFAULT);
return -EFAULT;
}
/* Copy event data */
buf += sizeof(struct cxl_event_header);
if (copy_to_user(buf, &pl->data, pl->data_size)) {
ctx->afu_driver_ops->event_delivered(ctx, pl, -EFAULT);
return -EFAULT;
}
ctx->afu_driver_ops->event_delivered(ctx, pl, 0); /* Success */
return event->header.size;
}
ssize_t afu_read(struct file *file, char __user *buf, size_t count,
loff_t *off)
{
struct cxl_context *ctx = file->private_data;
struct cxl_event_afu_driver_reserved *pl = NULL;
struct cxl_event event;
unsigned long flags;
int rc;
DEFINE_WAIT(wait);
if (!cxl_ops->link_ok(ctx->afu->adapter, ctx->afu))
return -EIO;
if (count < CXL_READ_MIN_SIZE)
return -EINVAL;
spin_lock_irqsave(&ctx->lock, flags);
for (;;) {
prepare_to_wait(&ctx->wq, &wait, TASK_INTERRUPTIBLE);
if (ctx_event_pending(ctx) || (ctx->status == CLOSED))
break;
if (!cxl_ops->link_ok(ctx->afu->adapter, ctx->afu)) {
rc = -EIO;
goto out;
}
if (file->f_flags & O_NONBLOCK) {
rc = -EAGAIN;
goto out;
}
if (signal_pending(current)) {
rc = -ERESTARTSYS;
goto out;
}
spin_unlock_irqrestore(&ctx->lock, flags);
pr_devel("afu_read going to sleep...\n");
schedule();
pr_devel("afu_read woken up\n");
spin_lock_irqsave(&ctx->lock, flags);
}
finish_wait(&ctx->wq, &wait);
memset(&event, 0, sizeof(event));
event.header.process_element = ctx->pe;
event.header.size = sizeof(struct cxl_event_header);
if (ctx->afu_driver_ops && atomic_read(&ctx->afu_driver_events)) {
pr_devel("afu_read delivering AFU driver specific event\n");
pl = ctx->afu_driver_ops->fetch_event(ctx);
atomic_dec(&ctx->afu_driver_events);
event.header.type = CXL_EVENT_AFU_DRIVER;
} else if (ctx->pending_irq) {
pr_devel("afu_read delivering AFU interrupt\n");
event.header.size += sizeof(struct cxl_event_afu_interrupt);
event.header.type = CXL_EVENT_AFU_INTERRUPT;
event.irq.irq = find_first_bit(ctx->irq_bitmap, ctx->irq_count) + 1;
clear_bit(event.irq.irq - 1, ctx->irq_bitmap);
if (bitmap_empty(ctx->irq_bitmap, ctx->irq_count))
ctx->pending_irq = false;
} else if (ctx->pending_fault) {
pr_devel("afu_read delivering data storage fault\n");
event.header.size += sizeof(struct cxl_event_data_storage);
event.header.type = CXL_EVENT_DATA_STORAGE;
event.fault.addr = ctx->fault_addr;
event.fault.dsisr = ctx->fault_dsisr;
ctx->pending_fault = false;
} else if (ctx->pending_afu_err) {
pr_devel("afu_read delivering afu error\n");
event.header.size += sizeof(struct cxl_event_afu_error);
event.header.type = CXL_EVENT_AFU_ERROR;
event.afu_error.error = ctx->afu_err;
ctx->pending_afu_err = false;
} else if (ctx->status == CLOSED) {
pr_devel("afu_read fatal error\n");
spin_unlock_irqrestore(&ctx->lock, flags);
return -EIO;
} else
WARN(1, "afu_read must be buggy\n");
spin_unlock_irqrestore(&ctx->lock, flags);
if (event.header.type == CXL_EVENT_AFU_DRIVER)
return afu_driver_event_copy(ctx, buf, &event, pl);
if (copy_to_user(buf, &event, event.header.size))
return -EFAULT;
return event.header.size;
out:
finish_wait(&ctx->wq, &wait);
spin_unlock_irqrestore(&ctx->lock, flags);
return rc;
}
/*
* Note: if this is updated, we need to update api.c to patch the new ones in
* too
*/
const struct file_operations afu_fops = {
.owner = THIS_MODULE,
.open = afu_open,
.poll = afu_poll,
.read = afu_read,
.release = afu_release,
.unlocked_ioctl = afu_ioctl,
.compat_ioctl = afu_compat_ioctl,
.mmap = afu_mmap,
};
static const struct file_operations afu_master_fops = {
.owner = THIS_MODULE,
.open = afu_master_open,
.poll = afu_poll,
.read = afu_read,
.release = afu_release,
.unlocked_ioctl = afu_ioctl,
.compat_ioctl = afu_compat_ioctl,
.mmap = afu_mmap,
};
static char *cxl_devnode(const struct device *dev, umode_t *mode)
{
if (cpu_has_feature(CPU_FTR_HVMODE) &&
CXL_DEVT_IS_CARD(dev->devt)) {
/*
* These minor numbers will eventually be used to program the
* PSL and AFUs once we have dynamic reprogramming support
*/
return NULL;
}
return kasprintf(GFP_KERNEL, "cxl/%s", dev_name(dev));
}
static const struct class cxl_class = {
.name = "cxl",
.devnode = cxl_devnode,
};
static int cxl_add_chardev(struct cxl_afu *afu, dev_t devt, struct cdev *cdev,
struct device **chardev, char *postfix, char *desc,
const struct file_operations *fops)
{
struct device *dev;
int rc;
cdev_init(cdev, fops);
rc = cdev_add(cdev, devt, 1);
if (rc) {
dev_err(&afu->dev, "Unable to add %s chardev: %i\n", desc, rc);
return rc;
}
dev = device_create(&cxl_class, &afu->dev, devt, afu,
"afu%i.%i%s", afu->adapter->adapter_num, afu->slice, postfix);
if (IS_ERR(dev)) {
rc = PTR_ERR(dev);
dev_err(&afu->dev, "Unable to create %s chardev in sysfs: %i\n", desc, rc);
goto err;
}
*chardev = dev;
return 0;
err:
cdev_del(cdev);
return rc;
}
int cxl_chardev_d_afu_add(struct cxl_afu *afu)
{
return cxl_add_chardev(afu, CXL_AFU_MKDEV_D(afu), &afu->afu_cdev_d,
&afu->chardev_d, "d", "dedicated",
&afu_master_fops); /* Uses master fops */
}
int cxl_chardev_m_afu_add(struct cxl_afu *afu)
{
return cxl_add_chardev(afu, CXL_AFU_MKDEV_M(afu), &afu->afu_cdev_m,
&afu->chardev_m, "m", "master",
&afu_master_fops);
}
int cxl_chardev_s_afu_add(struct cxl_afu *afu)
{
return cxl_add_chardev(afu, CXL_AFU_MKDEV_S(afu), &afu->afu_cdev_s,
&afu->chardev_s, "s", "shared",
&afu_fops);
}
void cxl_chardev_afu_remove(struct cxl_afu *afu)
{
if (afu->chardev_d) {
cdev_del(&afu->afu_cdev_d);
device_unregister(afu->chardev_d);
afu->chardev_d = NULL;
}
if (afu->chardev_m) {
cdev_del(&afu->afu_cdev_m);
device_unregister(afu->chardev_m);
afu->chardev_m = NULL;
}
if (afu->chardev_s) {
cdev_del(&afu->afu_cdev_s);
device_unregister(afu->chardev_s);
afu->chardev_s = NULL;
}
}
int cxl_register_afu(struct cxl_afu *afu)
{
afu->dev.class = &cxl_class;
return device_register(&afu->dev);
}
int cxl_register_adapter(struct cxl *adapter)
{
adapter->dev.class = &cxl_class;
/*
* Future: When we support dynamically reprogramming the PSL & AFU we
* will expose the interface to do that via a chardev:
* adapter->dev.devt = CXL_CARD_MKDEV(adapter);
*/
return device_register(&adapter->dev);
}
dev_t cxl_get_dev(void)
{
return cxl_dev;
}
int __init cxl_file_init(void)
{
int rc;
/*
* If these change we really need to update API. Either change some
* flags or update API version number CXL_API_VERSION.
*/
BUILD_BUG_ON(CXL_API_VERSION != 3);
BUILD_BUG_ON(sizeof(struct cxl_ioctl_start_work) != 64);
BUILD_BUG_ON(sizeof(struct cxl_event_header) != 8);
BUILD_BUG_ON(sizeof(struct cxl_event_afu_interrupt) != 8);
BUILD_BUG_ON(sizeof(struct cxl_event_data_storage) != 32);
BUILD_BUG_ON(sizeof(struct cxl_event_afu_error) != 16);
if ((rc = alloc_chrdev_region(&cxl_dev, 0, CXL_NUM_MINORS, "cxl"))) {
pr_err("Unable to allocate CXL major number: %i\n", rc);
return rc;
}
pr_devel("CXL device allocated, MAJOR %i\n", MAJOR(cxl_dev));
rc = class_register(&cxl_class);
if (rc) {
pr_err("Unable to create CXL class\n");
goto err;
}
return 0;
err:
unregister_chrdev_region(cxl_dev, CXL_NUM_MINORS);
return rc;
}
void cxl_file_exit(void)
{
unregister_chrdev_region(cxl_dev, CXL_NUM_MINORS);
class_unregister(&cxl_class);
}

538
drivers/misc/cxl/flash.c
View File

@ -1,538 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/semaphore.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/of.h>
#include <asm/rtas.h>
#include "cxl.h"
#include "hcalls.h"
#define DOWNLOAD_IMAGE 1
#define VALIDATE_IMAGE 2
struct ai_header {
u16 version;
u8 reserved0[6];
u16 vendor;
u16 device;
u16 subsystem_vendor;
u16 subsystem;
u64 image_offset;
u64 image_length;
u8 reserved1[96];
};
static struct semaphore sem;
static unsigned long *buffer[CXL_AI_MAX_ENTRIES];
static struct sg_list *le;
static u64 continue_token;
static unsigned int transfer;
struct update_props_workarea {
__be32 phandle;
__be32 state;
__be64 reserved;
__be32 nprops;
} __packed;
struct update_nodes_workarea {
__be32 state;
__be64 unit_address;
__be32 reserved;
} __packed;
#define DEVICE_SCOPE 3
#define NODE_ACTION_MASK 0xff000000
#define NODE_COUNT_MASK 0x00ffffff
#define OPCODE_DELETE 0x01000000
#define OPCODE_UPDATE 0x02000000
#define OPCODE_ADD 0x03000000
static int rcall(int token, char *buf, s32 scope)
{
int rc;
spin_lock(&rtas_data_buf_lock);
memcpy(rtas_data_buf, buf, RTAS_DATA_BUF_SIZE);
rc = rtas_call(token, 2, 1, NULL, rtas_data_buf, scope);
memcpy(buf, rtas_data_buf, RTAS_DATA_BUF_SIZE);
spin_unlock(&rtas_data_buf_lock);
return rc;
}
static int update_property(struct device_node *dn, const char *name,
u32 vd, char *value)
{
struct property *new_prop;
u32 *val;
int rc;
new_prop = kzalloc(sizeof(*new_prop), GFP_KERNEL);
if (!new_prop)
return -ENOMEM;
new_prop->name = kstrdup(name, GFP_KERNEL);
if (!new_prop->name) {
kfree(new_prop);
return -ENOMEM;
}
new_prop->length = vd;
new_prop->value = kzalloc(new_prop->length, GFP_KERNEL);
if (!new_prop->value) {
kfree(new_prop->name);
kfree(new_prop);
return -ENOMEM;
}
memcpy(new_prop->value, value, vd);
val = (u32 *)new_prop->value;
rc = cxl_update_properties(dn, new_prop);
pr_devel("%pOFn: update property (%s, length: %i, value: %#x)\n",
dn, name, vd, be32_to_cpu(*val));
if (rc) {
kfree(new_prop->name);
kfree(new_prop->value);
kfree(new_prop);
}
return rc;
}
static int update_node(__be32 phandle, s32 scope)
{
struct update_props_workarea *upwa;
struct device_node *dn;
int i, rc, ret;
char *prop_data;
char *buf;
int token;
u32 nprops;
u32 vd;
token = rtas_token("ibm,update-properties");
if (token == RTAS_UNKNOWN_SERVICE)
return -EINVAL;
buf = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
if (!buf)
return -ENOMEM;
dn = of_find_node_by_phandle(be32_to_cpu(phandle));
if (!dn) {
kfree(buf);
return -ENOENT;
}
upwa = (struct update_props_workarea *)&buf[0];
upwa->phandle = phandle;
do {
rc = rcall(token, buf, scope);
if (rc < 0)
break;
prop_data = buf + sizeof(*upwa);
nprops = be32_to_cpu(upwa->nprops);
if (*prop_data == 0) {
prop_data++;
vd = be32_to_cpu(*(__be32 *)prop_data);
prop_data += vd + sizeof(vd);
nprops--;
}
for (i = 0; i < nprops; i++) {
char *prop_name;
prop_name = prop_data;
prop_data += strlen(prop_name) + 1;
vd = be32_to_cpu(*(__be32 *)prop_data);
prop_data += sizeof(vd);
if ((vd != 0x00000000) && (vd != 0x80000000)) {
ret = update_property(dn, prop_name, vd,
prop_data);
if (ret)
pr_err("cxl: Could not update property %s - %i\n",
prop_name, ret);
prop_data += vd;
}
}
} while (rc == 1);
of_node_put(dn);
kfree(buf);
return rc;
}
static int update_devicetree(struct cxl *adapter, s32 scope)
{
struct update_nodes_workarea *unwa;
u32 action, node_count;
int token, rc, i;
__be32 *data, phandle;
char *buf;
token = rtas_token("ibm,update-nodes");
if (token == RTAS_UNKNOWN_SERVICE)
return -EINVAL;
buf = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
if (!buf)
return -ENOMEM;
unwa = (struct update_nodes_workarea *)&buf[0];
unwa->unit_address = cpu_to_be64(adapter->guest->handle);
do {
rc = rcall(token, buf, scope);
if (rc && rc != 1)
break;
data = (__be32 *)buf + 4;
while (be32_to_cpu(*data) & NODE_ACTION_MASK) {
action = be32_to_cpu(*data) & NODE_ACTION_MASK;
node_count = be32_to_cpu(*data) & NODE_COUNT_MASK;
pr_devel("device reconfiguration - action: %#x, nodes: %#x\n",
action, node_count);
data++;
for (i = 0; i < node_count; i++) {
phandle = *data++;
switch (action) {
case OPCODE_DELETE:
/* nothing to do */
break;
case OPCODE_UPDATE:
update_node(phandle, scope);
break;
case OPCODE_ADD:
/* nothing to do, just move pointer */
data++;
break;
}
}
}
} while (rc == 1);
kfree(buf);
return 0;
}
static int handle_image(struct cxl *adapter, int operation,
long (*fct)(u64, u64, u64, u64 *),
struct cxl_adapter_image *ai)
{
size_t mod, s_copy, len_chunk = 0;
struct ai_header *header = NULL;
unsigned int entries = 0, i;
void *dest, *from;
int rc = 0, need_header;
/* base adapter image header */
need_header = (ai->flags & CXL_AI_NEED_HEADER);
if (need_header) {
header = kzalloc(sizeof(struct ai_header), GFP_KERNEL);
if (!header)
return -ENOMEM;
header->version = cpu_to_be16(1);
header->vendor = cpu_to_be16(adapter->guest->vendor);
header->device = cpu_to_be16(adapter->guest->device);
header->subsystem_vendor = cpu_to_be16(adapter->guest->subsystem_vendor);
header->subsystem = cpu_to_be16(adapter->guest->subsystem);
header->image_offset = cpu_to_be64(CXL_AI_HEADER_SIZE);
header->image_length = cpu_to_be64(ai->len_image);
}
/* number of entries in the list */
len_chunk = ai->len_data;
if (need_header)
len_chunk += CXL_AI_HEADER_SIZE;
entries = len_chunk / CXL_AI_BUFFER_SIZE;
mod = len_chunk % CXL_AI_BUFFER_SIZE;
if (mod)
entries++;
if (entries > CXL_AI_MAX_ENTRIES) {
rc = -EINVAL;
goto err;
}
/* < -- MAX_CHUNK_SIZE = 4096 * 256 = 1048576 bytes -->
* chunk 0 ----------------------------------------------------
* | header | data |
* ----------------------------------------------------
* chunk 1 ----------------------------------------------------
* | data |
* ----------------------------------------------------
* ....
* chunk n ----------------------------------------------------
* | data |
* ----------------------------------------------------
*/
from = (void *) ai->data;
for (i = 0; i < entries; i++) {
dest = buffer[i];
s_copy = CXL_AI_BUFFER_SIZE;
if ((need_header) && (i == 0)) {
/* add adapter image header */
memcpy(buffer[i], header, sizeof(struct ai_header));
s_copy = CXL_AI_BUFFER_SIZE - CXL_AI_HEADER_SIZE;
dest += CXL_AI_HEADER_SIZE; /* image offset */
}
if ((i == (entries - 1)) && mod)
s_copy = mod;
/* copy data */
if (copy_from_user(dest, from, s_copy))
goto err;
/* fill in the list */
le[i].phys_addr = cpu_to_be64(virt_to_phys(buffer[i]));
le[i].len = cpu_to_be64(CXL_AI_BUFFER_SIZE);
if ((i == (entries - 1)) && mod)
le[i].len = cpu_to_be64(mod);
from += s_copy;
}
pr_devel("%s (op: %i, need header: %i, entries: %i, token: %#llx)\n",
__func__, operation, need_header, entries, continue_token);
/*
* download/validate the adapter image to the coherent
* platform facility
*/
rc = fct(adapter->guest->handle, virt_to_phys(le), entries,
&continue_token);
if (rc == 0) /* success of download/validation operation */
continue_token = 0;
err:
kfree(header);
return rc;
}
static int transfer_image(struct cxl *adapter, int operation,
struct cxl_adapter_image *ai)
{
int rc = 0;
int afu;
switch (operation) {
case DOWNLOAD_IMAGE:
rc = handle_image(adapter, operation,
&cxl_h_download_adapter_image, ai);
if (rc < 0) {
pr_devel("resetting adapter\n");
cxl_h_reset_adapter(adapter->guest->handle);
}
return rc;
case VALIDATE_IMAGE:
rc = handle_image(adapter, operation,
&cxl_h_validate_adapter_image, ai);
if (rc < 0) {
pr_devel("resetting adapter\n");
cxl_h_reset_adapter(adapter->guest->handle);
return rc;
}
if (rc == 0) {
pr_devel("remove current afu\n");
for (afu = 0; afu < adapter->slices; afu++)
cxl_guest_remove_afu(adapter->afu[afu]);
pr_devel("resetting adapter\n");
cxl_h_reset_adapter(adapter->guest->handle);
/* The entire image has now been
* downloaded and the validation has
* been successfully performed.
* After that, the partition should call
* ibm,update-nodes and
* ibm,update-properties to receive the
* current configuration
*/
rc = update_devicetree(adapter, DEVICE_SCOPE);
transfer = 1;
}
return rc;
}
return -EINVAL;
}
static long ioctl_transfer_image(struct cxl *adapter, int operation,
struct cxl_adapter_image __user *uai)
{
struct cxl_adapter_image ai;
pr_devel("%s\n", __func__);
if (copy_from_user(&ai, uai, sizeof(struct cxl_adapter_image)))
return -EFAULT;
/*
* Make sure reserved fields and bits are set to 0
*/
if (ai.reserved1 || ai.reserved2 || ai.reserved3 || ai.reserved4 ||
(ai.flags & ~CXL_AI_ALL))
return -EINVAL;
return transfer_image(adapter, operation, &ai);
}
static int device_open(struct inode *inode, struct file *file)
{
int adapter_num = CXL_DEVT_ADAPTER(inode->i_rdev);
struct cxl *adapter;
int rc = 0, i;
pr_devel("in %s\n", __func__);
BUG_ON(sizeof(struct ai_header) != CXL_AI_HEADER_SIZE);
/* Allows one process to open the device by using a semaphore */
if (down_interruptible(&sem) != 0)
return -EPERM;
if (!(adapter = get_cxl_adapter(adapter_num))) {
rc = -ENODEV;
goto err_unlock;
}
file->private_data = adapter;
continue_token = 0;
transfer = 0;
for (i = 0; i < CXL_AI_MAX_ENTRIES; i++)
buffer[i] = NULL;
/* aligned buffer containing list entries which describes up to
* 1 megabyte of data (256 entries of 4096 bytes each)
* Logical real address of buffer 0 - Buffer 0 length in bytes
* Logical real address of buffer 1 - Buffer 1 length in bytes
* Logical real address of buffer 2 - Buffer 2 length in bytes
* ....
* ....
* Logical real address of buffer N - Buffer N length in bytes
*/
le = (struct sg_list *)get_zeroed_page(GFP_KERNEL);
if (!le) {
rc = -ENOMEM;
goto err;
}
for (i = 0; i < CXL_AI_MAX_ENTRIES; i++) {
buffer[i] = (unsigned long *)get_zeroed_page(GFP_KERNEL);
if (!buffer[i]) {
rc = -ENOMEM;
goto err1;
}
}
return 0;
err1:
for (i = 0; i < CXL_AI_MAX_ENTRIES; i++) {
if (buffer[i])
free_page((unsigned long) buffer[i]);
}
if (le)
free_page((unsigned long) le);
err:
put_device(&adapter->dev);
err_unlock:
up(&sem);
return rc;
}
static long device_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
struct cxl *adapter = file->private_data;
pr_devel("in %s\n", __func__);
if (cmd == CXL_IOCTL_DOWNLOAD_IMAGE)
return ioctl_transfer_image(adapter,
DOWNLOAD_IMAGE,
(struct cxl_adapter_image __user *)arg);
else if (cmd == CXL_IOCTL_VALIDATE_IMAGE)
return ioctl_transfer_image(adapter,
VALIDATE_IMAGE,
(struct cxl_adapter_image __user *)arg);
else
return -EINVAL;
}
static int device_close(struct inode *inode, struct file *file)
{
struct cxl *adapter = file->private_data;
int i;
pr_devel("in %s\n", __func__);
for (i = 0; i < CXL_AI_MAX_ENTRIES; i++) {
if (buffer[i])
free_page((unsigned long) buffer[i]);
}
if (le)
free_page((unsigned long) le);
up(&sem);
put_device(&adapter->dev);
continue_token = 0;
/* reload the module */
if (transfer)
cxl_guest_reload_module(adapter);
else {
pr_devel("resetting adapter\n");
cxl_h_reset_adapter(adapter->guest->handle);
}
transfer = 0;
return 0;
}
static const struct file_operations fops = {
.owner = THIS_MODULE,
.open = device_open,
.unlocked_ioctl = device_ioctl,
.compat_ioctl = compat_ptr_ioctl,
.release = device_close,
};
void cxl_guest_remove_chardev(struct cxl *adapter)
{
cdev_del(&adapter->guest->cdev);
}
int cxl_guest_add_chardev(struct cxl *adapter)
{
dev_t devt;
int rc;
devt = MKDEV(MAJOR(cxl_get_dev()), CXL_CARD_MINOR(adapter));
cdev_init(&adapter->guest->cdev, &fops);
if ((rc = cdev_add(&adapter->guest->cdev, devt, 1))) {
dev_err(&adapter->dev,
"Unable to add chardev on adapter (card%i): %i\n",
adapter->adapter_num, rc);
goto err;
}
adapter->dev.devt = devt;
sema_init(&sem, 1);
err:
return rc;
}

1208
drivers/misc/cxl/guest.c
View File

File diff suppressed because it is too large Load Diff

643
drivers/misc/cxl/hcalls.c
View File

@ -1,643 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2015 IBM Corp.
*/
#include <linux/compiler.h>
#include <linux/types.h>
#include <linux/delay.h>
#include <asm/byteorder.h>
#include "hcalls.h"
#include "trace.h"
#define CXL_HCALL_TIMEOUT 60000
#define CXL_HCALL_TIMEOUT_DOWNLOAD 120000
#define H_ATTACH_CA_PROCESS 0x344
#define H_CONTROL_CA_FUNCTION 0x348
#define H_DETACH_CA_PROCESS 0x34C
#define H_COLLECT_CA_INT_INFO 0x350
#define H_CONTROL_CA_FAULTS 0x354
#define H_DOWNLOAD_CA_FUNCTION 0x35C
#define H_DOWNLOAD_CA_FACILITY 0x364
#define H_CONTROL_CA_FACILITY 0x368
#define H_CONTROL_CA_FUNCTION_RESET 1 /* perform a reset */
#define H_CONTROL_CA_FUNCTION_SUSPEND_PROCESS 2 /* suspend a process from being executed */
#define H_CONTROL_CA_FUNCTION_RESUME_PROCESS 3 /* resume a process to be executed */
#define H_CONTROL_CA_FUNCTION_READ_ERR_STATE 4 /* read the error state */
#define H_CONTROL_CA_FUNCTION_GET_AFU_ERR 5 /* collect the AFU error buffer */
#define H_CONTROL_CA_FUNCTION_GET_CONFIG 6 /* collect configuration record */
#define H_CONTROL_CA_FUNCTION_GET_DOWNLOAD_STATE 7 /* query to return download status */
#define H_CONTROL_CA_FUNCTION_TERMINATE_PROCESS 8 /* terminate the process before completion */
#define H_CONTROL_CA_FUNCTION_COLLECT_VPD 9 /* collect VPD */
#define H_CONTROL_CA_FUNCTION_GET_FUNCTION_ERR_INT 11 /* read the function-wide error data based on an interrupt */
#define H_CONTROL_CA_FUNCTION_ACK_FUNCTION_ERR_INT 12 /* acknowledge function-wide error data based on an interrupt */
#define H_CONTROL_CA_FUNCTION_GET_ERROR_LOG 13 /* retrieve the Platform Log ID (PLID) of an error log */
#define H_CONTROL_CA_FAULTS_RESPOND_PSL 1
#define H_CONTROL_CA_FAULTS_RESPOND_AFU 2
#define H_CONTROL_CA_FACILITY_RESET 1 /* perform a reset */
#define H_CONTROL_CA_FACILITY_COLLECT_VPD 2 /* collect VPD */
#define H_DOWNLOAD_CA_FACILITY_DOWNLOAD 1 /* download adapter image */
#define H_DOWNLOAD_CA_FACILITY_VALIDATE 2 /* validate adapter image */
#define _CXL_LOOP_HCALL(call, rc, retbuf, fn, ...) \
{ \
unsigned int delay, total_delay = 0; \
u64 token = 0; \
\
memset(retbuf, 0, sizeof(retbuf)); \
while (1) { \
rc = call(fn, retbuf, __VA_ARGS__, token); \
token = retbuf[0]; \
if (rc != H_BUSY && !H_IS_LONG_BUSY(rc)) \
break; \
\
if (rc == H_BUSY) \
delay = 10; \
else \
delay = get_longbusy_msecs(rc); \
\
total_delay += delay; \
if (total_delay > CXL_HCALL_TIMEOUT) { \
WARN(1, "Warning: Giving up waiting for CXL hcall " \
"%#x after %u msec\n", fn, total_delay); \
rc = H_BUSY; \
break; \
} \
msleep(delay); \
} \
}
#define CXL_H_WAIT_UNTIL_DONE(...) _CXL_LOOP_HCALL(plpar_hcall, __VA_ARGS__)
#define CXL_H9_WAIT_UNTIL_DONE(...) _CXL_LOOP_HCALL(plpar_hcall9, __VA_ARGS__)
#define _PRINT_MSG(rc, format, ...) \
{ \
if ((rc != H_SUCCESS) && (rc != H_CONTINUE)) \
pr_err(format, __VA_ARGS__); \
else \
pr_devel(format, __VA_ARGS__); \
} \
static char *afu_op_names[] = {
"UNKNOWN_OP", /* 0 undefined */
"RESET", /* 1 */
"SUSPEND_PROCESS", /* 2 */
"RESUME_PROCESS", /* 3 */
"READ_ERR_STATE", /* 4 */
"GET_AFU_ERR", /* 5 */
"GET_CONFIG", /* 6 */
"GET_DOWNLOAD_STATE", /* 7 */
"TERMINATE_PROCESS", /* 8 */
"COLLECT_VPD", /* 9 */
"UNKNOWN_OP", /* 10 undefined */
"GET_FUNCTION_ERR_INT", /* 11 */
"ACK_FUNCTION_ERR_INT", /* 12 */
"GET_ERROR_LOG", /* 13 */
};
static char *control_adapter_op_names[] = {
"UNKNOWN_OP", /* 0 undefined */
"RESET", /* 1 */
"COLLECT_VPD", /* 2 */
};
static char *download_op_names[] = {
"UNKNOWN_OP", /* 0 undefined */
"DOWNLOAD", /* 1 */
"VALIDATE", /* 2 */
};
static char *op_str(unsigned int op, char *name_array[], int array_len)
{
if (op >= array_len)
return "UNKNOWN_OP";
return name_array[op];
}
#define OP_STR(op, name_array) op_str(op, name_array, ARRAY_SIZE(name_array))
#define OP_STR_AFU(op) OP_STR(op, afu_op_names)
#define OP_STR_CONTROL_ADAPTER(op) OP_STR(op, control_adapter_op_names)
#define OP_STR_DOWNLOAD_ADAPTER(op) OP_STR(op, download_op_names)
long cxl_h_attach_process(u64 unit_address,
struct cxl_process_element_hcall *element,
u64 *process_token, u64 *mmio_addr, u64 *mmio_size)
{
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
long rc;
CXL_H_WAIT_UNTIL_DONE(rc, retbuf, H_ATTACH_CA_PROCESS, unit_address, virt_to_phys(element));
_PRINT_MSG(rc, "cxl_h_attach_process(%#.16llx, %#.16lx): %li\n",
unit_address, virt_to_phys(element), rc);
trace_cxl_hcall_attach(unit_address, virt_to_phys(element), retbuf[0], retbuf[1], retbuf[2], rc);
pr_devel("token: 0x%.8lx mmio_addr: 0x%lx mmio_size: 0x%lx\nProcess Element Structure:\n",
retbuf[0], retbuf[1], retbuf[2]);
cxl_dump_debug_buffer(element, sizeof(*element));
switch (rc) {
case H_SUCCESS: /* The process info is attached to the coherent platform function */
*process_token = retbuf[0];
if (mmio_addr)
*mmio_addr = retbuf[1];
if (mmio_size)
*mmio_size = retbuf[2];
return 0;
case H_PARAMETER: /* An incorrect parameter was supplied. */
case H_FUNCTION: /* The function is not supported. */
return -EINVAL;
case H_AUTHORITY: /* The partition does not have authority to perform this hcall */
case H_RESOURCE: /* The coherent platform function does not have enough additional resource to attach the process */
case H_HARDWARE: /* A hardware event prevented the attach operation */
case H_STATE: /* The coherent platform function is not in a valid state */
case H_BUSY:
return -EBUSY;
default:
WARN(1, "Unexpected return code: %lx", rc);
return -EINVAL;
}
}
/*
* cxl_h_detach_process - Detach a process element from a coherent
* platform function.
*/
long cxl_h_detach_process(u64 unit_address, u64 process_token)
{
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
long rc;
CXL_H_WAIT_UNTIL_DONE(rc, retbuf, H_DETACH_CA_PROCESS, unit_address, process_token);
_PRINT_MSG(rc, "cxl_h_detach_process(%#.16llx, 0x%.8llx): %li\n", unit_address, process_token, rc);
trace_cxl_hcall_detach(unit_address, process_token, rc);
switch (rc) {
case H_SUCCESS: /* The process was detached from the coherent platform function */
return 0;
case H_PARAMETER: /* An incorrect parameter was supplied. */
return -EINVAL;
case H_AUTHORITY: /* The partition does not have authority to perform this hcall */
case H_RESOURCE: /* The function has page table mappings for MMIO */
case H_HARDWARE: /* A hardware event prevented the detach operation */
case H_STATE: /* The coherent platform function is not in a valid state */
case H_BUSY:
return -EBUSY;
default:
WARN(1, "Unexpected return code: %lx", rc);
return -EINVAL;
}
}
/*
* cxl_h_control_function - This H_CONTROL_CA_FUNCTION hypervisor call allows
* the partition to manipulate or query
* certain coherent platform function behaviors.
*/
static long cxl_h_control_function(u64 unit_address, u64 op,
u64 p1, u64 p2, u64 p3, u64 p4, u64 *out)
{
unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
long rc;
CXL_H9_WAIT_UNTIL_DONE(rc, retbuf, H_CONTROL_CA_FUNCTION, unit_address, op, p1, p2, p3, p4);
_PRINT_MSG(rc, "cxl_h_control_function(%#.16llx, %s(%#llx, %#llx, %#llx, %#llx, R4: %#lx)): %li\n",
unit_address, OP_STR_AFU(op), p1, p2, p3, p4, retbuf[0], rc);
trace_cxl_hcall_control_function(unit_address, OP_STR_AFU(op), p1, p2, p3, p4, retbuf[0], rc);
switch (rc) {
case H_SUCCESS: /* The operation is completed for the coherent platform function */
if ((op == H_CONTROL_CA_FUNCTION_GET_FUNCTION_ERR_INT ||
op == H_CONTROL_CA_FUNCTION_READ_ERR_STATE ||
op == H_CONTROL_CA_FUNCTION_COLLECT_VPD))
*out = retbuf[0];
return 0;
case H_PARAMETER: /* An incorrect parameter was supplied. */
case H_FUNCTION: /* The function is not supported. */
case H_NOT_FOUND: /* The operation supplied was not valid */
case H_NOT_AVAILABLE: /* The operation cannot be performed because the AFU has not been downloaded */
case H_SG_LIST: /* An block list entry was invalid */
return -EINVAL;
case H_AUTHORITY: /* The partition does not have authority to perform this hcall */
case H_RESOURCE: /* The function has page table mappings for MMIO */
case H_HARDWARE: /* A hardware event prevented the attach operation */
case H_STATE: /* The coherent platform function is not in a valid state */
case H_BUSY:
return -EBUSY;
default:
WARN(1, "Unexpected return code: %lx", rc);
return -EINVAL;
}
}
/*
* cxl_h_reset_afu - Perform a reset to the coherent platform function.
*/
long cxl_h_reset_afu(u64 unit_address)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_RESET,
0, 0, 0, 0,
NULL);
}
/*
* cxl_h_suspend_process - Suspend a process from being executed
* Parameter1 = process-token as returned from H_ATTACH_CA_PROCESS when
* process was attached.
*/
long cxl_h_suspend_process(u64 unit_address, u64 process_token)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_SUSPEND_PROCESS,
process_token, 0, 0, 0,
NULL);
}
/*
* cxl_h_resume_process - Resume a process to be executed
* Parameter1 = process-token as returned from H_ATTACH_CA_PROCESS when
* process was attached.
*/
long cxl_h_resume_process(u64 unit_address, u64 process_token)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_RESUME_PROCESS,
process_token, 0, 0, 0,
NULL);
}
/*
* cxl_h_read_error_state - Checks the error state of the coherent
* platform function.
* R4 contains the error state
*/
long cxl_h_read_error_state(u64 unit_address, u64 *state)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_READ_ERR_STATE,
0, 0, 0, 0,
state);
}
/*
* cxl_h_get_afu_err - collect the AFU error buffer
* Parameter1 = byte offset into error buffer to retrieve, valid values
* are between 0 and (ibm,error-buffer-size - 1)
* Parameter2 = 4K aligned real address of error buffer, to be filled in
* Parameter3 = length of error buffer, valid values are 4K or less
*/
long cxl_h_get_afu_err(u64 unit_address, u64 offset,
u64 buf_address, u64 len)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_GET_AFU_ERR,
offset, buf_address, len, 0,
NULL);
}
/*
* cxl_h_get_config - collect configuration record for the
* coherent platform function
* Parameter1 = # of configuration record to retrieve, valid values are
* between 0 and (ibm,#config-records - 1)
* Parameter2 = byte offset into configuration record to retrieve,
* valid values are between 0 and (ibm,config-record-size - 1)
* Parameter3 = 4K aligned real address of configuration record buffer,
* to be filled in
* Parameter4 = length of configuration buffer, valid values are 4K or less
*/
long cxl_h_get_config(u64 unit_address, u64 cr_num, u64 offset,
u64 buf_address, u64 len)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_GET_CONFIG,
cr_num, offset, buf_address, len,
NULL);
}
/*
* cxl_h_terminate_process - Terminate the process before completion
* Parameter1 = process-token as returned from H_ATTACH_CA_PROCESS when
* process was attached.
*/
long cxl_h_terminate_process(u64 unit_address, u64 process_token)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_TERMINATE_PROCESS,
process_token, 0, 0, 0,
NULL);
}
/*
* cxl_h_collect_vpd - Collect VPD for the coherent platform function.
* Parameter1 = # of VPD record to retrieve, valid values are between 0
* and (ibm,#config-records - 1).
* Parameter2 = 4K naturally aligned real buffer containing block
* list entries
* Parameter3 = number of block list entries in the block list, valid
* values are between 0 and 256
*/
long cxl_h_collect_vpd(u64 unit_address, u64 record, u64 list_address,
u64 num, u64 *out)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_COLLECT_VPD,
record, list_address, num, 0,
out);
}
/*
* cxl_h_get_fn_error_interrupt - Read the function-wide error data based on an interrupt
*/
long cxl_h_get_fn_error_interrupt(u64 unit_address, u64 *reg)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_GET_FUNCTION_ERR_INT,
0, 0, 0, 0, reg);
}
/*
* cxl_h_ack_fn_error_interrupt - Acknowledge function-wide error data
* based on an interrupt
* Parameter1 = value to write to the function-wide error interrupt register
*/
long cxl_h_ack_fn_error_interrupt(u64 unit_address, u64 value)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_ACK_FUNCTION_ERR_INT,
value, 0, 0, 0,
NULL);
}
/*
* cxl_h_get_error_log - Retrieve the Platform Log ID (PLID) of
* an error log
*/
long cxl_h_get_error_log(u64 unit_address, u64 value)
{
return cxl_h_control_function(unit_address,
H_CONTROL_CA_FUNCTION_GET_ERROR_LOG,
0, 0, 0, 0,
NULL);
}
/*
* cxl_h_collect_int_info - Collect interrupt info about a coherent
* platform function after an interrupt occurred.
*/
long cxl_h_collect_int_info(u64 unit_address, u64 process_token,
struct cxl_irq_info *info)
{
long rc;
BUG_ON(sizeof(*info) != sizeof(unsigned long[PLPAR_HCALL9_BUFSIZE]));
rc = plpar_hcall9(H_COLLECT_CA_INT_INFO, (unsigned long *) info,
unit_address, process_token);
_PRINT_MSG(rc, "cxl_h_collect_int_info(%#.16llx, 0x%llx): %li\n",
unit_address, process_token, rc);
trace_cxl_hcall_collect_int_info(unit_address, process_token, rc);
switch (rc) {
case H_SUCCESS: /* The interrupt info is returned in return registers. */
pr_devel("dsisr:%#llx, dar:%#llx, dsr:%#llx, pid_tid:%#llx, afu_err:%#llx, errstat:%#llx\n",
info->dsisr, info->dar, info->dsr, info->reserved,
info->afu_err, info->errstat);
return 0;
case H_PARAMETER: /* An incorrect parameter was supplied. */
return -EINVAL;
case H_AUTHORITY: /* The partition does not have authority to perform this hcall. */
case H_HARDWARE: /* A hardware event prevented the collection of the interrupt info.*/
case H_STATE: /* The coherent platform function is not in a valid state to collect interrupt info. */
return -EBUSY;
default:
WARN(1, "Unexpected return code: %lx", rc);
return -EINVAL;
}
}
/*
* cxl_h_control_faults - Control the operation of a coherent platform
* function after a fault occurs.
*
* Parameters
* control-mask: value to control the faults
* looks like PSL_TFC_An shifted >> 32
* reset-mask: mask to control reset of function faults
* Set reset_mask = 1 to reset PSL errors
*/
long cxl_h_control_faults(u64 unit_address, u64 process_token,
u64 control_mask, u64 reset_mask)
{
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
long rc;
memset(retbuf, 0, sizeof(retbuf));
rc = plpar_hcall(H_CONTROL_CA_FAULTS, retbuf, unit_address,
H_CONTROL_CA_FAULTS_RESPOND_PSL, process_token,
control_mask, reset_mask);
_PRINT_MSG(rc, "cxl_h_control_faults(%#.16llx, 0x%llx, %#llx, %#llx): %li (%#lx)\n",
unit_address, process_token, control_mask, reset_mask,
rc, retbuf[0]);
trace_cxl_hcall_control_faults(unit_address, process_token,
control_mask, reset_mask, retbuf[0], rc);
switch (rc) {
case H_SUCCESS: /* Faults were successfully controlled for the function. */
return 0;
case H_PARAMETER: /* An incorrect parameter was supplied. */
return -EINVAL;
case H_HARDWARE: /* A hardware event prevented the control of faults. */
case H_STATE: /* The function was in an invalid state. */
case H_AUTHORITY: /* The partition does not have authority to perform this hcall; the coherent platform facilities may need to be licensed. */
return -EBUSY;
case H_FUNCTION: /* The function is not supported */
case H_NOT_FOUND: /* The operation supplied was not valid */
return -EINVAL;
default:
WARN(1, "Unexpected return code: %lx", rc);
return -EINVAL;
}
}
/*
* cxl_h_control_facility - This H_CONTROL_CA_FACILITY hypervisor call
* allows the partition to manipulate or query
* certain coherent platform facility behaviors.
*/
static long cxl_h_control_facility(u64 unit_address, u64 op,
u64 p1, u64 p2, u64 p3, u64 p4, u64 *out)
{
unsigned long retbuf[PLPAR_HCALL9_BUFSIZE];
long rc;
CXL_H9_WAIT_UNTIL_DONE(rc, retbuf, H_CONTROL_CA_FACILITY, unit_address, op, p1, p2, p3, p4);
_PRINT_MSG(rc, "cxl_h_control_facility(%#.16llx, %s(%#llx, %#llx, %#llx, %#llx, R4: %#lx)): %li\n",
unit_address, OP_STR_CONTROL_ADAPTER(op), p1, p2, p3, p4, retbuf[0], rc);
trace_cxl_hcall_control_facility(unit_address, OP_STR_CONTROL_ADAPTER(op), p1, p2, p3, p4, retbuf[0], rc);
switch (rc) {
case H_SUCCESS: /* The operation is completed for the coherent platform facility */
if (op == H_CONTROL_CA_FACILITY_COLLECT_VPD)
*out = retbuf[0];
return 0;
case H_PARAMETER: /* An incorrect parameter was supplied. */
case H_FUNCTION: /* The function is not supported. */
case H_NOT_FOUND: /* The operation supplied was not valid */
case H_NOT_AVAILABLE: /* The operation cannot be performed because the AFU has not been downloaded */
case H_SG_LIST: /* An block list entry was invalid */
return -EINVAL;
case H_AUTHORITY: /* The partition does not have authority to perform this hcall */
case H_RESOURCE: /* The function has page table mappings for MMIO */
case H_HARDWARE: /* A hardware event prevented the attach operation */
case H_STATE: /* The coherent platform facility is not in a valid state */
case H_BUSY:
return -EBUSY;
default:
WARN(1, "Unexpected return code: %lx", rc);
return -EINVAL;
}
}
/*
* cxl_h_reset_adapter - Perform a reset to the coherent platform facility.
*/
long cxl_h_reset_adapter(u64 unit_address)
{
return cxl_h_control_facility(unit_address,
H_CONTROL_CA_FACILITY_RESET,
0, 0, 0, 0,
NULL);
}
/*
* cxl_h_collect_vpd - Collect VPD for the coherent platform function.
* Parameter1 = 4K naturally aligned real buffer containing block
* list entries
* Parameter2 = number of block list entries in the block list, valid
* values are between 0 and 256
*/
long cxl_h_collect_vpd_adapter(u64 unit_address, u64 list_address,
u64 num, u64 *out)
{
return cxl_h_control_facility(unit_address,
H_CONTROL_CA_FACILITY_COLLECT_VPD,
list_address, num, 0, 0,
out);
}
/*
* cxl_h_download_facility - This H_DOWNLOAD_CA_FACILITY
* hypervisor call provide platform support for
* downloading a base adapter image to the coherent
* platform facility, and for validating the entire
* image after the download.
* Parameters
* op: operation to perform to the coherent platform function
* Download: operation = 1, the base image in the coherent platform
* facility is first erased, and then
* programmed using the image supplied
* in the scatter/gather list.
* Validate: operation = 2, the base image in the coherent platform
* facility is compared with the image
* supplied in the scatter/gather list.
* list_address: 4K naturally aligned real buffer containing
* scatter/gather list entries.
* num: number of block list entries in the scatter/gather list.
*/
static long cxl_h_download_facility(u64 unit_address, u64 op,
u64 list_address, u64 num,
u64 *out)
{
unsigned long retbuf[PLPAR_HCALL_BUFSIZE];
unsigned int delay, total_delay = 0;
u64 token = 0;
long rc;
if (*out != 0)
token = *out;
memset(retbuf, 0, sizeof(retbuf));
while (1) {
rc = plpar_hcall(H_DOWNLOAD_CA_FACILITY, retbuf,
unit_address, op, list_address, num,
token);
token = retbuf[0];
if (rc != H_BUSY && !H_IS_LONG_BUSY(rc))
break;
if (rc != H_BUSY) {
delay = get_longbusy_msecs(rc);
total_delay += delay;
if (total_delay > CXL_HCALL_TIMEOUT_DOWNLOAD) {
WARN(1, "Warning: Giving up waiting for CXL hcall "
"%#x after %u msec\n",
H_DOWNLOAD_CA_FACILITY, total_delay);
rc = H_BUSY;
break;
}
msleep(delay);
}
}
_PRINT_MSG(rc, "cxl_h_download_facility(%#.16llx, %s(%#llx, %#llx), %#lx): %li\n",
unit_address, OP_STR_DOWNLOAD_ADAPTER(op), list_address, num, retbuf[0], rc);
trace_cxl_hcall_download_facility(unit_address, OP_STR_DOWNLOAD_ADAPTER(op), list_address, num, retbuf[0], rc);
switch (rc) {
case H_SUCCESS: /* The operation is completed for the coherent platform facility */
return 0;
case H_PARAMETER: /* An incorrect parameter was supplied */
case H_FUNCTION: /* The function is not supported. */
case H_SG_LIST: /* An block list entry was invalid */
case H_BAD_DATA: /* Image verification failed */
return -EINVAL;
case H_AUTHORITY: /* The partition does not have authority to perform this hcall */
case H_RESOURCE: /* The function has page table mappings for MMIO */
case H_HARDWARE: /* A hardware event prevented the attach operation */
case H_STATE: /* The coherent platform facility is not in a valid state */
case H_BUSY:
return -EBUSY;
case H_CONTINUE:
*out = retbuf[0];
return 1; /* More data is needed for the complete image */
default:
WARN(1, "Unexpected return code: %lx", rc);
return -EINVAL;
}
}
/*
* cxl_h_download_adapter_image - Download the base image to the coherent
* platform facility.
*/
long cxl_h_download_adapter_image(u64 unit_address,
u64 list_address, u64 num,
u64 *out)
{
return cxl_h_download_facility(unit_address,
H_DOWNLOAD_CA_FACILITY_DOWNLOAD,
list_address, num, out);
}
/*
* cxl_h_validate_adapter_image - Validate the base image in the coherent
* platform facility.
*/
long cxl_h_validate_adapter_image(u64 unit_address,
u64 list_address, u64 num,
u64 *out)
{
return cxl_h_download_facility(unit_address,
H_DOWNLOAD_CA_FACILITY_VALIDATE,
list_address, num, out);
}

200
drivers/misc/cxl/hcalls.h
View File

@ -1,200 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright 2015 IBM Corp.
*/
#ifndef _HCALLS_H
#define _HCALLS_H
#include <linux/types.h>
#include <asm/byteorder.h>
#include <asm/hvcall.h>
#include "cxl.h"
#define SG_BUFFER_SIZE 4096
#define SG_MAX_ENTRIES 256
struct sg_list {
u64 phys_addr;
u64 len;
};
/*
* This is straight out of PAPR, but replacing some of the compound fields with
* a single field, where they were identical to the register layout.
*
* The 'flags' parameter regroups the various bit-fields
*/
#define CXL_PE_CSRP_VALID (1ULL << 63)
#define CXL_PE_PROBLEM_STATE (1ULL << 62)
#define CXL_PE_SECONDARY_SEGMENT_TBL_SRCH (1ULL << 61)
#define CXL_PE_TAGS_ACTIVE (1ULL << 60)
#define CXL_PE_USER_STATE (1ULL << 59)
#define CXL_PE_TRANSLATION_ENABLED (1ULL << 58)
#define CXL_PE_64_BIT (1ULL << 57)
#define CXL_PE_PRIVILEGED_PROCESS (1ULL << 56)
#define CXL_PROCESS_ELEMENT_VERSION 1
struct cxl_process_element_hcall {
__be64 version;
__be64 flags;
u8 reserved0[12];
__be32 pslVirtualIsn;
u8 applicationVirtualIsnBitmap[256];
u8 reserved1[144];
struct cxl_process_element_common common;
u8 reserved4[12];
} __packed;
#define H_STATE_NORMAL 1
#define H_STATE_DISABLE 2
#define H_STATE_TEMP_UNAVAILABLE 3
#define H_STATE_PERM_UNAVAILABLE 4
/* NOTE: element must be a logical real address, and must be pinned */
long cxl_h_attach_process(u64 unit_address, struct cxl_process_element_hcall *element,
u64 *process_token, u64 *mmio_addr, u64 *mmio_size);
/**
* cxl_h_detach_process - Detach a process element from a coherent
* platform function.
*/
long cxl_h_detach_process(u64 unit_address, u64 process_token);
/**
* cxl_h_reset_afu - Perform a reset to the coherent platform function.
*/
long cxl_h_reset_afu(u64 unit_address);
/**
* cxl_h_suspend_process - Suspend a process from being executed
* Parameter1 = process-token as returned from H_ATTACH_CA_PROCESS when
* process was attached.
*/
long cxl_h_suspend_process(u64 unit_address, u64 process_token);
/**
* cxl_h_resume_process - Resume a process to be executed
* Parameter1 = process-token as returned from H_ATTACH_CA_PROCESS when
* process was attached.
*/
long cxl_h_resume_process(u64 unit_address, u64 process_token);
/**
* cxl_h_read_error_state - Reads the error state of the coherent
* platform function.
* R4 contains the error state
*/
long cxl_h_read_error_state(u64 unit_address, u64 *state);
/**
* cxl_h_get_afu_err - collect the AFU error buffer
* Parameter1 = byte offset into error buffer to retrieve, valid values
* are between 0 and (ibm,error-buffer-size - 1)
* Parameter2 = 4K aligned real address of error buffer, to be filled in
* Parameter3 = length of error buffer, valid values are 4K or less
*/
long cxl_h_get_afu_err(u64 unit_address, u64 offset, u64 buf_address, u64 len);
/**
* cxl_h_get_config - collect configuration record for the
* coherent platform function
* Parameter1 = # of configuration record to retrieve, valid values are
* between 0 and (ibm,#config-records - 1)
* Parameter2 = byte offset into configuration record to retrieve,
* valid values are between 0 and (ibm,config-record-size - 1)
* Parameter3 = 4K aligned real address of configuration record buffer,
* to be filled in
* Parameter4 = length of configuration buffer, valid values are 4K or less
*/
long cxl_h_get_config(u64 unit_address, u64 cr_num, u64 offset,
u64 buf_address, u64 len);
/**
* cxl_h_terminate_process - Terminate the process before completion
* Parameter1 = process-token as returned from H_ATTACH_CA_PROCESS when
* process was attached.
*/
long cxl_h_terminate_process(u64 unit_address, u64 process_token);
/**
* cxl_h_collect_vpd - Collect VPD for the coherent platform function.
* Parameter1 = # of VPD record to retrieve, valid values are between 0
* and (ibm,#config-records - 1).
* Parameter2 = 4K naturally aligned real buffer containing block
* list entries
* Parameter3 = number of block list entries in the block list, valid
* values are between 0 and 256
*/
long cxl_h_collect_vpd(u64 unit_address, u64 record, u64 list_address,
u64 num, u64 *out);
/**
* cxl_h_get_fn_error_interrupt - Read the function-wide error data based on an interrupt
*/
long cxl_h_get_fn_error_interrupt(u64 unit_address, u64 *reg);
/**
* cxl_h_ack_fn_error_interrupt - Acknowledge function-wide error data
* based on an interrupt
* Parameter1 = value to write to the function-wide error interrupt register
*/
long cxl_h_ack_fn_error_interrupt(u64 unit_address, u64 value);
/**
* cxl_h_get_error_log - Retrieve the Platform Log ID (PLID) of
* an error log
*/
long cxl_h_get_error_log(u64 unit_address, u64 value);
/**
* cxl_h_collect_int_info - Collect interrupt info about a coherent
* platform function after an interrupt occurred.
*/
long cxl_h_collect_int_info(u64 unit_address, u64 process_token,
struct cxl_irq_info *info);
/**
* cxl_h_control_faults - Control the operation of a coherent platform
* function after a fault occurs.
*
* Parameters
* control-mask: value to control the faults
* looks like PSL_TFC_An shifted >> 32
* reset-mask: mask to control reset of function faults
* Set reset_mask = 1 to reset PSL errors
*/
long cxl_h_control_faults(u64 unit_address, u64 process_token,
u64 control_mask, u64 reset_mask);
/**
* cxl_h_reset_adapter - Perform a reset to the coherent platform facility.
*/
long cxl_h_reset_adapter(u64 unit_address);
/**
* cxl_h_collect_vpd - Collect VPD for the coherent platform function.
* Parameter1 = 4K naturally aligned real buffer containing block
* list entries
* Parameter2 = number of block list entries in the block list, valid
* values are between 0 and 256
*/
long cxl_h_collect_vpd_adapter(u64 unit_address, u64 list_address,
u64 num, u64 *out);
/**
* cxl_h_download_adapter_image - Download the base image to the coherent
* platform facility.
*/
long cxl_h_download_adapter_image(u64 unit_address,
u64 list_address, u64 num,
u64 *out);
/**
* cxl_h_validate_adapter_image - Validate the base image in the coherent
* platform facility.
*/
long cxl_h_validate_adapter_image(u64 unit_address,
u64 list_address, u64 num,
u64 *out);
#endif /* _HCALLS_H */

450
drivers/misc/cxl/irq.c
View File

@ -1,450 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/interrupt.h>
#include <linux/irqdomain.h>
#include <linux/workqueue.h>
#include <linux/sched.h>
#include <linux/wait.h>
#include <linux/slab.h>
#include <linux/pid.h>
#include <asm/cputable.h>
#include <misc/cxl-base.h>
#include "cxl.h"
#include "trace.h"
static int afu_irq_range_start(void)
{
if (cpu_has_feature(CPU_FTR_HVMODE))
return 1;
return 0;
}
static irqreturn_t schedule_cxl_fault(struct cxl_context *ctx, u64 dsisr, u64 dar)
{
ctx->dsisr = dsisr;
ctx->dar = dar;
schedule_work(&ctx->fault_work);
return IRQ_HANDLED;
}
irqreturn_t cxl_irq_psl9(int irq, struct cxl_context *ctx, struct cxl_irq_info *irq_info)
{
u64 dsisr, dar;
dsisr = irq_info->dsisr;
dar = irq_info->dar;
trace_cxl_psl9_irq(ctx, irq, dsisr, dar);
pr_devel("CXL interrupt %i for afu pe: %i DSISR: %#llx DAR: %#llx\n", irq, ctx->pe, dsisr, dar);
if (dsisr & CXL_PSL9_DSISR_An_TF) {
pr_devel("CXL interrupt: Scheduling translation fault handling for later (pe: %i)\n", ctx->pe);
return schedule_cxl_fault(ctx, dsisr, dar);
}
if (dsisr & CXL_PSL9_DSISR_An_PE)
return cxl_ops->handle_psl_slice_error(ctx, dsisr,
irq_info->errstat);
if (dsisr & CXL_PSL9_DSISR_An_AE) {
pr_devel("CXL interrupt: AFU Error 0x%016llx\n", irq_info->afu_err);
if (ctx->pending_afu_err) {
/*
* This shouldn't happen - the PSL treats these errors
* as fatal and will have reset the AFU, so there's not
* much point buffering multiple AFU errors.
* OTOH if we DO ever see a storm of these come in it's
* probably best that we log them somewhere:
*/
dev_err_ratelimited(&ctx->afu->dev, "CXL AFU Error undelivered to pe %i: 0x%016llx\n",
ctx->pe, irq_info->afu_err);
} else {
spin_lock(&ctx->lock);
ctx->afu_err = irq_info->afu_err;
ctx->pending_afu_err = 1;
spin_unlock(&ctx->lock);
wake_up_all(&ctx->wq);
}
cxl_ops->ack_irq(ctx, CXL_PSL_TFC_An_A, 0);
return IRQ_HANDLED;
}
if (dsisr & CXL_PSL9_DSISR_An_OC)
pr_devel("CXL interrupt: OS Context Warning\n");
WARN(1, "Unhandled CXL PSL IRQ\n");
return IRQ_HANDLED;
}
irqreturn_t cxl_irq_psl8(int irq, struct cxl_context *ctx, struct cxl_irq_info *irq_info)
{
u64 dsisr, dar;
dsisr = irq_info->dsisr;
dar = irq_info->dar;
trace_cxl_psl_irq(ctx, irq, dsisr, dar);
pr_devel("CXL interrupt %i for afu pe: %i DSISR: %#llx DAR: %#llx\n", irq, ctx->pe, dsisr, dar);
if (dsisr & CXL_PSL_DSISR_An_DS) {
/*
* We don't inherently need to sleep to handle this, but we do
* need to get a ref to the task's mm, which we can't do from
* irq context without the potential for a deadlock since it
* takes the task_lock. An alternate option would be to keep a
* reference to the task's mm the entire time it has cxl open,
* but to do that we need to solve the issue where we hold a
* ref to the mm, but the mm can hold a ref to the fd after an
* mmap preventing anything from being cleaned up.
*/
pr_devel("Scheduling segment miss handling for later pe: %i\n", ctx->pe);
return schedule_cxl_fault(ctx, dsisr, dar);
}
if (dsisr & CXL_PSL_DSISR_An_M)
pr_devel("CXL interrupt: PTE not found\n");
if (dsisr & CXL_PSL_DSISR_An_P)
pr_devel("CXL interrupt: Storage protection violation\n");
if (dsisr & CXL_PSL_DSISR_An_A)
pr_devel("CXL interrupt: AFU lock access to write through or cache inhibited storage\n");
if (dsisr & CXL_PSL_DSISR_An_S)
pr_devel("CXL interrupt: Access was afu_wr or afu_zero\n");
if (dsisr & CXL_PSL_DSISR_An_K)
pr_devel("CXL interrupt: Access not permitted by virtual page class key protection\n");
if (dsisr & CXL_PSL_DSISR_An_DM) {
/*
* In some cases we might be able to handle the fault
* immediately if hash_page would succeed, but we still need
* the task's mm, which as above we can't get without a lock
*/
pr_devel("Scheduling page fault handling for later pe: %i\n", ctx->pe);
return schedule_cxl_fault(ctx, dsisr, dar);
}
if (dsisr & CXL_PSL_DSISR_An_ST)
WARN(1, "CXL interrupt: Segment Table PTE not found\n");
if (dsisr & CXL_PSL_DSISR_An_UR)
pr_devel("CXL interrupt: AURP PTE not found\n");
if (dsisr & CXL_PSL_DSISR_An_PE)
return cxl_ops->handle_psl_slice_error(ctx, dsisr,
irq_info->errstat);
if (dsisr & CXL_PSL_DSISR_An_AE) {
pr_devel("CXL interrupt: AFU Error 0x%016llx\n", irq_info->afu_err);
if (ctx->pending_afu_err) {
/*
* This shouldn't happen - the PSL treats these errors
* as fatal and will have reset the AFU, so there's not
* much point buffering multiple AFU errors.
* OTOH if we DO ever see a storm of these come in it's
* probably best that we log them somewhere:
*/
dev_err_ratelimited(&ctx->afu->dev, "CXL AFU Error "
"undelivered to pe %i: 0x%016llx\n",
ctx->pe, irq_info->afu_err);
} else {
spin_lock(&ctx->lock);
ctx->afu_err = irq_info->afu_err;
ctx->pending_afu_err = true;
spin_unlock(&ctx->lock);
wake_up_all(&ctx->wq);
}
cxl_ops->ack_irq(ctx, CXL_PSL_TFC_An_A, 0);
return IRQ_HANDLED;
}
if (dsisr & CXL_PSL_DSISR_An_OC)
pr_devel("CXL interrupt: OS Context Warning\n");
WARN(1, "Unhandled CXL PSL IRQ\n");
return IRQ_HANDLED;
}
static irqreturn_t cxl_irq_afu(int irq, void *data)
{
struct cxl_context *ctx = data;
irq_hw_number_t hwirq = irqd_to_hwirq(irq_get_irq_data(irq));
int irq_off, afu_irq = 0;
__u16 range;
int r;
/*
* Look for the interrupt number.
* On bare-metal, we know range 0 only contains the PSL
* interrupt so we could start counting at range 1 and initialize
* afu_irq at 1.
* In a guest, range 0 also contains AFU interrupts, so it must
* be counted for. Therefore we initialize afu_irq at 0 to take into
* account the PSL interrupt.
*
* For code-readability, it just seems easier to go over all
* the ranges on bare-metal and guest. The end result is the same.
*/
for (r = 0; r < CXL_IRQ_RANGES; r++) {
irq_off = hwirq - ctx->irqs.offset[r];
range = ctx->irqs.range[r];
if (irq_off >= 0 && irq_off < range) {
afu_irq += irq_off;
break;
}
afu_irq += range;
}
if (unlikely(r >= CXL_IRQ_RANGES)) {
WARN(1, "Received AFU IRQ out of range for pe %i (virq %i hwirq %lx)\n",
ctx->pe, irq, hwirq);
return IRQ_HANDLED;
}
trace_cxl_afu_irq(ctx, afu_irq, irq, hwirq);
pr_devel("Received AFU interrupt %i for pe: %i (virq %i hwirq %lx)\n",
afu_irq, ctx->pe, irq, hwirq);
if (unlikely(!ctx->irq_bitmap)) {
WARN(1, "Received AFU IRQ for context with no IRQ bitmap\n");
return IRQ_HANDLED;
}
spin_lock(&ctx->lock);
set_bit(afu_irq - 1, ctx->irq_bitmap);
ctx->pending_irq = true;
spin_unlock(&ctx->lock);
wake_up_all(&ctx->wq);
return IRQ_HANDLED;
}
unsigned int cxl_map_irq(struct cxl *adapter, irq_hw_number_t hwirq,
irq_handler_t handler, void *cookie, const char *name)
{
unsigned int virq;
int result;
/* IRQ Domain? */
virq = irq_create_mapping(NULL, hwirq);
if (!virq) {
dev_warn(&adapter->dev, "cxl_map_irq: irq_create_mapping failed\n");
return 0;
}
if (cxl_ops->setup_irq)
cxl_ops->setup_irq(adapter, hwirq, virq);
pr_devel("hwirq %#lx mapped to virq %u\n", hwirq, virq);
result = request_irq(virq, handler, 0, name, cookie);
if (result) {
dev_warn(&adapter->dev, "cxl_map_irq: request_irq failed: %i\n", result);
return 0;
}
return virq;
}
void cxl_unmap_irq(unsigned int virq, void *cookie)
{
free_irq(virq, cookie);
}
int cxl_register_one_irq(struct cxl *adapter,
irq_handler_t handler,
void *cookie,
irq_hw_number_t *dest_hwirq,
unsigned int *dest_virq,
const char *name)
{
int hwirq, virq;
if ((hwirq = cxl_ops->alloc_one_irq(adapter)) < 0)
return hwirq;
if (!(virq = cxl_map_irq(adapter, hwirq, handler, cookie, name)))
goto err;
*dest_hwirq = hwirq;
*dest_virq = virq;
return 0;
err:
cxl_ops->release_one_irq(adapter, hwirq);
return -ENOMEM;
}
void afu_irq_name_free(struct cxl_context *ctx)
{
struct cxl_irq_name *irq_name, *tmp;
list_for_each_entry_safe(irq_name, tmp, &ctx->irq_names, list) {
kfree(irq_name->name);
list_del(&irq_name->list);
kfree(irq_name);
}
}
int afu_allocate_irqs(struct cxl_context *ctx, u32 count)
{
int rc, r, i, j = 1;
struct cxl_irq_name *irq_name;
int alloc_count;
/*
* In native mode, range 0 is reserved for the multiplexed
* PSL interrupt. It has been allocated when the AFU was initialized.
*
* In a guest, the PSL interrupt is not mutliplexed, but per-context,
* and is the first interrupt from range 0. It still needs to be
* allocated, so bump the count by one.
*/
if (cpu_has_feature(CPU_FTR_HVMODE))
alloc_count = count;
else
alloc_count = count + 1;
if ((rc = cxl_ops->alloc_irq_ranges(&ctx->irqs, ctx->afu->adapter,
alloc_count)))
return rc;
if (cpu_has_feature(CPU_FTR_HVMODE)) {
/* Multiplexed PSL Interrupt */
ctx->irqs.offset[0] = ctx->afu->native->psl_hwirq;
ctx->irqs.range[0] = 1;
}
ctx->irq_count = count;
ctx->irq_bitmap = bitmap_zalloc(count, GFP_KERNEL);
if (!ctx->irq_bitmap)
goto out;
/*
* Allocate names first. If any fail, bail out before allocating
* actual hardware IRQs.
*/
for (r = afu_irq_range_start(); r < CXL_IRQ_RANGES; r++) {
for (i = 0; i < ctx->irqs.range[r]; i++) {
irq_name = kmalloc(sizeof(struct cxl_irq_name),
GFP_KERNEL);
if (!irq_name)
goto out;
irq_name->name = kasprintf(GFP_KERNEL, "cxl-%s-pe%i-%i",
dev_name(&ctx->afu->dev),
ctx->pe, j);
if (!irq_name->name) {
kfree(irq_name);
goto out;
}
/* Add to tail so next look get the correct order */
list_add_tail(&irq_name->list, &ctx->irq_names);
j++;
}
}
return 0;
out:
cxl_ops->release_irq_ranges(&ctx->irqs, ctx->afu->adapter);
bitmap_free(ctx->irq_bitmap);
afu_irq_name_free(ctx);
return -ENOMEM;
}
static void afu_register_hwirqs(struct cxl_context *ctx)
{
irq_hw_number_t hwirq;
struct cxl_irq_name *irq_name;
int r, i;
irqreturn_t (*handler)(int irq, void *data);
/* We've allocated all memory now, so let's do the irq allocations */
irq_name = list_first_entry(&ctx->irq_names, struct cxl_irq_name, list);
for (r = afu_irq_range_start(); r < CXL_IRQ_RANGES; r++) {
hwirq = ctx->irqs.offset[r];
for (i = 0; i < ctx->irqs.range[r]; hwirq++, i++) {
if (r == 0 && i == 0)
/*
* The very first interrupt of range 0 is
* always the PSL interrupt, but we only
* need to connect a handler for guests,
* because there's one PSL interrupt per
* context.
* On bare-metal, the PSL interrupt is
* multiplexed and was setup when the AFU
* was configured.
*/
handler = cxl_ops->psl_interrupt;
else
handler = cxl_irq_afu;
cxl_map_irq(ctx->afu->adapter, hwirq, handler, ctx,
irq_name->name);
irq_name = list_next_entry(irq_name, list);
}
}
}
int afu_register_irqs(struct cxl_context *ctx, u32 count)
{
int rc;
rc = afu_allocate_irqs(ctx, count);
if (rc)
return rc;
afu_register_hwirqs(ctx);
return 0;
}
void afu_release_irqs(struct cxl_context *ctx, void *cookie)
{
irq_hw_number_t hwirq;
unsigned int virq;
int r, i;
for (r = afu_irq_range_start(); r < CXL_IRQ_RANGES; r++) {
hwirq = ctx->irqs.offset[r];
for (i = 0; i < ctx->irqs.range[r]; hwirq++, i++) {
virq = irq_find_mapping(NULL, hwirq);
if (virq)
cxl_unmap_irq(virq, cookie);
}
}
afu_irq_name_free(ctx);
cxl_ops->release_irq_ranges(&ctx->irqs, ctx->afu->adapter);
ctx->irq_count = 0;
}
void cxl_afu_decode_psl_serr(struct cxl_afu *afu, u64 serr)
{
dev_crit(&afu->dev,
"PSL Slice error received. Check AFU for root cause.\n");
dev_crit(&afu->dev, "PSL_SERR_An: 0x%016llx\n", serr);
if (serr & CXL_PSL_SERR_An_afuto)
dev_crit(&afu->dev, "AFU MMIO Timeout\n");
if (serr & CXL_PSL_SERR_An_afudis)
dev_crit(&afu->dev,
"MMIO targeted Accelerator that was not enabled\n");
if (serr & CXL_PSL_SERR_An_afuov)
dev_crit(&afu->dev, "AFU CTAG Overflow\n");
if (serr & CXL_PSL_SERR_An_badsrc)
dev_crit(&afu->dev, "Bad Interrupt Source\n");
if (serr & CXL_PSL_SERR_An_badctx)
dev_crit(&afu->dev, "Bad Context Handle\n");
if (serr & CXL_PSL_SERR_An_llcmdis)
dev_crit(&afu->dev, "LLCMD to Disabled AFU\n");
if (serr & CXL_PSL_SERR_An_llcmdto)
dev_crit(&afu->dev, "LLCMD Timeout to AFU\n");
if (serr & CXL_PSL_SERR_An_afupar)
dev_crit(&afu->dev, "AFU MMIO Parity Error\n");
if (serr & CXL_PSL_SERR_An_afudup)
dev_crit(&afu->dev, "AFU MMIO Duplicate CTAG Error\n");
if (serr & CXL_PSL_SERR_An_AE)
dev_crit(&afu->dev,
"AFU asserted JDONE with JERROR in AFU Directed Mode\n");
}

383
drivers/misc/cxl/main.c
View File

@ -1,383 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/spinlock.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/device.h>
#include <linux/mutex.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/mm.h>
#include <linux/of.h>
#include <linux/slab.h>
#include <linux/idr.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
#include <linux/sched/task.h>
#include <asm/cputable.h>
#include <asm/mmu.h>
#include <misc/cxl-base.h>
#include "cxl.h"
#include "trace.h"
static DEFINE_SPINLOCK(adapter_idr_lock);
static DEFINE_IDR(cxl_adapter_idr);
uint cxl_verbose;
module_param_named(verbose, cxl_verbose, uint, 0600);
MODULE_PARM_DESC(verbose, "Enable verbose dmesg output");
const struct cxl_backend_ops *cxl_ops;
int cxl_afu_slbia(struct cxl_afu *afu)
{
unsigned long timeout = jiffies + (HZ * CXL_TIMEOUT);
pr_devel("cxl_afu_slbia issuing SLBIA command\n");
cxl_p2n_write(afu, CXL_SLBIA_An, CXL_TLB_SLB_IQ_ALL);
while (cxl_p2n_read(afu, CXL_SLBIA_An) & CXL_TLB_SLB_P) {
if (time_after_eq(jiffies, timeout)) {
dev_warn(&afu->dev, "WARNING: CXL AFU SLBIA timed out!\n");
return -EBUSY;
}
/* If the adapter has gone down, we can assume that we
* will PERST it and that will invalidate everything.
*/
if (!cxl_ops->link_ok(afu->adapter, afu))
return -EIO;
cpu_relax();
}
return 0;
}
static inline void _cxl_slbia(struct cxl_context *ctx, struct mm_struct *mm)
{
unsigned long flags;
if (ctx->mm != mm)
return;
pr_devel("%s matched mm - card: %i afu: %i pe: %i\n", __func__,
ctx->afu->adapter->adapter_num, ctx->afu->slice, ctx->pe);
spin_lock_irqsave(&ctx->sste_lock, flags);
trace_cxl_slbia(ctx);
memset(ctx->sstp, 0, ctx->sst_size);
spin_unlock_irqrestore(&ctx->sste_lock, flags);
mb();
cxl_afu_slbia(ctx->afu);
}
static inline void cxl_slbia_core(struct mm_struct *mm)
{
struct cxl *adapter;
struct cxl_afu *afu;
struct cxl_context *ctx;
int card, slice, id;
pr_devel("%s called\n", __func__);
spin_lock(&adapter_idr_lock);
idr_for_each_entry(&cxl_adapter_idr, adapter, card) {
/* XXX: Make this lookup faster with link from mm to ctx */
spin_lock(&adapter->afu_list_lock);
for (slice = 0; slice < adapter->slices; slice++) {
afu = adapter->afu[slice];
if (!afu || !afu->enabled)
continue;
rcu_read_lock();
idr_for_each_entry(&afu->contexts_idr, ctx, id)
_cxl_slbia(ctx, mm);
rcu_read_unlock();
}
spin_unlock(&adapter->afu_list_lock);
}
spin_unlock(&adapter_idr_lock);
}
static struct cxl_calls cxl_calls = {
.cxl_slbia = cxl_slbia_core,
.owner = THIS_MODULE,
};
int cxl_alloc_sst(struct cxl_context *ctx)
{
unsigned long vsid;
u64 ea_mask, size, sstp0, sstp1;
sstp0 = 0;
sstp1 = 0;
ctx->sst_size = PAGE_SIZE;
ctx->sst_lru = 0;
ctx->sstp = (struct cxl_sste *)get_zeroed_page(GFP_KERNEL);
if (!ctx->sstp) {
pr_err("cxl_alloc_sst: Unable to allocate segment table\n");
return -ENOMEM;
}
pr_devel("SSTP allocated at 0x%p\n", ctx->sstp);
vsid = get_kernel_vsid((u64)ctx->sstp, mmu_kernel_ssize) << 12;
sstp0 |= (u64)mmu_kernel_ssize << CXL_SSTP0_An_B_SHIFT;
sstp0 |= (SLB_VSID_KERNEL | mmu_psize_defs[mmu_linear_psize].sllp) << 50;
size = (((u64)ctx->sst_size >> 8) - 1) << CXL_SSTP0_An_SegTableSize_SHIFT;
if (unlikely(size & ~CXL_SSTP0_An_SegTableSize_MASK)) {
WARN(1, "Impossible segment table size\n");
return -EINVAL;
}
sstp0 |= size;
if (mmu_kernel_ssize == MMU_SEGSIZE_256M)
ea_mask = 0xfffff00ULL;
else
ea_mask = 0xffffffff00ULL;
sstp0 |= vsid >> (50-14); /* Top 14 bits of VSID */
sstp1 |= (vsid << (64-(50-14))) & ~ea_mask;
sstp1 |= (u64)ctx->sstp & ea_mask;
sstp1 |= CXL_SSTP1_An_V;
pr_devel("Looked up %#llx: slbfee. %#llx (ssize: %x, vsid: %#lx), copied to SSTP0: %#llx, SSTP1: %#llx\n",
(u64)ctx->sstp, (u64)ctx->sstp & ESID_MASK, mmu_kernel_ssize, vsid, sstp0, sstp1);
/* Store calculated sstp hardware points for use later */
ctx->sstp0 = sstp0;
ctx->sstp1 = sstp1;
return 0;
}
/* print buffer content as integers when debugging */
void cxl_dump_debug_buffer(void *buf, size_t buf_len)
{
#ifdef DEBUG
int i, *ptr;
/*
* We want to regroup up to 4 integers per line, which means they
* need to be in the same pr_devel() statement
*/
ptr = (int *) buf;
for (i = 0; i * 4 < buf_len; i += 4) {
if ((i + 3) * 4 < buf_len)
pr_devel("%.8x %.8x %.8x %.8x\n", ptr[i], ptr[i + 1],
ptr[i + 2], ptr[i + 3]);
else if ((i + 2) * 4 < buf_len)
pr_devel("%.8x %.8x %.8x\n", ptr[i], ptr[i + 1],
ptr[i + 2]);
else if ((i + 1) * 4 < buf_len)
pr_devel("%.8x %.8x\n", ptr[i], ptr[i + 1]);
else
pr_devel("%.8x\n", ptr[i]);
}
#endif /* DEBUG */
}
/* Find a CXL adapter by it's number and increase it's refcount */
struct cxl *get_cxl_adapter(int num)
{
struct cxl *adapter;
spin_lock(&adapter_idr_lock);
if ((adapter = idr_find(&cxl_adapter_idr, num)))
get_device(&adapter->dev);
spin_unlock(&adapter_idr_lock);
return adapter;
}
static int cxl_alloc_adapter_nr(struct cxl *adapter)
{
int i;
idr_preload(GFP_KERNEL);
spin_lock(&adapter_idr_lock);
i = idr_alloc(&cxl_adapter_idr, adapter, 0, 0, GFP_NOWAIT);
spin_unlock(&adapter_idr_lock);
idr_preload_end();
if (i < 0)
return i;
adapter->adapter_num = i;
return 0;
}
void cxl_remove_adapter_nr(struct cxl *adapter)
{
idr_remove(&cxl_adapter_idr, adapter->adapter_num);
}
struct cxl *cxl_alloc_adapter(void)
{
struct cxl *adapter;
if (!(adapter = kzalloc(sizeof(struct cxl), GFP_KERNEL)))
return NULL;
spin_lock_init(&adapter->afu_list_lock);
if (cxl_alloc_adapter_nr(adapter))
goto err1;
if (dev_set_name(&adapter->dev, "card%i", adapter->adapter_num))
goto err2;
/* start with context lock taken */
atomic_set(&adapter->contexts_num, -1);
return adapter;
err2:
cxl_remove_adapter_nr(adapter);
err1:
kfree(adapter);
return NULL;
}
struct cxl_afu *cxl_alloc_afu(struct cxl *adapter, int slice)
{
struct cxl_afu *afu;
if (!(afu = kzalloc(sizeof(struct cxl_afu), GFP_KERNEL)))
return NULL;
afu->adapter = adapter;
afu->dev.parent = &adapter->dev;
afu->dev.release = cxl_ops->release_afu;
afu->slice = slice;
idr_init(&afu->contexts_idr);
mutex_init(&afu->contexts_lock);
spin_lock_init(&afu->afu_cntl_lock);
atomic_set(&afu->configured_state, -1);
afu->prefault_mode = CXL_PREFAULT_NONE;
afu->irqs_max = afu->adapter->user_irqs;
return afu;
}
int cxl_afu_select_best_mode(struct cxl_afu *afu)
{
if (afu->modes_supported & CXL_MODE_DIRECTED)
return cxl_ops->afu_activate_mode(afu, CXL_MODE_DIRECTED);
if (afu->modes_supported & CXL_MODE_DEDICATED)
return cxl_ops->afu_activate_mode(afu, CXL_MODE_DEDICATED);
dev_warn(&afu->dev, "No supported programming modes available\n");
/* We don't fail this so the user can inspect sysfs */
return 0;
}
int cxl_adapter_context_get(struct cxl *adapter)
{
int rc;
rc = atomic_inc_unless_negative(&adapter->contexts_num);
return rc ? 0 : -EBUSY;
}
void cxl_adapter_context_put(struct cxl *adapter)
{
atomic_dec_if_positive(&adapter->contexts_num);
}
int cxl_adapter_context_lock(struct cxl *adapter)
{
int rc;
/* no active contexts -> contexts_num == 0 */
rc = atomic_cmpxchg(&adapter->contexts_num, 0, -1);
return rc ? -EBUSY : 0;
}
void cxl_adapter_context_unlock(struct cxl *adapter)
{
int val = atomic_cmpxchg(&adapter->contexts_num, -1, 0);
/*
* contexts lock taken -> contexts_num == -1
* If not true then show a warning and force reset the lock.
* This will happen when context_unlock was requested without
* doing a context_lock.
*/
if (val != -1) {
atomic_set(&adapter->contexts_num, 0);
WARN(1, "Adapter context unlocked with %d active contexts",
val);
}
}
static int __init init_cxl(void)
{
int rc = 0;
if (!tlbie_capable)
return -EINVAL;
if ((rc = cxl_file_init()))
return rc;
cxl_debugfs_init();
/*
* we don't register the callback on P9. slb callack is only
* used for the PSL8 MMU and CX4.
*/
if (cxl_is_power8()) {
rc = register_cxl_calls(&cxl_calls);
if (rc)
goto err;
}
if (cpu_has_feature(CPU_FTR_HVMODE)) {
cxl_ops = &cxl_native_ops;
rc = pci_register_driver(&cxl_pci_driver);
}
#ifdef CONFIG_PPC_PSERIES
else {
cxl_ops = &cxl_guest_ops;
rc = platform_driver_register(&cxl_of_driver);
}
#endif
if (rc)
goto err1;
return 0;
err1:
if (cxl_is_power8())
unregister_cxl_calls(&cxl_calls);
err:
cxl_debugfs_exit();
cxl_file_exit();
return rc;
}
static void exit_cxl(void)
{
if (cpu_has_feature(CPU_FTR_HVMODE))
pci_unregister_driver(&cxl_pci_driver);
#ifdef CONFIG_PPC_PSERIES
else
platform_driver_unregister(&cxl_of_driver);
#endif
cxl_debugfs_exit();
cxl_file_exit();
if (cxl_is_power8())
unregister_cxl_calls(&cxl_calls);
idr_destroy(&cxl_adapter_idr);
}
module_init(init_cxl);
module_exit(exit_cxl);
MODULE_DESCRIPTION("IBM Coherent Accelerator");
MODULE_AUTHOR("Ian Munsie <imunsie@au1.ibm.com>");
MODULE_LICENSE("GPL");

1592
drivers/misc/cxl/native.c
View File

File diff suppressed because it is too large Load Diff

346
drivers/misc/cxl/of.c
View File

@ -1,346 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2015 IBM Corp.
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_platform.h>
#include "cxl.h"
static int read_phys_addr(struct device_node *np, char *prop_name,
struct cxl_afu *afu)
{
int i, len, entry_size, naddr, nsize, type;
u64 addr, size;
const __be32 *prop;
naddr = of_n_addr_cells(np);
nsize = of_n_size_cells(np);
prop = of_get_property(np, prop_name, &len);
if (prop) {
entry_size = naddr + nsize;
for (i = 0; i < (len / 4); i += entry_size, prop += entry_size) {
type = be32_to_cpu(prop[0]);
addr = of_read_number(prop, naddr);
size = of_read_number(&prop[naddr], nsize);
switch (type) {
case 0: /* unit address */
afu->guest->handle = addr;
break;
case 1: /* p2 area */
afu->guest->p2n_phys += addr;
afu->guest->p2n_size = size;
break;
case 2: /* problem state area */
afu->psn_phys += addr;
afu->adapter->ps_size = size;
break;
default:
pr_err("Invalid address type %d found in %s property of AFU\n",
type, prop_name);
return -EINVAL;
}
}
}
return 0;
}
static int read_vpd(struct cxl *adapter, struct cxl_afu *afu)
{
char vpd[256];
int rc;
size_t len = sizeof(vpd);
memset(vpd, 0, len);
if (adapter)
rc = cxl_guest_read_adapter_vpd(adapter, vpd, len);
else
rc = cxl_guest_read_afu_vpd(afu, vpd, len);
if (rc > 0) {
cxl_dump_debug_buffer(vpd, rc);
rc = 0;
}
return rc;
}
int cxl_of_read_afu_handle(struct cxl_afu *afu, struct device_node *afu_np)
{
return of_property_read_reg(afu_np, 0, &afu->guest->handle, NULL);
}
int cxl_of_read_afu_properties(struct cxl_afu *afu, struct device_node *np)
{
int i, rc;
u16 device_id, vendor_id;
u32 val = 0, class_code;
/* Properties are read in the same order as listed in PAPR */
rc = read_phys_addr(np, "reg", afu);
if (rc)
return rc;
rc = read_phys_addr(np, "assigned-addresses", afu);
if (rc)
return rc;
if (afu->psn_phys == 0)
afu->psa = false;
else
afu->psa = true;
of_property_read_u32(np, "ibm,#processes", &afu->max_procs_virtualised);
if (cxl_verbose)
read_vpd(NULL, afu);
of_property_read_u32(np, "ibm,max-ints-per-process", &afu->guest->max_ints);
afu->irqs_max = afu->guest->max_ints;
if (!of_property_read_u32(np, "ibm,min-ints-per-process", &afu->pp_irqs)) {
/* One extra interrupt for the PSL interrupt is already
* included. Remove it now to keep only AFU interrupts and
* match the native case.
*/
afu->pp_irqs--;
}
of_property_read_u64(np, "ibm,error-buffer-size", &afu->eb_len);
afu->eb_offset = 0;
of_property_read_u64(np, "ibm,config-record-size", &afu->crs_len);
afu->crs_offset = 0;
of_property_read_u32(np, "ibm,#config-records", &afu->crs_num);
if (cxl_verbose) {
for (i = 0; i < afu->crs_num; i++) {
rc = cxl_ops->afu_cr_read16(afu, i, PCI_DEVICE_ID,
&device_id);
if (!rc)
pr_info("record %d - device-id: %#x\n",
i, device_id);
rc = cxl_ops->afu_cr_read16(afu, i, PCI_VENDOR_ID,
&vendor_id);
if (!rc)
pr_info("record %d - vendor-id: %#x\n",
i, vendor_id);
rc = cxl_ops->afu_cr_read32(afu, i, PCI_CLASS_REVISION,
&class_code);
if (!rc) {
class_code >>= 8;
pr_info("record %d - class-code: %#x\n",
i, class_code);
}
}
}
/*
* if "ibm,process-mmio" doesn't exist then per-process mmio is
* not supported
*/
val = 0;
if (!of_property_read_u32(np, "ibm,process-mmio", &val) && val == 1)
afu->pp_psa = true;
else
afu->pp_psa = false;
if (!of_property_read_u32(np, "ibm,function-error-interrupt", &val))
afu->serr_hwirq = val;
pr_devel("AFU handle: %#llx\n", afu->guest->handle);
pr_devel("p2n_phys: %#llx (size %#llx)\n",
afu->guest->p2n_phys, afu->guest->p2n_size);
pr_devel("psn_phys: %#llx (size %#llx)\n",
afu->psn_phys, afu->adapter->ps_size);
pr_devel("Max number of processes virtualised=%i\n",
afu->max_procs_virtualised);
pr_devel("Per-process irqs min=%i, max=%i\n", afu->pp_irqs,
afu->irqs_max);
pr_devel("Slice error interrupt=%#lx\n", afu->serr_hwirq);
return 0;
}
static int read_adapter_irq_config(struct cxl *adapter, struct device_node *np)
{
const __be32 *ranges;
int len, nranges, i;
struct irq_avail *cur;
ranges = of_get_property(np, "interrupt-ranges", &len);
if (ranges == NULL || len < (2 * sizeof(int)))
return -EINVAL;
/*
* encoded array of two cells per entry, each cell encoded as
* with encode-int
*/
nranges = len / (2 * sizeof(int));
if (nranges == 0 || (nranges * 2 * sizeof(int)) != len)
return -EINVAL;
adapter->guest->irq_avail = kcalloc(nranges, sizeof(struct irq_avail),
GFP_KERNEL);
if (adapter->guest->irq_avail == NULL)
return -ENOMEM;
adapter->guest->irq_base_offset = be32_to_cpu(ranges[0]);
for (i = 0; i < nranges; i++) {
cur = &adapter->guest->irq_avail[i];
cur->offset = be32_to_cpu(ranges[i * 2]);
cur->range = be32_to_cpu(ranges[i * 2 + 1]);
cur->bitmap = bitmap_zalloc(cur->range, GFP_KERNEL);
if (cur->bitmap == NULL)
goto err;
if (cur->offset < adapter->guest->irq_base_offset)
adapter->guest->irq_base_offset = cur->offset;
if (cxl_verbose)
pr_info("available IRQ range: %#lx-%#lx (%lu)\n",
cur->offset, cur->offset + cur->range - 1,
cur->range);
}
adapter->guest->irq_nranges = nranges;
spin_lock_init(&adapter->guest->irq_alloc_lock);
return 0;
err:
for (i--; i >= 0; i--) {
cur = &adapter->guest->irq_avail[i];
bitmap_free(cur->bitmap);
}
kfree(adapter->guest->irq_avail);
adapter->guest->irq_avail = NULL;
return -ENOMEM;
}
int cxl_of_read_adapter_handle(struct cxl *adapter, struct device_node *np)
{
return of_property_read_reg(np, 0, &adapter->guest->handle, NULL);
}
int cxl_of_read_adapter_properties(struct cxl *adapter, struct device_node *np)
{
int rc;
const char *p;
u32 val = 0;
/* Properties are read in the same order as listed in PAPR */
if ((rc = read_adapter_irq_config(adapter, np)))
return rc;
if (!of_property_read_u32(np, "ibm,caia-version", &val)) {
adapter->caia_major = (val & 0xFF00) >> 8;
adapter->caia_minor = val & 0xFF;
}
if (!of_property_read_u32(np, "ibm,psl-revision", &val))
adapter->psl_rev = val;
if (!of_property_read_string(np, "status", &p)) {
adapter->guest->status = kasprintf(GFP_KERNEL, "%s", p);
if (adapter->guest->status == NULL)
return -ENOMEM;
}
if (!of_property_read_u32(np, "vendor-id", &val))
adapter->guest->vendor = val;
if (!of_property_read_u32(np, "device-id", &val))
adapter->guest->device = val;
if (!of_property_read_u32(np, "subsystem-vendor-id", &val))
adapter->guest->subsystem_vendor = val;
if (!of_property_read_u32(np, "subsystem-id", &val))
adapter->guest->subsystem = val;
if (cxl_verbose)
read_vpd(adapter, NULL);
return 0;
}
static void cxl_of_remove(struct platform_device *pdev)
{
struct cxl *adapter;
int afu;
adapter = dev_get_drvdata(&pdev->dev);
for (afu = 0; afu < adapter->slices; afu++)
cxl_guest_remove_afu(adapter->afu[afu]);
cxl_guest_remove_adapter(adapter);
}
static void cxl_of_shutdown(struct platform_device *pdev)
{
cxl_of_remove(pdev);
}
int cxl_of_probe(struct platform_device *pdev)
{
struct device_node *np = NULL;
struct device_node *afu_np = NULL;
struct cxl *adapter = NULL;
int ret;
int slice = 0, slice_ok = 0;
dev_err_once(&pdev->dev, "DEPRECATION: cxl is deprecated and will be removed in a future kernel release\n");
pr_devel("in %s\n", __func__);
np = pdev->dev.of_node;
if (np == NULL)
return -ENODEV;
/* init adapter */
adapter = cxl_guest_init_adapter(np, pdev);
if (IS_ERR(adapter)) {
dev_err(&pdev->dev, "guest_init_adapter failed: %li\n", PTR_ERR(adapter));
return PTR_ERR(adapter);
}
/* init afu */
for_each_child_of_node(np, afu_np) {
if ((ret = cxl_guest_init_afu(adapter, slice, afu_np)))
dev_err(&pdev->dev, "AFU %i failed to initialise: %i\n",
slice, ret);
else
slice_ok++;
slice++;
}
if (slice_ok == 0) {
dev_info(&pdev->dev, "No active AFU");
adapter->slices = 0;
}
return 0;
}
static const struct of_device_id cxl_of_match[] = {
{ .compatible = "ibm,coherent-platform-facility",},
{},
};
MODULE_DEVICE_TABLE(of, cxl_of_match);
struct platform_driver cxl_of_driver = {
.driver = {
.name = "cxl_of",
.of_match_table = cxl_of_match,
.owner = THIS_MODULE
},
.probe = cxl_of_probe,
.remove = cxl_of_remove,
.shutdown = cxl_of_shutdown,
};

2103
drivers/misc/cxl/pci.c
View File

File diff suppressed because it is too large Load Diff

771
drivers/misc/cxl/sysfs.c
View File

@ -1,771 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/sysfs.h>
#include <linux/pci_regs.h>
#include "cxl.h"
#define to_afu_chardev_m(d) dev_get_drvdata(d)
/********* Adapter attributes **********************************************/
static ssize_t caia_version_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
return scnprintf(buf, PAGE_SIZE, "%i.%i\n", adapter->caia_major,
adapter->caia_minor);
}
static ssize_t psl_revision_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
return scnprintf(buf, PAGE_SIZE, "%i\n", adapter->psl_rev);
}
static ssize_t base_image_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
return scnprintf(buf, PAGE_SIZE, "%i\n", adapter->base_image);
}
static ssize_t image_loaded_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
if (adapter->user_image_loaded)
return scnprintf(buf, PAGE_SIZE, "user\n");
return scnprintf(buf, PAGE_SIZE, "factory\n");
}
static ssize_t psl_timebase_synced_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
u64 psl_tb, delta;
/* Recompute the status only in native mode */
if (cpu_has_feature(CPU_FTR_HVMODE)) {
psl_tb = adapter->native->sl_ops->timebase_read(adapter);
delta = abs(mftb() - psl_tb);
/* CORE TB and PSL TB difference <= 16usecs ? */
adapter->psl_timebase_synced = (tb_to_ns(delta) < 16000) ? true : false;
pr_devel("PSL timebase %s - delta: 0x%016llx\n",
(tb_to_ns(delta) < 16000) ? "synchronized" :
"not synchronized", tb_to_ns(delta));
}
return scnprintf(buf, PAGE_SIZE, "%i\n", adapter->psl_timebase_synced);
}
static ssize_t tunneled_ops_supported_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
return scnprintf(buf, PAGE_SIZE, "%i\n", adapter->tunneled_ops_supported);
}
static ssize_t reset_adapter_store(struct device *device,
struct device_attribute *attr,
const char *buf, size_t count)
{
struct cxl *adapter = to_cxl_adapter(device);
int rc;
int val;
rc = sscanf(buf, "%i", &val);
if ((rc != 1) || (val != 1 && val != -1))
return -EINVAL;
/*
* See if we can lock the context mapping that's only allowed
* when there are no contexts attached to the adapter. Once
* taken this will also prevent any context from getting activated.
*/
if (val == 1) {
rc = cxl_adapter_context_lock(adapter);
if (rc)
goto out;
rc = cxl_ops->adapter_reset(adapter);
/* In case reset failed release context lock */
if (rc)
cxl_adapter_context_unlock(adapter);
} else if (val == -1) {
/* Perform a forced adapter reset */
rc = cxl_ops->adapter_reset(adapter);
}
out:
return rc ? rc : count;
}
static ssize_t load_image_on_perst_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
if (!adapter->perst_loads_image)
return scnprintf(buf, PAGE_SIZE, "none\n");
if (adapter->perst_select_user)
return scnprintf(buf, PAGE_SIZE, "user\n");
return scnprintf(buf, PAGE_SIZE, "factory\n");
}
static ssize_t load_image_on_perst_store(struct device *device,
struct device_attribute *attr,
const char *buf, size_t count)
{
struct cxl *adapter = to_cxl_adapter(device);
int rc;
if (!strncmp(buf, "none", 4))
adapter->perst_loads_image = false;
else if (!strncmp(buf, "user", 4)) {
adapter->perst_select_user = true;
adapter->perst_loads_image = true;
} else if (!strncmp(buf, "factory", 7)) {
adapter->perst_select_user = false;
adapter->perst_loads_image = true;
} else
return -EINVAL;
if ((rc = cxl_update_image_control(adapter)))
return rc;
return count;
}
static ssize_t perst_reloads_same_image_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl *adapter = to_cxl_adapter(device);
return scnprintf(buf, PAGE_SIZE, "%i\n", adapter->perst_same_image);
}
static ssize_t perst_reloads_same_image_store(struct device *device,
struct device_attribute *attr,
const char *buf, size_t count)
{
struct cxl *adapter = to_cxl_adapter(device);
int rc;
int val;
rc = sscanf(buf, "%i", &val);
if ((rc != 1) || !(val == 1 || val == 0))
return -EINVAL;
adapter->perst_same_image = (val == 1);
return count;
}
static struct device_attribute adapter_attrs[] = {
__ATTR_RO(caia_version),
__ATTR_RO(psl_revision),
__ATTR_RO(base_image),
__ATTR_RO(image_loaded),
__ATTR_RO(psl_timebase_synced),
__ATTR_RO(tunneled_ops_supported),
__ATTR_RW(load_image_on_perst),
__ATTR_RW(perst_reloads_same_image),
__ATTR(reset, S_IWUSR, NULL, reset_adapter_store),
};
/********* AFU master specific attributes **********************************/
static ssize_t mmio_size_show_master(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_afu_chardev_m(device);
return scnprintf(buf, PAGE_SIZE, "%llu\n", afu->adapter->ps_size);
}
static ssize_t pp_mmio_off_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_afu_chardev_m(device);
return scnprintf(buf, PAGE_SIZE, "%llu\n", afu->native->pp_offset);
}
static ssize_t pp_mmio_len_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_afu_chardev_m(device);
return scnprintf(buf, PAGE_SIZE, "%llu\n", afu->pp_size);
}
static struct device_attribute afu_master_attrs[] = {
__ATTR(mmio_size, S_IRUGO, mmio_size_show_master, NULL),
__ATTR_RO(pp_mmio_off),
__ATTR_RO(pp_mmio_len),
};
/********* AFU attributes **************************************************/
static ssize_t mmio_size_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_cxl_afu(device);
if (afu->pp_size)
return scnprintf(buf, PAGE_SIZE, "%llu\n", afu->pp_size);
return scnprintf(buf, PAGE_SIZE, "%llu\n", afu->adapter->ps_size);
}
static ssize_t reset_store_afu(struct device *device,
struct device_attribute *attr,
const char *buf, size_t count)
{
struct cxl_afu *afu = to_cxl_afu(device);
int rc;
/* Not safe to reset if it is currently in use */
mutex_lock(&afu->contexts_lock);
if (!idr_is_empty(&afu->contexts_idr)) {
rc = -EBUSY;
goto err;
}
if ((rc = cxl_ops->afu_reset(afu)))
goto err;
rc = count;
err:
mutex_unlock(&afu->contexts_lock);
return rc;
}
static ssize_t irqs_min_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_cxl_afu(device);
return scnprintf(buf, PAGE_SIZE, "%i\n", afu->pp_irqs);
}
static ssize_t irqs_max_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_cxl_afu(device);
return scnprintf(buf, PAGE_SIZE, "%i\n", afu->irqs_max);
}
static ssize_t irqs_max_store(struct device *device,
struct device_attribute *attr,
const char *buf, size_t count)
{
struct cxl_afu *afu = to_cxl_afu(device);
ssize_t ret;
int irqs_max;
ret = sscanf(buf, "%i", &irqs_max);
if (ret != 1)
return -EINVAL;
if (irqs_max < afu->pp_irqs)
return -EINVAL;
if (cpu_has_feature(CPU_FTR_HVMODE)) {
if (irqs_max > afu->adapter->user_irqs)
return -EINVAL;
} else {
/* pHyp sets a per-AFU limit */
if (irqs_max > afu->guest->max_ints)
return -EINVAL;
}
afu->irqs_max = irqs_max;
return count;
}
static ssize_t modes_supported_show(struct device *device,
struct device_attribute *attr, char *buf)
{
struct cxl_afu *afu = to_cxl_afu(device);
char *p = buf, *end = buf + PAGE_SIZE;
if (afu->modes_supported & CXL_MODE_DEDICATED)
p += scnprintf(p, end - p, "dedicated_process\n");
if (afu->modes_supported & CXL_MODE_DIRECTED)
p += scnprintf(p, end - p, "afu_directed\n");
return (p - buf);
}
static ssize_t prefault_mode_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_cxl_afu(device);
switch (afu->prefault_mode) {
case CXL_PREFAULT_WED:
return scnprintf(buf, PAGE_SIZE, "work_element_descriptor\n");
case CXL_PREFAULT_ALL:
return scnprintf(buf, PAGE_SIZE, "all\n");
default:
return scnprintf(buf, PAGE_SIZE, "none\n");
}
}
static ssize_t prefault_mode_store(struct device *device,
struct device_attribute *attr,
const char *buf, size_t count)
{
struct cxl_afu *afu = to_cxl_afu(device);
enum prefault_modes mode = -1;
if (!strncmp(buf, "none", 4))
mode = CXL_PREFAULT_NONE;
else {
if (!radix_enabled()) {
/* only allowed when not in radix mode */
if (!strncmp(buf, "work_element_descriptor", 23))
mode = CXL_PREFAULT_WED;
if (!strncmp(buf, "all", 3))
mode = CXL_PREFAULT_ALL;
} else {
dev_err(device, "Cannot prefault with radix enabled\n");
}
}
if (mode == -1)
return -EINVAL;
afu->prefault_mode = mode;
return count;
}
static ssize_t mode_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct cxl_afu *afu = to_cxl_afu(device);
if (afu->current_mode == CXL_MODE_DEDICATED)
return scnprintf(buf, PAGE_SIZE, "dedicated_process\n");
if (afu->current_mode == CXL_MODE_DIRECTED)
return scnprintf(buf, PAGE_SIZE, "afu_directed\n");
return scnprintf(buf, PAGE_SIZE, "none\n");
}
static ssize_t mode_store(struct device *device, struct device_attribute *attr,
const char *buf, size_t count)
{
struct cxl_afu *afu = to_cxl_afu(device);
int old_mode, mode = -1;
int rc = -EBUSY;
/* can't change this if we have a user */
mutex_lock(&afu->contexts_lock);
if (!idr_is_empty(&afu->contexts_idr))
goto err;
if (!strncmp(buf, "dedicated_process", 17))
mode = CXL_MODE_DEDICATED;
if (!strncmp(buf, "afu_directed", 12))
mode = CXL_MODE_DIRECTED;
if (!strncmp(buf, "none", 4))
mode = 0;
if (mode == -1) {
rc = -EINVAL;
goto err;
}
/*
* afu_deactivate_mode needs to be done outside the lock, prevent
* other contexts coming in before we are ready:
*/
old_mode = afu->current_mode;
afu->current_mode = 0;
afu->num_procs = 0;
mutex_unlock(&afu->contexts_lock);
if ((rc = cxl_ops->afu_deactivate_mode(afu, old_mode)))
return rc;
if ((rc = cxl_ops->afu_activate_mode(afu, mode)))
return rc;
return count;
err:
mutex_unlock(&afu->contexts_lock);
return rc;
}
static ssize_t api_version_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
return scnprintf(buf, PAGE_SIZE, "%i\n", CXL_API_VERSION);
}
static ssize_t api_version_compatible_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
return scnprintf(buf, PAGE_SIZE, "%i\n", CXL_API_VERSION_COMPATIBLE);
}
static ssize_t afu_eb_read(struct file *filp, struct kobject *kobj,
const struct bin_attribute *bin_attr, char *buf,
loff_t off, size_t count)
{
struct cxl_afu *afu = to_cxl_afu(kobj_to_dev(kobj));
return cxl_ops->afu_read_err_buffer(afu, buf, off, count);
}
static struct device_attribute afu_attrs[] = {
__ATTR_RO(mmio_size),
__ATTR_RO(irqs_min),
__ATTR_RW(irqs_max),
__ATTR_RO(modes_supported),
__ATTR_RW(mode),
__ATTR_RW(prefault_mode),
__ATTR_RO(api_version),
__ATTR_RO(api_version_compatible),
__ATTR(reset, S_IWUSR, NULL, reset_store_afu),
};
int cxl_sysfs_adapter_add(struct cxl *adapter)
{
struct device_attribute *dev_attr;
int i, rc;
for (i = 0; i < ARRAY_SIZE(adapter_attrs); i++) {
dev_attr = &adapter_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_ADAPTER_ATTRS)) {
if ((rc = device_create_file(&adapter->dev, dev_attr)))
goto err;
}
}
return 0;
err:
for (i--; i >= 0; i--) {
dev_attr = &adapter_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_ADAPTER_ATTRS))
device_remove_file(&adapter->dev, dev_attr);
}
return rc;
}
void cxl_sysfs_adapter_remove(struct cxl *adapter)
{
struct device_attribute *dev_attr;
int i;
for (i = 0; i < ARRAY_SIZE(adapter_attrs); i++) {
dev_attr = &adapter_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_ADAPTER_ATTRS))
device_remove_file(&adapter->dev, dev_attr);
}
}
struct afu_config_record {
struct kobject kobj;
struct bin_attribute config_attr;
struct list_head list;
int cr;
u16 device;
u16 vendor;
u32 class;
};
#define to_cr(obj) container_of(obj, struct afu_config_record, kobj)
static ssize_t vendor_show(struct kobject *kobj,
struct kobj_attribute *attr, char *buf)
{
struct afu_config_record *cr = to_cr(kobj);
return scnprintf(buf, PAGE_SIZE, "0x%.4x\n", cr->vendor);
}
static ssize_t device_show(struct kobject *kobj,
struct kobj_attribute *attr, char *buf)
{
struct afu_config_record *cr = to_cr(kobj);
return scnprintf(buf, PAGE_SIZE, "0x%.4x\n", cr->device);
}
static ssize_t class_show(struct kobject *kobj,
struct kobj_attribute *attr, char *buf)
{
struct afu_config_record *cr = to_cr(kobj);
return scnprintf(buf, PAGE_SIZE, "0x%.6x\n", cr->class);
}
static ssize_t afu_read_config(struct file *filp, struct kobject *kobj,
const struct bin_attribute *bin_attr, char *buf,
loff_t off, size_t count)
{
struct afu_config_record *cr = to_cr(kobj);
struct cxl_afu *afu = to_cxl_afu(kobj_to_dev(kobj->parent));
u64 i, j, val, rc;
for (i = 0; i < count;) {
rc = cxl_ops->afu_cr_read64(afu, cr->cr, off & ~0x7, &val);
if (rc)
val = ~0ULL;
for (j = off & 0x7; j < 8 && i < count; i++, j++, off++)
buf[i] = (val >> (j * 8)) & 0xff;
}
return count;
}
static struct kobj_attribute vendor_attribute =
__ATTR_RO(vendor);
static struct kobj_attribute device_attribute =
__ATTR_RO(device);
static struct kobj_attribute class_attribute =
__ATTR_RO(class);
static struct attribute *afu_cr_attrs[] = {
&vendor_attribute.attr,
&device_attribute.attr,
&class_attribute.attr,
NULL,
};
ATTRIBUTE_GROUPS(afu_cr);
static void release_afu_config_record(struct kobject *kobj)
{
struct afu_config_record *cr = to_cr(kobj);
kfree(cr);
}
static const struct kobj_type afu_config_record_type = {
.sysfs_ops = &kobj_sysfs_ops,
.release = release_afu_config_record,
.default_groups = afu_cr_groups,
};
static struct afu_config_record *cxl_sysfs_afu_new_cr(struct cxl_afu *afu, int cr_idx)
{
struct afu_config_record *cr;
int rc;
cr = kzalloc(sizeof(struct afu_config_record), GFP_KERNEL);
if (!cr)
return ERR_PTR(-ENOMEM);
cr->cr = cr_idx;
rc = cxl_ops->afu_cr_read16(afu, cr_idx, PCI_DEVICE_ID, &cr->device);
if (rc)
goto err;
rc = cxl_ops->afu_cr_read16(afu, cr_idx, PCI_VENDOR_ID, &cr->vendor);
if (rc)
goto err;
rc = cxl_ops->afu_cr_read32(afu, cr_idx, PCI_CLASS_REVISION, &cr->class);
if (rc)
goto err;
cr->class >>= 8;
/*
* Export raw AFU PCIe like config record. For now this is read only by
* root - we can expand that later to be readable by non-root and maybe
* even writable provided we have a good use-case. Once we support
* exposing AFUs through a virtual PHB they will get that for free from
* Linux' PCI infrastructure, but until then it's not clear that we
* need it for anything since the main use case is just identifying
* AFUs, which can be done via the vendor, device and class attributes.
*/
sysfs_bin_attr_init(&cr->config_attr);
cr->config_attr.attr.name = "config";
cr->config_attr.attr.mode = S_IRUSR;
cr->config_attr.size = afu->crs_len;
cr->config_attr.read_new = afu_read_config;
rc = kobject_init_and_add(&cr->kobj, &afu_config_record_type,
&afu->dev.kobj, "cr%i", cr->cr);
if (rc)
goto err1;
rc = sysfs_create_bin_file(&cr->kobj, &cr->config_attr);
if (rc)
goto err1;
rc = kobject_uevent(&cr->kobj, KOBJ_ADD);
if (rc)
goto err2;
return cr;
err2:
sysfs_remove_bin_file(&cr->kobj, &cr->config_attr);
err1:
kobject_put(&cr->kobj);
return ERR_PTR(rc);
err:
kfree(cr);
return ERR_PTR(rc);
}
void cxl_sysfs_afu_remove(struct cxl_afu *afu)
{
struct device_attribute *dev_attr;
struct afu_config_record *cr, *tmp;
int i;
/* remove the err buffer bin attribute */
if (afu->eb_len)
device_remove_bin_file(&afu->dev, &afu->attr_eb);
for (i = 0; i < ARRAY_SIZE(afu_attrs); i++) {
dev_attr = &afu_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_AFU_ATTRS))
device_remove_file(&afu->dev, &afu_attrs[i]);
}
list_for_each_entry_safe(cr, tmp, &afu->crs, list) {
sysfs_remove_bin_file(&cr->kobj, &cr->config_attr);
kobject_put(&cr->kobj);
}
}
int cxl_sysfs_afu_add(struct cxl_afu *afu)
{
struct device_attribute *dev_attr;
struct afu_config_record *cr;
int i, rc;
INIT_LIST_HEAD(&afu->crs);
for (i = 0; i < ARRAY_SIZE(afu_attrs); i++) {
dev_attr = &afu_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_AFU_ATTRS)) {
if ((rc = device_create_file(&afu->dev, &afu_attrs[i])))
goto err;
}
}
/* conditionally create the add the binary file for error info buffer */
if (afu->eb_len) {
sysfs_attr_init(&afu->attr_eb.attr);
afu->attr_eb.attr.name = "afu_err_buff";
afu->attr_eb.attr.mode = S_IRUGO;
afu->attr_eb.size = afu->eb_len;
afu->attr_eb.read_new = afu_eb_read;
rc = device_create_bin_file(&afu->dev, &afu->attr_eb);
if (rc) {
dev_err(&afu->dev,
"Unable to create eb attr for the afu. Err(%d)\n",
rc);
goto err;
}
}
for (i = 0; i < afu->crs_num; i++) {
cr = cxl_sysfs_afu_new_cr(afu, i);
if (IS_ERR(cr)) {
rc = PTR_ERR(cr);
goto err1;
}
list_add(&cr->list, &afu->crs);
}
return 0;
err1:
cxl_sysfs_afu_remove(afu);
return rc;
err:
/* reset the eb_len as we havent created the bin attr */
afu->eb_len = 0;
for (i--; i >= 0; i--) {
dev_attr = &afu_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_AFU_ATTRS))
device_remove_file(&afu->dev, &afu_attrs[i]);
}
return rc;
}
int cxl_sysfs_afu_m_add(struct cxl_afu *afu)
{
struct device_attribute *dev_attr;
int i, rc;
for (i = 0; i < ARRAY_SIZE(afu_master_attrs); i++) {
dev_attr = &afu_master_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_AFU_MASTER_ATTRS)) {
if ((rc = device_create_file(afu->chardev_m, &afu_master_attrs[i])))
goto err;
}
}
return 0;
err:
for (i--; i >= 0; i--) {
dev_attr = &afu_master_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_AFU_MASTER_ATTRS))
device_remove_file(afu->chardev_m, &afu_master_attrs[i]);
}
return rc;
}
void cxl_sysfs_afu_m_remove(struct cxl_afu *afu)
{
struct device_attribute *dev_attr;
int i;
for (i = 0; i < ARRAY_SIZE(afu_master_attrs); i++) {
dev_attr = &afu_master_attrs[i];
if (cxl_ops->support_attributes(dev_attr->attr.name,
CXL_AFU_MASTER_ATTRS))
device_remove_file(afu->chardev_m, &afu_master_attrs[i]);
}
}

9
drivers/misc/cxl/trace.c
View File

@ -1,9 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2015 IBM Corp.
*/
#ifndef __CHECKER__
#define CREATE_TRACE_POINTS
#include "trace.h"
#endif

691
drivers/misc/cxl/trace.h
View File

@ -1,691 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright 2015 IBM Corp.
*/
#undef TRACE_SYSTEM
#define TRACE_SYSTEM cxl
#if !defined(_CXL_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
#define _CXL_TRACE_H
#include <linux/tracepoint.h>
#include "cxl.h"
#define dsisr_psl9_flags(flags) \
__print_flags(flags, "|", \
{ CXL_PSL9_DSISR_An_CO_MASK, "FR" }, \
{ CXL_PSL9_DSISR_An_TF, "TF" }, \
{ CXL_PSL9_DSISR_An_PE, "PE" }, \
{ CXL_PSL9_DSISR_An_AE, "AE" }, \
{ CXL_PSL9_DSISR_An_OC, "OC" }, \
{ CXL_PSL9_DSISR_An_S, "S" })
#define DSISR_FLAGS \
{ CXL_PSL_DSISR_An_DS, "DS" }, \
{ CXL_PSL_DSISR_An_DM, "DM" }, \
{ CXL_PSL_DSISR_An_ST, "ST" }, \
{ CXL_PSL_DSISR_An_UR, "UR" }, \
{ CXL_PSL_DSISR_An_PE, "PE" }, \
{ CXL_PSL_DSISR_An_AE, "AE" }, \
{ CXL_PSL_DSISR_An_OC, "OC" }, \
{ CXL_PSL_DSISR_An_M, "M" }, \
{ CXL_PSL_DSISR_An_P, "P" }, \
{ CXL_PSL_DSISR_An_A, "A" }, \
{ CXL_PSL_DSISR_An_S, "S" }, \
{ CXL_PSL_DSISR_An_K, "K" }
#define TFC_FLAGS \
{ CXL_PSL_TFC_An_A, "A" }, \
{ CXL_PSL_TFC_An_C, "C" }, \
{ CXL_PSL_TFC_An_AE, "AE" }, \
{ CXL_PSL_TFC_An_R, "R" }
#define LLCMD_NAMES \
{ CXL_SPA_SW_CMD_TERMINATE, "TERMINATE" }, \
{ CXL_SPA_SW_CMD_REMOVE, "REMOVE" }, \
{ CXL_SPA_SW_CMD_SUSPEND, "SUSPEND" }, \
{ CXL_SPA_SW_CMD_RESUME, "RESUME" }, \
{ CXL_SPA_SW_CMD_ADD, "ADD" }, \
{ CXL_SPA_SW_CMD_UPDATE, "UPDATE" }
#define AFU_COMMANDS \
{ 0, "DISABLE" }, \
{ CXL_AFU_Cntl_An_E, "ENABLE" }, \
{ CXL_AFU_Cntl_An_RA, "RESET" }
#define PSL_COMMANDS \
{ CXL_PSL_SCNTL_An_Pc, "PURGE" }, \
{ CXL_PSL_SCNTL_An_Sc, "SUSPEND" }
DECLARE_EVENT_CLASS(cxl_pe_class,
TP_PROTO(struct cxl_context *ctx),
TP_ARGS(ctx),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
),
TP_printk("afu%i.%i pe=%i",
__entry->card,
__entry->afu,
__entry->pe
)
);
TRACE_EVENT(cxl_attach,
TP_PROTO(struct cxl_context *ctx, u64 wed, s16 num_interrupts, u64 amr),
TP_ARGS(ctx, wed, num_interrupts, amr),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(pid_t, pid)
__field(u64, wed)
__field(u64, amr)
__field(s16, num_interrupts)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->pid = pid_nr(ctx->pid);
__entry->wed = wed;
__entry->amr = amr;
__entry->num_interrupts = num_interrupts;
),
TP_printk("afu%i.%i pid=%i pe=%i wed=0x%016llx irqs=%i amr=0x%llx",
__entry->card,
__entry->afu,
__entry->pid,
__entry->pe,
__entry->wed,
__entry->num_interrupts,
__entry->amr
)
);
DEFINE_EVENT(cxl_pe_class, cxl_detach,
TP_PROTO(struct cxl_context *ctx),
TP_ARGS(ctx)
);
TRACE_EVENT(cxl_afu_irq,
TP_PROTO(struct cxl_context *ctx, int afu_irq, int virq, irq_hw_number_t hwirq),
TP_ARGS(ctx, afu_irq, virq, hwirq),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(u16, afu_irq)
__field(int, virq)
__field(irq_hw_number_t, hwirq)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->afu_irq = afu_irq;
__entry->virq = virq;
__entry->hwirq = hwirq;
),
TP_printk("afu%i.%i pe=%i afu_irq=%i virq=%i hwirq=0x%lx",
__entry->card,
__entry->afu,
__entry->pe,
__entry->afu_irq,
__entry->virq,
__entry->hwirq
)
);
TRACE_EVENT(cxl_psl9_irq,
TP_PROTO(struct cxl_context *ctx, int irq, u64 dsisr, u64 dar),
TP_ARGS(ctx, irq, dsisr, dar),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(int, irq)
__field(u64, dsisr)
__field(u64, dar)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->irq = irq;
__entry->dsisr = dsisr;
__entry->dar = dar;
),
TP_printk("afu%i.%i pe=%i irq=%i dsisr=0x%016llx dsisr=%s dar=0x%016llx",
__entry->card,
__entry->afu,
__entry->pe,
__entry->irq,
__entry->dsisr,
dsisr_psl9_flags(__entry->dsisr),
__entry->dar
)
);
TRACE_EVENT(cxl_psl_irq,
TP_PROTO(struct cxl_context *ctx, int irq, u64 dsisr, u64 dar),
TP_ARGS(ctx, irq, dsisr, dar),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(int, irq)
__field(u64, dsisr)
__field(u64, dar)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->irq = irq;
__entry->dsisr = dsisr;
__entry->dar = dar;
),
TP_printk("afu%i.%i pe=%i irq=%i dsisr=%s dar=0x%016llx",
__entry->card,
__entry->afu,
__entry->pe,
__entry->irq,
__print_flags(__entry->dsisr, "|", DSISR_FLAGS),
__entry->dar
)
);
TRACE_EVENT(cxl_psl_irq_ack,
TP_PROTO(struct cxl_context *ctx, u64 tfc),
TP_ARGS(ctx, tfc),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(u64, tfc)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->tfc = tfc;
),
TP_printk("afu%i.%i pe=%i tfc=%s",
__entry->card,
__entry->afu,
__entry->pe,
__print_flags(__entry->tfc, "|", TFC_FLAGS)
)
);
TRACE_EVENT(cxl_ste_miss,
TP_PROTO(struct cxl_context *ctx, u64 dar),
TP_ARGS(ctx, dar),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(u64, dar)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->dar = dar;
),
TP_printk("afu%i.%i pe=%i dar=0x%016llx",
__entry->card,
__entry->afu,
__entry->pe,
__entry->dar
)
);
TRACE_EVENT(cxl_ste_write,
TP_PROTO(struct cxl_context *ctx, unsigned int idx, u64 e, u64 v),
TP_ARGS(ctx, idx, e, v),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(unsigned int, idx)
__field(u64, e)
__field(u64, v)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->idx = idx;
__entry->e = e;
__entry->v = v;
),
TP_printk("afu%i.%i pe=%i SSTE[%i] E=0x%016llx V=0x%016llx",
__entry->card,
__entry->afu,
__entry->pe,
__entry->idx,
__entry->e,
__entry->v
)
);
TRACE_EVENT(cxl_pte_miss,
TP_PROTO(struct cxl_context *ctx, u64 dsisr, u64 dar),
TP_ARGS(ctx, dsisr, dar),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(u64, dsisr)
__field(u64, dar)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->dsisr = dsisr;
__entry->dar = dar;
),
TP_printk("afu%i.%i pe=%i dsisr=%s dar=0x%016llx",
__entry->card,
__entry->afu,
__entry->pe,
__print_flags(__entry->dsisr, "|", DSISR_FLAGS),
__entry->dar
)
);
TRACE_EVENT(cxl_llcmd,
TP_PROTO(struct cxl_context *ctx, u64 cmd),
TP_ARGS(ctx, cmd),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(u64, cmd)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->cmd = cmd;
),
TP_printk("afu%i.%i pe=%i cmd=%s",
__entry->card,
__entry->afu,
__entry->pe,
__print_symbolic_u64(__entry->cmd, LLCMD_NAMES)
)
);
TRACE_EVENT(cxl_llcmd_done,
TP_PROTO(struct cxl_context *ctx, u64 cmd, int rc),
TP_ARGS(ctx, cmd, rc),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u16, pe)
__field(u64, cmd)
__field(int, rc)
),
TP_fast_assign(
__entry->card = ctx->afu->adapter->adapter_num;
__entry->afu = ctx->afu->slice;
__entry->pe = ctx->pe;
__entry->rc = rc;
__entry->cmd = cmd;
),
TP_printk("afu%i.%i pe=%i cmd=%s rc=%i",
__entry->card,
__entry->afu,
__entry->pe,
__print_symbolic_u64(__entry->cmd, LLCMD_NAMES),
__entry->rc
)
);
DECLARE_EVENT_CLASS(cxl_afu_psl_ctrl,
TP_PROTO(struct cxl_afu *afu, u64 cmd),
TP_ARGS(afu, cmd),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u64, cmd)
),
TP_fast_assign(
__entry->card = afu->adapter->adapter_num;
__entry->afu = afu->slice;
__entry->cmd = cmd;
),
TP_printk("afu%i.%i cmd=%s",
__entry->card,
__entry->afu,
__print_symbolic_u64(__entry->cmd, AFU_COMMANDS)
)
);
DECLARE_EVENT_CLASS(cxl_afu_psl_ctrl_done,
TP_PROTO(struct cxl_afu *afu, u64 cmd, int rc),
TP_ARGS(afu, cmd, rc),
TP_STRUCT__entry(
__field(u8, card)
__field(u8, afu)
__field(u64, cmd)
__field(int, rc)
),
TP_fast_assign(
__entry->card = afu->adapter->adapter_num;
__entry->afu = afu->slice;
__entry->rc = rc;
__entry->cmd = cmd;
),
TP_printk("afu%i.%i cmd=%s rc=%i",
__entry->card,
__entry->afu,
__print_symbolic_u64(__entry->cmd, AFU_COMMANDS),
__entry->rc
)
);
DEFINE_EVENT(cxl_afu_psl_ctrl, cxl_afu_ctrl,
TP_PROTO(struct cxl_afu *afu, u64 cmd),
TP_ARGS(afu, cmd)
);
DEFINE_EVENT(cxl_afu_psl_ctrl_done, cxl_afu_ctrl_done,
TP_PROTO(struct cxl_afu *afu, u64 cmd, int rc),
TP_ARGS(afu, cmd, rc)
);
DEFINE_EVENT_PRINT(cxl_afu_psl_ctrl, cxl_psl_ctrl,
TP_PROTO(struct cxl_afu *afu, u64 cmd),
TP_ARGS(afu, cmd),
TP_printk("psl%i.%i cmd=%s",
__entry->card,
__entry->afu,
__print_symbolic_u64(__entry->cmd, PSL_COMMANDS)
)
);
DEFINE_EVENT_PRINT(cxl_afu_psl_ctrl_done, cxl_psl_ctrl_done,
TP_PROTO(struct cxl_afu *afu, u64 cmd, int rc),
TP_ARGS(afu, cmd, rc),
TP_printk("psl%i.%i cmd=%s rc=%i",
__entry->card,
__entry->afu,
__print_symbolic_u64(__entry->cmd, PSL_COMMANDS),
__entry->rc
)
);
DEFINE_EVENT(cxl_pe_class, cxl_slbia,
TP_PROTO(struct cxl_context *ctx),
TP_ARGS(ctx)
);
TRACE_EVENT(cxl_hcall,
TP_PROTO(u64 unit_address, u64 process_token, long rc),
TP_ARGS(unit_address, process_token, rc),
TP_STRUCT__entry(
__field(u64, unit_address)
__field(u64, process_token)
__field(long, rc)
),
TP_fast_assign(
__entry->unit_address = unit_address;
__entry->process_token = process_token;
__entry->rc = rc;
),
TP_printk("unit_address=0x%016llx process_token=0x%016llx rc=%li",
__entry->unit_address,
__entry->process_token,
__entry->rc
)
);
TRACE_EVENT(cxl_hcall_control,
TP_PROTO(u64 unit_address, char *fct, u64 p1, u64 p2, u64 p3,
u64 p4, unsigned long r4, long rc),
TP_ARGS(unit_address, fct, p1, p2, p3, p4, r4, rc),
TP_STRUCT__entry(
__field(u64, unit_address)
__field(char *, fct)
__field(u64, p1)
__field(u64, p2)
__field(u64, p3)
__field(u64, p4)
__field(unsigned long, r4)
__field(long, rc)
),
TP_fast_assign(
__entry->unit_address = unit_address;
__entry->fct = fct;
__entry->p1 = p1;
__entry->p2 = p2;
__entry->p3 = p3;
__entry->p4 = p4;
__entry->r4 = r4;
__entry->rc = rc;
),
TP_printk("unit_address=%#.16llx %s(%#llx, %#llx, %#llx, %#llx, R4: %#lx)): %li",
__entry->unit_address,
__entry->fct,
__entry->p1,
__entry->p2,
__entry->p3,
__entry->p4,
__entry->r4,
__entry->rc
)
);
TRACE_EVENT(cxl_hcall_attach,
TP_PROTO(u64 unit_address, u64 phys_addr, unsigned long process_token,
unsigned long mmio_addr, unsigned long mmio_size, long rc),
TP_ARGS(unit_address, phys_addr, process_token,
mmio_addr, mmio_size, rc),
TP_STRUCT__entry(
__field(u64, unit_address)
__field(u64, phys_addr)
__field(unsigned long, process_token)
__field(unsigned long, mmio_addr)
__field(unsigned long, mmio_size)
__field(long, rc)
),
TP_fast_assign(
__entry->unit_address = unit_address;
__entry->phys_addr = phys_addr;
__entry->process_token = process_token;
__entry->mmio_addr = mmio_addr;
__entry->mmio_size = mmio_size;
__entry->rc = rc;
),
TP_printk("unit_address=0x%016llx phys_addr=0x%016llx "
"token=0x%.8lx mmio_addr=0x%lx mmio_size=0x%lx rc=%li",
__entry->unit_address,
__entry->phys_addr,
__entry->process_token,
__entry->mmio_addr,
__entry->mmio_size,
__entry->rc
)
);
DEFINE_EVENT(cxl_hcall, cxl_hcall_detach,
TP_PROTO(u64 unit_address, u64 process_token, long rc),
TP_ARGS(unit_address, process_token, rc)
);
DEFINE_EVENT(cxl_hcall_control, cxl_hcall_control_function,
TP_PROTO(u64 unit_address, char *fct, u64 p1, u64 p2, u64 p3,
u64 p4, unsigned long r4, long rc),
TP_ARGS(unit_address, fct, p1, p2, p3, p4, r4, rc)
);
DEFINE_EVENT(cxl_hcall, cxl_hcall_collect_int_info,
TP_PROTO(u64 unit_address, u64 process_token, long rc),
TP_ARGS(unit_address, process_token, rc)
);
TRACE_EVENT(cxl_hcall_control_faults,
TP_PROTO(u64 unit_address, u64 process_token,
u64 control_mask, u64 reset_mask, unsigned long r4,
long rc),
TP_ARGS(unit_address, process_token,
control_mask, reset_mask, r4, rc),
TP_STRUCT__entry(
__field(u64, unit_address)
__field(u64, process_token)
__field(u64, control_mask)
__field(u64, reset_mask)
__field(unsigned long, r4)
__field(long, rc)
),
TP_fast_assign(
__entry->unit_address = unit_address;
__entry->process_token = process_token;
__entry->control_mask = control_mask;
__entry->reset_mask = reset_mask;
__entry->r4 = r4;
__entry->rc = rc;
),
TP_printk("unit_address=0x%016llx process_token=0x%llx "
"control_mask=%#llx reset_mask=%#llx r4=%#lx rc=%li",
__entry->unit_address,
__entry->process_token,
__entry->control_mask,
__entry->reset_mask,
__entry->r4,
__entry->rc
)
);
DEFINE_EVENT(cxl_hcall_control, cxl_hcall_control_facility,
TP_PROTO(u64 unit_address, char *fct, u64 p1, u64 p2, u64 p3,
u64 p4, unsigned long r4, long rc),
TP_ARGS(unit_address, fct, p1, p2, p3, p4, r4, rc)
);
TRACE_EVENT(cxl_hcall_download_facility,
TP_PROTO(u64 unit_address, char *fct, u64 list_address, u64 num,
unsigned long r4, long rc),
TP_ARGS(unit_address, fct, list_address, num, r4, rc),
TP_STRUCT__entry(
__field(u64, unit_address)
__field(char *, fct)
__field(u64, list_address)
__field(u64, num)
__field(unsigned long, r4)
__field(long, rc)
),
TP_fast_assign(
__entry->unit_address = unit_address;
__entry->fct = fct;
__entry->list_address = list_address;
__entry->num = num;
__entry->r4 = r4;
__entry->rc = rc;
),
TP_printk("%#.16llx, %s(%#llx, %#llx), %#lx): %li",
__entry->unit_address,
__entry->fct,
__entry->list_address,
__entry->num,
__entry->r4,
__entry->rc
)
);
#endif /* _CXL_TRACE_H */
/* This part must be outside protection */
#undef TRACE_INCLUDE_PATH
#define TRACE_INCLUDE_PATH .
#define TRACE_INCLUDE_FILE trace
#include <trace/define_trace.h>

309
drivers/misc/cxl/vphb.c
View File

@ -1,309 +0,0 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* Copyright 2014 IBM Corp.
*/
#include <linux/pci.h>
#include <misc/cxl.h>
#include "cxl.h"
static int cxl_pci_probe_mode(struct pci_bus *bus)
{
return PCI_PROBE_NORMAL;
}
static int cxl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type)
{
return -ENODEV;
}
static void cxl_teardown_msi_irqs(struct pci_dev *pdev)
{
/*
* MSI should never be set but need still need to provide this call
* back.
*/
}
static bool cxl_pci_enable_device_hook(struct pci_dev *dev)
{
struct pci_controller *phb;
struct cxl_afu *afu;
struct cxl_context *ctx;
phb = pci_bus_to_host(dev->bus);
afu = (struct cxl_afu *)phb->private_data;
if (!cxl_ops->link_ok(afu->adapter, afu)) {
dev_warn(&dev->dev, "%s: Device link is down, refusing to enable AFU\n", __func__);
return false;
}
dev->dev.archdata.dma_offset = PAGE_OFFSET;
/*
* Allocate a context to do cxl things too. If we eventually do real
* DMA ops, we'll need a default context to attach them to
*/
ctx = cxl_dev_context_init(dev);
if (IS_ERR(ctx))
return false;
dev->dev.archdata.cxl_ctx = ctx;
return (cxl_ops->afu_check_and_enable(afu) == 0);
}
static void cxl_pci_disable_device(struct pci_dev *dev)
{
struct cxl_context *ctx = cxl_get_context(dev);
if (ctx) {
if (ctx->status == STARTED) {
dev_err(&dev->dev, "Default context started\n");
return;
}
dev->dev.archdata.cxl_ctx = NULL;
cxl_release_context(ctx);
}
}
static void cxl_pci_reset_secondary_bus(struct pci_dev *dev)
{
/* Should we do an AFU reset here ? */
}
static int cxl_pcie_cfg_record(u8 bus, u8 devfn)
{
return (bus << 8) + devfn;
}
static inline struct cxl_afu *pci_bus_to_afu(struct pci_bus *bus)
{
struct pci_controller *phb = bus ? pci_bus_to_host(bus) : NULL;
return phb ? phb->private_data : NULL;
}
static void cxl_afu_configured_put(struct cxl_afu *afu)
{
atomic_dec_if_positive(&afu->configured_state);
}
static bool cxl_afu_configured_get(struct cxl_afu *afu)
{
return atomic_inc_unless_negative(&afu->configured_state);
}
static inline int cxl_pcie_config_info(struct pci_bus *bus, unsigned int devfn,
struct cxl_afu *afu, int *_record)
{
int record;
record = cxl_pcie_cfg_record(bus->number, devfn);
if (record > afu->crs_num)
return PCIBIOS_DEVICE_NOT_FOUND;
*_record = record;
return 0;
}
static int cxl_pcie_read_config(struct pci_bus *bus, unsigned int devfn,
int offset, int len, u32 *val)
{
int rc, record;
struct cxl_afu *afu;
u8 val8;
u16 val16;
u32 val32;
afu = pci_bus_to_afu(bus);
/* Grab a reader lock on afu. */
if (afu == NULL || !cxl_afu_configured_get(afu))
return PCIBIOS_DEVICE_NOT_FOUND;
rc = cxl_pcie_config_info(bus, devfn, afu, &record);
if (rc)
goto out;
switch (len) {
case 1:
rc = cxl_ops->afu_cr_read8(afu, record, offset, &val8);
*val = val8;
break;
case 2:
rc = cxl_ops->afu_cr_read16(afu, record, offset, &val16);
*val = val16;
break;
case 4:
rc = cxl_ops->afu_cr_read32(afu, record, offset, &val32);
*val = val32;
break;
default:
WARN_ON(1);
}
out:
cxl_afu_configured_put(afu);
return rc ? PCIBIOS_DEVICE_NOT_FOUND : 0;
}
static int cxl_pcie_write_config(struct pci_bus *bus, unsigned int devfn,
int offset, int len, u32 val)
{
int rc, record;
struct cxl_afu *afu;
afu = pci_bus_to_afu(bus);
/* Grab a reader lock on afu. */
if (afu == NULL || !cxl_afu_configured_get(afu))
return PCIBIOS_DEVICE_NOT_FOUND;
rc = cxl_pcie_config_info(bus, devfn, afu, &record);
if (rc)
goto out;
switch (len) {
case 1:
rc = cxl_ops->afu_cr_write8(afu, record, offset, val & 0xff);
break;
case 2:
rc = cxl_ops->afu_cr_write16(afu, record, offset, val & 0xffff);
break;
case 4:
rc = cxl_ops->afu_cr_write32(afu, record, offset, val);
break;
default:
WARN_ON(1);
}
out:
cxl_afu_configured_put(afu);
return rc ? PCIBIOS_SET_FAILED : 0;
}
static struct pci_ops cxl_pcie_pci_ops =
{
.read = cxl_pcie_read_config,
.write = cxl_pcie_write_config,
};
static struct pci_controller_ops cxl_pci_controller_ops =
{
.probe_mode = cxl_pci_probe_mode,
.enable_device_hook = cxl_pci_enable_device_hook,
.disable_device = cxl_pci_disable_device,
.release_device = cxl_pci_disable_device,
.reset_secondary_bus = cxl_pci_reset_secondary_bus,
.setup_msi_irqs = cxl_setup_msi_irqs,
.teardown_msi_irqs = cxl_teardown_msi_irqs,
};
int cxl_pci_vphb_add(struct cxl_afu *afu)
{
struct pci_controller *phb;
struct device_node *vphb_dn;
struct device *parent;
/*
* If there are no AFU configuration records we won't have anything to
* expose under the vPHB, so skip creating one, returning success since
* this is still a valid case. This will also opt us out of EEH
* handling since we won't have anything special to do if there are no
* kernel drivers attached to the vPHB, and EEH handling is not yet
* supported in the peer model.
*/
if (!afu->crs_num)
return 0;
/* The parent device is the adapter. Reuse the device node of
* the adapter.
* We don't seem to care what device node is used for the vPHB,
* but tools such as lsvpd walk up the device parents looking
* for a valid location code, so we might as well show devices
* attached to the adapter as being located on that adapter.
*/
parent = afu->adapter->dev.parent;
vphb_dn = parent->of_node;
/* Alloc and setup PHB data structure */
phb = pcibios_alloc_controller(vphb_dn);
if (!phb)
return -ENODEV;
/* Setup parent in sysfs */
phb->parent = parent;
/* Setup the PHB using arch provided callback */
phb->ops = &cxl_pcie_pci_ops;
phb->cfg_addr = NULL;
phb->cfg_data = NULL;
phb->private_data = afu;
phb->controller_ops = cxl_pci_controller_ops;
/* Scan the bus */
pcibios_scan_phb(phb);
if (phb->bus == NULL)
return -ENXIO;
/* Set release hook on root bus */
pci_set_host_bridge_release(to_pci_host_bridge(phb->bus->bridge),
pcibios_free_controller_deferred,
(void *) phb);
/* Claim resources. This might need some rework as well depending
* whether we are doing probe-only or not, like assigning unassigned
* resources etc...
*/
pcibios_claim_one_bus(phb->bus);
/* Add probed PCI devices to the device model */
pci_bus_add_devices(phb->bus);
afu->phb = phb;
return 0;
}
void cxl_pci_vphb_remove(struct cxl_afu *afu)
{
struct pci_controller *phb;
/* If there is no configuration record we won't have one of these */
if (!afu || !afu->phb)
return;
phb = afu->phb;
afu->phb = NULL;
pci_remove_root_bus(phb->bus);
/*
* We don't free phb here - that's handled by
* pcibios_free_controller_deferred()
*/
}
bool cxl_pci_is_vphb_device(struct pci_dev *dev)
{
struct pci_controller *phb;
phb = pci_bus_to_host(dev->bus);
return (phb->ops == &cxl_pcie_pci_ops);
}
struct cxl_afu *cxl_pci_to_afu(struct pci_dev *dev)
{
struct pci_controller *phb;
phb = pci_bus_to_host(dev->bus);
return (struct cxl_afu *)phb->private_data;
}
EXPORT_SYMBOL_GPL(cxl_pci_to_afu);
unsigned int cxl_pci_to_cfg_record(struct pci_dev *dev)
{
return cxl_pcie_cfg_record(dev->bus->number, dev->devfn);
}
EXPORT_SYMBOL_GPL(cxl_pci_to_cfg_record);

48
include/misc/cxl-base.h
View File

@ -1,48 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright 2014 IBM Corp.
*/
#ifndef _MISC_CXL_BASE_H
#define _MISC_CXL_BASE_H
#ifdef CONFIG_CXL_BASE
#define CXL_IRQ_RANGES 4
struct cxl_irq_ranges {
irq_hw_number_t offset[CXL_IRQ_RANGES];
irq_hw_number_t range[CXL_IRQ_RANGES];
};
extern atomic_t cxl_use_count;
static inline bool cxl_ctx_in_use(void)
{
return (atomic_read(&cxl_use_count) != 0);
}
static inline void cxl_ctx_get(void)
{
atomic_inc(&cxl_use_count);
}
static inline void cxl_ctx_put(void)
{
atomic_dec(&cxl_use_count);
}
struct cxl_afu *cxl_afu_get(struct cxl_afu *afu);
void cxl_afu_put(struct cxl_afu *afu);
void cxl_slbia(struct mm_struct *mm);
#else /* CONFIG_CXL_BASE */
static inline bool cxl_ctx_in_use(void) { return false; }
static inline struct cxl_afu *cxl_afu_get(struct cxl_afu *afu) { return NULL; }
static inline void cxl_afu_put(struct cxl_afu *afu) {}
static inline void cxl_slbia(struct mm_struct *mm) {}
#endif /* CONFIG_CXL_BASE */
#endif

265
include/misc/cxl.h
View File

@ -1,265 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright 2015 IBM Corp.
*/
#ifndef _MISC_CXL_H
#define _MISC_CXL_H
#include <linux/pci.h>
#include <linux/poll.h>
#include <linux/interrupt.h>
#include <uapi/misc/cxl.h>
/*
* This documents the in kernel API for driver to use CXL. It allows kernel
* drivers to bind to AFUs using an AFU configuration record exposed as a PCI
* configuration record.
*
* This API enables control over AFU and contexts which can't be part of the
* generic PCI API. This API is agnostic to the actual AFU.
*/
/* Get the AFU associated with a pci_dev */
struct cxl_afu *cxl_pci_to_afu(struct pci_dev *dev);
/* Get the AFU conf record number associated with a pci_dev */
unsigned int cxl_pci_to_cfg_record(struct pci_dev *dev);
/*
* Context lifetime overview:
*
* An AFU context may be inited and then started and stopped multiple times
* before it's released. ie.
* - cxl_dev_context_init()
* - cxl_start_context()
* - cxl_stop_context()
* - cxl_start_context()
* - cxl_stop_context()
* ...repeat...
* - cxl_release_context()
* Once released, a context can't be started again.
*
* One context is inited by the cxl driver for every pci_dev. This is to be
* used as a default kernel context. cxl_get_context() will get this
* context. This context will be released by PCI hot unplug, so doesn't need to
* be released explicitly by drivers.
*
* Additional kernel contexts may be inited using cxl_dev_context_init().
* These must be released using cxl_context_detach().
*
* Once a context has been inited, IRQs may be configured. Firstly these IRQs
* must be allocated (cxl_allocate_afu_irqs()), then individually mapped to
* specific handlers (cxl_map_afu_irq()).
*
* These IRQs can be unmapped (cxl_unmap_afu_irq()) and finally released
* (cxl_free_afu_irqs()).
*
* The AFU can be reset (cxl_afu_reset()). This will cause the PSL/AFU
* hardware to lose track of all contexts. It's upto the caller of
* cxl_afu_reset() to restart these contexts.
*/
/*
* On pci_enabled_device(), the cxl driver will init a single cxl context for
* use by the driver. It doesn't start this context (as that will likely
* generate DMA traffic for most AFUs).
*
* This gets the default context associated with this pci_dev. This context
* doesn't need to be released as this will be done by the PCI subsystem on hot
* unplug.
*/
struct cxl_context *cxl_get_context(struct pci_dev *dev);
/*
* Allocate and initalise a context associated with a AFU PCI device. This
* doesn't start the context in the AFU.
*/
struct cxl_context *cxl_dev_context_init(struct pci_dev *dev);
/*
* Release and free a context. Context should be stopped before calling.
*/
int cxl_release_context(struct cxl_context *ctx);
/*
* Set and get private data associated with a context. Allows drivers to have a
* back pointer to some useful structure.
*/
int cxl_set_priv(struct cxl_context *ctx, void *priv);
void *cxl_get_priv(struct cxl_context *ctx);
/*
* Allocate AFU interrupts for this context. num=0 will allocate the default
* for this AFU as given in the AFU descriptor. This number doesn't include the
* interrupt 0 (CAIA defines AFU IRQ 0 for page faults). Each interrupt to be
* used must map a handler with cxl_map_afu_irq.
*/
int cxl_allocate_afu_irqs(struct cxl_context *cxl, int num);
/* Free allocated interrupts */
void cxl_free_afu_irqs(struct cxl_context *cxl);
/*
* Map a handler for an AFU interrupt associated with a particular context. AFU
* IRQS numbers start from 1 (CAIA defines AFU IRQ 0 for page faults). cookie
* is private data is that will be provided to the interrupt handler.
*/
int cxl_map_afu_irq(struct cxl_context *cxl, int num,
irq_handler_t handler, void *cookie, char *name);
/* unmap mapped IRQ handlers */
void cxl_unmap_afu_irq(struct cxl_context *cxl, int num, void *cookie);
/*
* Start work on the AFU. This starts an cxl context and associates it with a
* task. task == NULL will make it a kernel context.
*/
int cxl_start_context(struct cxl_context *ctx, u64 wed,
struct task_struct *task);
/*
* Stop a context and remove it from the PSL
*/
int cxl_stop_context(struct cxl_context *ctx);
/* Reset the AFU */
int cxl_afu_reset(struct cxl_context *ctx);
/*
* Set a context as a master context.
* This sets the default problem space area mapped as the full space, rather
* than just the per context area (for slaves).
*/
void cxl_set_master(struct cxl_context *ctx);
/*
* Map and unmap the AFU Problem Space area. The amount and location mapped
* depends on if this context is a master or slave.
*/
void __iomem *cxl_psa_map(struct cxl_context *ctx);
void cxl_psa_unmap(void __iomem *addr);
/* Get the process element for this context */
int cxl_process_element(struct cxl_context *ctx);
/*
* These calls allow drivers to create their own file descriptors and make them
* identical to the cxl file descriptor user API. An example use case:
*
* struct file_operations cxl_my_fops = {};
* ......
* // Init the context
* ctx = cxl_dev_context_init(dev);
* if (IS_ERR(ctx))
* return PTR_ERR(ctx);
* // Create and attach a new file descriptor to my file ops
* file = cxl_get_fd(ctx, &cxl_my_fops, &fd);
* // Start context
* rc = cxl_start_work(ctx, &work.work);
* if (rc) {
* fput(file);
* put_unused_fd(fd);
* return -ENODEV;
* }
* // No error paths after installing the fd
* fd_install(fd, file);
* return fd;
*
* This inits a context, and gets a file descriptor and associates some file
* ops to that file descriptor. If the file ops are blank, the cxl driver will
* fill them in with the default ones that mimic the standard user API. Once
* completed, the file descriptor can be installed. Once the file descriptor is
* installed, it's visible to the user so no errors must occur past this point.
*
* If cxl_fd_release() file op call is installed, the context will be stopped
* and released when the fd is released. Hence the driver won't need to manage
* this itself.
*/
/*
* Take a context and associate it with my file ops. Returns the associated
* file and file descriptor. Any file ops which are blank are filled in by the
* cxl driver with the default ops to mimic the standard API.
*/
struct file *cxl_get_fd(struct cxl_context *ctx, struct file_operations *fops,
int *fd);
/* Get the context associated with this file */
struct cxl_context *cxl_fops_get_context(struct file *file);
/*
* Start a context associated a struct cxl_ioctl_start_work used by the
* standard cxl user API.
*/
int cxl_start_work(struct cxl_context *ctx,
struct cxl_ioctl_start_work *work);
/*
* Export all the existing fops so drivers can use them
*/
int cxl_fd_open(struct inode *inode, struct file *file);
int cxl_fd_release(struct inode *inode, struct file *file);
long cxl_fd_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
int cxl_fd_mmap(struct file *file, struct vm_area_struct *vm);
__poll_t cxl_fd_poll(struct file *file, struct poll_table_struct *poll);
ssize_t cxl_fd_read(struct file *file, char __user *buf, size_t count,
loff_t *off);
/*
* For EEH, a driver may want to assert a PERST will reload the same image
* from flash into the FPGA.
*
* This is a property of the entire adapter, not a single AFU, so drivers
* should set this property with care!
*/
void cxl_perst_reloads_same_image(struct cxl_afu *afu,
bool perst_reloads_same_image);
/*
* Read the VPD for the card where the AFU resides
*/
ssize_t cxl_read_adapter_vpd(struct pci_dev *dev, void *buf, size_t count);
/*
* AFU driver ops allow an AFU driver to create their own events to pass to
* userspace through the file descriptor as a simpler alternative to overriding
* the read() and poll() calls that works with the generic cxl events. These
* events are given priority over the generic cxl events, so they will be
* delivered first if multiple types of events are pending.
*
* The AFU driver must call cxl_context_events_pending() to notify the cxl
* driver that new events are ready to be delivered for a specific context.
* cxl_context_events_pending() will adjust the current count of AFU driver
* events for this context, and wake up anyone waiting on the context wait
* queue.
*
* The cxl driver will then call fetch_event() to get a structure defining
* the size and address of the driver specific event data. The cxl driver
* will build a cxl header with type and process_element fields filled in,
* and header.size set to sizeof(struct cxl_event_header) + data_size.
* The total size of the event is limited to CXL_READ_MIN_SIZE (4K).
*
* fetch_event() is called with a spin lock held, so it must not sleep.
*
* The cxl driver will then deliver the event to userspace, and finally
* call event_delivered() to return the status of the operation, identified
* by cxl context and AFU driver event data pointers.
* 0 Success
* -EFAULT copy_to_user() has failed
* -EINVAL Event data pointer is NULL, or event size is greater than
* CXL_READ_MIN_SIZE.
*/
struct cxl_afu_driver_ops {
struct cxl_event_afu_driver_reserved *(*fetch_event) (
struct cxl_context *ctx);
void (*event_delivered) (struct cxl_context *ctx,
struct cxl_event_afu_driver_reserved *event,
int rc);
};
/*
* Associate the above driver ops with a specific context.
* Reset the current count of AFU driver events.
*/
void cxl_set_driver_ops(struct cxl_context *ctx,
struct cxl_afu_driver_ops *ops);
/* Notify cxl driver that new events are ready to be delivered for context */
void cxl_context_events_pending(struct cxl_context *ctx,
unsigned int new_events);
#endif /* _MISC_CXL_H */

129
include/misc/cxllib.h
View File

@ -1,129 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* Copyright 2017 IBM Corp.
*/
#ifndef _MISC_CXLLIB_H
#define _MISC_CXLLIB_H
#include <linux/pci.h>
#include <asm/reg.h>
/*
* cxl driver exports a in-kernel 'library' API which can be called by
* other drivers to help interacting with an IBM XSL.
*/
/*
* tells whether capi is supported on the PCIe slot where the
* device is seated
*
* Input:
* dev: device whose slot needs to be checked
* flags: 0 for the time being
*/
bool cxllib_slot_is_supported(struct pci_dev *dev, unsigned long flags);
/*
* Returns the configuration parameters to be used by the XSL or device
*
* Input:
* dev: device, used to find PHB
* Output:
* struct cxllib_xsl_config:
* version
* capi BAR address, i.e. 0x2000000000000-0x2FFFFFFFFFFFF
* capi BAR size
* data send control (XSL_DSNCTL)
* dummy read address (XSL_DRA)
*/
#define CXL_XSL_CONFIG_VERSION1 1
struct cxllib_xsl_config {
u32 version; /* format version for register encoding */
u32 log_bar_size;/* log size of the capi_window */
u64 bar_addr; /* address of the start of capi window */
u64 dsnctl; /* matches definition of XSL_DSNCTL */
u64 dra; /* real address that can be used for dummy read */
};
int cxllib_get_xsl_config(struct pci_dev *dev, struct cxllib_xsl_config *cfg);
/*
* Activate capi for the pci host bridge associated with the device.
* Can be extended to deactivate once we know how to do it.
* Device must be ready to accept messages from the CAPP unit and
* respond accordingly (TLB invalidates, ...)
*
* PHB is switched to capi mode through calls to skiboot.
* CAPP snooping is activated
*
* Input:
* dev: device whose PHB should switch mode
* mode: mode to switch to i.e. CAPI or PCI
* flags: options related to the mode
*/
enum cxllib_mode {
CXL_MODE_CXL,
CXL_MODE_PCI,
};
#define CXL_MODE_NO_DMA 0
#define CXL_MODE_DMA_TVT0 1
#define CXL_MODE_DMA_TVT1 2
int cxllib_switch_phb_mode(struct pci_dev *dev, enum cxllib_mode mode,
unsigned long flags);
/*
* Set the device for capi DMA.
* Define its dma_ops and dma offset so that allocations will be using TVT#1
*
* Input:
* dev: device to set
* flags: options. CXL_MODE_DMA_TVT1 should be used
*/
int cxllib_set_device_dma(struct pci_dev *dev, unsigned long flags);
/*
* Get the Process Element structure for the given thread
*
* Input:
* task: task_struct for the context of the translation
* translation_mode: whether addresses should be translated
* Output:
* attr: attributes to fill up the Process Element structure from CAIA
*/
struct cxllib_pe_attributes {
u64 sr;
u32 lpid;
u32 tid;
u32 pid;
};
#define CXL_TRANSLATED_MODE 0
#define CXL_REAL_MODE 1
int cxllib_get_PE_attributes(struct task_struct *task,
unsigned long translation_mode, struct cxllib_pe_attributes *attr);
/*
* Handle memory fault.
* Fault in all the pages of the specified buffer for the permissions
* provided in ‘flags’
*
* Shouldn't be called from interrupt context
*
* Input:
* mm: struct mm for the thread faulting the pages
* addr: base address of the buffer to page in
* size: size of the buffer to page in
* flags: permission requested (DSISR_ISSTORE...)
*/
int cxllib_handle_fault(struct mm_struct *mm, u64 addr, u64 size, u64 flags);
#endif /* _MISC_CXLLIB_H */

156
include/uapi/misc/cxl.h
View File

@ -1,156 +0,0 @@
/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
/*
* Copyright 2014 IBM Corp.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*/
#ifndef _UAPI_MISC_CXL_H
#define _UAPI_MISC_CXL_H
#include <linux/types.h>
#include <linux/ioctl.h>
struct cxl_ioctl_start_work {
__u64 flags;
__u64 work_element_descriptor;
__u64 amr;
__s16 num_interrupts;
__u16 tid;
__s32 reserved1;
__u64 reserved2;
__u64 reserved3;
__u64 reserved4;
__u64 reserved5;
};
#define CXL_START_WORK_AMR 0x0000000000000001ULL
#define CXL_START_WORK_NUM_IRQS 0x0000000000000002ULL
#define CXL_START_WORK_ERR_FF 0x0000000000000004ULL
#define CXL_START_WORK_TID 0x0000000000000008ULL
#define CXL_START_WORK_ALL (CXL_START_WORK_AMR |\
CXL_START_WORK_NUM_IRQS |\
CXL_START_WORK_ERR_FF |\
CXL_START_WORK_TID)
/* Possible modes that an afu can be in */
#define CXL_MODE_DEDICATED 0x1
#define CXL_MODE_DIRECTED 0x2
/* possible flags for the cxl_afu_id flags field */
#define CXL_AFUID_FLAG_SLAVE 0x1 /* In directed-mode afu is in slave mode */
struct cxl_afu_id {
__u64 flags; /* One of CXL_AFUID_FLAG_X */
__u32 card_id;
__u32 afu_offset;
__u32 afu_mode; /* one of the CXL_MODE_X */
__u32 reserved1;
__u64 reserved2;
__u64 reserved3;
__u64 reserved4;
__u64 reserved5;
__u64 reserved6;
};
/* base adapter image header is included in the image */
#define CXL_AI_NEED_HEADER 0x0000000000000001ULL
#define CXL_AI_ALL CXL_AI_NEED_HEADER
#define CXL_AI_HEADER_SIZE 128
#define CXL_AI_BUFFER_SIZE 4096
#define CXL_AI_MAX_ENTRIES 256
#define CXL_AI_MAX_CHUNK_SIZE (CXL_AI_BUFFER_SIZE * CXL_AI_MAX_ENTRIES)
struct cxl_adapter_image {
__u64 flags;
__u64 data;
__u64 len_data;
__u64 len_image;
__u64 reserved1;
__u64 reserved2;
__u64 reserved3;
__u64 reserved4;
};
/* ioctl numbers */
#define CXL_MAGIC 0xCA
/* AFU devices */
#define CXL_IOCTL_START_WORK _IOW(CXL_MAGIC, 0x00, struct cxl_ioctl_start_work)
#define CXL_IOCTL_GET_PROCESS_ELEMENT _IOR(CXL_MAGIC, 0x01, __u32)
#define CXL_IOCTL_GET_AFU_ID _IOR(CXL_MAGIC, 0x02, struct cxl_afu_id)
/* adapter devices */
#define CXL_IOCTL_DOWNLOAD_IMAGE _IOW(CXL_MAGIC, 0x0A, struct cxl_adapter_image)
#define CXL_IOCTL_VALIDATE_IMAGE _IOW(CXL_MAGIC, 0x0B, struct cxl_adapter_image)
#define CXL_READ_MIN_SIZE 0x1000 /* 4K */
/* Events from read() */
enum cxl_event_type {
CXL_EVENT_RESERVED = 0,
CXL_EVENT_AFU_INTERRUPT = 1,
CXL_EVENT_DATA_STORAGE = 2,
CXL_EVENT_AFU_ERROR = 3,
CXL_EVENT_AFU_DRIVER = 4,
};
struct cxl_event_header {
__u16 type;
__u16 size;
__u16 process_element;
__u16 reserved1;
};
struct cxl_event_afu_interrupt {
__u16 flags;
__u16 irq; /* Raised AFU interrupt number */
__u32 reserved1;
};
struct cxl_event_data_storage {
__u16 flags;
__u16 reserved1;
__u32 reserved2;
__u64 addr;
__u64 dsisr;
__u64 reserved3;
};
struct cxl_event_afu_error {
__u16 flags;
__u16 reserved1;
__u32 reserved2;
__u64 error;
};
struct cxl_event_afu_driver_reserved {
/*
* Defines the buffer passed to the cxl driver by the AFU driver.
*
* This is not ABI since the event header.size passed to the user for
* existing events is set in the read call to sizeof(cxl_event_header)
* + sizeof(whatever event is being dispatched) and the user is already
* required to use a 4K buffer on the read call.
*
* Of course the contents will be ABI, but that's up the AFU driver.
*/
__u32 data_size;
__u8 data[];
};
struct cxl_event {
struct cxl_event_header header;
union {
struct cxl_event_afu_interrupt irq;
struct cxl_event_data_storage fault;
struct cxl_event_afu_error afu_error;
struct cxl_event_afu_driver_reserved afu_driver_event;
};
};
#endif /* _UAPI_MISC_CXL_H */