Lines Matching refs:the
9 memory and generate page faults. As a result, the host interface to
10 an adapter running in CAPI mode does not require the data buffers to
11 be mapped to the device's memory (IOMMU bypass) nor does it require
16 This abstraction simplifies the infrastructure and programming
21 directly talk to a device (network or storage) bypassing the typical
25 The CXL Flash Adapter Driver is a kernel module that sits in the
26 SCSI stack as a low level device driver (below the SCSI disk and
27 protocol drivers) for the IBM CXL Flash Adapter. This driver is
28 responsible for the initialization of the adapter, setting up the
30 communicates directly the Flash Accelerator Functional Unit (AFU)
34 operation at the device (LUN) level:
37 regular disk device (i.e.: /dev/sdc). This is the default mode.
41 specifies the means of accessing the device and provides for
42 either raw access to the entire LUN (referred to as direct
44 partition of the LUN (referred to as virtual LUN access). The
46 by special translation services provided by the Flash AFU.
53 granted to it by the kernel or hypervisor allowing it to perform AFU
55 directly in each user I/O, but at the minimum is involved in the
56 initial setup before the user application is allowed to send requests
57 directly to the AFU.
59 The CXL Flash Adapter Driver establishes a master context with the
76 This driver configures itself into the SCSI software stack as an
77 adapter driver. The driver is the only entity that is considered a
78 Trusted Process to program the Provisioning and Control and Global
79 areas in the MMIO Space shown above. The master context driver
80 discovers all LUNs attached to the CXL Flash adapter and instantiates
85 written to a specification provided by the block library may get
86 access to the Flash from user space (without requiring a system call).
90 two modes for accessing the block device.
94 virtual LUNs. The virtual LUNs may be resized as long as the sum of
95 the sizes of all the virtual LUNs, along with the meta-data associated
96 with it does not exceed the physical capacity.
98 The second mode is called the physical mode. In this mode a single
99 block device (/dev/sdb) may be opened directly by the block library
100 and the entire space for the LUN is available to the application.
102 Only the physical mode provides persistence of the data. i.e. The
103 data written to the block device will survive application exit and
105 not survive after the application terminates or the system reboots).
111 Applications intending to get access to the CXL Flash from user
112 space should use the block library, as it abstracts the details of
113 interfacing directly with the cxlflash driver that are necessary for
116 implemented as IOCTLs, that are provided by the cxlflash driver
119 the interface between the block library and the cxlflash driver,
130 Users, such as the block library, that wish to interface with a flash
131 device (LUN) via user space access need to use the services provided
132 by the cxlflash driver. As these services are implemented as ioctls,
134 the communication channel between a user and the kernel. This file
135 descriptor is obtained by opening the device special file associated
136 with the scsi disk device (/dev/sdb) that was created during LUN
137 discovery. As per the location of the cxlflash driver within the
138 SCSI protocol stack, this open is actually not seen by the cxlflash
139 driver. Upon successful open, the user receives a file descriptor
140 (herein referred to as fd1) that should be used for issuing the
149 This ioctl obtains, initializes, and starts a context using the CXL
151 to uniquely identify the context and its allocated resources. The
153 referred to as fd2) that is used by the block library to initiate
154 memory mapped I/O (via mmap()) to the CXL flash device and poll for
156 this driver and not the CXL kernel services to allow for intermediary
157 notification and access in the event of a non-user-initiated close(),
159 detail in the description for the DK_CXLFLASH_DETACH ioctl.
161 There are a few important aspects regarding the "tokens" (context id
162 and fd2) that are provided back to the user:
164 - These tokens are only valid for the process under which they
166 to use the context id or file descriptor created by its parent
169 - These tokens are only valid for the lifetime of the context and
170 the process under which they were created. Once either is
171 destroyed, the tokens are to be considered stale and subsequent
174 - When a context is no longer needed, the user shall detach from
175 the context via the DK_CXLFLASH_DETACH ioctl.
177 - A close on fd2 will invalidate the tokens. This operation is not
178 required by the user.
182 This ioctl is responsible for transitioning the LUN to direct
183 (physical) mode access and configuring the AFU for direct access from
184 user space on a per-context basis. Additionally, the block size and
185 last logical block address (LBA) are returned to the user.
190 requests to use the LUN in a different mode are denied.
193 entry to the AFU's resource handle table. The index of the entry is
194 treated as a resource handle that is returned to the user. The user
195 is then able to use the handle to reference the LUN during I/O.
199 This ioctl is responsible for transitioning the LUN to virtual mode
200 of access and configuring the AFU for virtual access from user space
201 on a per-context basis. Additionally, the block size and last logical
202 block address (LBA) are returned to the user.
207 requests to use the LUN in a different mode are denied.
210 an entry to the AFU's resource handle table. The index of the entry
211 is treated as a resource handle that is returned to the user. The
212 user is then able to use the handle to reference the LUN during I/O.
214 By default, the virtual LUN is created with a size of 0. The user
215 would need to use the DK_CXLFLASH_VLUN_RESIZE ioctl to adjust the grow
216 the virtual LUN to a desired size. To avoid having to perform this
217 resize for the initial creation of the virtual LUN, the user has the
218 option of specifying a size as part of the DK_CXLFLASH_USER_VIRTUAL
219 ioctl, such that when success is returned to the user, the
221 storage. This is reflected by the last LBA being a non-zero value.
227 mode. Upon success, an updated last LBA is returned to the user
228 indicating the new size of the virtual LUN associated with the
231 The partitioning of virtual LUNs is jointly mediated by the cxlflash
232 driver and the AFU. An allocation table is kept for each LUN that is
233 operating in the virtual mode and used to program a LUN translation
234 table that the AFU references when provided with a resource handle.
240 thought of as the inverse of the DK_CXLFLASH_USER_DIRECT or
241 DK_CXLFLASH_USER_VIRTUAL ioctls. Upon success, the resource handle
242 is no longer valid and the entry in the resource handle table is
245 As part of the release process for virtual LUNs, the virtual LUN
246 is first resized to 0 to clear out and free the translation tables
247 associated with the virtual LUN reference.
251 This ioctl is responsible for unregistering a context with the
253 not explicitly released via the DK_CXLFLASH_RELEASE ioctl. Upon
254 success, all "tokens" which had been provided to the user from the
262 forks. Upon success, the child process (which invoked the ioctl)
263 will have access to the same LUNs via the same resource handle(s)
264 and fd2 as the parent, but under a different context.
268 for the child process. This ioctl simplifies the state management
270 forks, child process can clone the parents context by first creating
272 perform the clone from the parent to the child.
275 translation tables are copied from the parent context to the child's
276 and then synced with the AFU.
280 This ioctl is used to detect various changes such as the capacity of
281 the disk changing, the number of LUNs visible changing, etc. In cases
282 where the changes affect the application (such as a LUN resize), the
283 cxlflash driver will report the changed state to the application.
286 changed in response to a check condition. As the user is operating out
287 of band from the kernel, they will see these types of events without
288 the kernel's knowledge. When encountered, the user's architected
297 of a specified user context. Any state associated with the user context
300 User contexts are put into an error condition when the device needs to
302 by seeing all 0xF's on an MMIO read. Upon encountering this, the
305 check if the device is operating normally. If a failure is returned
306 from this ioctl, the user is expected to gracefully clean up their
307 context via release/detach ioctls. Until they do, the context they
308 hold is not relinquished. The user may also optionally exit the process
309 at which time the context/resources they held will be freed as part of
310 the release fop.