1Remote Processor Framework 2 31. Introduction 4 5Modern SoCs typically have heterogeneous remote processor devices in asymmetric 6multiprocessing (AMP) configurations, which may be running different instances 7of operating system, whether it's Linux or any other flavor of real-time OS. 8 9OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. 10In a typical configuration, the dual cortex-A9 is running Linux in a SMP 11configuration, and each of the other three cores (two M3 cores and a DSP) 12is running its own instance of RTOS in an AMP configuration. 13 14The remoteproc framework allows different platforms/architectures to 15control (power on, load firmware, power off) those remote processors while 16abstracting the hardware differences, so the entire driver doesn't need to be 17duplicated. In addition, this framework also adds rpmsg virtio devices 18for remote processors that supports this kind of communication. This way, 19platform-specific remoteproc drivers only need to provide a few low-level 20handlers, and then all rpmsg drivers will then just work 21(for more information about the virtio-based rpmsg bus and its drivers, 22please read Documentation/rpmsg.txt). 23Registration of other types of virtio devices is now also possible. Firmwares 24just need to publish what kind of virtio devices do they support, and then 25remoteproc will add those devices. This makes it possible to reuse the 26existing virtio drivers with remote processor backends at a minimal development 27cost. 28 292. User API 30 31 int rproc_boot(struct rproc *rproc) 32 - Boot a remote processor (i.e. load its firmware, power it on, ...). 33 If the remote processor is already powered on, this function immediately 34 returns (successfully). 35 Returns 0 on success, and an appropriate error value otherwise. 36 Note: to use this function you should already have a valid rproc 37 handle. There are several ways to achieve that cleanly (devres, pdata, 38 the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we 39 might also consider using dev_archdata for this). 40 41 void rproc_shutdown(struct rproc *rproc) 42 - Power off a remote processor (previously booted with rproc_boot()). 43 In case @rproc is still being used by an additional user(s), then 44 this function will just decrement the power refcount and exit, 45 without really powering off the device. 46 Every call to rproc_boot() must (eventually) be accompanied by a call 47 to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. 48 Notes: 49 - we're not decrementing the rproc's refcount, only the power refcount. 50 which means that the @rproc handle stays valid even after 51 rproc_shutdown() returns, and users can still use it with a subsequent 52 rproc_boot(), if needed. 53 543. Typical usage 55 56#include <linux/remoteproc.h> 57 58/* in case we were given a valid 'rproc' handle */ 59int dummy_rproc_example(struct rproc *my_rproc) 60{ 61 int ret; 62 63 /* let's power on and boot our remote processor */ 64 ret = rproc_boot(my_rproc); 65 if (ret) { 66 /* 67 * something went wrong. handle it and leave. 68 */ 69 } 70 71 /* 72 * our remote processor is now powered on... give it some work 73 */ 74 75 /* let's shut it down now */ 76 rproc_shutdown(my_rproc); 77} 78 794. API for implementors 80 81 struct rproc *rproc_alloc(struct device *dev, const char *name, 82 const struct rproc_ops *ops, 83 const char *firmware, int len) 84 - Allocate a new remote processor handle, but don't register 85 it yet. Required parameters are the underlying device, the 86 name of this remote processor, platform-specific ops handlers, 87 the name of the firmware to boot this rproc with, and the 88 length of private data needed by the allocating rproc driver (in bytes). 89 90 This function should be used by rproc implementations during 91 initialization of the remote processor. 92 After creating an rproc handle using this function, and when ready, 93 implementations should then call rproc_add() to complete 94 the registration of the remote processor. 95 On success, the new rproc is returned, and on failure, NULL. 96 97 Note: _never_ directly deallocate @rproc, even if it was not registered 98 yet. Instead, when you need to unroll rproc_alloc(), use rproc_put(). 99 100 void rproc_put(struct rproc *rproc) 101 - Free an rproc handle that was allocated by rproc_alloc. 102 This function essentially unrolls rproc_alloc(), by decrementing the 103 rproc's refcount. It doesn't directly free rproc; that would happen 104 only if there are no other references to rproc and its refcount now 105 dropped to zero. 106 107 int rproc_add(struct rproc *rproc) 108 - Register @rproc with the remoteproc framework, after it has been 109 allocated with rproc_alloc(). 110 This is called by the platform-specific rproc implementation, whenever 111 a new remote processor device is probed. 112 Returns 0 on success and an appropriate error code otherwise. 113 Note: this function initiates an asynchronous firmware loading 114 context, which will look for virtio devices supported by the rproc's 115 firmware. 116 If found, those virtio devices will be created and added, so as a result 117 of registering this remote processor, additional virtio drivers might get 118 probed. 119 120 int rproc_del(struct rproc *rproc) 121 - Unroll rproc_add(). 122 This function should be called when the platform specific rproc 123 implementation decides to remove the rproc device. it should 124 _only_ be called if a previous invocation of rproc_add() 125 has completed successfully. 126 127 After rproc_del() returns, @rproc is still valid, and its 128 last refcount should be decremented by calling rproc_put(). 129 130 Returns 0 on success and -EINVAL if @rproc isn't valid. 131 132 void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) 133 - Report a crash in a remoteproc 134 This function must be called every time a crash is detected by the 135 platform specific rproc implementation. This should not be called from a 136 non-remoteproc driver. This function can be called from atomic/interrupt 137 context. 138 1395. Implementation callbacks 140 141These callbacks should be provided by platform-specific remoteproc 142drivers: 143 144/** 145 * struct rproc_ops - platform-specific device handlers 146 * @start: power on the device and boot it 147 * @stop: power off the device 148 * @kick: kick a virtqueue (virtqueue id given as a parameter) 149 */ 150struct rproc_ops { 151 int (*start)(struct rproc *rproc); 152 int (*stop)(struct rproc *rproc); 153 void (*kick)(struct rproc *rproc, int vqid); 154}; 155 156Every remoteproc implementation should at least provide the ->start and ->stop 157handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler 158should be provided as well. 159 160The ->start() handler takes an rproc handle and should then power on the 161device and boot it (use rproc->priv to access platform-specific private data). 162The boot address, in case needed, can be found in rproc->bootaddr (remoteproc 163core puts there the ELF entry point). 164On success, 0 should be returned, and on failure, an appropriate error code. 165 166The ->stop() handler takes an rproc handle and powers the device down. 167On success, 0 is returned, and on failure, an appropriate error code. 168 169The ->kick() handler takes an rproc handle, and an index of a virtqueue 170where new message was placed in. Implementations should interrupt the remote 171processor and let it know it has pending messages. Notifying remote processors 172the exact virtqueue index to look in is optional: it is easy (and not 173too expensive) to go through the existing virtqueues and look for new buffers 174in the used rings. 175 1766. Binary Firmware Structure 177 178At this point remoteproc only supports ELF32 firmware binaries. However, 179it is quite expected that other platforms/devices which we'd want to 180support with this framework will be based on different binary formats. 181 182When those use cases show up, we will have to decouple the binary format 183from the framework core, so we can support several binary formats without 184duplicating common code. 185 186When the firmware is parsed, its various segments are loaded to memory 187according to the specified device address (might be a physical address 188if the remote processor is accessing memory directly). 189 190In addition to the standard ELF segments, most remote processors would 191also include a special section which we call "the resource table". 192 193The resource table contains system resources that the remote processor 194requires before it should be powered on, such as allocation of physically 195contiguous memory, or iommu mapping of certain on-chip peripherals. 196Remotecore will only power up the device after all the resource table's 197requirement are met. 198 199In addition to system resources, the resource table may also contain 200resource entries that publish the existence of supported features 201or configurations by the remote processor, such as trace buffers and 202supported virtio devices (and their configurations). 203 204The resource table begins with this header: 205 206/** 207 * struct resource_table - firmware resource table header 208 * @ver: version number 209 * @num: number of resource entries 210 * @reserved: reserved (must be zero) 211 * @offset: array of offsets pointing at the various resource entries 212 * 213 * The header of the resource table, as expressed by this structure, 214 * contains a version number (should we need to change this format in the 215 * future), the number of available resource entries, and their offsets 216 * in the table. 217 */ 218struct resource_table { 219 u32 ver; 220 u32 num; 221 u32 reserved[2]; 222 u32 offset[0]; 223} __packed; 224 225Immediately following this header are the resource entries themselves, 226each of which begins with the following resource entry header: 227 228/** 229 * struct fw_rsc_hdr - firmware resource entry header 230 * @type: resource type 231 * @data: resource data 232 * 233 * Every resource entry begins with a 'struct fw_rsc_hdr' header providing 234 * its @type. The content of the entry itself will immediately follow 235 * this header, and it should be parsed according to the resource type. 236 */ 237struct fw_rsc_hdr { 238 u32 type; 239 u8 data[0]; 240} __packed; 241 242Some resources entries are mere announcements, where the host is informed 243of specific remoteproc configuration. Other entries require the host to 244do something (e.g. allocate a system resource). Sometimes a negotiation 245is expected, where the firmware requests a resource, and once allocated, 246the host should provide back its details (e.g. address of an allocated 247memory region). 248 249Here are the various resource types that are currently supported: 250 251/** 252 * enum fw_resource_type - types of resource entries 253 * 254 * @RSC_CARVEOUT: request for allocation of a physically contiguous 255 * memory region. 256 * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. 257 * @RSC_TRACE: announces the availability of a trace buffer into which 258 * the remote processor will be writing logs. 259 * @RSC_VDEV: declare support for a virtio device, and serve as its 260 * virtio header. 261 * @RSC_LAST: just keep this one at the end 262 * 263 * Please note that these values are used as indices to the rproc_handle_rsc 264 * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to 265 * check the validity of an index before the lookup table is accessed, so 266 * please update it as needed. 267 */ 268enum fw_resource_type { 269 RSC_CARVEOUT = 0, 270 RSC_DEVMEM = 1, 271 RSC_TRACE = 2, 272 RSC_VDEV = 3, 273 RSC_LAST = 4, 274}; 275 276For more details regarding a specific resource type, please see its 277dedicated structure in include/linux/remoteproc.h. 278 279We also expect that platform-specific resource entries will show up 280at some point. When that happens, we could easily add a new RSC_PLATFORM 281type, and hand those resources to the platform-specific rproc driver to handle. 282 2837. Virtio and remoteproc 284 285The firmware should provide remoteproc information about virtio devices 286that it supports, and their configurations: a RSC_VDEV resource entry 287should specify the virtio device id (as in virtio_ids.h), virtio features, 288virtio config space, vrings information, etc. 289 290When a new remote processor is registered, the remoteproc framework 291will look for its resource table and will register the virtio devices 292it supports. A firmware may support any number of virtio devices, and 293of any type (a single remote processor can also easily support several 294rpmsg virtio devices this way, if desired). 295 296Of course, RSC_VDEV resource entries are only good enough for static 297allocation of virtio devices. Dynamic allocations will also be made possible 298using the rpmsg bus (similar to how we already do dynamic allocations of 299rpmsg channels; read more about it in rpmsg.txt). 300