![Ohad Ben-Cohen](/assets/img/avatar_default.png)
The resource table is an array of 'struct fw_resource' members, where each resource entry is expressed as a single member of that array. This approach got us this far, but it has a few drawbacks: 1. Different resource entries end up overloading the same members of 'struct fw_resource' with different meanings. The resulting code is error prone and hard to read and maintain. 2. It's impossible to extend 'struct fw_resource' without breaking the existing firmware images (and we already want to: we can't introduce the new virito device resource entry with the current scheme). 3. It doesn't scale: 'struct fw_resource' must be as big as the largest resource entry type. As a result, smaller resource entries end up utilizing only small part of it. This is fixed by defining a dedicated structure for every resource type, and then converting the resource table to a list of type-value members. Instead of a rigid array of homogeneous structs, the resource table is turned into a collection of heterogeneous structures. This way: 1. Resource entries consume exactly the amount of bytes they need. 2. It's easy to extend: just create a new resource entry structure, and assign it a new type. 3. The code is easier to read and maintain: the structures' members names are meaningful. While we're at it, this patch has several other resource table changes: 1. The resource table gains a simple header which contains the number of entries in the table and their offsets within the table. This makes the parsing code simpler and easier to read. 2. A version member is added to the resource table. Should we change the format again, we'll bump up this version to prevent breakage with existing firmware images. 3. The VRING and VIRTIO_DEV resource entries are combined to a single VDEV entry. This paves the way to supporting multiple VDEV entries. 4. Since we don't really support 64-bit rprocs yet, convert two stray u64 members to u32. Signed-off-by: Ohad Ben-Cohen <ohad@wizery.com> Cc: Brian Swetland <swetland@google.com> Cc: Iliyan Malchev <malchev@google.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Grant Likely <grant.likely@secretlab.ca> Cc: Rusty Russell <rusty@rustcorp.com.au> Cc: Mark Grosen <mgrosen@ti.com> Cc: John Williams <john.williams@petalogix.com> Cc: Michal Simek <monstr@monstr.eu> Cc: Loic PALLARDY <loic.pallardy@stericsson.com> Cc: Ludovic BARRE <ludovic.barre@stericsson.com> Cc: Omar Ramirez Luna <omar.luna@linaro.org> Cc: Guzman Lugo Fernando <fernando.lugo@ti.com> Cc: Anna Suman <s-anna@ti.com> Cc: Clark Rob <rob@ti.com> Cc: Stephen Boyd <sboyd@codeaurora.org> Cc: Saravana Kannan <skannan@codeaurora.org> Cc: David Brown <davidb@codeaurora.org> Cc: Kieran Bingham <kieranbingham@gmail.com> Cc: Tony Lindgren <tony@atomide.com>
320 lines
14 KiB
Plaintext
320 lines
14 KiB
Plaintext
Remote Processor Framework
|
|
|
|
1. Introduction
|
|
|
|
Modern SoCs typically have heterogeneous remote processor devices in asymmetric
|
|
multiprocessing (AMP) configurations, which may be running different instances
|
|
of operating system, whether it's Linux or any other flavor of real-time OS.
|
|
|
|
OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP.
|
|
In a typical configuration, the dual cortex-A9 is running Linux in a SMP
|
|
configuration, and each of the other three cores (two M3 cores and a DSP)
|
|
is running its own instance of RTOS in an AMP configuration.
|
|
|
|
The remoteproc framework allows different platforms/architectures to
|
|
control (power on, load firmware, power off) those remote processors while
|
|
abstracting the hardware differences, so the entire driver doesn't need to be
|
|
duplicated. In addition, this framework also adds rpmsg virtio devices
|
|
for remote processors that supports this kind of communication. This way,
|
|
platform-specific remoteproc drivers only need to provide a few low-level
|
|
handlers, and then all rpmsg drivers will then just work
|
|
(for more information about the virtio-based rpmsg bus and its drivers,
|
|
please read Documentation/rpmsg.txt).
|
|
|
|
2. User API
|
|
|
|
int rproc_boot(struct rproc *rproc)
|
|
- Boot a remote processor (i.e. load its firmware, power it on, ...).
|
|
If the remote processor is already powered on, this function immediately
|
|
returns (successfully).
|
|
Returns 0 on success, and an appropriate error value otherwise.
|
|
Note: to use this function you should already have a valid rproc
|
|
handle. There are several ways to achieve that cleanly (devres, pdata,
|
|
the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we
|
|
might also consider using dev_archdata for this). See also
|
|
rproc_get_by_name() below.
|
|
|
|
void rproc_shutdown(struct rproc *rproc)
|
|
- Power off a remote processor (previously booted with rproc_boot()).
|
|
In case @rproc is still being used by an additional user(s), then
|
|
this function will just decrement the power refcount and exit,
|
|
without really powering off the device.
|
|
Every call to rproc_boot() must (eventually) be accompanied by a call
|
|
to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug.
|
|
Notes:
|
|
- we're not decrementing the rproc's refcount, only the power refcount.
|
|
which means that the @rproc handle stays valid even after
|
|
rproc_shutdown() returns, and users can still use it with a subsequent
|
|
rproc_boot(), if needed.
|
|
- don't call rproc_shutdown() to unroll rproc_get_by_name(), exactly
|
|
because rproc_shutdown() _does not_ decrement the refcount of @rproc.
|
|
To decrement the refcount of @rproc, use rproc_put() (but _only_ if
|
|
you acquired @rproc using rproc_get_by_name()).
|
|
|
|
struct rproc *rproc_get_by_name(const char *name)
|
|
- Find an rproc handle using the remote processor's name, and then
|
|
boot it. If it's already powered on, then just immediately return
|
|
(successfully). Returns the rproc handle on success, and NULL on failure.
|
|
This function increments the remote processor's refcount, so always
|
|
use rproc_put() to decrement it back once rproc isn't needed anymore.
|
|
Note: currently rproc_get_by_name() and rproc_put() are not used anymore
|
|
by the rpmsg bus and its drivers. We need to scrutinize the use cases
|
|
that still need them, and see if we can migrate them to use the non
|
|
name-based boot/shutdown interface.
|
|
|
|
void rproc_put(struct rproc *rproc)
|
|
- Decrement @rproc's power refcount and shut it down if it reaches zero
|
|
(essentially by just calling rproc_shutdown), and then decrement @rproc's
|
|
validity refcount too.
|
|
After this function returns, @rproc may _not_ be used anymore, and its
|
|
handle should be considered invalid.
|
|
This function should be called _iff_ the @rproc handle was grabbed by
|
|
calling rproc_get_by_name().
|
|
|
|
3. Typical usage
|
|
|
|
#include <linux/remoteproc.h>
|
|
|
|
/* in case we were given a valid 'rproc' handle */
|
|
int dummy_rproc_example(struct rproc *my_rproc)
|
|
{
|
|
int ret;
|
|
|
|
/* let's power on and boot our remote processor */
|
|
ret = rproc_boot(my_rproc);
|
|
if (ret) {
|
|
/*
|
|
* something went wrong. handle it and leave.
|
|
*/
|
|
}
|
|
|
|
/*
|
|
* our remote processor is now powered on... give it some work
|
|
*/
|
|
|
|
/* let's shut it down now */
|
|
rproc_shutdown(my_rproc);
|
|
}
|
|
|
|
4. API for implementors
|
|
|
|
struct rproc *rproc_alloc(struct device *dev, const char *name,
|
|
const struct rproc_ops *ops,
|
|
const char *firmware, int len)
|
|
- Allocate a new remote processor handle, but don't register
|
|
it yet. Required parameters are the underlying device, the
|
|
name of this remote processor, platform-specific ops handlers,
|
|
the name of the firmware to boot this rproc with, and the
|
|
length of private data needed by the allocating rproc driver (in bytes).
|
|
|
|
This function should be used by rproc implementations during
|
|
initialization of the remote processor.
|
|
After creating an rproc handle using this function, and when ready,
|
|
implementations should then call rproc_register() to complete
|
|
the registration of the remote processor.
|
|
On success, the new rproc is returned, and on failure, NULL.
|
|
|
|
Note: _never_ directly deallocate @rproc, even if it was not registered
|
|
yet. Instead, if you just need to unroll rproc_alloc(), use rproc_free().
|
|
|
|
void rproc_free(struct rproc *rproc)
|
|
- Free an rproc handle that was allocated by rproc_alloc.
|
|
This function should _only_ be used if @rproc was only allocated,
|
|
but not registered yet.
|
|
If @rproc was already successfully registered (by calling
|
|
rproc_register()), then use rproc_unregister() instead.
|
|
|
|
int rproc_register(struct rproc *rproc)
|
|
- Register @rproc with the remoteproc framework, after it has been
|
|
allocated with rproc_alloc().
|
|
This is called by the platform-specific rproc implementation, whenever
|
|
a new remote processor device is probed.
|
|
Returns 0 on success and an appropriate error code otherwise.
|
|
Note: this function initiates an asynchronous firmware loading
|
|
context, which will look for virtio devices supported by the rproc's
|
|
firmware.
|
|
If found, those virtio devices will be created and added, so as a result
|
|
of registering this remote processor, additional virtio drivers might get
|
|
probed.
|
|
Currently, though, we only support a single RPMSG virtio vdev per remote
|
|
processor.
|
|
|
|
int rproc_unregister(struct rproc *rproc)
|
|
- Unregister a remote processor, and decrement its refcount.
|
|
If its refcount drops to zero, then @rproc will be freed. If not,
|
|
it will be freed later once the last reference is dropped.
|
|
|
|
This function should be called when the platform specific rproc
|
|
implementation decides to remove the rproc device. it should
|
|
_only_ be called if a previous invocation of rproc_register()
|
|
has completed successfully.
|
|
|
|
After rproc_unregister() returns, @rproc is _not_ valid anymore and
|
|
it shouldn't be used. More specifically, don't call rproc_free()
|
|
or try to directly free @rproc after rproc_unregister() returns;
|
|
none of these are needed, and calling them is a bug.
|
|
|
|
Returns 0 on success and -EINVAL if @rproc isn't valid.
|
|
|
|
5. Implementation callbacks
|
|
|
|
These callbacks should be provided by platform-specific remoteproc
|
|
drivers:
|
|
|
|
/**
|
|
* struct rproc_ops - platform-specific device handlers
|
|
* @start: power on the device and boot it
|
|
* @stop: power off the device
|
|
* @kick: kick a virtqueue (virtqueue id given as a parameter)
|
|
*/
|
|
struct rproc_ops {
|
|
int (*start)(struct rproc *rproc);
|
|
int (*stop)(struct rproc *rproc);
|
|
void (*kick)(struct rproc *rproc, int vqid);
|
|
};
|
|
|
|
Every remoteproc implementation should at least provide the ->start and ->stop
|
|
handlers. If rpmsg functionality is also desired, then the ->kick handler
|
|
should be provided as well.
|
|
|
|
The ->start() handler takes an rproc handle and should then power on the
|
|
device and boot it (use rproc->priv to access platform-specific private data).
|
|
The boot address, in case needed, can be found in rproc->bootaddr (remoteproc
|
|
core puts there the ELF entry point).
|
|
On success, 0 should be returned, and on failure, an appropriate error code.
|
|
|
|
The ->stop() handler takes an rproc handle and powers the device down.
|
|
On success, 0 is returned, and on failure, an appropriate error code.
|
|
|
|
The ->kick() handler takes an rproc handle, and an index of a virtqueue
|
|
where new message was placed in. Implementations should interrupt the remote
|
|
processor and let it know it has pending messages. Notifying remote processors
|
|
the exact virtqueue index to look in is optional: it is easy (and not
|
|
too expensive) to go through the existing virtqueues and look for new buffers
|
|
in the used rings.
|
|
|
|
6. Binary Firmware Structure
|
|
|
|
At this point remoteproc only supports ELF32 firmware binaries. However,
|
|
it is quite expected that other platforms/devices which we'd want to
|
|
support with this framework will be based on different binary formats.
|
|
|
|
When those use cases show up, we will have to decouple the binary format
|
|
from the framework core, so we can support several binary formats without
|
|
duplicating common code.
|
|
|
|
When the firmware is parsed, its various segments are loaded to memory
|
|
according to the specified device address (might be a physical address
|
|
if the remote processor is accessing memory directly).
|
|
|
|
In addition to the standard ELF segments, most remote processors would
|
|
also include a special section which we call "the resource table".
|
|
|
|
The resource table contains system resources that the remote processor
|
|
requires before it should be powered on, such as allocation of physically
|
|
contiguous memory, or iommu mapping of certain on-chip peripherals.
|
|
Remotecore will only power up the device after all the resource table's
|
|
requirement are met.
|
|
|
|
In addition to system resources, the resource table may also contain
|
|
resource entries that publish the existence of supported features
|
|
or configurations by the remote processor, such as trace buffers and
|
|
supported virtio devices (and their configurations).
|
|
|
|
The resource table begins with this header:
|
|
|
|
/**
|
|
* struct resource_table - firmware resource table header
|
|
* @ver: version number
|
|
* @num: number of resource entries
|
|
* @reserved: reserved (must be zero)
|
|
* @offset: array of offsets pointing at the various resource entries
|
|
*
|
|
* The header of the resource table, as expressed by this structure,
|
|
* contains a version number (should we need to change this format in the
|
|
* future), the number of available resource entries, and their offsets
|
|
* in the table.
|
|
*/
|
|
struct resource_table {
|
|
u32 ver;
|
|
u32 num;
|
|
u32 reserved[2];
|
|
u32 offset[0];
|
|
} __packed;
|
|
|
|
Immediately following this header are the resource entries themselves,
|
|
each of which begins with the following resource entry header:
|
|
|
|
/**
|
|
* struct fw_rsc_hdr - firmware resource entry header
|
|
* @type: resource type
|
|
* @data: resource data
|
|
*
|
|
* Every resource entry begins with a 'struct fw_rsc_hdr' header providing
|
|
* its @type. The content of the entry itself will immediately follow
|
|
* this header, and it should be parsed according to the resource type.
|
|
*/
|
|
struct fw_rsc_hdr {
|
|
u32 type;
|
|
u8 data[0];
|
|
} __packed;
|
|
|
|
Some resources entries are mere announcements, where the host is informed
|
|
of specific remoteproc configuration. Other entries require the host to
|
|
do something (e.g. allocate a system resource). Sometimes a negotiation
|
|
is expected, where the firmware requests a resource, and once allocated,
|
|
the host should provide back its details (e.g. address of an allocated
|
|
memory region).
|
|
|
|
Here are the various resource types that are currently supported:
|
|
|
|
/**
|
|
* enum fw_resource_type - types of resource entries
|
|
*
|
|
* @RSC_CARVEOUT: request for allocation of a physically contiguous
|
|
* memory region.
|
|
* @RSC_DEVMEM: request to iommu_map a memory-based peripheral.
|
|
* @RSC_TRACE: announces the availability of a trace buffer into which
|
|
* the remote processor will be writing logs.
|
|
* @RSC_VDEV: declare support for a virtio device, and serve as its
|
|
* virtio header.
|
|
* @RSC_LAST: just keep this one at the end
|
|
*
|
|
* Please note that these values are used as indices to the rproc_handle_rsc
|
|
* lookup table, so please keep them sane. Moreover, @RSC_LAST is used to
|
|
* check the validity of an index before the lookup table is accessed, so
|
|
* please update it as needed.
|
|
*/
|
|
enum fw_resource_type {
|
|
RSC_CARVEOUT = 0,
|
|
RSC_DEVMEM = 1,
|
|
RSC_TRACE = 2,
|
|
RSC_VDEV = 3,
|
|
RSC_LAST = 4,
|
|
};
|
|
|
|
For more details regarding a specific resource type, please see its
|
|
dedicated structure in include/linux/remoteproc.h.
|
|
|
|
We also expect that platform-specific resource entries will show up
|
|
at some point. When that happens, we could easily add a new RSC_PLATFORM
|
|
type, and hand those resources to the platform-specific rproc driver to handle.
|
|
|
|
7. Virtio and remoteproc
|
|
|
|
The firmware should provide remoteproc information about virtio devices
|
|
that it supports, and their configurations: a RSC_VDEV resource entry
|
|
should specify the virtio device id (as in virtio_ids.h), virtio features,
|
|
virtio config space, vrings information, etc.
|
|
|
|
When a new remote processor is registered, the remoteproc framework
|
|
will look for its resource table and will register the virtio devices
|
|
it supports. A firmware may support any number of virtio devices, and
|
|
of any type (a single remote processor can also easily support several
|
|
rpmsg virtio devices this way, if desired).
|
|
|
|
Of course, RSC_VDEV resource entries are only good enough for static
|
|
allocation of virtio devices. Dynamic allocations will also be made possible
|
|
using the rpmsg bus (similar to how we already do dynamic allocations of
|
|
rpmsg channels; read more about it in rpmsg.txt).
|