Commit graph

44 commits

Author SHA1 Message Date
Edward O'Callaghan e88a614c40 drm/amdkfd: Pass 'struct queue_propertices' by reference
Allow init_queue() to take 'struct queue_properties' by reference.

Signed-off-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2016-09-19 20:58:35 +03:00
Daniel Vetter a104299b94 drm/amdkfd: Clean up inline handling
- inline functions need to be static inline, otherwise gcc can opt to
  not inline and the linker gets unhappy.
- no forward decls for inline functions, just include the right headers.

Cc: Oded Gabbay <oded.gabbay@gmail.com>
Cc: Ben Goz <ben.goz@amd.com>
Reviewed-by: Oded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Link: http://patchwork.freedesktop.org/patch/msgid/1466500235-21282-2-git-send-email-daniel.vetter@ffwll.ch
2016-06-21 21:32:52 +02:00
Ben Goz a82918f18a drm/amdkfd: make reset wavefronts per process per device
This commit moves the reset wavefront flag to per process per device
data structure, so we can support multiple devices.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-06-06 22:26:39 +03:00
Ben Goz c3447e8150 drm/amdkfd: Enforce kill all waves on process termination
This commit makes sure that on process termination, after
we're destroying all the active queues, we're killing all the
existing wave front of the current process.

By doing this we're making sure that if any of the CUs were blocked
by infinite loop we're enforcing it to end the shader explicitly.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-06-03 11:34:47 +03:00
Yair Shachar 788bf83db3 drm/amdkfd: Add wave control operation to debugger
The wave control operation supports several command types executed upon
existing wave fronts that belong to the currently debugged process.

The available commands are:

HALT   - Freeze wave front(s) execution
RESUME - Resume freezed wave front(s) execution
KILL   - Kill existing wave front(s)

Signed-off-by: Yair Shachar <yair.shachar@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-06-03 11:33:06 +03:00
Yair Shachar fbeb661bfa drm/amdkfd: Add skeleton H/W debugger module support
This patch adds the skeleton H/W debugger module support. This code
enables registration and unregistration of a single HSA process at a
time.

The module saves the process's pasid and use it to verify that only the
registered process is allowed to execute debugger operations through the
kernel driver.

v2: rename get_dbgmgr_mutex to kfd_get_dbgmgr_mutex to namespace it

Signed-off-by: Yair Shachar <yair.shachar@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-06-03 11:32:28 +03:00
Yair Shachar 992839ad64 drm/amdkfd: Add static user-mode queues support
This patch adds support for static user-mode queues in QCM.
Queues which are designated as static can NOT be preempted by
the CP microcode when it is executing its scheduling algorithm.

This is needed for supporting the debugger feature, because we
can't allow the CP to preempt queues which are currently being debugged.

The number of queues that can be designated as static is limited by the
number of HQDs (Hardware Queue Descriptors).

Signed-off-by: Yair Shachar <yair.shachar@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-06-03 11:32:28 +03:00
Joe Perches f761d8bd80 drm/amdkfd: Use DECLARE_BITMAP
Use the generic mechanism to declare a bitmap instead of unsigned long.

It seems that "struct kfd_process.allocated_queue_bitmap" is unused.
Maybe it could be deleted instead.

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-06-03 11:31:12 +03:00
Oded Gabbay 81663016db drm/amdkfd: Add module parameter of send_sigterm
This patch adds a new kernel module parameter to amdkfd,
called send_sigterm.

This parameter specifies whether amdkfd should send the
SIGTERM signal to an HSA process, when the following conditions
occur:

1. The GPU triggers an exception regarding a kernel that was
   issued by this process.

2. The HSA process isn't waiting on an event that handles
   this exception.

The default behavior is not to send a SIGTERM and suffice
with a dmesg error print.

Reviewed-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-05-19 13:02:28 +03:00
Alexey Skidanov 930c5ff439 drm/amdkfd: Add bad opcode exception handling
Signed-off-by: Alexey Skidanov <alexey.skidanov@gmail.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-05-19 13:02:28 +03:00
Alexey Skidanov 59d3e8be87 drm/amdkfd: Add memory exception handling
This patch adds Peripheral Page Request (PPR) failure processing
and reporting.

Bad address or pointer to a system memory block with inappropriate
read/write permission cause such PPR failure during a user queue
processing. PPR request handling is done by IOMMU driver notifying
AMDKFD module on PPR failure.

The process triggering a PPR failure will be notified by
appropriate event or SIGTERM signal will be sent to it.

v3:
- Change all bool fields in struct kfd_memory_exception_failure to
  uint32_t

Signed-off-by: Alexey Skidanov <alexey.skidanov@gmail.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-05-19 13:02:27 +03:00
Andrew Lewycky f3a398183f drm/amdkfd: Add the events module
This patch adds the events module (kfd_events.c) and the interrupt
handle module for Kaveri (cik_event_interrupt.c).

The patch updates the interrupt_is_wanted(), so that it now calls the
interrupt isr function specific for the device that received the
interrupt. That function(implemented in cik_event_interrupt.c)
returns whether this interrupt is of interest to us or not.

The patch also updates the interrupt_wq(), so that it now calls the
device's specific wq function, which checks the interrupt source
and tries to signal relevant events.

v2:

Increase limit of signal events to 4096 per process
Remove bitfields from struct cik_ih_ring_entry
Rename radeon_kfd_event_mmap to kfd_event_mmap
Add debug prints to allocate_free_slot and allocate_signal_page
Make allocate_event_notification_slot return a correct value
Add warning prints to create_signal_event
Remove error print from IOCTL path
Reformatted debug prints in kfd_event_mmap
Map correct size (as received from mmap) in kfd_event_mmap

v3:

Reduce limit of signal events back to 256 per process
Fix allocation of kernel memory for signal events

Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-05-19 13:02:26 +03:00
Andrew Lewycky 2249d55827 drm/amdkfd: Add interrupt handling module
This patch adds the interrupt handling module, kfd_interrupt.c, and its
related members in different data structures to the amdkfd driver.

The amdkfd interrupt module maintains an internal interrupt ring
per amdkfd device. The internal interrupt ring contains interrupts
that needs further handling. The extra handling is deferred to
a later time through a workqueue.

There's no acknowledgment for the interrupts we use. The hardware
simply queues a new interrupt each time without waiting.

The fixed-size internal queue means that it's possible for us to lose
interrupts because we have no back-pressure to the hardware.

However, only interrupts that are "wanted" by amdkfd, are copied into
the amdkfd s/w interrupt ring, in order to minimize the chances
for overflow of the ring.

Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@gmail.com>
2015-05-19 12:13:39 +03:00
Xihan Zhang cea405b172 drm/amdkfd: Add multiple kgd support
The current code can only support one kgd instance. We have to
support multiple kgd instances in one system. i.e two amdgpu or two
radeon or one amdgpu + one radeon or more than two kgd instances.

Signed-off-by: Xihan Zhang <xihan.zhang@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2015-03-25 14:02:05 +02:00
Oded Gabbay 0d9200874c drm/amdkfd: Remove unused field from struct qcm_process_device
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Ben Goz <ben.goz@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-03-25 14:02:05 +02:00
Dave Airlie b3869b17fd Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux into drm-next
This backmerges drm-fixes into drm-next mainly for the amdkfd
stuff, I'm not 100% confident, but it builds and the amdkfd
folks can fix anything up.

Signed-off-by: Dave Airlie <airlied@redhat.com>
Conflicts:
	drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
	drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
2015-01-29 11:45:31 +10:00
Dave Airlie 281d1bbd34 Merge remote-tracking branch 'origin/master' into drm-next
Backmerge Linus tree after rc5 + drm-fixes went in.

There were a few amdkfd conflicts I wanted to avoid,
and Ben requested this for nouveau also.

Conflicts:
	drivers/gpu/drm/amd/amdkfd/Makefile
	drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
	drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c
	drivers/gpu/drm/amd/amdkfd/kfd_priv.h
	drivers/gpu/drm/amd/include/kgd_kfd_interface.h
	drivers/gpu/drm/i915/intel_runtime_pm.c
	drivers/gpu/drm/radeon/radeon_kfd.c
2015-01-22 10:44:41 +10:00
Oded Gabbay b8cbab042c drm/amdkfd: Allow user to limit only queues per device
This patch replaces the two current amdkfd module parameters with a new one.

The current parameters that are being replaced are:

- Maximum number of HSA processes
- Maximum number of queues per process

The new parameter that replaces them is called "Maximum queues per device"

This replacement achieves two goals:

- Allows the user to have as many HSA processes as it wants (until
  a maximum of 512 HSA processes in Kaveri).

- Removes the limitation the user had on maximum number of queues per HSA
  process. E.g. the user can now have processes which only have one queue and
  other processes which have hundreds of queues, while before the user
  couldn't have more than 128 queues per process (as default).

The default value of the new parameter is 4096 (32 * 128, which were the
defaults of the old parameters). There is almost no additional GART memory
required for the default case. As a reminder, this amount of queues requires a
little bit below 4MB of GART memory.

v2:
In addition, This patch defines a new counter for queues accounting in the DQM
structure. This is done because the current counter only counts active queues
which allows the user to create more queues than the
max_num_of_queues_per_device module parameter allows.

However, we need the current counter for the runlist packet build process, so
the solution is to have a dedicated counter for this accounting.

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Ben Goz <ben.goz@amd.com>
2015-01-18 13:18:01 +02:00
Michel Dänzer 6ee0ad2a7f drm/amdkfd: Drop interrupt SW ring buffer
The work queue couldn't reliably prevent the SW ring buffer from
overflowing, so dmesg was spammed by

 kfd kfd: Interrupt ring overflow, dropping interrupt.

messages when running e.g. the Atlantis Substance demo from
https://wiki.unrealengine.com/Linux_Demos on Kaveri.

Since the SW ring buffer doesn't actually do anything at this point, just
remove it for now. When actual interrupt processing code is added to
amdkfd, it should try to do things immediately and only defer to work
queues when necessary.

Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2015-01-08 13:27:15 +09:00
Oded Gabbay 76baee6c73 drm/amdkfd: rewrite kfd_ioctl() according to drm_ioctl()
This patch changes kfd_ioctl() to be very similar to drm_ioctl().

The patch defines an array of amdkfd_ioctls, which maps IOCTL definition to the
ioctl function.

The kfd_ioctl() uses that mapping to call the appropriate ioctl function,
through a function pointer.

This patch also declares a new typedef for the ioctl function pointer.

v2: Renamed KFD_COMMAND_(START|END) to AMDKFD_...

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
2015-01-06 19:44:36 +02:00
Alexey Skidanov 52a5fdce13 amdkfd: Instead of using get function, use container_of
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-11-19 17:07:00 +02:00
Oded Gabbay 5cd78de526 amdkfd: add __iomem attribute to doorbell_ptr
This patch was done due to sparse warning. It changes the definition of
doorbell_ptr in queue_properties to be with __iomem attribute, so it would
match the type which the doorbell module functions are returning.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-11-20 16:14:56 +02:00
Ben Goz 4b8f589b05 drm/amdkfd: Change MQD manager to be H/W specific
The MQDs for CI and VI are different. Therefore, the MQD manager module need to
be H/W specific.

This patch splits the current MQD manager into three files:

- kfd_mqd_manager.c, which contains common functions and initializes the
  specific mqd manager module according to the H/W

- kfd_mqd_manager_cik.c, which contains Kaveri specific functions. This is
  basically the old kfd_mqd_manager.c

- kfd_mqd_manager_vi.c, which will contain VI specific functions. Currently it
  is not implemented except for returning NULL on initialization.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-01-04 11:24:25 +02:00
Ben Goz 0da7558c69 drm/amdkfd: Add asic property to kfd_device_info
This patch adds a new property to kfd_device_info structure. That structure
holds information that is H/W specific.

The new property is called asic_family and its purpose is to distinguish
between different asic families in amdkfd operations, mainly in QCM (queue
control & management)

This patch also adds a new enum, to select different ASICs. We set the current
kfd_device_info instance as Kaveri and create a new instance which describes
the new AMD APU, codenamed 'Carrizo'.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-01-01 17:10:01 +02:00
Ben Goz 85d258f9a7 drm/amdkfd: Make KFD_MQD_TYPE enum types H/W agnostic
As the MQD types are common across all AMD GPUs/APUs, let's remove the CIK part
from the name.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-01-04 10:36:30 +02:00
Ben Goz ff3d04a171 drm/amdkfd: Add new VI-specific queue properties
This patch adds new fields to the queue_properties structure. The new fields
are relevant only for queues running on AMD GPU VI architecture.

The eop_ring_buffer_address and eop_ring_buffer_size describe an
end-of-pipe queue which is assigned to the MQD. In CI, the EOP queue was per
pipeline and in VI it is per queue.

The ctx_save_restore_area_address and ctx_save_restore_area_size describe a
memory area that is designated to allow the CP to do context save/restore in
mid-wave state.

This patch also modifies the set_queue_properties_from_user() (called from
kfd_ioctl_create_queue()) to check and copy those new parameters.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-01-04 10:37:18 +02:00
Oded Gabbay 6e81090b2e drm/amdkfd: Add kfd gtt sub-allocator functions
This patch adds new kfd gtt sub-allocator functions that service the amdkfd
driver when it wants to use gtt memory.

The sub-allocator uses a bitmap to handle the memory area that was transferred
to it during init. It divides the memory area into chunks, according to chunk
size parameter.

The allocation function will allocate contiguous chunks from that memory area,
according to the requested size. If the requested size is smaller than the
chunk size, a single chunk will be allocated.

v2: Do some more verifications on parameters that are passed into
kfd_gtt_sa_init()

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alexey Skidanov <Alexey.skidanov@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-01-09 22:26:08 +02:00
Oded Gabbay 36b5c08f09 drm/amdkfd: Add gtt sa related data to kfd_dev struct
This patch adds new fields to kfd_dev struct that are necessary for the new kfd
gtt sa module

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alexey Skidanov <Alexey.skidanov@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-01-09 22:26:08 +02:00
Ben Goz 77669eb87a drm/amdkfd: Add SDMA mqd support
This patch adds support for SDMA mqd operations:
- init_mqd_sdma
- uninit_mqd_sdma
- load_mqd_sdma
- update_mqd_sdma
- destroy_mqd_sdma
- is_occupied_sdma

It also adds SDMA queue information to some private structures of amdkfd.

v3: Use the new names of some of the defines.

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
2015-01-09 22:26:04 +02:00
Alexey Skidanov 093c7d8cfd drm/amdkfd: Process-device data creation and lookup split
This patch splits the current kfd_get_process_device_data() to two
functions, one that specifically creates a pdd and another one which
just do lookup.

This is done to enhance the readability and maintainability of the code.

Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2015-01-09 22:25:58 +02:00
Alexey Skidanov f7c826ad38 drm/amdkfd: Add number of watch points to topology
This patch adds the number of watch points to the node capabilities in the
topology module

Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2015-01-09 22:25:55 +02:00
Alexey Skidanov 775921edc1 amdkfd: Implement the Get Process Aperture IOCTL
v3: Fixed debug messages

Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 01:49:36 +03:00
Andrew Lewycky b3f5e6b441 amdkfd: Add interrupt handling module
This patch adds the interrupt handling module, in kfd_interrupt.c, and its
related members in different data structures to the amdkfd driver.

The amdkfd interrupt module maintains an internal interrupt ring per amdkfd
device. The internal interrupt ring contains interrupts that needs further
handling. The extra handling is deferred to a later time through a workqueue.

There's no acknowledgment for the interrupts we use. The hardware simply queues
a new interrupt each time without waiting.

The fixed-size internal queue means that it's possible for us to lose
interrupts because we have no back-pressure to the hardware.

v3:

Move amdkfd from drm/radeon/ to drm/amd/
Change device init
Made sure spin lock is taken only if init is complete
Moved bool field to the end of the structure

Signed-off-by: Andrew Lewycky <Andrew.Lewycky@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 01:37:30 +03:00
Ben Goz 64c7f8cf79 amdkfd: Add device queue manager module
The queue scheduler divides into two sections, one section is process bounded
and the other section is device bounded.
The device bounded section is handled by this module.
The DQM module handles queue setup, update and tear-down from the device side.
It also supports suspend/resume operation.

v3: Changed device_init, added the use of the new gart allocation functions an
Added documentation.

v4:

Fixed a race in DQM queue scheduler where dqm->lock must be held when accessing
dqm->queue_count and dqm->processes_count. This fixes runlist IB allocation
failures when DQM is under load.

Fixed race in DQM queue destruction where queues being destroyed must be
removed from qpd->queues_list prior to preemption, or concurrent queue
creation activity may reschedule them while their MQD is destroyed.

Fixed EOP queue size setting in CP_HPD_EOP_CONTROL, because the size is
specified as (log2(size_dwords)-1). The previous calculation assumed the
size was specified in bytes, which caused interference between EOP queues
when multiple MEC pipelines were active.

v5:

Move amdkfd from drm/radeon/ to drm/amd/
Change format of mqd structure to match latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime
Remove unused unmap_queue function
Various fixes (Style, typos)

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Jay Cornwall <jay.cornwall@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 01:27:00 +03:00
Ben Goz 45102048f7 amdkfd: Add process queue manager module
The queue scheduler divides into two sections, one section is process bounded
and the other section is device bounded.
The process bounded section is handled by this module. The PQM handles usermode
queue setup, updates and tear-down.

v3:

Used kernel parameter to limit queues per process instead of define
Added use of doorbell address from user

v4:

Modified pqm_create_queue so that only when creating usermode queues the
driver should return the queue properties to the userspace.

Added an info message print when no more queues can be opened because of the
queue per process limitation

v5:

Move amdkfd from drm/radeon/ to drm/amd/
Various fixes

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 01:04:10 +03:00
Ben Goz 241f24f823 amdkfd: Add packet manager module
The packet manager module builds PM4 packets for the sole use of the CP
scheduler. Those packets are used by the HIQ to submit runlists to the CP.

v3:

Removed include of cik_mqds.h
Changed lower_32/upper_32 calls to use linux macros
Used new gart allocation functions
Added documentation

v5:

Move amdkfd from drm/radeon/ to drm/amd/
Change format of mqd structure to match latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime
Always chain runlist if you have more than 1 process or if you have
over-subscription over the number of queues.
Various fixes (typos, style)

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 00:55:28 +03:00
Ben Goz 31c21fece7 amdkfd: Add module parameter of scheduling policy
This patch adds a new parameter to the amdkfd driver. This parameter enables
the user to select the scheduling policy of the CP. The choices are:

* CP Scheduling with support for over-subscription
* CP Scheduling without support for over-subscription
* Without CP Scheduling

Note that the third option (Without CP scheduling) is only for debug purposes
and bringup of new H/W. As such, it is _not_ guaranteed to work at all times on
all H/W versions.

v3: Fixed description of parameter, changed the permissions to read_only, added
a verification of the value and added documentation

v5: Set default sched_policy to HWS as it is now supported by firmware

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 00:48:28 +03:00
Ben Goz ed6e6a3487 amdkfd: Add kernel queue module
The kernel queue module enables the amdkfd to establish kernel queues, not
exposed to user space.

The kernel queues are used for HIQ (HSA Interface Queue) and DIQ (Debug
Interface Queue) operations

v3: Removed use of internal typedefs and added use of the new gart allocation
functions

v4: Fixed a miscalculation in kernel queue wrapping

v5:

Move amdkfd from drm/radeon/ to drm/amd/
Change format of mqd structure to match latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime
Add define for kernel queue size
Various fixes

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 00:45:35 +03:00
Ben Goz 6e99df5741 amdkfd: Add mqd_manager module
The mqd_manager module handles MQD data structures.
MQD stands for Memory Queue Descriptor, which is used by the H/W to
keep the usermode queue state in memory.

v3:

Removed new typedefs
Removed pragma pack 4
Remove cik_mqds.h file
Changed lower_32/upper_32 calls to use linux macros
Used new gart allocation functions
Added documentation

v4:

Added missing initialization of the addr field in init_mqd()

Setting the hqd persistent.preload_req bit ON so that when queues switches
on/off, their context will kept and read from the mqd when the cp reassign
them, and thus the dispatched workload context kept consistent without any
interrupts.

v5:

Move amdkfd from drm/radeon/ to drm/amd/
Change format of mqd structure to match latest KV firmware
Add support for AQL queues creation to enable working with open-source HSA
runtime.
Various fixes

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 00:36:17 +03:00
Ben Goz ed8aab4594 amdkfd: Add queue module
The queue module enables allocating and initializing queues uniformly.

v3: Removed typedef and redundant memset call. Broke long pr_debug print to one
liners and Added documentation.

v5: Move amdkfd from drm/radeon/ to drm/amd/

Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 00:18:51 +03:00
Oded Gabbay b17f068a09 amdkfd: Add binding/unbinding calls to amd_iommu driver
This patch adds the functions to bind and unbind pasid
from a device through the amd_iommu driver.

The unbind function is called when the mm_struct of the
process is released.

The bind function is not called here because it is called
only in the IOCTLs which are not yet implemented at this
stage of the patchset.

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-17 00:06:27 +03:00
Oded Gabbay 19f6d2a660 amdkfd: Add basic modules to amdkfd
This patch adds the process module and three helper modules:

- kfd_process, which handles process which open /dev/kfd

- kfd_doorbell, which provides helper functions for doorbell allocation,
  release and mapping to userspace

- kfd_pasid, which provides helper functions for pasid allocation and release

- kfd_aperture, which provides helper functions for managing the LDS, Local GPU
  memory and Scratch memory apertures of the process

This patch only contains the basic kfd_process module, which doesn't contain
the reference to the queue scheduler. This was done to allow easier code review.

Also, this patch doesn't contain the calls to the IOMMU driver for binding the
pasid to the device. Again, this was done to allow easier code review

The kfd_process object is created when a process opens /dev/kfd and is closed
when the mm_struct of that process is teared-down.

v3:

Removed kfd_vidmem.c file
Replaced direct mmput call to mmu_notifier release
Removed typedefs
Moved bool field to end of the structure
Added new kernel params for gart usage limitation
Added initialization of sa manager
Fixed debug messages
Remove support for LDS in 32 bit
Changed code to support mmap of doorbell pages from userspace
Added documentation for apertures

v4: Replaced RCU by SRCU for kfd_process list management

v5:

Move amdkfd from drm/radeon/ to drm/amd/
Rename kfd_aperture.c to kfd_flat_memory.c
Protect against multiple init calls
MQD size is H/W dependent so moved it to device info structure
Rename kfd_mem_obj structure's members
Use delayed function for process tear-down

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-16 23:25:31 +03:00
Evgeny Pinchuk 5b5c4e40a3 amdkfd: Add topology module to amdkfd
This patch adds the topology module to the driver. The topology is exposed to
userspace through the sysfs.

The calls to add and remove a device to/from topology are done by the radeon
driver.

v3:

The CPU information, that is provided in the topology section of the amdkfd
driver, is extracted from the CRAT table. Unlike the CPU information located
in /sys/devices/system/cpu/cpu*, which is extracted from the SRAT table.

While the CPU information provided by the CRAT and the SRAT tables might be
identical, the node topology might be different. The SRAT table contains the
topology of CPU nodes only. The CRAT table contains the topology of CPU and GPU
nodes together (and can be interleaved). For example CPU node 1 in SRAT can be
CPU node 3 in CRAT. Furthermore it's worth to mention that the CRAT table
contains only HSA compatible nodes (nodes which are compliant with the HSA
spec).

To recap, amdkfd exposes a different kind of topology than the one exposed by
/sys/devices/system/cpu/cpu even though it may contain similar information.

v4:

The topology module doesn't support uevent handling and doesn't notify the
userspace about runtime modifications. It is up to the userspace to acquire
snapshots of the topology information created by the amdkfd and exposed
in sysfs.

The following is an example of how the topology looks on a Kaveri A10-7850K
system with amdkfd installed:

/sys/devices/virtual/kfd/kfd/
|
--- topology/
      |
      |--- generation_id
      |--- system_properties
      |--- nodes/
            |
            |--- 0/
                 |
                 |--- gpu_id
                 |--- name
                 |--- properties
                 |--- caches/
                      |
                      |--- 0/
                           |
                           |--- properties
                      |--- 1/
                           |
                           |--- properties
                      |--- 2/
                           |
                           |--- properties
                 |--- io_links/
                      |
                 |--- mem_banks/
                      |
                      |--- 0/
                           |
                           |--- properties
                      |--- 1/
                           |
                           |--- properties
                      |--- 2/
                           |
                           |--- properties
                      |--- 3/
                           |
                           |--- properties

v5:

Move amdkfd from drm/radeon/ to drm/amd/

Add a check if dev->gpu pointer is null before accessing it in the
node_show function in kfd_topology.c
This situation may occur when amdkfd is loaded and there is a GPU with a CRAT
table, but that GPU isn't supported by amdkfd

Signed-off-by: Evgeny Pinchuk <evgeny.pinchuk@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-16 21:22:32 +03:00
Oded Gabbay 4a488a7ad7 amdkfd: Add amdkfd skeleton driver
This patch adds the amdkfd skeleton driver. The driver does nothing except
define a /dev/kfd device.

It returns -ENODEV on all amdkfd IOCTLs.

v3: Move bool field to the end of structure, removed the pmc ioctls and added
a meaningful error message for ioctl error.

v5:

Create a new folder drm/amd and move amdkfd from drm/radeon/ to drm/amd/
Remove scheduler_class from kfd_priv.h as it was never used
Add skeleton implementation of the Get Version IOCTL

v6:
Update module version to the correct number and remove the "default m" from the
Kconfig file

Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-16 21:08:55 +03:00