This makes it less likely to run into an ENOMEM because
VM page tables are evicted last.
v2: move the BOs in the LRU tail after validation
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This merges '5b726e06d6e8309e5c9ef4109a32caf27c71dfc8' into drm-next
Just to resolve some merges to make Daniel's life easier.
Signed-off-by: DAve Airlie <airlied@redhat.com>
[airlied: fixup build problems on arm - added errno.h include]
* 'drm-next-4.5' of git://people.freedesktop.org/~agd5f/linux: (152 commits)
amd/powerplay: fix copy paste typo in hardwaremanager.c
amd/powerplay: disable powerplay by default initially
amd/powerplay: don't enable ucode fan control if vbios has no fan table
drm/amd/powerplay: show gpu load when print gpu performance for Cz. (v2)
drm/amd/powerplay: check whether need to enable thermal control. (v2)
drm/amd/powerplay: add point check to avoid NULL point hang.
drm/amdgpu/powerplay: Program a calculated value as Deep Sleep clock.
drm/amd/powerplay: Don't return an error if fan table is missing
drm/powerplay/hwmgr: log errors in tonga_hwmgr_backend_init
drm/powerplay: add debugging output to processpptables.c
drm/powerplay: add debugging output to tonga_processpptables.c
amd/powerplay: Add structures required to report configuration change
amd/powerplay: Fix get dal power level
amd\powerplay Implement get dal power level
drm/amd/powerplay: display gpu load when print performance for tonga.
drm/amdgpu/powerplay: enable sysfs and debugfs interfaces late
drm/amd/powerplay: move shared function of vi to hwmgr. (v2)
drm/amd/powerplay: check whether enable dpm in powerplay.
drm/amd/powerplay: fix bug that dpm funcs in debugfs/sysfs missing.
drm/amd/powerplay: fix boolreturn.cocci warnings
...
This avoids allocating it on the fly.
v2: fix grammar in comment
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
This way we avoid the extra allocation for the page directory entry.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
They share the reservation object with the page directory anyway.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
This fixes a random corruption under memory pressure. We need to fence
the BO for the user fence as well, otherwise it might be swapped out
and the GPU could write the fence value to an undesired location.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Cc: stable@vger.kernel.org
No need for a GEM reference here.
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Unify the two code path again, cause they do pretty much the same thing.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <davdi1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
We need to clear parser.ibs and num_ibs before amd_sched_fence_create,
otherwise the IB could be freed twice if fence creates fails.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Before this patch the scheduler fence was created when we push the job
into the queue, so we could only get the fence after pushing it.
The mutex now was necessary to prevent the thread pushing the jobs to
the hardware from running faster than the thread pushing the jobs into
the queue.
Otherwise the thread pushing jobs into the queue would have accessed
possible freed up memory when it tries to get a reference to the fence.
So what you get in the end is thread A:
mutex_lock(&job->lock);
...
Kick of thread B.
...
mutex_unlock(&job->lock);
And thread B:
mutex_lock(&job->lock);
....
mutex_unlock(&job->lock);
kfree(job);
I'm actually not sure if I'm still up to date on this, but this usage
pattern used to be not allowed with mutexes. See here as well
https://lwn.net/Articles/575460/.
v2: remove unrelated changes, fix missing owner
v3: rebased, add more commit message
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
No need any more to allocate that structure dynamically, just put it on the
stack. This is a start to cleanup some of the scheduler fallouts.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Fixing a memory leak when the scheduler is enabled.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
- Updated register headers for GFX 8.1 for Stoney
- Add some new CZ revisions
- minor pageflip optimizations
- Fencing clean up
- Warning fix
- More fence cleanup
- oops fix
- Fiji fixes
* 'drm-next-4.4' of git://people.freedesktop.org/~agd5f/linux: (29 commits)
drm/amdgpu: group together common fence implementation
drm/amdgpu: remove AMDGPU_FENCE_OWNER_MOVE
drm/amdgpu: remove now unused fence functions
drm/amdgpu: fix fence fallback check
drm/amdgpu: fix stoping the scheduler timeout
drm/amdgpu: cleanup on error in amdgpu_cs_ioctl()
drm/amdgpu: update Fiji's Golden setting
drm/amdgpu: update Fiji's rev id
drm/amdgpu: extract common code in vi_common_early_init
drm/amd/scheduler: don't oops on failure to load
drm/amdgpu: don't oops on failure to load (v2)
drm/amdgpu: don't VT switch on suspend
drm/amdgpu: Make amdgpu_mn functions inline
drm/amdgpu: remove amdgpu_fence_ref/unref
drm/amdgpu: use common fence for sync
drm/amdgpu: use the new fence_is_later
drm/amdgpu: use common fences for VMID management v2
drm/amdgpu: move ring_from_fence to common code
drm/amdgpu: switch to common fence_wait_any_timeout v2
drm/amdgpu: remove unneeded fence functions
...
We recently changed the locking in this function and now there is a
missing unlock on error. Also there are some other resources that we
should probably release as well...
Fixes: f48b2659f5 ('drm/amdgpu: fix the broken vm->mutex V2')
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
More amdgpu and radeon stuff for drm-next. Stoney support is the big change.
The rest is just bug fixes and code cleanups. The Stoney stuff is pretty
low impact with respect to existing chips.
* 'drm-next-4.4' of git://people.freedesktop.org/~agd5f/linux:
drm/amdgpu: change VM size default to 64GB
drm/amdgpu: add Stoney pci ids
drm/amdgpu: update the core VI support for Stoney
drm/amdgpu: add VCE support for Stoney (v2)
drm/amdgpu: add UVD support for Stoney
drm/amdgpu: add GFX support for Stoney (v2)
drm/amdgpu: add SDMA support for Stoney (v2)
drm/amdgpu: add DCE support for Stoney
drm/amdgpu: Update SMC/DPM for Stoney
drm/amdgpu: add GMC support for Stoney
drm/amdgpu: add Stoney chip family
drm/amdgpu: fix the broken vm->mutex V2
drm/amdgpu: remove the unnecessary parameter adev for amdgpu_fence_wait_any()
drm/amdgpu: remove the exclusive lock
drm/amdgpu: remove old lockup detection infrastructure
drm: fix trivial typos
drm/amdgpu/dce: simplify suspend/resume
drm/amdgpu/gfx8: set TC_WB_ACTION_EN in RELEASE_MEM packet
drm/radeon: Use rdev->gem.mutex to protect hyperz/cmask owners
fix the vm->mutex and ww_mutex confilcts.
vm->mutex is always token first, then ww_mutex.
V2: remove unneccessary checking for pt bo.
Change-Id: Iea56e183752c02831126d06d2f5b7a474a6e4743
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
This makes it much easier to find when userspace misses to send some buffers.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Rework the sdma structures in the driver to
consolidate all of the sdma info into a single
structure and allow for asics that may have
different numbers of sdma instances.
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The new amdgpu driver passes a user space pointer in a 64-bit structure
member, which is the correct way to do it, but it attempts to
directly cast it to a __user pointer in the kernel, which causes
a warning in three places:
drm/amd/amdgpu/amdgpu_cs.c: In function 'amdgpu_cs_parser_init':
drm/amd/amdgpu/amdgpu_cs.c:180:21: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
chunk_array_user = (uint64_t __user *)(cs->in.chunks);
This changes all three to add an intermediate cast to 'unsigned long'
as other drivers do. This avoids the warning and works correctly on
both 32-bit and 64-bit architectures.
Fixes: e60b344f6c ("drm/amdgpu: optimize amdgpu_parser_init")
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The "i" variable should be signed or it leads to a crash in the error
handling code.
Fixes: 1d263474c4 ('drm/amdgpu: unwind properly in amdgpu_cs_parser_init()')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The amdgpu_cs_parser_init() function doesn't clean up after itself but
instead the caller uses a free everything function amdgpu_cs_parser_fini()
on failure. This style of error handling is often buggy. In this
example, we call "drm_free_large(parser->chunks[i].kdata);" when it is
an unintialized pointer or when "parser->chunks" is NULL.
I fixed this bug by adding unwind code so that it frees everything that
it allocates.
I also mode some other very minor changes:
1) Renamed "r" to "ret".
2) Moved the chunk_array allocation to the start of the function.
3) Removed some initializers which are no longer needed.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reported-by: Ilja Van Sprundel <ivansprundel@ioactive.com>
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Embed the scheduler into the ring structure instead of allocating it.
Use the ring name directly instead of the id.
v2: rebased, whitespace cleanup
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>
Just to be consistent with the other members.
v2: rename the ring member as well.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com> (v1)
Reviewed-by: Chunming Zhou<david1.zhou@amd.com>
Use container_of rather than casting.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: David Zhou <david1.zhou@amd.com>
Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Use consistent naming across functions.
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: David Zhou <david1.zhou@amd.com>
Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
This allows for multiple BOs to have the same reservation object.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Some buffers (UVD/VM page tables) must be placed in VRAM,
but the byte restriction for moving buffers didn't took this
into account.
Port of radeon commit 4b09556660.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Calling schedule() is probably the worse things we can do.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Simply not used any more. Only keep 32bit atomic for fence sequence numbering.
v2: trivial rebase
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com> (v1)
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Stop double freeing the the BO list by pulling the content
of amdgpu_cs_parser_prepare_job() into the IOCTL function again.
v2: better commit message
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
The problem now is that we don't necessarily call amdgpu_ib_get()
in some error paths and so work with uninitialized data.
Better require that the memory is already zeroed.
v2: better commit message
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com> (v1)
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Pagetables can be moved and therefore the page directory update can be necessary
for the current cs even if none of the the bo's are moved. In that scenario
there is no fence between the sdma0 and gfx ring, so we add one.
v2 (chk): rebased
Signed-off-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
bo_list_clone() will take a lot of time when bo_list hold too much
elements, like above 7000
Signed-off-by: Monk.Liu <monk.liu@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Jammy Zhou <jammy.zhou@amd.com>
fix fence is released when pass to **fence sometimes.
add reference for it.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
scheduler fence is based on kernel fence framework.
v2: squash in Christian's build fix
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
Avoiding a couple of casts.
v2: rename c_entity to entity as well
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
We didn't initialized the mutex in the cloned bo list resulting in nice
warnings from lockdep. Also fixes error handling in this function.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
It is clean to update last_queued_v_seq in the scheduler module
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Fix the code alignment, etc.
v2: rebase the code
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
It is theoretically possible that a swapped out BO gets the
same GTT address, but different backing pages while being swapped in.
Instead just use another VA state to note updated areas.
Ported from not upstream yet radeon commit with the same name.
v2: fix some bugs in the original implementation found in the radeon code.
v3: squash in VCE/UVD fix
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
if enabling scheduler, then the queued seq is assigned
when pushing job before emitting job.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
it is possible that the callback isn't defined sometimes.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Christian K?nig <christian.koenig@amd.com>
user mode will still use pte ring as a normal ring.
if the prepare job generates another command(update pte) on its ring in scheduler,
then will kill scheduler which is going to waiting later job but pending running job.
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
BO validation is moved to scheduler except usrptr which must be validated
in user process
Signed-off-by: Chunming Zhou <david1.zhou@amd.com>
Acked-by: Christian K?nig <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
use kmalloc_array instead of kcalloc where appropriate and other
cleanups.
Signed-off-by: monk.liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
We need to return the sequence number to userspace
even when we don't use user fences.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
This is a prerequisite for the GPU scheduler to make the order
of submission independent from the order of execution.
v2: properly implement the locking
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
This also requires some error handling from the callers of that function.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Just to make sure userspace don't send nonsense to the kernel.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
v2: remove unrelated whitespace change, fix C comment
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
And use common fence infrastructure for the wait.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
amdgpu_cs_find_mapping doesn't work without all buffers being validated,
so the TTM validation must be done first.
v2: only use amdgpu_cs_find_mapping for UVD/VCE VM emulation
Signed-off-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
bo_va may un-initialized, fix it.
Signed-off-by: monk.liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Jammy Zhou <jammy.zhou@amd.com>
Properly protect the state and also handle submission failures.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
check the filp is not robust, and sometimes different contexts may
have same filp value.
v2: check both filp and ctx_id
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
It is required to support fence per context.
v2: add amdgpu_ctx_get/put
v3: improve get/put
v4: squash hlock fix
Signed-off-by: Jammy Zhou <Jammy.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Still need to handle ibs BO and validate them even bo_list is NULL
Signed-off-by: Monk.Liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This adds the non-asic specific core driver code.
v2: remove extra kconfig option
v3: implement minor fixes from Fengguang Wu
v4: fix cast in amdgpu_ucode.c
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Jammy Zhou <Jammy.Zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Christian König