The change (008fa749e0) that moved the
node release code to a separate function broke death notifications in
some cases. When it encountered a reference without a death
notification request, it would skip looking at the remaining
references, and therefore fail to send death notifications for them.
Cc: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Cc: stable <stable@vger.kernel.org> # 3.10
Signed-off-by: Arve Hjønnevåg <arve@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Jeremy Compostella <jeremy.compostella@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
We dereference "heap->task" before checking if it's an ERR_PTR.
Fixes: ea313b5f88 ('gpu: ion: Also shrink memory cached in the deferred free list')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
We should be returning -ENOMEM here instead of zero.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
There is a race condition
Assume we have *one* sync_fence object, with *one* sync_pt
which belongs to *one* sync_timeline, given this condition,
sync_timeline->kref will have two counts, one for sync_timeline
(implicit) and another for sync_pt.
Assume following is the situation on CPU
Theead-1 : (Thread which calls sync_timeline_destroy())
-> (some function calls)
-> sync_timeline_destory()
-> sync_timeline_signal() (CPU is inside this
function after putting reference to sync_timeline)
At this time Thread-2 comes and does following
Thread-2 : (fclose on fence fd)
> sync_fence_release() -> because of fclose() on fence object
-> sync_fence_free()
-> sync_pt_free()
-> kref_put(&pt->parent->kref, sync_timeline_free);
-> sync_timeline_free() (CPU is inside this because
this time kref will be zero after _put)
Thread-2 will free sync_timeline object before Thread-1
has finished its work inside sync_timeline_signal.
With this change we signals all sync_pt before putting
reference to sync_timeline object.
Cc: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: Prakash Kamliya <pkamliya@codeaurora.org>
[jstultz: minor commit subject tweak]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add #include <linux/device.h> to fix the following warning seen
with gcc 4.7.3:
In file included from drivers/staging/android/ion/ion_heap.c:26:0:
drivers/staging/android/ion/ion_priv.h:358:21: warning: ‘struct device’ declared inside parameter list [enabled by default]
drivers/staging/android/ion/ion_priv.h:358:21: warning: its scope is only this definition or declaration, which is probably not what you want [enabled by default]
Cc: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The compat ioctl for ION_IOC_FREE currently passes allocation data
instead of the free data. Correct this.
Cc: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: Laura Abbott <lauraa@codeaurora.org>
[jstultz: Folded in a small build fix]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Fix a few bugs in ion_system_heap:
Initialize the list node in the info block.
Don't store size_remaining in a signed long, allocating >2GB
could overflow, resulting in a call to sg_alloc_table with
nents=0 which panics. alloc_largest_available will never
return a block larger than size_remanining, so it can never
go negative.
Limit a single allocation to half of all memory. Prevents a
large allocation from taking down the whole system.
Cc: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: Colin Cross <ccross@android.com>
[jstultz: Minor commit subject tweak]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Avoid holding ashmem_mutex across code that can page fault. Page faults
grab the mmap_sem for the process, which are also held by mmap calls
prior to calling ashmem_mmap, which locks ashmem_mutex. The reversed
order of locking between the two can deadlock.
The calls that can page fault are read() and the ASHMEM_SET_NAME and
ASHMEM_GET_NAME ioctls. Move the code that accesses userspace pages
outside the ashmem_mutex.
Cc: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: Todd Poynor <toddpoynor@google.com>
[jstultz: minor commit message tweaks]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Before this change, a timeline would only be removed from the timeline
list *after* the sync driver had its release_obj() called. However, the
driver's release_obj() may free resources needed by print_obj().
Although the timeline list is locked when print_obj() is called, it is
not locked when release_obj() is called. If one CPU was in print_obj()
when another was in release_obj(), the print_obj() may make unsafe
accesses.
It is not actually necessary to hold the timeline list lock when calling
release_obj() if the call is made after the timeline is unlinked from
the list, since there is no possibility another thread could be in --
or enter -- print_obj() for that timeline.
This change moves the release_obj() call to after the timeline is
unlinked, preventing the above race from occurring.
Cc: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: Alistair Strachan <alistair.strachan@imgtec.com>
[jstultz: minor commit subject tweak]
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The ION_DUMMY option is bool, and hence this code is either
present or absent. It will never be modular, so using
module_init as an alias for __initcall is rather misleading.
Fix this up now, so that we can relocate module_init from
init.h into module.h in the future. If we don't do this, we'd
have to add module.h to obviously non-modular code, and that
would be a worse thing.
Note that direct use of __initcall is discouraged, vs. one
of the priority categorized subgroups. As __initcall gets
mapped onto device_initcall, our use of device_initcall
directly in this change means that the runtime impact is
zero -- it will remain at level 6 in initcall ordering.
Cc: Colin Cross <ccross@android.com>
Cc: Jesse Barker <jesse.barker@arm.com>
Cc: Android Kernel Team <kernel-team@android.com>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
use ARRAY_SIZE to count number of heaps in static array
Signed-off-by: Tomas Winkler <tomas.winkler@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Need add "linux/io.h" to pass compiling under metag architecture with
allmodconfig (which use the default 'virt_to_phys'), the related error:
CC drivers/staging/android/ion/ion_dummy_driver.o
drivers/staging/android/ion/ion_dummy_driver.c: In function 'ion_dummy_init':
drivers/staging/android/ion/ion_dummy_driver.c:81: error: implicit declaration of function 'virt_to_phys'
Signed-off-by: Chen Gang <gang.chen.5i5j@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add support to the dummy driver for basic carveout and chunk heaps.
Since we're generating these heaps at module_init, and we want
this driver to be generic enough to be tested on any arch, we
don't have the ability to alloc bootmem, so both of these heaps
are conventionally allocated using alloc_pages(), which limits us
to 4M in size.
Should look into using CMA for heap allocation eventually, but
this provides enough to test the basic functionality of the
heaps.
Cc: Colin Cross <ccross@android.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jesse Barker <jesse.barker@arm.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Provide a basic dummy driver to register the ion device
and to install basic SYSTEM and SYSTEM_CONTIG heaps.
This allows for basic testing with ION without having
access to drivers or systems that have been enabled to use
ION.
Cc: Colin Cross <ccross@android.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Jesse Barker <jesse.barker@arm.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Prior to subitting this, Colin reworked the compat_ioctl support
for the ion_test driver, moving the structure to be the same size
on both 32 and 64 bit architectures.
Two small things were left out. The compat_ioctl ptr assignment,
and the fact that despite having uniform sized types in the
structure, the structure pads out to different sizes on different
arches.
This patch resolves this issue by adding a padding entry after
the write flag, and adding the compat_ioctl ptr.
Changes in v2:
- Add a padding int rather then making write a u64
Acked-by: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
RT_MUTEXES can be configured out of the kernel, causing compile
problems with ION.
To quote Colin:
"rt_mutexes were added with the deferred freeing feature. Heaps need
to return zeroed memory to userspace, but zeroing the memory on every
allocation was causing performance issues. We added a SCHED_IDLE
thread to zero memory in the background after freeing, but locking the
heap from the SCHED_IDLE thread might block a high priority allocation
thread for a long time.
The lock is only used to protect the heap's free_list and
free_list_size members, and is not held for any long or sleeping
operations. Converting to a spinlock should prevent priority
inversion without using the rt_mutex. I'd also rename it to free_lock
to so it doesn't get used as a general heap lock."
Thus this patch converts the rt_mutex usage to a spinlock and
renames the lock free_lock to be more clear as to its use.
I also had to change a bit of logic in ion_heap_freelist_drain()
to safely avoid list corruption.
Acked-by: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Reported-by: Jim Davis <jim.epost@gmail.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The kbuild test robot reported:
drivers/staging/android/ion/ion_system_heap.c:122 alloc_largest_available() error: potential null dereference 'info'. (kmalloc returns null)
Where the pointer returned from kmalloc goes unchecked for failure.
This patch checks the return for NULL, and reworks the logic, as
suggested by Colin, so we allocate the page_info structure first.
Acked-by: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Reported-by: kbuild test robot <fengguang.wu@intel.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The kbuild test robot reported a build issue w/ ION on m68k:
drivers/staging/android/ion/ion.c: In function 'ion_reserve':
drivers/staging/android/ion/ion.c:1526:4: error: implicit declaration of function 'memblock_alloc_base' [-Werror=implicit-function-declaration]
drivers/staging/android/ion/ion.c:1528:11: error: 'MEMBLOCK_ALLOC_ANYWHERE' undeclared (first use in this function)
drivers/staging/android/ion/ion.c:1528:11: note: each undeclared identifier is reported only once for each function it appears in
drivers/staging/android/ion/ion.c:1537:4: error: implicit declaration of function 'memblock_reserve' [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
This is caused by ION using memblock functionality which m68k doesn't support.
This patch adds a HAVE_MEMBLOCK dependency to the ION config.
Acked-by: Colin Cross <ccross@android.com>
Cc: Android Kernel Team <kernel-team@android.com>
Reported-by: kbuild test robot <fengguang.wu@intel.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Fixes the following sparse warnings:
drivers/staging/android/ion/tegra/tegra_ion.c:23:19: warning:
symbol 'idev' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:24:19: warning:
symbol 'tegra_user_mapper' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:25:5: warning:
symbol 'num_heaps' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:26:17: warning:
symbol 'heaps' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:28:5: warning:
symbol 'tegra_ion_probe' was not declared. Should it be static?
drivers/staging/android/ion/tegra/tegra_ion.c:66:5: warning:
symbol 'tegra_ion_remove' was not declared. Should it be static?
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
ion_test.h should not define ion_user_handle_t, and defining it
causes a warning:
In file included from drivers/staging/android/ion/ion_test.c:31:
drivers/staging/android/ion/../uapi/ion_test.h:23: error: redefinition of typedef 'ion_user_handle_t'
drivers/staging/android/ion/../uapi/ion.h:23: note: previous declaration of 'ion_user_handle_t' was here
Reported-by: Andrew Morton <akpm@linux-foundation.org>
Cc: John Stultz <john.stultz@linaro.org>
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Now that ION builds, reenable it in the build.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
ION doesn't export the proper symbols for it to be a module. This
causes build issues when ION is configured as a module.
Since Andorid kernels rarely use modules (I think recent policy
requires no modules?), go ahead and set the ION config to a bool
from the tristate option.
If folks decide ION as a module is important, we will have to go
through and export the various needed symbols.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Update the ION system heap shrinker to use the new count/scan
interfaces that landed in 3.12
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Mostly just to quiet checkpatch warnings, be more verbose
in describing the ION config option.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Just some simple cleanups to address whitespace issues and
other issues found w/ checkpatch.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Implement ion_cma_unmap_kernel, ion will call it unconditionally.
Use correct gfp flags when calling dma_alloc_coherent so it doesn't
try to use atomic DMA memory.
Check for invalid alignment when allocating.
Reject cached allocations - the cpu address returned by
dma_alloc_coherent is always going to be an uncached mapping, so
map_kernel will not see data written by a cached userspace mapping.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add ion_heap_pages_zero for ion heaps to use to zero pages
during initialization or allocation, when a struct ion_buffer
may not be available. Use it from the chunk heap and carveout
heaps.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The carveout heap wasn't zeroing its buffers after use.
Create the sg_table during allocate instead of map_dma, to allow
using the sg_table during free, and call ion_heap_buffer_zero
during free. Also fixes a missing kfree when destroying the
table.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
There is no reason to use kzalloc, just call alloc_pages directly.
Change the GFP from GFP_KERNEL to include __GFP_HIGH, to allow it
to return contiguous pages from highmem. virt_to_* functions
aren't valid on highmem pages, so store the struct page * in an
sg_table in buffer->priv_virt like most other heaps, and replace
virt_to_* with page_to_*.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Now that ion_vm_fault uses vm_insert_pfn instead of vm_insert_page
cached buffers can be supported in any heap. Remove the checks
in the chunk and system heaps.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Now that ion_vm_fault doesn't need a struct page with a nonzero
refcount, there is no need allocate heap memory for cached pages using
split_page. Remove the ion_heap_alloc_pages and ion_heap_free_pages
helpers in favor of direct calls to alloc_pages and __free_pages,
and remove the special handling in the system heap.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Most ion userspace mappings are created with remap_pfn_range. Use
vm_insert_pfn instead of vm_insert_page to make faulted cached
mappings look more like uncached mappings.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Check the return value of remap_pfn_range and return an error if
it fails.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
When the shrinkers are called with GFP_HIGH free low memory first,
it is more important to have free than high memory.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
ion_heap_buffer_zero can spend a long time in unmap_kernel_range
if it has to broadcast a tlb flush to every cpu for every page.
Modify it to batch pages into a larger region to clear using a
single mapping. This may cause the mapping size to change if
the buffer size is not a multiple of the mapping size, so
switch to allocating the address space for each chunk. This
allows us to use vm_map_ram to handle the allocation and mapping
together.
The number of pages to zero using a single mapping is set to 32
to hit the fastpath in vm_map_ram.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Ion will compile and run on other platforms now, remove the
dependency on ARM.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
In testing ion system heap allocations, I ran across two issues:
1) Not k*z*allocing the sg table. This can cause trouble if
we end up trying call sg_alloc_table() with too many entries,
then sg_alloc_table() internally fails and tries to free what it
thinks is internal table structure, which causes bad pointer
traversals.
2) The second list_for_each_entry probably should be _safe,
since I was seeing strange lock warnings and oopses on occasion.
This seems to resolve it, but could use some extra checking.
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
The `buffer' variable is being used after being freed. Fix this.
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Convert the ion ioctls to use _IOW instead of _IOWR where
appropriate, and factor out the copy_from_user and copy_to_user
based on the _IOC_DIR bits. For the existing incorrect ioctls,
add a function to wrap _IOC_DIR to return the corrected value.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
ion_system_contig_heap buffers have an sglist, just call
ion_heap_map_user to map it.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Use %z for size_t and %pa for dma_addr_t to avoid warnings in printks.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
phys_to_page and __phys_to_pfn don't exist on all platforms.
Use a combination of pfn_to_page, PFN_DOWN, page_to_pfn, and
virt_to_page to get the same results.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
ion_heap_map_kernel already implements mapping a scatterlist of
pages into the kernel, and all heaps are required to have struct
pages associated with them, so delete the functions that use
__arm_ioremap and use ion_heap_map_kernel instead.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Use idr_alloc instead if idr_pre_get/idr_get_new_above, and
remove idr_remove_all.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Add a /dev/ion-test device that will be created if CONFIG_ION_TEST
is set. The device accepts a dma_buf fd and allows reading and
writing to the backing memory using DMA-like apis or kernel mapping
apis. Can be used to test the dma_buf mapping ops, including
the ion implementations, from userspace.
Signed-off-by: Colin Cross <ccross@android.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Arve Hjønnevåg