Commit e66eed651f ("list: remove prefetching from regular list
iterators") removed the include of prefetch.h from list.h, which
uncovered several cases that had apparently relied on that rather
obscure header file dependency.
So this fixes things up a bit, using
grep -L linux/prefetch.h $(git grep -l '[^a-z_]prefetchw*(' -- '*.[ch]')
grep -L 'prefetchw*(' $(git grep -l 'linux/prefetch.h' -- '*.[ch]')
to guide us in finding files that either need <linux/prefetch.h>
inclusion, or have it despite not needing it.
There are more of them around (mostly network drivers), but this gets
many core ones.
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Simplify the code that checks for correct assignment of GRU contexts to
users.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add anticipatory TLB dropins for GRU TLB misses that occur on BCOPY
instructions that copy large amounts of data.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Update the GRU irq allocate/free functions to use the latest upstream
infrastructure.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Fix several bugs related to prefetch, ordering & speculation:
- GRU cch_allocate() instruction causes cacheable memory
to be created. Add a barriers to prevent speculation
from prefetching data before it exists.
- Add memory barriers before cache-flush instructions to ensure
that previously stored data is included in the line flushed to memory.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Under some conditions, mmu_notifier_register() will fail to register a
mmu_notifier. Fix the GRU driver to correctly handle these failures.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This patch builds on the infrastructure introduced in the patches that
allow user specification of GRU blades & chiplets for context allocation.
This patch simplifies the algorithms for migrating GRU contexts between
blades.
No new functionality is introduced.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add support to the GRU driver to allow users to specify the blade &
chiplet for allocation of GRU contexts. Add new statistics for context
loading/unloading/retargeting. Also deleted a few GRU stats that were no
longer being unused.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add table & user request infrastructure that is needed to allow users to
specify the blade and chiplet for allocation of GRU contexts. Use of this
information is in a subsequent patch.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Fix the copyright statements in a couple of GRU files. No functional
changes are being made.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add a user request to specify the gru instruction slice parameter for user
contexts.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Collect GRU statistics for each user GRU context. Statistics are kept for
TLB misses & content resource contention. Add user request for retrieving
the statistics.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add support for interrupts generated by GRU instruction completion.
Previously, the only interrupts were for TLB misses. The hardware also
supports interrupts on instruction completion. This will be supported for
instructions issued by the kernel.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Check whether the gru state being loaded into a gru is from a new context
or a previously unloaded context. If new, simply zero out the hardware
context; if unloaded and valid, reload the old state.
This change is primarily for reloading kernel contexts where the previous
is not required to be saved.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Support alocation of GRU contexts that contain zero DSR or CBR resources.
Some instructions do not require DSR resources. Contexts without CBR
resources are useful for diagnostics.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Change the way GRU resources are assigned for kernel threads. GRU
contexts for kernel threads are now allocated on demand and can be stolen
by user processes when idle. This allows MPI jobs to use ALL of the GRU
resources when the kernel is not using them.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Change the interface to cch_allocate so that it can be used to allocate
GRU contexts for kernel threads. Kernel threads use the GRU in unmapped
mode and do not require ASIDs for the GRU TLB.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Remove "static" from the functions for loading/unloading GRU contexts.
These functions will be called from other GRU files. Fix bug in unlocking
gru context.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Change the interface to gru_alloc_gts() so that it can be used to allocate
GRU contexts for kernel threads. Kernel threads do not have vdata
structures for the GRU contexts. The GRU resource count are now passed
explicitly instead of inside the vdata structure.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Bug fixes for GRU exception handling. Additional fields from the CBR must
be returned to the user to allow the user to correctly diagnose GRU
exceptions.
Handle endcase in TFH TLB miss handling. Verify that TFH actually
indicates a pending exception.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Add multiple pagesize support to the GRU driver.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This patch fixes bugs related to ASID (context id) management in the GRU
driver. These changes are all internal to the SGI GRU driver and have no
effect on the base kernel.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Misc trivial GRU drivers fixes:
- fix long lines
- eliminate extra whitespace
- eliminate compiler warning
- better validation of invalidate user parameters
- bug fix for GRU TLB flush (not the cpu TLB flush)
These changes are all internal to the SGI GRU driver and have no effect
on the base kernel.
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
A few minor updates for the GRU driver.
- documentation changes found in code reviews
- changes to #ifdefs to make them recognized by "unifdef"
(used in simulator testing)
- change GRU context load/unload to prefetch data
[akpm@linux-foundation.org: fix typo in comment]
Signed-off-by: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Fixes problems identified in a code review:
- add comment with high level dscription of the GRU
- prepend "gru_" to all global names
- delete unused function
- couple of trivial bug fixes
[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Jack Steiner <steiner@sgi.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Linus Torvalds