redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity

First drm/i915 feature batch heading for v4.18: 

- drm-next backmerge to fix build (Rodrigo)
 - GPU documentation improvements (Kevin)
 - GuC and HuC refactoring, host/GuC communication, logging, fixes, and more
   (mostly Michal and Michał, also Jackie, Michel and Piotr)
 - PSR and PSR2 enabling and fixes (DK, José, Rodrigo and Chris)
 - Selftest updates (Chris, Daniele)
 - DPLL management refactoring (Lucas)
 - DP MST fixes (Lyude and DK)
 - Watermark refactoring and changes to support NV12 (Mahesh)
 - NV12 prep work (Chandra)
 - Icelake Combo PHY enablers (Manasi)
 - Perf OA refactoring and ICL enabling (Lionel)
 - ICL enabling (Oscar, Paulo, Nabendu, Mika, Kelvin, Michel)
 - Workarounds refactoring (Oscar)
 - HDCP fixes and improvements (Ramalingam, Radhakrishna)
 - Power management fixes (Imre)
 - Various display fixes (Maarten, Ville, Vidya, Jani, Gaurav)
 - debugfs for FIFO underrun clearing (Maarten)
 - Execlist improvements (Chris)
 - Reset improvements (Chris)
 - Plenty of things here and there I overlooked and/or didn't understand... (Everyone)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEFWWmW3ewYy4RJOWc05gHnSar7m8FAlrQokEACgkQ05gHnSar
 7m+Y0w//Y+I4AsqJcmkcNuE+H3IAzK9Dw3noh8+biV4O1tJCR+obTdS3JifAPF98
 wRwsYjV3mgZRgGn6seTEUD9cgT5uLkYLfMWuO4SsxJr85bBgV4N2OXBMaU7Hw/WO
 lPrMHfvG6vIt4ZiLlxuS3SzH7+QTvPtIS3caRAIlcLL917gmkmGvqCB4XkznAG4D
 b4G07JJoasRozPP9SljwvTDKrcsY/ehPeBpU1N7iGIG3eXMgLjunt/QdNU+dA0tr
 mk6tHkV7zUnI+Y4is7QHnnhem299tYY/WWtaHEhRCe2qJRw/e+o7mfsiwmIENEUe
 7K7Zi+x78gIwPxTR2QefphQA2JLZKUOGJHzVy2bAkIhfwf5bJw/qmUdfGEVNCe9r
 EAllcreqvHZI8eqaqABC+5zgOozzy07+jWoD3K/5FGLDeSZTvfL4NQk91x06C84C
 yT9lvamvZBHUXYh715d3DEhZg4UwbZLTZQjJLDn8yQ/Rw5NSuaD5GlHiopner1j/
 lxZakbEHiuTdrHtk4JEho/fup1dE3uYuAQJyAjVYdZm5IVxWQW67XZxep0vwQdN5
 K4oGh02Npn3/G3KUFS6kud67P5yzccx+xqu8Ey6Lc7e3yF0FuxsM9OXEL3l18fJB
 tg17jThrALhONh7s0byyW0Kt/AHIrK2YLSj0xPzzlOVA8m2W/tw=
 =beK0
 -----END PGP SIGNATURE-----

Merge tag 'drm-intel-next-2018-04-13' of git://anongit.freedesktop.org/drm/drm-intel into drm-next

First drm/i915 feature batch heading for v4.18:

- drm-next backmerge to fix build (Rodrigo)
- GPU documentation improvements (Kevin)
- GuC and HuC refactoring, host/GuC communication, logging, fixes, and more
  (mostly Michal and Michał, also Jackie, Michel and Piotr)
- PSR and PSR2 enabling and fixes (DK, José, Rodrigo and Chris)
- Selftest updates (Chris, Daniele)
- DPLL management refactoring (Lucas)
- DP MST fixes (Lyude and DK)
- Watermark refactoring and changes to support NV12 (Mahesh)
- NV12 prep work (Chandra)
- Icelake Combo PHY enablers (Manasi)
- Perf OA refactoring and ICL enabling (Lionel)
- ICL enabling (Oscar, Paulo, Nabendu, Mika, Kelvin, Michel)
- Workarounds refactoring (Oscar)
- HDCP fixes and improvements (Ramalingam, Radhakrishna)
- Power management fixes (Imre)
- Various display fixes (Maarten, Ville, Vidya, Jani, Gaurav)
- debugfs for FIFO underrun clearing (Maarten)
- Execlist improvements (Chris)
- Reset improvements (Chris)
- Plenty of things here and there I overlooked and/or didn't understand... (Everyone)

Signed-off-by: Dave Airlie <airlied@redhat.com>
Link: https://patchwork.freedesktop.org/patch/msgid/87lgd2cze8.fsf@intel.com
hifive-unleashed-5.1
Dave Airlie 2018-05-04 10:31:10 +10:00
parent 0ab390262c fadec6eefe
commit 8eb008c808
87 changed files with 7267 additions and 3732 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

141
Documentation/gpu/i915.rst
View File

@ -58,6 +58,12 @@ Intel GVT-g Host Support(vGPU device model)
.. kernel-doc:: drivers/gpu/drm/i915/intel_gvt.c
:internal:
Workarounds
-----------
.. kernel-doc:: drivers/gpu/drm/i915/intel_workarounds.c
:doc: Hardware workarounds
Display Hardware Handling
=========================
@ -249,6 +255,103 @@ Memory Management and Command Submission
This sections covers all things related to the GEM implementation in the
i915 driver.
Intel GPU Basics
----------------
An Intel GPU has multiple engines. There are several engine types.
- RCS engine is for rendering 3D and performing compute, this is named
`I915_EXEC_RENDER` in user space.
- BCS is a blitting (copy) engine, this is named `I915_EXEC_BLT` in user
space.
- VCS is a video encode and decode engine, this is named `I915_EXEC_BSD`
in user space
- VECS is video enhancement engine, this is named `I915_EXEC_VEBOX` in user
space.
- The enumeration `I915_EXEC_DEFAULT` does not refer to specific engine;
instead it is to be used by user space to specify a default rendering
engine (for 3D) that may or may not be the same as RCS.
The Intel GPU family is a family of integrated GPU's using Unified
Memory Access. For having the GPU "do work", user space will feed the
GPU batch buffers via one of the ioctls `DRM_IOCTL_I915_GEM_EXECBUFFER2`
or `DRM_IOCTL_I915_GEM_EXECBUFFER2_WR`. Most such batchbuffers will
instruct the GPU to perform work (for example rendering) and that work
needs memory from which to read and memory to which to write. All memory
is encapsulated within GEM buffer objects (usually created with the ioctl
`DRM_IOCTL_I915_GEM_CREATE`). An ioctl providing a batchbuffer for the GPU
to create will also list all GEM buffer objects that the batchbuffer reads
and/or writes. For implementation details of memory management see
`GEM BO Management Implementation Details`_.
The i915 driver allows user space to create a context via the ioctl
`DRM_IOCTL_I915_GEM_CONTEXT_CREATE` which is identified by a 32-bit
integer. Such a context should be viewed by user-space as -loosely-
analogous to the idea of a CPU process of an operating system. The i915
driver guarantees that commands issued to a fixed context are to be
executed so that writes of a previously issued command are seen by
reads of following commands. Actions issued between different contexts
(even if from the same file descriptor) are NOT given that guarantee
and the only way to synchronize across contexts (even from the same
file descriptor) is through the use of fences. At least as far back as
Gen4, also have that a context carries with it a GPU HW context;
the HW context is essentially (most of atleast) the state of a GPU.
In addition to the ordering guarantees, the kernel will restore GPU
state via HW context when commands are issued to a context, this saves
user space the need to restore (most of atleast) the GPU state at the
start of each batchbuffer. The non-deprecated ioctls to submit batchbuffer
work can pass that ID (in the lower bits of drm_i915_gem_execbuffer2::rsvd1)
to identify what context to use with the command.
The GPU has its own memory management and address space. The kernel
driver maintains the memory translation table for the GPU. For older
GPUs (i.e. those before Gen8), there is a single global such translation
table, a global Graphics Translation Table (GTT). For newer generation
GPUs each context has its own translation table, called Per-Process
Graphics Translation Table (PPGTT). Of important note, is that although
PPGTT is named per-process it is actually per context. When user space
submits a batchbuffer, the kernel walks the list of GEM buffer objects
used by the batchbuffer and guarantees that not only is the memory of
each such GEM buffer object resident but it is also present in the
(PP)GTT. If the GEM buffer object is not yet placed in the (PP)GTT,
then it is given an address. Two consequences of this are: the kernel
needs to edit the batchbuffer submitted to write the correct value of
the GPU address when a GEM BO is assigned a GPU address and the kernel
might evict a different GEM BO from the (PP)GTT to make address room
for another GEM BO. Consequently, the ioctls submitting a batchbuffer
for execution also include a list of all locations within buffers that
refer to GPU-addresses so that the kernel can edit the buffer correctly.
This process is dubbed relocation.
GEM BO Management Implementation Details
----------------------------------------
.. kernel-doc:: drivers/gpu/drm/i915/i915_vma.h
:doc: Virtual Memory Address
Buffer Object Eviction
----------------------
This section documents the interface functions for evicting buffer
objects to make space available in the virtual gpu address spaces. Note
that this is mostly orthogonal to shrinking buffer objects caches, which
has the goal to make main memory (shared with the gpu through the
unified memory architecture) available.
.. kernel-doc:: drivers/gpu/drm/i915/i915_gem_evict.c
:internal:
Buffer Object Memory Shrinking
------------------------------
This section documents the interface function for shrinking memory usage
of buffer object caches. Shrinking is used to make main memory
available. Note that this is mostly orthogonal to evicting buffer
objects, which has the goal to make space in gpu virtual address spaces.
.. kernel-doc:: drivers/gpu/drm/i915/i915_gem_shrinker.c
:internal:
Batchbuffer Parsing
-------------------
@ -267,6 +370,12 @@ Batchbuffer Pools
.. kernel-doc:: drivers/gpu/drm/i915/i915_gem_batch_pool.c
:internal:
User Batchbuffer Execution
--------------------------
.. kernel-doc:: drivers/gpu/drm/i915/i915_gem_execbuffer.c
:doc: User command execution
Logical Rings, Logical Ring Contexts and Execlists
--------------------------------------------------
@ -312,28 +421,14 @@ Object Tiling IOCTLs
.. kernel-doc:: drivers/gpu/drm/i915/i915_gem_tiling.c
:doc: buffer object tiling
Buffer Object Eviction
----------------------
WOPCM
=====
This section documents the interface functions for evicting buffer
objects to make space available in the virtual gpu address spaces. Note
that this is mostly orthogonal to shrinking buffer objects caches, which
has the goal to make main memory (shared with the gpu through the
unified memory architecture) available.
WOPCM Layout
------------
.. kernel-doc:: drivers/gpu/drm/i915/i915_gem_evict.c
:internal:
Buffer Object Memory Shrinking
------------------------------
This section documents the interface function for shrinking memory usage
of buffer object caches. Shrinking is used to make main memory
available. Note that this is mostly orthogonal to evicting buffer
objects, which has the goal to make space in gpu virtual address spaces.
.. kernel-doc:: drivers/gpu/drm/i915/i915_gem_shrinker.c
:internal:
.. kernel-doc:: drivers/gpu/drm/i915/intel_wopcm.c
:doc: WOPCM Layout
GuC
===
@ -359,6 +454,12 @@ GuC Firmware Layout
.. kernel-doc:: drivers/gpu/drm/i915/intel_guc_fwif.h
:doc: GuC Firmware Layout
GuC Address Space
-----------------
.. kernel-doc:: drivers/gpu/drm/i915/intel_guc.c
:doc: GuC Address Space
Tracing
=======

13
drivers/gpu/drm/i915/Kconfig.debug
View File

@ -25,6 +25,7 @@ config DRM_I915_DEBUG
select X86_MSR # used by igt/pm_rpm
select DRM_VGEM # used by igt/prime_vgem (dmabuf interop checks)
select DRM_DEBUG_MM if DRM=y
select STACKDEPOT if DRM=y # for DRM_DEBUG_MM
select DRM_DEBUG_MM_SELFTEST
select SW_SYNC # signaling validation framework (igt/syncobj*)
select DRM_I915_SW_FENCE_DEBUG_OBJECTS
@ -89,6 +90,18 @@ config DRM_I915_SW_FENCE_CHECK_DAG
If in doubt, say "N".
config DRM_I915_DEBUG_GUC
bool "Enable additional driver debugging for GuC"
depends on DRM_I915
default n
help
Choose this option to turn on extra driver debugging that may affect
performance but will help resolve GuC related issues.
Recommended for driver developers only.
If in doubt, say "N".
config DRM_I915_SELFTEST
bool "Enable selftests upon driver load"
depends on DRM_I915

11
drivers/gpu/drm/i915/Makefile
View File

@ -12,7 +12,7 @@
# Note the danger in using -Wall -Wextra is that when CI updates gcc we
# will most likely get a sudden build breakage... Hopefully we will fix
# new warnings before CI updates!
subdir-ccflags-y := -Wall -Wextra
subdir-ccflags-y := -Wall -Wextra -Wvla
subdir-ccflags-y += $(call cc-disable-warning, unused-parameter)
subdir-ccflags-y += $(call cc-disable-warning, type-limits)
subdir-ccflags-y += $(call cc-disable-warning, missing-field-initializers)
@ -43,7 +43,8 @@ i915-y := i915_drv.o \
intel_csr.o \
intel_device_info.o \
intel_pm.o \
intel_runtime_pm.o
intel_runtime_pm.o \
intel_workarounds.o
i915-$(CONFIG_COMPAT) += i915_ioc32.o
i915-$(CONFIG_DEBUG_FS) += i915_debugfs.o intel_pipe_crc.o
@ -79,7 +80,8 @@ i915-y += i915_cmd_parser.o \
intel_lrc.o \
intel_mocs.o \
intel_ringbuffer.o \
intel_uncore.o
intel_uncore.o \
intel_wopcm.o
# general-purpose microcontroller (GuC) support
i915-y += intel_uc.o \
@ -171,7 +173,8 @@ i915-y += i915_perf.o \
i915_oa_glk.o \
i915_oa_cflgt2.o \
i915_oa_cflgt3.o \
i915_oa_cnl.o
i915_oa_cnl.o \
i915_oa_icl.o
ifeq ($(CONFIG_DRM_I915_GVT),y)
i915-y += intel_gvt.o

13
drivers/gpu/drm/i915/gvt/debugfs.c
View File

@ -122,18 +122,7 @@ static int vgpu_mmio_diff_show(struct seq_file *s, void *unused)
seq_printf(s, "Total: %d, Diff: %d\n", param.total, param.diff);
return 0;
}
static int vgpu_mmio_diff_open(struct inode *inode, struct file *file)
{
return single_open(file, vgpu_mmio_diff_show, inode->i_private);
}
static const struct file_operations vgpu_mmio_diff_fops = {
.open = vgpu_mmio_diff_open,
.read = seq_read,
.llseek = seq_lseek,
.release = single_release,
};
DEFINE_SHOW_ATTRIBUTE(vgpu_mmio_diff);
/**
* intel_gvt_debugfs_add_vgpu - register debugfs entries for a vGPU

318
drivers/gpu/drm/i915/i915_debugfs.c
View File

@ -1215,20 +1215,20 @@ static int i915_frequency_info(struct seq_file *m, void *unused)
max_freq = (IS_GEN9_LP(dev_priv) ? rp_state_cap >> 0 :
rp_state_cap >> 16) & 0xff;
max_freq *= (IS_GEN9_BC(dev_priv) ||
IS_CANNONLAKE(dev_priv) ? GEN9_FREQ_SCALER : 1);
INTEL_GEN(dev_priv) >= 10 ? GEN9_FREQ_SCALER : 1);
seq_printf(m, "Lowest (RPN) frequency: %dMHz\n",
intel_gpu_freq(dev_priv, max_freq));
max_freq = (rp_state_cap & 0xff00) >> 8;
max_freq *= (IS_GEN9_BC(dev_priv) ||
IS_CANNONLAKE(dev_priv) ? GEN9_FREQ_SCALER : 1);
INTEL_GEN(dev_priv) >= 10 ? GEN9_FREQ_SCALER : 1);
seq_printf(m, "Nominal (RP1) frequency: %dMHz\n",
intel_gpu_freq(dev_priv, max_freq));
max_freq = (IS_GEN9_LP(dev_priv) ? rp_state_cap >> 16 :
rp_state_cap >> 0) & 0xff;
max_freq *= (IS_GEN9_BC(dev_priv) ||
IS_CANNONLAKE(dev_priv) ? GEN9_FREQ_SCALER : 1);
INTEL_GEN(dev_priv) >= 10 ? GEN9_FREQ_SCALER : 1);
seq_printf(m, "Max non-overclocked (RP0) frequency: %dMHz\n",
intel_gpu_freq(dev_priv, max_freq));
seq_printf(m, "Max overclocked frequency: %dMHz\n",
@ -1796,9 +1796,9 @@ static int i915_ring_freq_table(struct seq_file *m, void *unused)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
struct intel_rps *rps = &dev_priv->gt_pm.rps;
int ret = 0;
int gpu_freq, ia_freq;
unsigned int max_gpu_freq, min_gpu_freq;
int gpu_freq, ia_freq;
int ret;
if (!HAS_LLC(dev_priv))
return -ENODEV;
@ -1809,13 +1809,12 @@ static int i915_ring_freq_table(struct seq_file *m, void *unused)
if (ret)
goto out;
if (IS_GEN9_BC(dev_priv) || IS_CANNONLAKE(dev_priv)) {
min_gpu_freq = rps->min_freq;
max_gpu_freq = rps->max_freq;
if (IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) {
/* Convert GT frequency to 50 HZ units */
min_gpu_freq = rps->min_freq_softlimit / GEN9_FREQ_SCALER;
max_gpu_freq = rps->max_freq_softlimit / GEN9_FREQ_SCALER;
} else {
min_gpu_freq = rps->min_freq_softlimit;
max_gpu_freq = rps->max_freq_softlimit;
min_gpu_freq /= GEN9_FREQ_SCALER;
max_gpu_freq /= GEN9_FREQ_SCALER;
}
seq_puts(m, "GPU freq (MHz)\tEffective CPU freq (MHz)\tEffective Ring freq (MHz)\n");
@ -1828,7 +1827,7 @@ static int i915_ring_freq_table(struct seq_file *m, void *unused)
seq_printf(m, "%d\t\t%d\t\t\t\t%d\n",
intel_gpu_freq(dev_priv, (gpu_freq *
(IS_GEN9_BC(dev_priv) ||
IS_CANNONLAKE(dev_priv) ?
INTEL_GEN(dev_priv) >= 10 ?
GEN9_FREQ_SCALER : 1))),
((ia_freq >> 0) & 0xff) * 100,
((ia_freq >> 8) & 0xff) * 100);
@ -1923,8 +1922,8 @@ static int i915_gem_framebuffer_info(struct seq_file *m, void *data)
static void describe_ctx_ring(struct seq_file *m, struct intel_ring *ring)
{
seq_printf(m, " (ringbuffer, space: %d, head: %u, tail: %u)",
ring->space, ring->head, ring->tail);
seq_printf(m, " (ringbuffer, space: %d, head: %u, tail: %u, emit: %u)",
ring->space, ring->head, ring->tail, ring->emit);
}
static int i915_context_status(struct seq_file *m, void *unused)
@ -2326,30 +2325,45 @@ static int i915_guc_load_status_info(struct seq_file *m, void *data)
return 0;
}
static const char *
stringify_guc_log_type(enum guc_log_buffer_type type)
{
switch (type) {
case GUC_ISR_LOG_BUFFER:
return "ISR";
case GUC_DPC_LOG_BUFFER:
return "DPC";
case GUC_CRASH_DUMP_LOG_BUFFER:
return "CRASH";
default:
MISSING_CASE(type);
}
return "";
}
static void i915_guc_log_info(struct seq_file *m,
struct drm_i915_private *dev_priv)
{
struct intel_guc *guc = &dev_priv->guc;
struct intel_guc_log *log = &dev_priv->guc.log;
enum guc_log_buffer_type type;
seq_puts(m, "\nGuC logging stats:\n");
if (!intel_guc_log_relay_enabled(log)) {
seq_puts(m, "GuC log relay disabled\n");
return;
}
seq_printf(m, "\tISR: flush count %10u, overflow count %10u\n",
guc->log.flush_count[GUC_ISR_LOG_BUFFER],
guc->log.total_overflow_count[GUC_ISR_LOG_BUFFER]);
seq_puts(m, "GuC logging stats:\n");
seq_printf(m, "\tDPC: flush count %10u, overflow count %10u\n",
guc->log.flush_count[GUC_DPC_LOG_BUFFER],
guc->log.total_overflow_count[GUC_DPC_LOG_BUFFER]);
seq_printf(m, "\tRelay full count: %u\n",
log->relay.full_count);
seq_printf(m, "\tCRASH: flush count %10u, overflow count %10u\n",
guc->log.flush_count[GUC_CRASH_DUMP_LOG_BUFFER],
guc->log.total_overflow_count[GUC_CRASH_DUMP_LOG_BUFFER]);
seq_printf(m, "\tTotal flush interrupt count: %u\n",
guc->log.flush_interrupt_count);
seq_printf(m, "\tCapture miss count: %u\n",
guc->log.capture_miss_count);
for (type = GUC_ISR_LOG_BUFFER; type < GUC_MAX_LOG_BUFFER; type++) {
seq_printf(m, "\t%s:\tflush count %10u, overflow count %10u\n",
stringify_guc_log_type(type),
log->stats[type].flush,
log->stats[type].sampled_overflow);
}
}
static void i915_guc_client_info(struct seq_file *m,
@ -2379,14 +2393,19 @@ static int i915_guc_info(struct seq_file *m, void *data)
struct drm_i915_private *dev_priv = node_to_i915(m->private);
const struct intel_guc *guc = &dev_priv->guc;
if (!USES_GUC_SUBMISSION(dev_priv))
if (!USES_GUC(dev_priv))
return -ENODEV;
i915_guc_log_info(m, dev_priv);
if (!USES_GUC_SUBMISSION(dev_priv))
return 0;
GEM_BUG_ON(!guc->execbuf_client);
seq_printf(m, "Doorbell map:\n");
seq_printf(m, "\nDoorbell map:\n");
seq_printf(m, "\t%*pb\n", GUC_NUM_DOORBELLS, guc->doorbell_bitmap);
seq_printf(m, "Doorbell next cacheline: 0x%x\n\n", guc->db_cacheline);
seq_printf(m, "Doorbell next cacheline: 0x%x\n", guc->db_cacheline);
seq_printf(m, "\nGuC execbuf client @ %p:\n", guc->execbuf_client);
i915_guc_client_info(m, dev_priv, guc->execbuf_client);
@ -2396,8 +2415,6 @@ static int i915_guc_info(struct seq_file *m, void *data)
i915_guc_client_info(m, dev_priv, guc->preempt_client);
}
i915_guc_log_info(m, dev_priv);
/* Add more as required ... */
return 0;
@ -2496,35 +2513,73 @@ static int i915_guc_log_dump(struct seq_file *m, void *data)
return 0;
}
static int i915_guc_log_control_get(void *data, u64 *val)
static int i915_guc_log_level_get(void *data, u64 *val)
{
struct drm_i915_private *dev_priv = data;
if (!HAS_GUC(dev_priv))
if (!USES_GUC(dev_priv))
return -ENODEV;
if (!dev_priv->guc.log.vma)
return -EINVAL;
*val = i915_modparams.guc_log_level;
*val = intel_guc_log_level_get(&dev_priv->guc.log);
return 0;
}
static int i915_guc_log_control_set(void *data, u64 val)
static int i915_guc_log_level_set(void *data, u64 val)
{
struct drm_i915_private *dev_priv = data;
if (!HAS_GUC(dev_priv))
if (!USES_GUC(dev_priv))
return -ENODEV;
return intel_guc_log_control(&dev_priv->guc, val);
return intel_guc_log_level_set(&dev_priv->guc.log, val);
}
DEFINE_SIMPLE_ATTRIBUTE(i915_guc_log_control_fops,
i915_guc_log_control_get, i915_guc_log_control_set,
DEFINE_SIMPLE_ATTRIBUTE(i915_guc_log_level_fops,
i915_guc_log_level_get, i915_guc_log_level_set,
"%lld\n");
static int i915_guc_log_relay_open(struct inode *inode, struct file *file)
{
struct drm_i915_private *dev_priv = inode->i_private;
if (!USES_GUC(dev_priv))
return -ENODEV;
file->private_data = &dev_priv->guc.log;
return intel_guc_log_relay_open(&dev_priv->guc.log);
}
static ssize_t
i915_guc_log_relay_write(struct file *filp,
const char __user *ubuf,
size_t cnt,
loff_t *ppos)
{
struct intel_guc_log *log = filp->private_data;
intel_guc_log_relay_flush(log);
return cnt;
}
static int i915_guc_log_relay_release(struct inode *inode, struct file *file)
{
struct drm_i915_private *dev_priv = inode->i_private;
intel_guc_log_relay_close(&dev_priv->guc.log);
return 0;
}
static const struct file_operations i915_guc_log_relay_fops = {
.owner = THIS_MODULE,
.open = i915_guc_log_relay_open,
.write = i915_guc_log_relay_write,
.release = i915_guc_log_relay_release,
};
static const char *psr2_live_status(u32 val)
{
static const char * const live_status[] = {
@ -2569,14 +2624,13 @@ static int i915_edp_psr_status(struct seq_file *m, void *data)
mutex_lock(&dev_priv->psr.lock);
seq_printf(m, "Enabled: %s\n", yesno((bool)dev_priv->psr.enabled));
seq_printf(m, "Active: %s\n", yesno(dev_priv->psr.active));
seq_printf(m, "Busy frontbuffer bits: 0x%03x\n",
dev_priv->psr.busy_frontbuffer_bits);
seq_printf(m, "Re-enable work scheduled: %s\n",
yesno(work_busy(&dev_priv->psr.work.work)));
if (HAS_DDI(dev_priv)) {
if (dev_priv->psr.psr2_support)
if (dev_priv->psr.psr2_enabled)
enabled = I915_READ(EDP_PSR2_CTL) & EDP_PSR2_ENABLE;
else
enabled = I915_READ(EDP_PSR_CTL) & EDP_PSR_ENABLE;
@ -2624,7 +2678,7 @@ static int i915_edp_psr_status(struct seq_file *m, void *data)
seq_printf(m, "Performance_Counter: %u\n", psrperf);
}
if (dev_priv->psr.psr2_support) {
if (dev_priv->psr.psr2_enabled) {
u32 psr2 = I915_READ(EDP_PSR2_STATUS);
seq_printf(m, "EDP_PSR2_STATUS: %x [%s]\n",
@ -3231,7 +3285,8 @@ static int i915_shared_dplls_info(struct seq_file *m, void *unused)
for (i = 0; i < dev_priv->num_shared_dpll; i++) {
struct intel_shared_dpll *pll = &dev_priv->shared_dplls[i];
seq_printf(m, "DPLL%i: %s, id: %i\n", i, pll->name, pll->id);
seq_printf(m, "DPLL%i: %s, id: %i\n", i, pll->info->name,
pll->info->id);
seq_printf(m, " crtc_mask: 0x%08x, active: 0x%x, on: %s\n",
pll->state.crtc_mask, pll->active_mask, yesno(pll->on));
seq_printf(m, " tracked hardware state:\n");
@ -3567,7 +3622,8 @@ static ssize_t i915_displayport_test_active_write(struct file *file,
static int i915_displayport_test_active_show(struct seq_file *m, void *data)
{
struct drm_device *dev = m->private;
struct drm_i915_private *dev_priv = m->private;
struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
@ -3601,10 +3657,8 @@ static int i915_displayport_test_active_show(struct seq_file *m, void *data)
static int i915_displayport_test_active_open(struct inode *inode,
struct file *file)
{
struct drm_i915_private *dev_priv = inode->i_private;
return single_open(file, i915_displayport_test_active_show,
&dev_priv->drm);
inode->i_private);
}
static const struct file_operations i915_displayport_test_active_fops = {
@ -3618,7 +3672,8 @@ static const struct file_operations i915_displayport_test_active_fops = {
static int i915_displayport_test_data_show(struct seq_file *m, void *data)
{
struct drm_device *dev = m->private;
struct drm_i915_private *dev_priv = m->private;
struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
@ -3657,26 +3712,12 @@ static int i915_displayport_test_data_show(struct seq_file *m, void *data)
return 0;
}
static int i915_displayport_test_data_open(struct inode *inode,
struct file *file)
{
struct drm_i915_private *dev_priv = inode->i_private;
return single_open(file, i915_displayport_test_data_show,
&dev_priv->drm);
}
static const struct file_operations i915_displayport_test_data_fops = {
.owner = THIS_MODULE,
.open = i915_displayport_test_data_open,
.read = seq_read,
.llseek = seq_lseek,
.release = single_release
};
DEFINE_SHOW_ATTRIBUTE(i915_displayport_test_data);
static int i915_displayport_test_type_show(struct seq_file *m, void *data)
{
struct drm_device *dev = m->private;
struct drm_i915_private *dev_priv = m->private;
struct drm_device *dev = &dev_priv->drm;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct intel_dp *intel_dp;
@ -3703,23 +3744,7 @@ static int i915_displayport_test_type_show(struct seq_file *m, void *data)
return 0;
}
static int i915_displayport_test_type_open(struct inode *inode,
struct file *file)
{
struct drm_i915_private *dev_priv = inode->i_private;
return single_open(file, i915_displayport_test_type_show,
&dev_priv->drm);
}
static const struct file_operations i915_displayport_test_type_fops = {
.owner = THIS_MODULE,
.open = i915_displayport_test_type_open,
.read = seq_read,
.llseek = seq_lseek,
.release = single_release
};
DEFINE_SHOW_ATTRIBUTE(i915_displayport_test_type);
static void wm_latency_show(struct seq_file *m, const uint16_t wm[8])
{
@ -3987,8 +4012,8 @@ i915_wedged_set(void *data, u64 val)
engine->hangcheck.stalled = true;
}
i915_handle_error(i915, val, "Manually set wedged engine mask = %llx",
val);
i915_handle_error(i915, val, I915_ERROR_CAPTURE,
"Manually set wedged engine mask = %llx", val);
wait_on_bit(&i915->gpu_error.flags,
I915_RESET_HANDOFF,
@ -4316,9 +4341,10 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_cache_sharing_fops,
static void cherryview_sseu_device_status(struct drm_i915_private *dev_priv,
struct sseu_dev_info *sseu)
{
int ss_max = 2;
#define SS_MAX 2
const int ss_max = SS_MAX;
u32 sig1[SS_MAX], sig2[SS_MAX];
int ss;
u32 sig1[ss_max], sig2[ss_max];
sig1[0] = I915_READ(CHV_POWER_SS0_SIG1);
sig1[1] = I915_READ(CHV_POWER_SS1_SIG1);
@ -4342,15 +4368,16 @@ static void cherryview_sseu_device_status(struct drm_i915_private *dev_priv,
sseu->eu_per_subslice = max_t(unsigned int,
sseu->eu_per_subslice, eu_cnt);
}
#undef SS_MAX
}
static void gen10_sseu_device_status(struct drm_i915_private *dev_priv,
struct sseu_dev_info *sseu)
{
#define SS_MAX 6
const struct intel_device_info *info = INTEL_INFO(dev_priv);
u32 s_reg[SS_MAX], eu_reg[2 * SS_MAX], eu_mask[2];
int s, ss;
u32 s_reg[info->sseu.max_slices];
u32 eu_reg[2 * info->sseu.max_subslices], eu_mask[2];
for (s = 0; s < info->sseu.max_slices; s++) {
/*
@ -4397,15 +4424,16 @@ static void gen10_sseu_device_status(struct drm_i915_private *dev_priv,
eu_cnt);
}
}
#undef SS_MAX
}
static void gen9_sseu_device_status(struct drm_i915_private *dev_priv,
struct sseu_dev_info *sseu)
{
#define SS_MAX 3
const struct intel_device_info *info = INTEL_INFO(dev_priv);
u32 s_reg[SS_MAX], eu_reg[2 * SS_MAX], eu_mask[2];
int s, ss;
u32 s_reg[info->sseu.max_slices];
u32 eu_reg[2 * info->sseu.max_subslices], eu_mask[2];
for (s = 0; s < info->sseu.max_slices; s++) {
s_reg[s] = I915_READ(GEN9_SLICE_PGCTL_ACK(s));
@ -4452,6 +4480,7 @@ static void gen9_sseu_device_status(struct drm_i915_private *dev_priv,
eu_cnt);
}
}
#undef SS_MAX
}
static void broadwell_sseu_device_status(struct drm_i915_private *dev_priv,
@ -4703,6 +4732,67 @@ static int i915_drrs_ctl_set(void *data, u64 val)
DEFINE_SIMPLE_ATTRIBUTE(i915_drrs_ctl_fops, NULL, i915_drrs_ctl_set, "%llu\n");
static ssize_t
i915_fifo_underrun_reset_write(struct file *filp,
const char __user *ubuf,
size_t cnt, loff_t *ppos)
{
struct drm_i915_private *dev_priv = filp->private_data;
struct intel_crtc *intel_crtc;
struct drm_device *dev = &dev_priv->drm;
int ret;
bool reset;
ret = kstrtobool_from_user(ubuf, cnt, &reset);
if (ret)
return ret;
if (!reset)
return cnt;
for_each_intel_crtc(dev, intel_crtc) {
struct drm_crtc_commit *commit;
struct intel_crtc_state *crtc_state;
ret = drm_modeset_lock_single_interruptible(&intel_crtc->base.mutex);
if (ret)
return ret;
crtc_state = to_intel_crtc_state(intel_crtc->base.state);
commit = crtc_state->base.commit;
if (commit) {
ret = wait_for_completion_interruptible(&commit->hw_done);
if (!ret)
ret = wait_for_completion_interruptible(&commit->flip_done);
}
if (!ret && crtc_state->base.active) {
DRM_DEBUG_KMS("Re-arming FIFO underruns on pipe %c\n",
pipe_name(intel_crtc->pipe));
intel_crtc_arm_fifo_underrun(intel_crtc, crtc_state);
}
drm_modeset_unlock(&intel_crtc->base.mutex);
if (ret)
return ret;
}
ret = intel_fbc_reset_underrun(dev_priv);
if (ret)
return ret;
return cnt;
}
static const struct file_operations i915_fifo_underrun_reset_ops = {
.owner = THIS_MODULE,
.open = simple_open,
.write = i915_fifo_underrun_reset_write,
.llseek = default_llseek,
};
static const struct drm_info_list i915_debugfs_list[] = {
{"i915_capabilities", i915_capabilities, 0},
{"i915_gem_objects", i915_gem_object_info, 0},
@ -4770,6 +4860,7 @@ static const struct i915_debugfs_files {
{"i915_error_state", &i915_error_state_fops},
{"i915_gpu_info", &i915_gpu_info_fops},
#endif
{"i915_fifo_underrun_reset", &i915_fifo_underrun_reset_ops},
{"i915_next_seqno", &i915_next_seqno_fops},
{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
@ -4779,7 +4870,8 @@ static const struct i915_debugfs_files {
{"i915_dp_test_data", &i915_displayport_test_data_fops},
{"i915_dp_test_type", &i915_displayport_test_type_fops},
{"i915_dp_test_active", &i915_displayport_test_active_fops},
{"i915_guc_log_control", &i915_guc_log_control_fops},
{"i915_guc_log_level", &i915_guc_log_level_fops},
{"i915_guc_log_relay", &i915_guc_log_relay_fops},
{"i915_hpd_storm_ctl", &i915_hpd_storm_ctl_fops},
{"i915_ipc_status", &i915_ipc_status_fops},
{"i915_drrs_ctl", &i915_drrs_ctl_fops}
@ -4876,19 +4968,7 @@ static int i915_dpcd_show(struct seq_file *m, void *data)
return 0;
}
static int i915_dpcd_open(struct inode *inode, struct file *file)
{
return single_open(file, i915_dpcd_show, inode->i_private);
}
static const struct file_operations i915_dpcd_fops = {
.owner = THIS_MODULE,
.open = i915_dpcd_open,
.read = seq_read,
.llseek = seq_lseek,
.release = single_release,
};
DEFINE_SHOW_ATTRIBUTE(i915_dpcd);
static int i915_panel_show(struct seq_file *m, void *data)
{
@ -4910,19 +4990,7 @@ static int i915_panel_show(struct seq_file *m, void *data)
return 0;
}
static int i915_panel_open(struct inode *inode, struct file *file)
{
return single_open(file, i915_panel_show, inode->i_private);
}
static const struct file_operations i915_panel_fops = {
.owner = THIS_MODULE,
.open = i915_panel_open,
.read = seq_read,
.llseek = seq_lseek,
.release = single_release,
};
DEFINE_SHOW_ATTRIBUTE(i915_panel);
/**
* i915_debugfs_connector_add - add i915 specific connector debugfs files

66
drivers/gpu/drm/i915/i915_drv.c
View File

@ -377,9 +377,9 @@ static int i915_getparam_ioctl(struct drm_device *dev, void *data,
value = INTEL_INFO(dev_priv)->sseu.min_eu_in_pool;
break;
case I915_PARAM_HUC_STATUS:
intel_runtime_pm_get(dev_priv);
value = I915_READ(HUC_STATUS2) & HUC_FW_VERIFIED;
intel_runtime_pm_put(dev_priv);
value = intel_huc_check_status(&dev_priv->huc);
if (value < 0)
return value;
break;
case I915_PARAM_MMAP_GTT_VERSION:
/* Though we've started our numbering from 1, and so class all
@ -695,11 +695,9 @@ static int i915_load_modeset_init(struct drm_device *dev)
if (ret)
goto cleanup_irq;
intel_uc_init_fw(dev_priv);
ret = i915_gem_init(dev_priv);
if (ret)
goto cleanup_uc;
goto cleanup_irq;
intel_setup_overlay(dev_priv);
@ -719,8 +717,6 @@ cleanup_gem:
if (i915_gem_suspend(dev_priv))
DRM_ERROR("failed to idle hardware; continuing to unload!\n");
i915_gem_fini(dev_priv);
cleanup_uc:
intel_uc_fini_fw(dev_priv);
cleanup_irq:
drm_irq_uninstall(dev);
intel_teardown_gmbus(dev_priv);
@ -922,16 +918,21 @@ static int i915_driver_init_early(struct drm_i915_private *dev_priv,
mutex_init(&dev_priv->wm.wm_mutex);
mutex_init(&dev_priv->pps_mutex);
intel_uc_init_early(dev_priv);
i915_memcpy_init_early(dev_priv);
ret = i915_workqueues_init(dev_priv);
if (ret < 0)
goto err_engines;
ret = i915_gem_init_early(dev_priv);
if (ret < 0)
goto err_workqueues;
/* This must be called before any calls to HAS_PCH_* */
intel_detect_pch(dev_priv);
intel_wopcm_init_early(&dev_priv->wopcm);
intel_uc_init_early(dev_priv);
intel_pm_setup(dev_priv);
intel_init_dpio(dev_priv);
intel_power_domains_init(dev_priv);
@ -940,18 +941,13 @@ static int i915_driver_init_early(struct drm_i915_private *dev_priv,
intel_init_display_hooks(dev_priv);
intel_init_clock_gating_hooks(dev_priv);
intel_init_audio_hooks(dev_priv);
ret = i915_gem_load_init(dev_priv);
if (ret < 0)
goto err_irq;
intel_display_crc_init(dev_priv);
intel_detect_preproduction_hw(dev_priv);
return 0;
err_irq:
intel_irq_fini(dev_priv);
err_workqueues:
i915_workqueues_cleanup(dev_priv);
err_engines:
i915_engines_cleanup(dev_priv);
@ -964,8 +960,9 @@ err_engines:
*/
static void i915_driver_cleanup_early(struct drm_i915_private *dev_priv)
{
i915_gem_load_cleanup(dev_priv);
intel_irq_fini(dev_priv);
intel_uc_cleanup_early(dev_priv);
i915_gem_cleanup_early(dev_priv);
i915_workqueues_cleanup(dev_priv);
i915_engines_cleanup(dev_priv);
}
@ -1035,6 +1032,10 @@ static int i915_driver_init_mmio(struct drm_i915_private *dev_priv)
intel_uncore_init(dev_priv);
intel_device_info_init_mmio(dev_priv);
intel_uncore_prune(dev_priv);
intel_uc_init_mmio(dev_priv);
ret = intel_engines_init_mmio(dev_priv);
@ -1077,8 +1078,6 @@ static void intel_sanitize_options(struct drm_i915_private *dev_priv)
i915_modparams.enable_ppgtt);
DRM_DEBUG_DRIVER("ppgtt mode: %i\n", i915_modparams.enable_ppgtt);
intel_uc_sanitize_options(dev_priv);
intel_gvt_sanitize_options(dev_priv);
}
@ -1244,7 +1243,6 @@ static void i915_driver_register(struct drm_i915_private *dev_priv)
/* Reveal our presence to userspace */
if (drm_dev_register(dev, 0) == 0) {
i915_debugfs_register(dev_priv);
i915_guc_log_register(dev_priv);
i915_setup_sysfs(dev_priv);
/* Depends on sysfs having been initialized */
@ -1304,7 +1302,6 @@ static void i915_driver_unregister(struct drm_i915_private *dev_priv)
i915_pmu_unregister(dev_priv);
i915_teardown_sysfs(dev_priv);
i915_guc_log_unregister(dev_priv);
drm_dev_unregister(&dev_priv->drm);
i915_gem_shrinker_unregister(dev_priv);
@ -1463,7 +1460,6 @@ void i915_driver_unload(struct drm_device *dev)
i915_reset_error_state(dev_priv);
i915_gem_fini(dev_priv);
intel_uc_fini_fw(dev_priv);
intel_fbc_cleanup_cfb(dev_priv);
intel_power_domains_fini(dev_priv);
@ -1876,7 +1872,8 @@ static int i915_resume_switcheroo(struct drm_device *dev)
/**
* i915_reset - reset chip after a hang
* @i915: #drm_i915_private to reset
* @flags: Instructions
* @stalled_mask: mask of the stalled engines with the guilty requests
* @reason: user error message for why we are resetting
*
* Reset the chip. Useful if a hang is detected. Marks the device as wedged
* on failure.
@ -1891,12 +1888,16 @@ static int i915_resume_switcheroo(struct drm_device *dev)
* - re-init interrupt state
* - re-init display
*/
void i915_reset(struct drm_i915_private *i915, unsigned int flags)
void i915_reset(struct drm_i915_private *i915,
unsigned int stalled_mask,
const char *reason)
{
struct i915_gpu_error *error = &i915->gpu_error;
int ret;
int i;
GEM_TRACE("flags=%lx\n", error->flags);
might_sleep();
lockdep_assert_held(&i915->drm.struct_mutex);
GEM_BUG_ON(!test_bit(I915_RESET_BACKOFF, &error->flags));
@ -1908,8 +1909,8 @@ void i915_reset(struct drm_i915_private *i915, unsigned int flags)
if (!i915_gem_unset_wedged(i915))
goto wakeup;
if (!(flags & I915_RESET_QUIET))
dev_notice(i915->drm.dev, "Resetting chip after gpu hang\n");
if (reason)
dev_notice(i915->drm.dev, "Resetting chip for %s\n", reason);
error->reset_count++;
disable_irq(i915->drm.irq);
@ -1952,7 +1953,7 @@ void i915_reset(struct drm_i915_private *i915, unsigned int flags)
goto error;
}
i915_gem_reset(i915);
i915_gem_reset(i915, stalled_mask);
intel_overlay_reset(i915);
/*
@ -1998,7 +1999,6 @@ taint:
error:
i915_gem_set_wedged(i915);
i915_retire_requests(i915);
intel_gpu_reset(i915, ALL_ENGINES);
goto finish;
}
@ -2011,7 +2011,7 @@ static inline int intel_gt_reset_engine(struct drm_i915_private *dev_priv,
/**
* i915_reset_engine - reset GPU engine to recover from a hang
* @engine: engine to reset
* @flags: options
* @msg: reason for GPU reset; or NULL for no dev_notice()
*
* Reset a specific GPU engine. Useful if a hang is detected.
* Returns zero on successful reset or otherwise an error code.
@ -2021,12 +2021,13 @@ static inline int intel_gt_reset_engine(struct drm_i915_private *dev_priv,
* - reset engine (which will force the engine to idle)
* - re-init/configure engine
*/
int i915_reset_engine(struct intel_engine_cs *engine, unsigned int flags)
int i915_reset_engine(struct intel_engine_cs *engine, const char *msg)
{
struct i915_gpu_error *error = &engine->i915->gpu_error;
struct i915_request *active_request;
int ret;
GEM_TRACE("%s flags=%lx\n", engine->name, error->flags);
GEM_BUG_ON(!test_bit(I915_RESET_ENGINE + engine->id, &error->flags));
active_request = i915_gem_reset_prepare_engine(engine);
@ -2036,10 +2037,9 @@ int i915_reset_engine(struct intel_engine_cs *engine, unsigned int flags)
goto out;
}
if (!(flags & I915_RESET_QUIET)) {
if (msg)
dev_notice(engine->i915->drm.dev,
"Resetting %s after gpu hang\n", engine->name);
}
"Resetting %s for %s\n", engine->name, msg);
error->reset_engine_count[engine->id]++;
if (!engine->i915->guc.execbuf_client)
@ -2059,7 +2059,7 @@ int i915_reset_engine(struct intel_engine_cs *engine, unsigned int flags)
* active request and can drop it, adjust head to skip the offending
* request to resume executing remaining requests in the queue.
*/
i915_gem_reset_engine(engine, active_request);
i915_gem_reset_engine(engine, active_request, true);
/*
* The engine and its registers (and workarounds in case of render)

389
drivers/gpu/drm/i915/i915_drv.h
View File

@ -64,6 +64,7 @@
#include "intel_opregion.h"
#include "intel_ringbuffer.h"
#include "intel_uncore.h"
#include "intel_wopcm.h"
#include "intel_uc.h"
#include "i915_gem.h"
@ -72,7 +73,7 @@
#include "i915_gem_object.h"
#include "i915_gem_gtt.h"
#include "i915_gem_timeline.h"
#include "i915_gpu_error.h"
#include "i915_request.h"
#include "i915_vma.h"
@ -83,8 +84,8 @@
#define DRIVER_NAME "i915"
#define DRIVER_DESC "Intel Graphics"
#define DRIVER_DATE "20180308"
#define DRIVER_TIMESTAMP 1520513379
#define DRIVER_DATE "20180413"
#define DRIVER_TIMESTAMP 1523611258
/* Use I915_STATE_WARN(x) and I915_STATE_WARN_ON() (rather than WARN() and
* WARN_ON()) for hw state sanity checks to check for unexpected conditions
@ -261,6 +262,7 @@ enum hpd_pin {
HPD_PORT_C,
HPD_PORT_D,
HPD_PORT_E,
HPD_PORT_F,
HPD_NUM_PINS
};
@ -453,172 +455,6 @@ struct intel_csr {
uint32_t allowed_dc_mask;
};
struct intel_display_error_state;
struct i915_gpu_state {
struct kref ref;
ktime_t time;
ktime_t boottime;
ktime_t uptime;
struct drm_i915_private *i915;
char error_msg[128];
bool simulated;
bool awake;
bool wakelock;
bool suspended;
int iommu;
u32 reset_count;
u32 suspend_count;
struct intel_device_info device_info;
struct intel_driver_caps driver_caps;
struct i915_params params;
struct i915_error_uc {
struct intel_uc_fw guc_fw;
struct intel_uc_fw huc_fw;
struct drm_i915_error_object *guc_log;
} uc;
/* Generic register state */
u32 eir;
u32 pgtbl_er;
u32 ier;
u32 gtier[4], ngtier;
u32 ccid;
u32 derrmr;
u32 forcewake;
u32 error; /* gen6+ */
u32 err_int; /* gen7 */
u32 fault_data0; /* gen8, gen9 */
u32 fault_data1; /* gen8, gen9 */
u32 done_reg;
u32 gac_eco;
u32 gam_ecochk;
u32 gab_ctl;
u32 gfx_mode;
u32 nfence;
u64 fence[I915_MAX_NUM_FENCES];
struct intel_overlay_error_state *overlay;
struct intel_display_error_state *display;
struct drm_i915_error_engine {
int engine_id;
/* Software tracked state */
bool idle;
bool waiting;
int num_waiters;
unsigned long hangcheck_timestamp;
bool hangcheck_stalled;
enum intel_engine_hangcheck_action hangcheck_action;
struct i915_address_space *vm;
int num_requests;
u32 reset_count;
/* position of active request inside the ring */
u32 rq_head, rq_post, rq_tail;
/* our own tracking of ring head and tail */
u32 cpu_ring_head;
u32 cpu_ring_tail;
u32 last_seqno;
/* Register state */
u32 start;
u32 tail;
u32 head;
u32 ctl;
u32 mode;
u32 hws;
u32 ipeir;
u32 ipehr;
u32 bbstate;
u32 instpm;
u32 instps;
u32 seqno;
u64 bbaddr;
u64 acthd;
u32 fault_reg;
u64 faddr;
u32 rc_psmi; /* sleep state */
u32 semaphore_mboxes[I915_NUM_ENGINES - 1];
struct intel_instdone instdone;
struct drm_i915_error_context {
char comm[TASK_COMM_LEN];
pid_t pid;
u32 handle;
u32 hw_id;
int priority;
int ban_score;
int active;
int guilty;
bool bannable;
} context;
struct drm_i915_error_object {
u64 gtt_offset;
u64 gtt_size;
int page_count;
int unused;
u32 *pages[0];
} *ringbuffer, *batchbuffer, *wa_batchbuffer, *ctx, *hws_page;
struct drm_i915_error_object **user_bo;
long user_bo_count;
struct drm_i915_error_object *wa_ctx;
struct drm_i915_error_object *default_state;
struct drm_i915_error_request {
long jiffies;
pid_t pid;
u32 context;
int priority;
int ban_score;
u32 seqno;
u32 head;
u32 tail;
} *requests, execlist[EXECLIST_MAX_PORTS];
unsigned int num_ports;
struct drm_i915_error_waiter {
char comm[TASK_COMM_LEN];
pid_t pid;
u32 seqno;
} *waiters;
struct {
u32 gfx_mode;
union {
u64 pdp[4];
u32 pp_dir_base;
};
} vm_info;
} engine[I915_NUM_ENGINES];
struct drm_i915_error_buffer {
u32 size;
u32 name;
u32 rseqno[I915_NUM_ENGINES], wseqno;
u64 gtt_offset;
u32 read_domains;
u32 write_domain;
s32 fence_reg:I915_MAX_NUM_FENCE_BITS;
u32 tiling:2;
u32 dirty:1;
u32 purgeable:1;
u32 userptr:1;
s32 engine:4;
u32 cache_level:3;
} *active_bo[I915_NUM_ENGINES], *pinned_bo;
u32 active_bo_count[I915_NUM_ENGINES], pinned_bo_count;
struct i915_address_space *active_vm[I915_NUM_ENGINES];
};
enum i915_cache_level {
I915_CACHE_NONE = 0,
I915_CACHE_LLC, /* also used for snoopable memory on non-LLC */
@ -766,12 +602,13 @@ struct i915_psr {
bool active;
struct delayed_work work;
unsigned busy_frontbuffer_bits;
bool psr2_support;
bool aux_frame_sync;
bool sink_psr2_support;
bool link_standby;
bool y_cord_support;
bool colorimetry_support;
bool alpm;
bool has_hw_tracking;
bool psr2_enabled;
u8 sink_sync_latency;
void (*enable_source)(struct intel_dp *,
const struct intel_crtc_state *);
@ -1146,16 +983,6 @@ struct i915_gem_mm {
u32 object_count;
};
struct drm_i915_error_state_buf {
struct drm_i915_private *i915;
unsigned bytes;
unsigned size;
int err;
u8 *buf;
loff_t start;
loff_t pos;
};
#define I915_IDLE_ENGINES_TIMEOUT (200) /* in ms */
#define I915_RESET_TIMEOUT (10 * HZ) /* 10s */
@ -1164,102 +991,6 @@ struct drm_i915_error_state_buf {
#define I915_ENGINE_DEAD_TIMEOUT (4 * HZ) /* Seqno, head and subunits dead */
#define I915_SEQNO_DEAD_TIMEOUT (12 * HZ) /* Seqno dead with active head */
struct i915_gpu_error {
/* For hangcheck timer */
#define DRM_I915_HANGCHECK_PERIOD 1500 /* in ms */
#define DRM_I915_HANGCHECK_JIFFIES msecs_to_jiffies(DRM_I915_HANGCHECK_PERIOD)
struct delayed_work hangcheck_work;
/* For reset and error_state handling. */
spinlock_t lock;
/* Protected by the above dev->gpu_error.lock. */
struct i915_gpu_state *first_error;
atomic_t pending_fb_pin;
unsigned long missed_irq_rings;
/**
* State variable controlling the reset flow and count
*
* This is a counter which gets incremented when reset is triggered,
*
* Before the reset commences, the I915_RESET_BACKOFF bit is set
* meaning that any waiters holding onto the struct_mutex should
* relinquish the lock immediately in order for the reset to start.
*
* If reset is not completed succesfully, the I915_WEDGE bit is
* set meaning that hardware is terminally sour and there is no
* recovery. All waiters on the reset_queue will be woken when
* that happens.
*
* This counter is used by the wait_seqno code to notice that reset
* event happened and it needs to restart the entire ioctl (since most
* likely the seqno it waited for won't ever signal anytime soon).
*
* This is important for lock-free wait paths, where no contended lock
* naturally enforces the correct ordering between the bail-out of the
* waiter and the gpu reset work code.
*/
unsigned long reset_count;
/**
* flags: Control various stages of the GPU reset
*
* #I915_RESET_BACKOFF - When we start a reset, we want to stop any
* other users acquiring the struct_mutex. To do this we set the
* #I915_RESET_BACKOFF bit in the error flags when we detect a reset
* and then check for that bit before acquiring the struct_mutex (in
* i915_mutex_lock_interruptible()?). I915_RESET_BACKOFF serves a
* secondary role in preventing two concurrent global reset attempts.
*
* #I915_RESET_HANDOFF - To perform the actual GPU reset, we need the
* struct_mutex. We try to acquire the struct_mutex in the reset worker,
* but it may be held by some long running waiter (that we cannot
* interrupt without causing trouble). Once we are ready to do the GPU
* reset, we set the I915_RESET_HANDOFF bit and wakeup any waiters. If
* they already hold the struct_mutex and want to participate they can
* inspect the bit and do the reset directly, otherwise the worker
* waits for the struct_mutex.
*
* #I915_RESET_ENGINE[num_engines] - Since the driver doesn't need to
* acquire the struct_mutex to reset an engine, we need an explicit
* flag to prevent two concurrent reset attempts in the same engine.
* As the number of engines continues to grow, allocate the flags from
* the most significant bits.
*
* #I915_WEDGED - If reset fails and we can no longer use the GPU,
* we set the #I915_WEDGED bit. Prior to command submission, e.g.
* i915_request_alloc(), this bit is checked and the sequence
* aborted (with -EIO reported to userspace) if set.
*/
unsigned long flags;
#define I915_RESET_BACKOFF 0
#define I915_RESET_HANDOFF 1
#define I915_RESET_MODESET 2
#define I915_WEDGED (BITS_PER_LONG - 1)
#define I915_RESET_ENGINE (I915_WEDGED - I915_NUM_ENGINES)
/** Number of times an engine has been reset */
u32 reset_engine_count[I915_NUM_ENGINES];
/**
* Waitqueue to signal when a hang is detected. Used to for waiters
* to release the struct_mutex for the reset to procede.
*/
wait_queue_head_t wait_queue;
/**
* Waitqueue to signal when the reset has completed. Used by clients
* that wait for dev_priv->mm.wedged to settle.
*/
wait_queue_head_t reset_queue;
/* For missed irq/seqno simulation. */
unsigned long test_irq_rings;
};
enum modeset_restore {
MODESET_ON_LID_OPEN,
MODESET_DONE,
@ -1451,11 +1182,12 @@ static inline bool skl_ddb_entry_equal(const struct skl_ddb_entry *e1,
}
struct skl_ddb_allocation {
struct skl_ddb_entry plane[I915_MAX_PIPES][I915_MAX_PLANES]; /* packed/uv */
struct skl_ddb_entry y_plane[I915_MAX_PIPES][I915_MAX_PLANES];
/* packed/y */
struct skl_ddb_entry plane[I915_MAX_PIPES][I915_MAX_PLANES];
struct skl_ddb_entry uv_plane[I915_MAX_PIPES][I915_MAX_PLANES];
};
struct skl_wm_values {
struct skl_ddb_values {
unsigned dirty_pipes;
struct skl_ddb_allocation ddb;
};
@ -1470,6 +1202,7 @@ struct skl_wm_level {
struct skl_wm_params {
bool x_tiled, y_tiled;
bool rc_surface;
bool is_planar;
uint32_t width;
uint8_t cpp;
uint32_t plane_pixel_rate;
@ -1860,6 +1593,8 @@ struct drm_i915_private {
struct intel_gvt *gvt;
struct intel_wopcm wopcm;
struct intel_huc huc;
struct intel_guc guc;
@ -2152,7 +1887,7 @@ struct drm_i915_private {
/* current hardware state */
union {
struct ilk_wm_values hw;
struct skl_wm_values skl_hw;
struct skl_ddb_values skl_hw;
struct vlv_wm_values vlv;
struct g4x_wm_values g4x;
};
@ -2392,6 +2127,11 @@ static inline struct drm_i915_private *kdev_to_i915(struct device *kdev)
return to_i915(dev_get_drvdata(kdev));
}
static inline struct drm_i915_private *wopcm_to_i915(struct intel_wopcm *wopcm)
{
return container_of(wopcm, struct drm_i915_private, wopcm);
}
static inline struct drm_i915_private *guc_to_i915(struct intel_guc *guc)
{
return container_of(guc, struct drm_i915_private, guc);
@ -2411,8 +2151,10 @@ static inline struct drm_i915_private *huc_to_i915(struct intel_huc *huc)
/* Iterator over subset of engines selected by mask */
#define for_each_engine_masked(engine__, dev_priv__, mask__, tmp__) \
for (tmp__ = mask__ & INTEL_INFO(dev_priv__)->ring_mask; \
tmp__ ? (engine__ = (dev_priv__)->engine[__mask_next_bit(tmp__)]), 1 : 0; )
for ((tmp__) = (mask__) & INTEL_INFO(dev_priv__)->ring_mask; \
(tmp__) ? \
((engine__) = (dev_priv__)->engine[__mask_next_bit(tmp__)]), 1 : \
0;)
enum hdmi_force_audio {
HDMI_AUDIO_OFF_DVI = -2, /* no aux data for HDMI-DVI converter */
@ -2963,10 +2705,11 @@ extern void i915_driver_unload(struct drm_device *dev);
extern int intel_gpu_reset(struct drm_i915_private *dev_priv, u32 engine_mask);
extern bool intel_has_gpu_reset(struct drm_i915_private *dev_priv);
#define I915_RESET_QUIET BIT(0)
extern void i915_reset(struct drm_i915_private *i915, unsigned int flags);
extern void i915_reset(struct drm_i915_private *i915,
unsigned int stalled_mask,
const char *reason);
extern int i915_reset_engine(struct intel_engine_cs *engine,
unsigned int flags);
const char *reason);
extern bool intel_has_reset_engine(struct drm_i915_private *dev_priv);
extern int intel_reset_guc(struct drm_i915_private *dev_priv);
@ -3014,10 +2757,12 @@ static inline void i915_queue_hangcheck(struct drm_i915_private *dev_priv)
&dev_priv->gpu_error.hangcheck_work, delay);
}
__printf(3, 4)
__printf(4, 5)
void i915_handle_error(struct drm_i915_private *dev_priv,
u32 engine_mask,
unsigned long flags,
const char *fmt, ...);
#define I915_ERROR_CAPTURE BIT(0)
extern void intel_irq_init(struct drm_i915_private *dev_priv);
extern void intel_irq_fini(struct drm_i915_private *dev_priv);
@ -3132,8 +2877,8 @@ int i915_gem_get_aperture_ioctl(struct drm_device *dev, void *data,
int i915_gem_wait_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
void i915_gem_sanitize(struct drm_i915_private *i915);
int i915_gem_load_init(struct drm_i915_private *dev_priv);
void i915_gem_load_cleanup(struct drm_i915_private *dev_priv);
int i915_gem_init_early(struct drm_i915_private *dev_priv);
void i915_gem_cleanup_early(struct drm_i915_private *dev_priv);
void i915_gem_load_init_fences(struct drm_i915_private *dev_priv);
int i915_gem_freeze(struct drm_i915_private *dev_priv);
int i915_gem_freeze_late(struct drm_i915_private *dev_priv);
@ -3388,13 +3133,15 @@ static inline u32 i915_reset_engine_count(struct i915_gpu_error *error,
struct i915_request *
i915_gem_reset_prepare_engine(struct intel_engine_cs *engine);
int i915_gem_reset_prepare(struct drm_i915_private *dev_priv);
void i915_gem_reset(struct drm_i915_private *dev_priv);
void i915_gem_reset(struct drm_i915_private *dev_priv,
unsigned int stalled_mask);
void i915_gem_reset_finish_engine(struct intel_engine_cs *engine);
void i915_gem_reset_finish(struct drm_i915_private *dev_priv);
void i915_gem_set_wedged(struct drm_i915_private *dev_priv);
bool i915_gem_unset_wedged(struct drm_i915_private *dev_priv);
void i915_gem_reset_engine(struct intel_engine_cs *engine,
struct i915_request *request);
struct i915_request *request,
bool stalled);
void i915_gem_init_mmio(struct drm_i915_private *i915);
int __must_check i915_gem_init(struct drm_i915_private *dev_priv);
@ -3589,64 +3336,6 @@ static inline int i915_debugfs_connector_add(struct drm_connector *connector)
static inline void intel_display_crc_init(struct drm_i915_private *dev_priv) {}
#endif
/* i915_gpu_error.c */
#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
__printf(2, 3)
void i915_error_printf(struct drm_i915_error_state_buf *e, const char *f, ...);
int i915_error_state_to_str(struct drm_i915_error_state_buf *estr,
const struct i915_gpu_state *gpu);
int i915_error_state_buf_init(struct drm_i915_error_state_buf *eb,
struct drm_i915_private *i915,
size_t count, loff_t pos);
static inline void i915_error_state_buf_release(
struct drm_i915_error_state_buf *eb)
{
kfree(eb->buf);
}
struct i915_gpu_state *i915_capture_gpu_state(struct drm_i915_private *i915);
void i915_capture_error_state(struct drm_i915_private *dev_priv,
u32 engine_mask,
const char *error_msg);
static inline struct i915_gpu_state *
i915_gpu_state_get(struct i915_gpu_state *gpu)
{
kref_get(&gpu->ref);
return gpu;
}
void __i915_gpu_state_free(struct kref *kref);
static inline void i915_gpu_state_put(struct i915_gpu_state *gpu)
{
if (gpu)
kref_put(&gpu->ref, __i915_gpu_state_free);
}
struct i915_gpu_state *i915_first_error_state(struct drm_i915_private *i915);
void i915_reset_error_state(struct drm_i915_private *i915);
#else
static inline void i915_capture_error_state(struct drm_i915_private *dev_priv,
u32 engine_mask,
const char *error_msg)
{
}
static inline struct i915_gpu_state *
i915_first_error_state(struct drm_i915_private *i915)
{
return NULL;
}
static inline void i915_reset_error_state(struct drm_i915_private *i915)
{
}
#endif
const char *i915_cache_level_str(struct drm_i915_private *i915, int type);
/* i915_cmd_parser.c */

237
drivers/gpu/drm/i915/i915_gem.c
View File

@ -35,6 +35,7 @@
#include "intel_drv.h"
#include "intel_frontbuffer.h"
#include "intel_mocs.h"
#include "intel_workarounds.h"
#include "i915_gemfs.h"
#include <linux/dma-fence-array.h>
#include <linux/kthread.h>
@ -136,6 +137,100 @@ int i915_mutex_lock_interruptible(struct drm_device *dev)
return 0;
}
static u32 __i915_gem_park(struct drm_i915_private *i915)
{
lockdep_assert_held(&i915->drm.struct_mutex);
GEM_BUG_ON(i915->gt.active_requests);
if (!i915->gt.awake)
return I915_EPOCH_INVALID;
GEM_BUG_ON(i915->gt.epoch == I915_EPOCH_INVALID);
/*
* Be paranoid and flush a concurrent interrupt to make sure
* we don't reactivate any irq tasklets after parking.
*
* FIXME: Note that even though we have waited for execlists to be idle,
* there may still be an in-flight interrupt even though the CSB
* is now empty. synchronize_irq() makes sure that a residual interrupt
* is completed before we continue, but it doesn't prevent the HW from
* raising a spurious interrupt later. To complete the shield we should
* coordinate disabling the CS irq with flushing the interrupts.
*/
synchronize_irq(i915->drm.irq);
intel_engines_park(i915);
i915_gem_timelines_park(i915);
i915_pmu_gt_parked(i915);
i915->gt.awake = false;
if (INTEL_GEN(i915) >= 6)
gen6_rps_idle(i915);
intel_display_power_put(i915, POWER_DOMAIN_GT_IRQ);
intel_runtime_pm_put(i915);
return i915->gt.epoch;
}
void i915_gem_park(struct drm_i915_private *i915)
{
lockdep_assert_held(&i915->drm.struct_mutex);
GEM_BUG_ON(i915->gt.active_requests);
if (!i915->gt.awake)
return;
/* Defer the actual call to __i915_gem_park() to prevent ping-pongs */
mod_delayed_work(i915->wq, &i915->gt.idle_work, msecs_to_jiffies(100));
}
void i915_gem_unpark(struct drm_i915_private *i915)
{
lockdep_assert_held(&i915->drm.struct_mutex);
GEM_BUG_ON(!i915->gt.active_requests);
if (i915->gt.awake)
return;
intel_runtime_pm_get_noresume(i915);
/*
* It seems that the DMC likes to transition between the DC states a lot
* when there are no connected displays (no active power domains) during
* command submission.
*
* This activity has negative impact on the performance of the chip with
* huge latencies observed in the interrupt handler and elsewhere.
*
* Work around it by grabbing a GT IRQ power domain whilst there is any
* GT activity, preventing any DC state transitions.
*/
intel_display_power_get(i915, POWER_DOMAIN_GT_IRQ);
i915->gt.awake = true;
if (unlikely(++i915->gt.epoch == 0)) /* keep 0 as invalid */
i915->gt.epoch = 1;
intel_enable_gt_powersave(i915);
i915_update_gfx_val(i915);
if (INTEL_GEN(i915) >= 6)
gen6_rps_busy(i915);
i915_pmu_gt_unparked(i915);
intel_engines_unpark(i915);
i915_queue_hangcheck(i915);
queue_delayed_work(i915->wq,
&i915->gt.retire_work,
round_jiffies_up_relative(HZ));
}
int
i915_gem_get_aperture_ioctl(struct drm_device *dev, void *data,
struct drm_file *file)
@ -2896,20 +2991,6 @@ i915_gem_find_active_request(struct intel_engine_cs *engine)
return active;
}
static bool engine_stalled(struct intel_engine_cs *engine)
{
if (!engine->hangcheck.stalled)
return false;
/* Check for possible seqno movement after hang declaration */
if (engine->hangcheck.seqno != intel_engine_get_seqno(engine)) {
DRM_DEBUG_DRIVER("%s pardoned\n", engine->name);
return false;
}
return true;
}
/*
* Ensure irq handler finishes, and not run again.
* Also return the active request so that we only search for it once.
@ -2998,6 +3079,7 @@ int i915_gem_reset_prepare(struct drm_i915_private *dev_priv)
}
i915_gem_revoke_fences(dev_priv);
intel_uc_sanitize(dev_priv);
return err;
}
@ -3047,7 +3129,8 @@ static void engine_skip_context(struct i915_request *request)
/* Returns the request if it was guilty of the hang */
static struct i915_request *
i915_gem_reset_request(struct intel_engine_cs *engine,
struct i915_request *request)
struct i915_request *request,
bool stalled)
{
/* The guilty request will get skipped on a hung engine.
*
@ -3070,7 +3153,15 @@ i915_gem_reset_request(struct intel_engine_cs *engine,
* subsequent hangs.
*/
if (engine_stalled(engine)) {
if (i915_request_completed(request)) {
GEM_TRACE("%s pardoned global=%d (fence %llx:%d), current %d\n",
engine->name, request->global_seqno,
request->fence.context, request->fence.seqno,
intel_engine_get_seqno(engine));
stalled = false;
}
if (stalled) {
i915_gem_context_mark_guilty(request->ctx);
skip_request(request);
@ -3101,7 +3192,8 @@ i915_gem_reset_request(struct intel_engine_cs *engine,
}
void i915_gem_reset_engine(struct intel_engine_cs *engine,
struct i915_request *request)
struct i915_request *request,
bool stalled)
{
/*
* Make sure this write is visible before we re-enable the interrupt
@ -3111,7 +3203,7 @@ void i915_gem_reset_engine(struct intel_engine_cs *engine,
smp_store_mb(engine->irq_posted, 0);
if (request)
request = i915_gem_reset_request(engine, request);
request = i915_gem_reset_request(engine, request, stalled);
if (request) {
DRM_DEBUG_DRIVER("resetting %s to restart from tail of request 0x%x\n",
@ -3122,7 +3214,8 @@ void i915_gem_reset_engine(struct intel_engine_cs *engine,
engine->reset_hw(engine, request);
}
void i915_gem_reset(struct drm_i915_private *dev_priv)
void i915_gem_reset(struct drm_i915_private *dev_priv,
unsigned int stalled_mask)
{
struct intel_engine_cs *engine;
enum intel_engine_id id;
@ -3134,7 +3227,9 @@ void i915_gem_reset(struct drm_i915_private *dev_priv)
for_each_engine(engine, dev_priv, id) {
struct i915_gem_context *ctx;
i915_gem_reset_engine(engine, engine->hangcheck.active_request);
i915_gem_reset_engine(engine,
engine->hangcheck.active_request,
stalled_mask & ENGINE_MASK(id));
ctx = fetch_and_zero(&engine->last_retired_context);
if (ctx)
engine->context_unpin(engine, ctx);
@ -3160,13 +3255,6 @@ void i915_gem_reset(struct drm_i915_private *dev_priv)
}
i915_gem_restore_fences(dev_priv);
if (dev_priv->gt.awake) {
intel_sanitize_gt_powersave(dev_priv);
intel_enable_gt_powersave(dev_priv);
if (INTEL_GEN(dev_priv) >= 6)
gen6_rps_busy(dev_priv);
}
}
void i915_gem_reset_finish_engine(struct intel_engine_cs *engine)
@ -3192,6 +3280,9 @@ void i915_gem_reset_finish(struct drm_i915_private *dev_priv)
static void nop_submit_request(struct i915_request *request)
{
GEM_TRACE("%s fence %llx:%d -> -EIO\n",
request->engine->name,
request->fence.context, request->fence.seqno);
dma_fence_set_error(&request->fence, -EIO);
i915_request_submit(request);
@ -3201,6 +3292,9 @@ static void nop_complete_submit_request(struct i915_request *request)
{
unsigned long flags;
GEM_TRACE("%s fence %llx:%d -> -EIO\n",
request->engine->name,
request->fence.context, request->fence.seqno);
dma_fence_set_error(&request->fence, -EIO);
spin_lock_irqsave(&request->engine->timeline->lock, flags);
@ -3214,6 +3308,8 @@ void i915_gem_set_wedged(struct drm_i915_private *i915)
struct intel_engine_cs *engine;
enum intel_engine_id id;
GEM_TRACE("start\n");
if (drm_debug & DRM_UT_DRIVER) {
struct drm_printer p = drm_debug_printer(__func__);
@ -3237,6 +3333,9 @@ void i915_gem_set_wedged(struct drm_i915_private *i915)
}
i915->caps.scheduler = 0;
/* Even if the GPU reset fails, it should still stop the engines */
intel_gpu_reset(i915, ALL_ENGINES);
/*
* Make sure no one is running the old callback before we proceed with
* cancelling requests and resetting the completion tracking. Otherwise
@ -3278,6 +3377,8 @@ void i915_gem_set_wedged(struct drm_i915_private *i915)
i915_gem_reset_finish_engine(engine);
}
GEM_TRACE("end\n");
wake_up_all(&i915->gpu_error.reset_queue);
}
@ -3290,7 +3391,10 @@ bool i915_gem_unset_wedged(struct drm_i915_private *i915)
if (!test_bit(I915_WEDGED, &i915->gpu_error.flags))
return true;
/* Before unwedging, make sure that all pending operations
GEM_TRACE("start\n");
/*
* Before unwedging, make sure that all pending operations
* are flushed and errored out - we may have requests waiting upon
* third party fences. We marked all inflight requests as EIO, and
* every execbuf since returned EIO, for consistency we want all
@ -3308,7 +3412,8 @@ bool i915_gem_unset_wedged(struct drm_i915_private *i915)
if (!rq)
continue;
/* We can't use our normal waiter as we want to
/*
* We can't use our normal waiter as we want to
* avoid recursively trying to handle the current
* reset. The basic dma_fence_default_wait() installs
* a callback for dma_fence_signal(), which is
@ -3323,8 +3428,11 @@ bool i915_gem_unset_wedged(struct drm_i915_private *i915)
return false;
}
}
i915_retire_requests(i915);
GEM_BUG_ON(i915->gt.active_requests);
/* Undo nop_submit_request. We prevent all new i915 requests from
/*
* Undo nop_submit_request. We prevent all new i915 requests from
* being queued (by disallowing execbuf whilst wedged) so having
* waited for all active requests above, we know the system is idle
* and do not have to worry about a thread being inside
@ -3335,6 +3443,8 @@ bool i915_gem_unset_wedged(struct drm_i915_private *i915)
intel_engines_reset_default_submission(i915);
i915_gem_contexts_lost(i915);
GEM_TRACE("end\n");
smp_mb__before_atomic(); /* complete takeover before enabling execbuf */
clear_bit(I915_WEDGED, &i915->gpu_error.flags);
@ -3473,36 +3583,9 @@ i915_gem_idle_work_handler(struct work_struct *work)
if (new_requests_since_last_retire(dev_priv))
goto out_unlock;
/*
* Be paranoid and flush a concurrent interrupt to make sure
* we don't reactivate any irq tasklets after parking.
*
* FIXME: Note that even though we have waited for execlists to be idle,
* there may still be an in-flight interrupt even though the CSB
* is now empty. synchronize_irq() makes sure that a residual interrupt
* is completed before we continue, but it doesn't prevent the HW from
* raising a spurious interrupt later. To complete the shield we should
* coordinate disabling the CS irq with flushing the interrupts.
*/
synchronize_irq(dev_priv->drm.irq);
epoch = __i915_gem_park(dev_priv);
intel_engines_park(dev_priv);
i915_gem_timelines_park(dev_priv);
i915_pmu_gt_parked(dev_priv);
GEM_BUG_ON(!dev_priv->gt.awake);
dev_priv->gt.awake = false;
epoch = dev_priv->gt.epoch;
GEM_BUG_ON(epoch == I915_EPOCH_INVALID);
rearm_hangcheck = false;
if (INTEL_GEN(dev_priv) >= 6)
gen6_rps_idle(dev_priv);
intel_display_power_put(dev_priv, POWER_DOMAIN_GT_IRQ);
intel_runtime_pm_put(dev_priv);
out_unlock:
mutex_unlock(&dev_priv->drm.struct_mutex);
@ -3666,16 +3749,7 @@ static int wait_for_engines(struct drm_i915_private *i915)
if (wait_for(intel_engines_are_idle(i915), I915_IDLE_ENGINES_TIMEOUT)) {
dev_err(i915->drm.dev,
"Failed to idle engines, declaring wedged!\n");
if (drm_debug & DRM_UT_DRIVER) {
struct drm_printer p = drm_debug_printer(__func__);
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, i915, id)
intel_engine_dump(engine, &p,
"%s\n", engine->name);
}
GEM_TRACE_DUMP();
i915_gem_set_wedged(i915);
return -EIO;
}
@ -4088,9 +4162,10 @@ out:
}
/*
* Prepare buffer for display plane (scanout, cursors, etc).
* Can be called from an uninterruptible phase (modesetting) and allows
* any flushes to be pipelined (for pageflips).
* Prepare buffer for display plane (scanout, cursors, etc). Can be called from
* an uninterruptible phase (modesetting) and allows any flushes to be pipelined
* (for pageflips). We only flush the caches while preparing the buffer for
* display, the callers are responsible for frontbuffer flush.
*/
struct i915_vma *
i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
@ -4146,9 +4221,7 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
vma->display_alignment = max_t(u64, vma->display_alignment, alignment);
/* Treat this as an end-of-frame, like intel_user_framebuffer_dirty() */
__i915_gem_object_flush_for_display(obj);
intel_fb_obj_flush(obj, ORIGIN_DIRTYFB);
/* It should now be out of any other write domains, and we can update
* the domain values for our changes.
@ -4973,6 +5046,7 @@ int i915_gem_suspend(struct drm_i915_private *dev_priv)
* machines is a good idea, we don't - just in case it leaves the
* machine in an unusable condition.
*/
intel_uc_sanitize(dev_priv);
i915_gem_sanitize(dev_priv);
intel_runtime_pm_put(dev_priv);
@ -5118,6 +5192,8 @@ int i915_gem_init_hw(struct drm_i915_private *dev_priv)
}
}
intel_gt_workarounds_apply(dev_priv);
i915_gem_init_swizzling(dev_priv);
/*
@ -5140,6 +5216,12 @@ int i915_gem_init_hw(struct drm_i915_private *dev_priv)
goto out;
}
ret = intel_wopcm_init_hw(&dev_priv->wopcm);
if (ret) {
DRM_ERROR("Enabling WOPCM failed (%d)\n", ret);
goto out;
}
/* We can't enable contexts until all firmware is loaded */
ret = intel_uc_init_hw(dev_priv);
if (ret) {
@ -5297,6 +5379,10 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
if (ret)
return ret;
ret = intel_wopcm_init(&dev_priv->wopcm);
if (ret)
return ret;
ret = intel_uc_init_misc(dev_priv);
if (ret)
return ret;
@ -5478,8 +5564,7 @@ static void i915_gem_init__mm(struct drm_i915_private *i915)
INIT_WORK(&i915->mm.free_work, __i915_gem_free_work);
}
int
i915_gem_load_init(struct drm_i915_private *dev_priv)
int i915_gem_init_early(struct drm_i915_private *dev_priv)
{
int err = -ENOMEM;
@ -5554,7 +5639,7 @@ err_out:
return err;
}
void i915_gem_load_cleanup(struct drm_i915_private *dev_priv)
void i915_gem_cleanup_early(struct drm_i915_private *dev_priv)
{
i915_gem_drain_freed_objects(dev_priv);
GEM_BUG_ON(!llist_empty(&dev_priv->mm.free_list));

7
drivers/gpu/drm/i915/i915_gem.h
View File

@ -27,6 +27,8 @@
#include <linux/bug.h>
struct drm_i915_private;
#ifdef CONFIG_DRM_I915_DEBUG_GEM
#define GEM_BUG_ON(condition) do { if (unlikely((condition))) { \
pr_err("%s:%d GEM_BUG_ON(%s)\n", \
@ -53,10 +55,15 @@
#if IS_ENABLED(CONFIG_DRM_I915_TRACE_GEM)
#define GEM_TRACE(...) trace_printk(__VA_ARGS__)
#define GEM_TRACE_DUMP() ftrace_dump(DUMP_ALL)
#else
#define GEM_TRACE(...) do { } while (0)
#define GEM_TRACE_DUMP() do { } while (0)
#endif
#define I915_NUM_ENGINES 8
void i915_gem_park(struct drm_i915_private *i915);
void i915_gem_unpark(struct drm_i915_private *i915);
#endif /* __I915_GEM_H__ */

30
drivers/gpu/drm/i915/i915_gem_batch_pool.c
View File

@ -1,29 +1,11 @@
/*
* Copyright © 2014 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
* SPDX-License-Identifier: MIT
*
* Copyright © 2014-2018 Intel Corporation
*/
#include "i915_drv.h"
#include "i915_gem_batch_pool.h"
#include "i915_drv.h"
/**
* DOC: batch pool
@ -41,11 +23,11 @@
/**
* i915_gem_batch_pool_init() - initialize a batch buffer pool
* @engine: the associated request submission engine
* @pool: the batch buffer pool
* @engine: the associated request submission engine
*/
void i915_gem_batch_pool_init(struct intel_engine_cs *engine,
struct i915_gem_batch_pool *pool)
void i915_gem_batch_pool_init(struct i915_gem_batch_pool *pool,
struct intel_engine_cs *engine)
{
int n;

29
drivers/gpu/drm/i915/i915_gem_batch_pool.h
View File

@ -1,31 +1,13 @@
/*
* Copyright © 2014 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
* SPDX-License-Identifier: MIT
*
* Copyright © 2014-2018 Intel Corporation
*/
#ifndef I915_GEM_BATCH_POOL_H
#define I915_GEM_BATCH_POOL_H
#include "i915_drv.h"
#include <linux/types.h>
struct intel_engine_cs;
@ -34,9 +16,8 @@ struct i915_gem_batch_pool {
struct list_head cache_list[4];
};
/* i915_gem_batch_pool.c */
void i915_gem_batch_pool_init(struct intel_engine_cs *engine,
struct i915_gem_batch_pool *pool);
void i915_gem_batch_pool_init(struct i915_gem_batch_pool *pool,
struct intel_engine_cs *engine);
void i915_gem_batch_pool_fini(struct i915_gem_batch_pool *pool);
struct drm_i915_gem_object*
i915_gem_batch_pool_get(struct i915_gem_batch_pool *pool, size_t size);

11
drivers/gpu/drm/i915/i915_gem_context.c
View File

@ -90,6 +90,7 @@
#include <drm/i915_drm.h>
#include "i915_drv.h"
#include "i915_trace.h"
#include "intel_workarounds.h"
#define ALL_L3_SLICES(dev) (1 << NUM_L3_SLICES(dev)) - 1
@ -318,12 +319,13 @@ __create_hw_context(struct drm_i915_private *dev_priv,
ctx->desc_template =
default_desc_template(dev_priv, dev_priv->mm.aliasing_ppgtt);
/* GuC requires the ring to be placed above GUC_WOPCM_TOP. If GuC is not
/*
* GuC requires the ring to be placed in Non-WOPCM memory. If GuC is not
* present or not in use we still need a small bias as ring wraparound
* at offset 0 sometimes hangs. No idea why.
*/
if (USES_GUC(dev_priv))
ctx->ggtt_offset_bias = GUC_WOPCM_TOP;
ctx->ggtt_offset_bias = dev_priv->guc.ggtt_pin_bias;
else
ctx->ggtt_offset_bias = I915_GTT_PAGE_SIZE;
@ -458,11 +460,16 @@ static bool needs_preempt_context(struct drm_i915_private *i915)
int i915_gem_contexts_init(struct drm_i915_private *dev_priv)
{
struct i915_gem_context *ctx;
int ret;
/* Reassure ourselves we are only called once */
GEM_BUG_ON(dev_priv->kernel_context);
GEM_BUG_ON(dev_priv->preempt_context);
ret = intel_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
INIT_LIST_HEAD(&dev_priv->contexts.list);
INIT_WORK(&dev_priv->contexts.free_work, contexts_free_worker);
init_llist_head(&dev_priv->contexts.free_list);

29
drivers/gpu/drm/i915/i915_gem_execbuffer.c
View File

@ -81,6 +81,35 @@ enum {
* but this remains just a hint as the kernel may choose a new location for
* any object in the future.
*
* At the level of talking to the hardware, submitting a batchbuffer for the
* GPU to execute is to add content to a buffer from which the HW
* command streamer is reading.
*
* 1. Add a command to load the HW context. For Logical Ring Contexts, i.e.
* Execlists, this command is not placed on the same buffer as the
* remaining items.
*
* 2. Add a command to invalidate caches to the buffer.
*
* 3. Add a batchbuffer start command to the buffer; the start command is
* essentially a token together with the GPU address of the batchbuffer
* to be executed.
*
* 4. Add a pipeline flush to the buffer.
*
* 5. Add a memory write command to the buffer to record when the GPU
* is done executing the batchbuffer. The memory write writes the
* global sequence number of the request, ``i915_request::global_seqno``;
* the i915 driver uses the current value in the register to determine
* if the GPU has completed the batchbuffer.
*
* 6. Add a user interrupt command to the buffer. This command instructs
* the GPU to issue an interrupt when the command, pipeline flush and
* memory write are completed.
*
* 7. Inform the hardware of the additional commands added to the buffer
* (by updating the tail pointer).
*
* Processing an execbuf ioctl is conceptually split up into a few phases.
*
* 1. Validation - Ensure all the pointers, handles and flags are valid.

163
drivers/gpu/drm/i915/i915_gem_stolen.c
View File

@ -121,8 +121,8 @@ static int i915_adjust_stolen(struct drm_i915_private *dev_priv,
if (stolen[0].start != stolen[1].start ||
stolen[0].end != stolen[1].end) {
DRM_DEBUG_KMS("GTT within stolen memory at %pR\n", &ggtt_res);
DRM_DEBUG_KMS("Stolen memory adjusted to %pR\n", dsm);
DRM_DEBUG_DRIVER("GTT within stolen memory at %pR\n", &ggtt_res);
DRM_DEBUG_DRIVER("Stolen memory adjusted to %pR\n", dsm);
}
}
@ -174,18 +174,19 @@ void i915_gem_cleanup_stolen(struct drm_device *dev)
}
static void g4x_get_stolen_reserved(struct drm_i915_private *dev_priv,
resource_size_t *base, resource_size_t *size)
resource_size_t *base,
resource_size_t *size)
{
uint32_t reg_val = I915_READ(IS_GM45(dev_priv) ?
CTG_STOLEN_RESERVED :
ELK_STOLEN_RESERVED);
u32 reg_val = I915_READ(IS_GM45(dev_priv) ?
CTG_STOLEN_RESERVED :
ELK_STOLEN_RESERVED);
resource_size_t stolen_top = dev_priv->dsm.end + 1;
if ((reg_val & G4X_STOLEN_RESERVED_ENABLE) == 0) {
*base = 0;
*size = 0;
DRM_DEBUG_DRIVER("%s_STOLEN_RESERVED = %08x\n",
IS_GM45(dev_priv) ? "CTG" : "ELK", reg_val);
if ((reg_val & G4X_STOLEN_RESERVED_ENABLE) == 0)
return;
}
/*
* Whether ILK really reuses the ELK register for this is unclear.
@ -193,30 +194,25 @@ static void g4x_get_stolen_reserved(struct drm_i915_private *dev_priv,
*/
WARN(IS_GEN5(dev_priv), "ILK stolen reserved found? 0x%08x\n", reg_val);
*base = (reg_val & G4X_STOLEN_RESERVED_ADDR2_MASK) << 16;
if (!(reg_val & G4X_STOLEN_RESERVED_ADDR2_MASK))
return;
*base = (reg_val & G4X_STOLEN_RESERVED_ADDR2_MASK) << 16;
WARN_ON((reg_val & G4X_STOLEN_RESERVED_ADDR1_MASK) < *base);
/* On these platforms, the register doesn't have a size field, so the
* size is the distance between the base and the top of the stolen
* memory. We also have the genuine case where base is zero and there's
* nothing reserved. */
if (*base == 0)
*size = 0;
else
*size = stolen_top - *base;
*size = stolen_top - *base;
}
static void gen6_get_stolen_reserved(struct drm_i915_private *dev_priv,
resource_size_t *base, resource_size_t *size)
resource_size_t *base,
resource_size_t *size)
{
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
u32 reg_val = I915_READ(GEN6_STOLEN_RESERVED);
if ((reg_val & GEN6_STOLEN_RESERVED_ENABLE) == 0) {
*base = 0;
*size = 0;
DRM_DEBUG_DRIVER("GEN6_STOLEN_RESERVED = %08x\n", reg_val);
if (!(reg_val & GEN6_STOLEN_RESERVED_ENABLE))
return;
}
*base = reg_val & GEN6_STOLEN_RESERVED_ADDR_MASK;
@ -239,17 +235,44 @@ static void gen6_get_stolen_reserved(struct drm_i915_private *dev_priv,
}
}
static void gen7_get_stolen_reserved(struct drm_i915_private *dev_priv,
resource_size_t *base, resource_size_t *size)
static void vlv_get_stolen_reserved(struct drm_i915_private *dev_priv,
resource_size_t *base,
resource_size_t *size)
{
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
u32 reg_val = I915_READ(GEN6_STOLEN_RESERVED);
resource_size_t stolen_top = dev_priv->dsm.end + 1;
if ((reg_val & GEN6_STOLEN_RESERVED_ENABLE) == 0) {
*base = 0;
*size = 0;
DRM_DEBUG_DRIVER("GEN6_STOLEN_RESERVED = %08x\n", reg_val);
if (!(reg_val & GEN6_STOLEN_RESERVED_ENABLE))
return;
switch (reg_val & GEN7_STOLEN_RESERVED_SIZE_MASK) {
default:
MISSING_CASE(reg_val & GEN7_STOLEN_RESERVED_SIZE_MASK);
case GEN7_STOLEN_RESERVED_1M:
*size = 1024 * 1024;
break;
}
/*
* On vlv, the ADDR_MASK portion is left as 0 and HW deduces the
* reserved location as (top - size).
*/
*base = stolen_top - *size;
}
static void gen7_get_stolen_reserved(struct drm_i915_private *dev_priv,
resource_size_t *base,
resource_size_t *size)
{
u32 reg_val = I915_READ(GEN6_STOLEN_RESERVED);
DRM_DEBUG_DRIVER("GEN6_STOLEN_RESERVED = %08x\n", reg_val);
if (!(reg_val & GEN6_STOLEN_RESERVED_ENABLE))
return;
*base = reg_val & GEN7_STOLEN_RESERVED_ADDR_MASK;
switch (reg_val & GEN7_STOLEN_RESERVED_SIZE_MASK) {
@ -266,15 +289,15 @@ static void gen7_get_stolen_reserved(struct drm_i915_private *dev_priv,
}
static void chv_get_stolen_reserved(struct drm_i915_private *dev_priv,
resource_size_t *base, resource_size_t *size)
resource_size_t *base,
resource_size_t *size)
{
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
u32 reg_val = I915_READ(GEN6_STOLEN_RESERVED);
if ((reg_val & GEN6_STOLEN_RESERVED_ENABLE) == 0) {
*base = 0;
*size = 0;
DRM_DEBUG_DRIVER("GEN6_STOLEN_RESERVED = %08x\n", reg_val);
if (!(reg_val & GEN6_STOLEN_RESERVED_ENABLE))
return;
}
*base = reg_val & GEN6_STOLEN_RESERVED_ADDR_MASK;
@ -298,29 +321,22 @@ static void chv_get_stolen_reserved(struct drm_i915_private *dev_priv,
}
static void bdw_get_stolen_reserved(struct drm_i915_private *dev_priv,
resource_size_t *base, resource_size_t *size)
resource_size_t *base,
resource_size_t *size)
{
uint32_t reg_val = I915_READ(GEN6_STOLEN_RESERVED);
resource_size_t stolen_top;
u32 reg_val = I915_READ(GEN6_STOLEN_RESERVED);
resource_size_t stolen_top = dev_priv->dsm.end + 1;
if ((reg_val & GEN6_STOLEN_RESERVED_ENABLE) == 0) {
*base = 0;
*size = 0;
DRM_DEBUG_DRIVER("GEN6_STOLEN_RESERVED = %08x\n", reg_val);
if (!(reg_val & GEN6_STOLEN_RESERVED_ENABLE))
return;
}
stolen_top = dev_priv->dsm.end + 1;
if (!(reg_val & GEN6_STOLEN_RESERVED_ADDR_MASK))
return;
*base = reg_val & GEN6_STOLEN_RESERVED_ADDR_MASK;
/* On these platforms, the register doesn't have a size field, so the
* size is the distance between the base and the top of the stolen
* memory. We also have the genuine case where base is zero and there's
* nothing reserved. */
if (*base == 0)
*size = 0;
else
*size = stolen_top - *base;
*size = stolen_top - *base;
}
int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
@ -353,7 +369,7 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
GEM_BUG_ON(dev_priv->dsm.end <= dev_priv->dsm.start);
stolen_top = dev_priv->dsm.end + 1;
reserved_base = 0;
reserved_base = stolen_top;
reserved_size = 0;
switch (INTEL_GEN(dev_priv)) {
@ -373,8 +389,12 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
&reserved_base, &reserved_size);
break;
case 7:
gen7_get_stolen_reserved(dev_priv,
&reserved_base, &reserved_size);
if (IS_VALLEYVIEW(dev_priv))
vlv_get_stolen_reserved(dev_priv,
&reserved_base, &reserved_size);
else
gen7_get_stolen_reserved(dev_priv,
&reserved_base, &reserved_size);
break;
default:
if (IS_LP(dev_priv))
@ -386,11 +406,16 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
break;
}
/* It is possible for the reserved base to be zero, but the register
* field for size doesn't have a zero option. */
if (reserved_base == 0) {
reserved_size = 0;
/*
* Our expectation is that the reserved space is at the top of the
* stolen region and *never* at the bottom. If we see !reserved_base,
* it likely means we failed to read the registers correctly.
*/
if (!reserved_base) {
DRM_ERROR("inconsistent reservation %pa + %pa; ignoring\n",
&reserved_base, &reserved_size);
reserved_base = stolen_top;
reserved_size = 0;
}
dev_priv->dsm_reserved =
@ -406,9 +431,9 @@ int i915_gem_init_stolen(struct drm_i915_private *dev_priv)
* memory, so just consider the start. */
reserved_total = stolen_top - reserved_base;
DRM_DEBUG_KMS("Memory reserved for graphics device: %lluK, usable: %lluK\n",
(u64)resource_size(&dev_priv->dsm) >> 10,
((u64)resource_size(&dev_priv->dsm) - reserved_total) >> 10);
DRM_DEBUG_DRIVER("Memory reserved for graphics device: %lluK, usable: %lluK\n",
(u64)resource_size(&dev_priv->dsm) >> 10,
((u64)resource_size(&dev_priv->dsm) - reserved_total) >> 10);
stolen_usable_start = 0;
/* WaSkipStolenMemoryFirstPage:bdw+ */
@ -580,8 +605,8 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_i915_private *dev_priv
lockdep_assert_held(&dev_priv->drm.struct_mutex);
DRM_DEBUG_KMS("creating preallocated stolen object: stolen_offset=%pa, gtt_offset=%pa, size=%pa\n",
&stolen_offset, &gtt_offset, &size);
DRM_DEBUG_DRIVER("creating preallocated stolen object: stolen_offset=%pa, gtt_offset=%pa, size=%pa\n",
&stolen_offset, &gtt_offset, &size);
/* KISS and expect everything to be page-aligned */
if (WARN_ON(size == 0) ||
@ -599,14 +624,14 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_i915_private *dev_priv
ret = drm_mm_reserve_node(&dev_priv->mm.stolen, stolen);
mutex_unlock(&dev_priv->mm.stolen_lock);
if (ret) {
DRM_DEBUG_KMS("failed to allocate stolen space\n");
DRM_DEBUG_DRIVER("failed to allocate stolen space\n");
kfree(stolen);
return NULL;
}
obj = _i915_gem_object_create_stolen(dev_priv, stolen);
if (obj == NULL) {
DRM_DEBUG_KMS("failed to allocate stolen object\n");
DRM_DEBUG_DRIVER("failed to allocate stolen object\n");
i915_gem_stolen_remove_node(dev_priv, stolen);
kfree(stolen);
return NULL;
@ -635,7 +660,7 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_i915_private *dev_priv
size, gtt_offset, obj->cache_level,
0);
if (ret) {
DRM_DEBUG_KMS("failed to allocate stolen GTT space\n");
DRM_DEBUG_DRIVER("failed to allocate stolen GTT space\n");
goto err_pages;
}

1
drivers/gpu/drm/i915/i915_gpu_error.c
View File

@ -32,6 +32,7 @@
#include <linux/zlib.h>
#include <drm/drm_print.h>
#include "i915_gpu_error.h"
#include "i915_drv.h"
static inline const struct intel_engine_cs *

362
drivers/gpu/drm/i915/i915_gpu_error.h 100644
View File

@ -0,0 +1,362 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright <EFBFBD> 2008-2018 Intel Corporation
*/
#ifndef _I915_GPU_ERROR_H_
#define _I915_GPU_ERROR_H_
#include <linux/kref.h>
#include <linux/ktime.h>
#include <linux/sched.h>
#include <drm/drm_mm.h>
#include "intel_device_info.h"
#include "intel_ringbuffer.h"
#include "intel_uc_fw.h"
#include "i915_gem.h"
#include "i915_gem_gtt.h"
#include "i915_params.h"
struct drm_i915_private;
struct intel_overlay_error_state;
struct intel_display_error_state;
struct i915_gpu_state {
struct kref ref;
ktime_t time;
ktime_t boottime;
ktime_t uptime;
struct drm_i915_private *i915;
char error_msg[128];
bool simulated;
bool awake;
bool wakelock;
bool suspended;
int iommu;
u32 reset_count;
u32 suspend_count;
struct intel_device_info device_info;
struct intel_driver_caps driver_caps;
struct i915_params params;
struct i915_error_uc {
struct intel_uc_fw guc_fw;
struct intel_uc_fw huc_fw;
struct drm_i915_error_object *guc_log;
} uc;
/* Generic register state */
u32 eir;
u32 pgtbl_er;
u32 ier;
u32 gtier[4], ngtier;
u32 ccid;
u32 derrmr;
u32 forcewake;
u32 error; /* gen6+ */
u32 err_int; /* gen7 */
u32 fault_data0; /* gen8, gen9 */
u32 fault_data1; /* gen8, gen9 */
u32 done_reg;
u32 gac_eco;
u32 gam_ecochk;
u32 gab_ctl;
u32 gfx_mode;
u32 nfence;
u64 fence[I915_MAX_NUM_FENCES];
struct intel_overlay_error_state *overlay;
struct intel_display_error_state *display;
struct drm_i915_error_engine {
int engine_id;
/* Software tracked state */
bool idle;
bool waiting;
int num_waiters;
unsigned long hangcheck_timestamp;
bool hangcheck_stalled;
enum intel_engine_hangcheck_action hangcheck_action;
struct i915_address_space *vm;
int num_requests;
u32 reset_count;
/* position of active request inside the ring */
u32 rq_head, rq_post, rq_tail;
/* our own tracking of ring head and tail */
u32 cpu_ring_head;
u32 cpu_ring_tail;
u32 last_seqno;
/* Register state */
u32 start;
u32 tail;
u32 head;
u32 ctl;
u32 mode;
u32 hws;
u32 ipeir;
u32 ipehr;
u32 bbstate;
u32 instpm;
u32 instps;
u32 seqno;
u64 bbaddr;
u64 acthd;
u32 fault_reg;
u64 faddr;
u32 rc_psmi; /* sleep state */
u32 semaphore_mboxes[I915_NUM_ENGINES - 1];
struct intel_instdone instdone;
struct drm_i915_error_context {
char comm[TASK_COMM_LEN];
pid_t pid;
u32 handle;
u32 hw_id;
int priority;
int ban_score;
int active;
int guilty;
bool bannable;
} context;
struct drm_i915_error_object {
u64 gtt_offset;
u64 gtt_size;
int page_count;
int unused;
u32 *pages[0];
} *ringbuffer, *batchbuffer, *wa_batchbuffer, *ctx, *hws_page;
struct drm_i915_error_object **user_bo;
long user_bo_count;
struct drm_i915_error_object *wa_ctx;
struct drm_i915_error_object *default_state;
struct drm_i915_error_request {
long jiffies;
pid_t pid;
u32 context;
int priority;
int ban_score;
u32 seqno;
u32 head;
u32 tail;
} *requests, execlist[EXECLIST_MAX_PORTS];
unsigned int num_ports;
struct drm_i915_error_waiter {
char comm[TASK_COMM_LEN];
pid_t pid;
u32 seqno;
} *waiters;
struct {
u32 gfx_mode;
union {
u64 pdp[4];
u32 pp_dir_base;
};
} vm_info;
} engine[I915_NUM_ENGINES];
struct drm_i915_error_buffer {
u32 size;
u32 name;
u32 rseqno[I915_NUM_ENGINES], wseqno;
u64 gtt_offset;
u32 read_domains;
u32 write_domain;
s32 fence_reg:I915_MAX_NUM_FENCE_BITS;
u32 tiling:2;
u32 dirty:1;
u32 purgeable:1;
u32 userptr:1;
s32 engine:4;
u32 cache_level:3;
} *active_bo[I915_NUM_ENGINES], *pinned_bo;
u32 active_bo_count[I915_NUM_ENGINES], pinned_bo_count;
struct i915_address_space *active_vm[I915_NUM_ENGINES];
};
struct i915_gpu_error {
/* For hangcheck timer */
#define DRM_I915_HANGCHECK_PERIOD 1500 /* in ms */
#define DRM_I915_HANGCHECK_JIFFIES msecs_to_jiffies(DRM_I915_HANGCHECK_PERIOD)
struct delayed_work hangcheck_work;
/* For reset and error_state handling. */
spinlock_t lock;
/* Protected by the above dev->gpu_error.lock. */
struct i915_gpu_state *first_error;
atomic_t pending_fb_pin;
unsigned long missed_irq_rings;
/**
* State variable controlling the reset flow and count
*
* This is a counter which gets incremented when reset is triggered,
*
* Before the reset commences, the I915_RESET_BACKOFF bit is set
* meaning that any waiters holding onto the struct_mutex should
* relinquish the lock immediately in order for the reset to start.
*
* If reset is not completed successfully, the I915_WEDGE bit is
* set meaning that hardware is terminally sour and there is no
* recovery. All waiters on the reset_queue will be woken when
* that happens.
*
* This counter is used by the wait_seqno code to notice that reset
* event happened and it needs to restart the entire ioctl (since most
* likely the seqno it waited for won't ever signal anytime soon).
*
* This is important for lock-free wait paths, where no contended lock
* naturally enforces the correct ordering between the bail-out of the
* waiter and the gpu reset work code.
*/
unsigned long reset_count;
/**
* flags: Control various stages of the GPU reset
*
* #I915_RESET_BACKOFF - When we start a reset, we want to stop any
* other users acquiring the struct_mutex. To do this we set the
* #I915_RESET_BACKOFF bit in the error flags when we detect a reset
* and then check for that bit before acquiring the struct_mutex (in
* i915_mutex_lock_interruptible()?). I915_RESET_BACKOFF serves a
* secondary role in preventing two concurrent global reset attempts.
*
* #I915_RESET_HANDOFF - To perform the actual GPU reset, we need the
* struct_mutex. We try to acquire the struct_mutex in the reset worker,
* but it may be held by some long running waiter (that we cannot
* interrupt without causing trouble). Once we are ready to do the GPU
* reset, we set the I915_RESET_HANDOFF bit and wakeup any waiters. If
* they already hold the struct_mutex and want to participate they can
* inspect the bit and do the reset directly, otherwise the worker
* waits for the struct_mutex.
*
* #I915_RESET_ENGINE[num_engines] - Since the driver doesn't need to
* acquire the struct_mutex to reset an engine, we need an explicit
* flag to prevent two concurrent reset attempts in the same engine.
* As the number of engines continues to grow, allocate the flags from
* the most significant bits.
*
* #I915_WEDGED - If reset fails and we can no longer use the GPU,
* we set the #I915_WEDGED bit. Prior to command submission, e.g.
* i915_request_alloc(), this bit is checked and the sequence
* aborted (with -EIO reported to userspace) if set.
*/
unsigned long flags;
#define I915_RESET_BACKOFF 0
#define I915_RESET_HANDOFF 1
#define I915_RESET_MODESET 2
#define I915_WEDGED (BITS_PER_LONG - 1)
#define I915_RESET_ENGINE (I915_WEDGED - I915_NUM_ENGINES)
/** Number of times an engine has been reset */
u32 reset_engine_count[I915_NUM_ENGINES];
/** Set of stalled engines with guilty requests, in the current reset */
u32 stalled_mask;
/** Reason for the current *global* reset */
const char *reason;
/**
* Waitqueue to signal when a hang is detected. Used to for waiters
* to release the struct_mutex for the reset to procede.
*/
wait_queue_head_t wait_queue;
/**
* Waitqueue to signal when the reset has completed. Used by clients
* that wait for dev_priv->mm.wedged to settle.
*/
wait_queue_head_t reset_queue;
/* For missed irq/seqno simulation. */
unsigned long test_irq_rings;
};
struct drm_i915_error_state_buf {
struct drm_i915_private *i915;
unsigned int bytes;
unsigned int size;
int err;
u8 *buf;
loff_t start;
loff_t pos;
};
#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
__printf(2, 3)
void i915_error_printf(struct drm_i915_error_state_buf *e, const char *f, ...);
int i915_error_state_to_str(struct drm_i915_error_state_buf *estr,
const struct i915_gpu_state *gpu);
int i915_error_state_buf_init(struct drm_i915_error_state_buf *eb,
struct drm_i915_private *i915,
size_t count, loff_t pos);
static inline void
i915_error_state_buf_release(struct drm_i915_error_state_buf *eb)
{
kfree(eb->buf);
}
struct i915_gpu_state *i915_capture_gpu_state(struct drm_i915_private *i915);
void i915_capture_error_state(struct drm_i915_private *dev_priv,
u32 engine_mask,
const char *error_msg);
static inline struct i915_gpu_state *
i915_gpu_state_get(struct i915_gpu_state *gpu)
{
kref_get(&gpu->ref);
return gpu;
}
void __i915_gpu_state_free(struct kref *kref);
static inline void i915_gpu_state_put(struct i915_gpu_state *gpu)
{
if (gpu)
kref_put(&gpu->ref, __i915_gpu_state_free);
}
struct i915_gpu_state *i915_first_error_state(struct drm_i915_private *i915);
void i915_reset_error_state(struct drm_i915_private *i915);
#else
static inline void i915_capture_error_state(struct drm_i915_private *dev_priv,
u32 engine_mask,
const char *error_msg)
{
}
static inline struct i915_gpu_state *
i915_first_error_state(struct drm_i915_private *i915)
{
return NULL;
}
static inline void i915_reset_error_state(struct drm_i915_private *i915)
{
}
#endif /* IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR) */
#endif /* _I915_GPU_ERROR_H_ */

374
drivers/gpu/drm/i915/i915_irq.c
View File

@ -243,6 +243,41 @@ void i915_hotplug_interrupt_update(struct drm_i915_private *dev_priv,
spin_unlock_irq(&dev_priv->irq_lock);
}
static u32
gen11_gt_engine_identity(struct drm_i915_private * const i915,
const unsigned int bank, const unsigned int bit);
static bool gen11_reset_one_iir(struct drm_i915_private * const i915,
const unsigned int bank,
const unsigned int bit)
{
void __iomem * const regs = i915->regs;
u32 dw;
lockdep_assert_held(&i915->irq_lock);
dw = raw_reg_read(regs, GEN11_GT_INTR_DW(bank));
if (dw & BIT(bit)) {
/*
* According to the BSpec, DW_IIR bits cannot be cleared without
* first servicing the Selector & Shared IIR registers.
*/
gen11_gt_engine_identity(i915, bank, bit);
/*
* We locked GT INT DW by reading it. If we want to (try
* to) recover from this succesfully, we need to clear
* our bit, otherwise we are locking the register for
* everybody.
*/
raw_reg_write(regs, GEN11_GT_INTR_DW(bank), BIT(bit));
return true;
}
return false;
}
/**
* ilk_update_display_irq - update DEIMR
* @dev_priv: driver private
@ -308,17 +343,29 @@ void gen5_disable_gt_irq(struct drm_i915_private *dev_priv, uint32_t mask)
static i915_reg_t gen6_pm_iir(struct drm_i915_private *dev_priv)
{
WARN_ON_ONCE(INTEL_GEN(dev_priv) >= 11);
return INTEL_GEN(dev_priv) >= 8 ? GEN8_GT_IIR(2) : GEN6_PMIIR;
}
static i915_reg_t gen6_pm_imr(struct drm_i915_private *dev_priv)
{
return INTEL_GEN(dev_priv) >= 8 ? GEN8_GT_IMR(2) : GEN6_PMIMR;
if (INTEL_GEN(dev_priv) >= 11)
return GEN11_GPM_WGBOXPERF_INTR_MASK;
else if (INTEL_GEN(dev_priv) >= 8)
return GEN8_GT_IMR(2);
else
return GEN6_PMIMR;
}
static i915_reg_t gen6_pm_ier(struct drm_i915_private *dev_priv)
{
return INTEL_GEN(dev_priv) >= 8 ? GEN8_GT_IER(2) : GEN6_PMIER;
if (INTEL_GEN(dev_priv) >= 11)
return GEN11_GPM_WGBOXPERF_INTR_ENABLE;
else if (INTEL_GEN(dev_priv) >= 8)
return GEN8_GT_IER(2);
else
return GEN6_PMIER;
}
/**
@ -400,6 +447,18 @@ static void gen6_disable_pm_irq(struct drm_i915_private *dev_priv, u32 disable_m
/* though a barrier is missing here, but don't really need a one */
}
void gen11_reset_rps_interrupts(struct drm_i915_private *dev_priv)
{
spin_lock_irq(&dev_priv->irq_lock);
while (gen11_reset_one_iir(dev_priv, 0, GEN11_GTPM))
;
dev_priv->gt_pm.rps.pm_iir = 0;
spin_unlock_irq(&dev_priv->irq_lock);
}
void gen6_reset_rps_interrupts(struct drm_i915_private *dev_priv)
{
spin_lock_irq(&dev_priv->irq_lock);
@ -415,12 +474,14 @@ void gen6_enable_rps_interrupts(struct drm_i915_private *dev_priv)
if (READ_ONCE(rps->interrupts_enabled))
return;
if (WARN_ON_ONCE(IS_GEN11(dev_priv)))
return;
spin_lock_irq(&dev_priv->irq_lock);
WARN_ON_ONCE(rps->pm_iir);
WARN_ON_ONCE(I915_READ(gen6_pm_iir(dev_priv)) & dev_priv->pm_rps_events);
if (INTEL_GEN(dev_priv) >= 11)
WARN_ON_ONCE(gen11_reset_one_iir(dev_priv, 0, GEN11_GTPM));
else
WARN_ON_ONCE(I915_READ(gen6_pm_iir(dev_priv)) & dev_priv->pm_rps_events);
rps->interrupts_enabled = true;
gen6_enable_pm_irq(dev_priv, dev_priv->pm_rps_events);
@ -434,9 +495,6 @@ void gen6_disable_rps_interrupts(struct drm_i915_private *dev_priv)
if (!READ_ONCE(rps->interrupts_enabled))
return;
if (WARN_ON_ONCE(IS_GEN11(dev_priv)))
return;
spin_lock_irq(&dev_priv->irq_lock);
rps->interrupts_enabled = false;
@ -453,7 +511,10 @@ void gen6_disable_rps_interrupts(struct drm_i915_private *dev_priv)
* state of the worker can be discarded.
*/
cancel_work_sync(&rps->work);
gen6_reset_rps_interrupts(dev_priv);
if (INTEL_GEN(dev_priv) >= 11)
gen11_reset_rps_interrupts(dev_priv);
else
gen6_reset_rps_interrupts(dev_priv);
}
void gen9_reset_guc_interrupts(struct drm_i915_private *dev_priv)
@ -1399,19 +1460,18 @@ static void snb_gt_irq_handler(struct drm_i915_private *dev_priv,
}
static void
gen8_cs_irq_handler(struct intel_engine_cs *engine, u32 iir, int test_shift)
gen8_cs_irq_handler(struct intel_engine_cs *engine, u32 iir)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
bool tasklet = false;
if (iir & (GT_CONTEXT_SWITCH_INTERRUPT << test_shift)) {
if (READ_ONCE(engine->execlists.active)) {
__set_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
tasklet = true;
}
if (iir & GT_CONTEXT_SWITCH_INTERRUPT) {
if (READ_ONCE(engine->execlists.active))
tasklet = !test_and_set_bit(ENGINE_IRQ_EXECLIST,
&engine->irq_posted);
}
if (iir & (GT_RENDER_USER_INTERRUPT << test_shift)) {
if (iir & GT_RENDER_USER_INTERRUPT) {
notify_ring(engine);
tasklet |= USES_GUC_SUBMISSION(engine->i915);
}
@ -1466,21 +1526,21 @@ static void gen8_gt_irq_handler(struct drm_i915_private *i915,
{
if (master_ctl & (GEN8_GT_RCS_IRQ | GEN8_GT_BCS_IRQ)) {
gen8_cs_irq_handler(i915->engine[RCS],
gt_iir[0], GEN8_RCS_IRQ_SHIFT);
gt_iir[0] >> GEN8_RCS_IRQ_SHIFT);
gen8_cs_irq_handler(i915->engine[BCS],
gt_iir[0], GEN8_BCS_IRQ_SHIFT);
gt_iir[0] >> GEN8_BCS_IRQ_SHIFT);
}
if (master_ctl & (GEN8_GT_VCS1_IRQ | GEN8_GT_VCS2_IRQ)) {
gen8_cs_irq_handler(i915->engine[VCS],
gt_iir[1], GEN8_VCS1_IRQ_SHIFT);
gt_iir[1] >> GEN8_VCS1_IRQ_SHIFT);
gen8_cs_irq_handler(i915->engine[VCS2],
gt_iir[1], GEN8_VCS2_IRQ_SHIFT);
gt_iir[1] >> GEN8_VCS2_IRQ_SHIFT);
}
if (master_ctl & GEN8_GT_VECS_IRQ) {
gen8_cs_irq_handler(i915->engine[VECS],
gt_iir[3], GEN8_VECS_IRQ_SHIFT);
gt_iir[3] >> GEN8_VECS_IRQ_SHIFT);
}
if (master_ctl & (GEN8_GT_PM_IRQ | GEN8_GT_GUC_IRQ)) {
@ -1627,7 +1687,7 @@ static void display_pipe_crc_irq_handler(struct drm_i915_private *dev_priv,
int head, tail;
spin_lock(&pipe_crc->lock);
if (pipe_crc->source) {
if (pipe_crc->source && !crtc->base.crc.opened) {
if (!pipe_crc->entries) {
spin_unlock(&pipe_crc->lock);
DRM_DEBUG_KMS("spurious interrupt\n");
@ -1667,7 +1727,7 @@ static void display_pipe_crc_irq_handler(struct drm_i915_private *dev_priv,
* On GEN8+ sometimes the second CRC is bonkers as well, so
* don't trust that one either.
*/
if (pipe_crc->skipped == 0 ||
if (pipe_crc->skipped <= 0 ||
(INTEL_GEN(dev_priv) >= 8 && pipe_crc->skipped == 1)) {
pipe_crc->skipped++;
spin_unlock(&pipe_crc->lock);
@ -1766,37 +1826,8 @@ static void gen6_rps_irq_handler(struct drm_i915_private *dev_priv, u32 pm_iir)
static void gen9_guc_irq_handler(struct drm_i915_private *dev_priv, u32 gt_iir)
{
if (gt_iir & GEN9_GUC_TO_HOST_INT_EVENT) {
/* Sample the log buffer flush related bits & clear them out now
* itself from the message identity register to minimize the
* probability of losing a flush interrupt, when there are back
* to back flush interrupts.
* There can be a new flush interrupt, for different log buffer
* type (like for ISR), whilst Host is handling one (for DPC).
* Since same bit is used in message register for ISR & DPC, it
* could happen that GuC sets the bit for 2nd interrupt but Host
* clears out the bit on handling the 1st interrupt.
*/
u32 msg, flush;
msg = I915_READ(SOFT_SCRATCH(15));
flush = msg & (INTEL_GUC_RECV_MSG_CRASH_DUMP_POSTED |
INTEL_GUC_RECV_MSG_FLUSH_LOG_BUFFER);
if (flush) {
/* Clear the message bits that are handled */
I915_WRITE(SOFT_SCRATCH(15), msg & ~flush);
/* Handle flush interrupt in bottom half */
queue_work(dev_priv->guc.log.runtime.flush_wq,
&dev_priv->guc.log.runtime.flush_work);
dev_priv->guc.log.flush_interrupt_count++;
} else {
/* Not clearing of unhandled event bits won't result in
* re-triggering of the interrupt.
*/
}
}
if (gt_iir & GEN9_GUC_TO_HOST_INT_EVENT)
intel_guc_to_host_event_handler(&dev_priv->guc);
}
static void i9xx_pipestat_irq_reset(struct drm_i915_private *dev_priv)
@ -2762,58 +2793,16 @@ static void __fini_wedge(struct wedge_me *w)
(W)->i915; \
__fini_wedge((W)))
static __always_inline void
gen11_cs_irq_handler(struct intel_engine_cs * const engine, const u32 iir)
{
gen8_cs_irq_handler(engine, iir, 0);
}
static void
gen11_gt_engine_irq_handler(struct drm_i915_private * const i915,
const unsigned int bank,
const unsigned int engine_n,
const u16 iir)
{
struct intel_engine_cs ** const engine = i915->engine;
switch (bank) {
case 0:
switch (engine_n) {
case GEN11_RCS0:
return gen11_cs_irq_handler(engine[RCS], iir);
case GEN11_BCS:
return gen11_cs_irq_handler(engine[BCS], iir);
}
case 1:
switch (engine_n) {
case GEN11_VCS(0):
return gen11_cs_irq_handler(engine[_VCS(0)], iir);
case GEN11_VCS(1):
return gen11_cs_irq_handler(engine[_VCS(1)], iir);
case GEN11_VCS(2):
return gen11_cs_irq_handler(engine[_VCS(2)], iir);
case GEN11_VCS(3):
return gen11_cs_irq_handler(engine[_VCS(3)], iir);
case GEN11_VECS(0):
return gen11_cs_irq_handler(engine[_VECS(0)], iir);
case GEN11_VECS(1):
return gen11_cs_irq_handler(engine[_VECS(1)], iir);
}
}
}
static u32
gen11_gt_engine_intr(struct drm_i915_private * const i915,
const unsigned int bank, const unsigned int bit)
gen11_gt_engine_identity(struct drm_i915_private * const i915,
const unsigned int bank, const unsigned int bit)
{
void __iomem * const regs = i915->regs;
u32 timeout_ts;
u32 ident;
lockdep_assert_held(&i915->irq_lock);
raw_reg_write(regs, GEN11_IIR_REG_SELECTOR(bank), BIT(bit));
/*
@ -2835,42 +2824,101 @@ gen11_gt_engine_intr(struct drm_i915_private * const i915,
raw_reg_write(regs, GEN11_INTR_IDENTITY_REG(bank),
GEN11_INTR_DATA_VALID);
return ident & GEN11_INTR_ENGINE_MASK;
return ident;
}
static void
gen11_other_irq_handler(struct drm_i915_private * const i915,
const u8 instance, const u16 iir)
{
if (instance == OTHER_GTPM_INSTANCE)
return gen6_rps_irq_handler(i915, iir);
WARN_ONCE(1, "unhandled other interrupt instance=0x%x, iir=0x%x\n",
instance, iir);
}
static void
gen11_engine_irq_handler(struct drm_i915_private * const i915,
const u8 class, const u8 instance, const u16 iir)
{
struct intel_engine_cs *engine;
if (instance <= MAX_ENGINE_INSTANCE)
engine = i915->engine_class[class][instance];
else
engine = NULL;
if (likely(engine))
return gen8_cs_irq_handler(engine, iir);
WARN_ONCE(1, "unhandled engine interrupt class=0x%x, instance=0x%x\n",
class, instance);
}
static void
gen11_gt_identity_handler(struct drm_i915_private * const i915,
const u32 identity)
{
const u8 class = GEN11_INTR_ENGINE_CLASS(identity);
const u8 instance = GEN11_INTR_ENGINE_INSTANCE(identity);
const u16 intr = GEN11_INTR_ENGINE_INTR(identity);
if (unlikely(!intr))
return;
if (class <= COPY_ENGINE_CLASS)
return gen11_engine_irq_handler(i915, class, instance, intr);
if (class == OTHER_CLASS)
return gen11_other_irq_handler(i915, instance, intr);
WARN_ONCE(1, "unknown interrupt class=0x%x, instance=0x%x, intr=0x%x\n",
class, instance, intr);
}
static void
gen11_gt_bank_handler(struct drm_i915_private * const i915,
const unsigned int bank)
{
void __iomem * const regs = i915->regs;
unsigned long intr_dw;
unsigned int bit;
lockdep_assert_held(&i915->irq_lock);
intr_dw = raw_reg_read(regs, GEN11_GT_INTR_DW(bank));
if (unlikely(!intr_dw)) {
DRM_ERROR("GT_INTR_DW%u blank!\n", bank);
return;
}
for_each_set_bit(bit, &intr_dw, 32) {
const u32 ident = gen11_gt_engine_identity(i915,
bank, bit);
gen11_gt_identity_handler(i915, ident);
}
/* Clear must be after shared has been served for engine */
raw_reg_write(regs, GEN11_GT_INTR_DW(bank), intr_dw);
}
static void
gen11_gt_irq_handler(struct drm_i915_private * const i915,
const u32 master_ctl)
{
void __iomem * const regs = i915->regs;
unsigned int bank;
spin_lock(&i915->irq_lock);
for (bank = 0; bank < 2; bank++) {
unsigned long intr_dw;
unsigned int bit;
if (!(master_ctl & GEN11_GT_DW_IRQ(bank)))
continue;
intr_dw = raw_reg_read(regs, GEN11_GT_INTR_DW(bank));
if (unlikely(!intr_dw)) {
DRM_ERROR("GT_INTR_DW%u blank!\n", bank);
continue;
}
for_each_set_bit(bit, &intr_dw, 32) {
const u16 iir = gen11_gt_engine_intr(i915, bank, bit);
if (unlikely(!iir))
continue;
gen11_gt_engine_irq_handler(i915, bank, bit, iir);
}
/* Clear must be after shared has been served for engine */
raw_reg_write(regs, GEN11_GT_INTR_DW(bank), intr_dw);
if (master_ctl & GEN11_GT_DW_IRQ(bank))
gen11_gt_bank_handler(i915, bank);
}
spin_unlock(&i915->irq_lock);
}
static irqreturn_t gen11_irq_handler(int irq, void *arg)
@ -2912,15 +2960,11 @@ static irqreturn_t gen11_irq_handler(int irq, void *arg)
return IRQ_HANDLED;
}
/**
* i915_reset_device - do process context error handling work
* @dev_priv: i915 device private
*
* Fire an error uevent so userspace can see that a hang or error
* was detected.
*/
static void i915_reset_device(struct drm_i915_private *dev_priv)
static void i915_reset_device(struct drm_i915_private *dev_priv,
u32 engine_mask,
const char *reason)
{
struct i915_gpu_error *error = &dev_priv->gpu_error;
struct kobject *kobj = &dev_priv->drm.primary->kdev->kobj;
char *error_event[] = { I915_ERROR_UEVENT "=1", NULL };
char *reset_event[] = { I915_RESET_UEVENT "=1", NULL };
@ -2936,29 +2980,35 @@ static void i915_reset_device(struct drm_i915_private *dev_priv)
i915_wedge_on_timeout(&w, dev_priv, 5*HZ) {
intel_prepare_reset(dev_priv);
error->reason = reason;
error->stalled_mask = engine_mask;
/* Signal that locked waiters should reset the GPU */
set_bit(I915_RESET_HANDOFF, &dev_priv->gpu_error.flags);
wake_up_all(&dev_priv->gpu_error.wait_queue);
smp_mb__before_atomic();
set_bit(I915_RESET_HANDOFF, &error->flags);
wake_up_all(&error->wait_queue);
/* Wait for anyone holding the lock to wakeup, without
* blocking indefinitely on struct_mutex.
*/
do {
if (mutex_trylock(&dev_priv->drm.struct_mutex)) {
i915_reset(dev_priv, 0);
i915_reset(dev_priv, engine_mask, reason);
mutex_unlock(&dev_priv->drm.struct_mutex);
}
} while (wait_on_bit_timeout(&dev_priv->gpu_error.flags,
} while (wait_on_bit_timeout(&error->flags,
I915_RESET_HANDOFF,
TASK_UNINTERRUPTIBLE,
1));
error->stalled_mask = 0;
error->reason = NULL;
intel_finish_reset(dev_priv);
}
if (!test_bit(I915_WEDGED, &dev_priv->gpu_error.flags))
kobject_uevent_env(kobj,
KOBJ_CHANGE, reset_done_event);
if (!test_bit(I915_WEDGED, &error->flags))
kobject_uevent_env(kobj, KOBJ_CHANGE, reset_done_event);
}
static void i915_clear_error_registers(struct drm_i915_private *dev_priv)
@ -2990,6 +3040,7 @@ static void i915_clear_error_registers(struct drm_i915_private *dev_priv)
* i915_handle_error - handle a gpu error
* @dev_priv: i915 device private
* @engine_mask: mask representing engines that are hung
* @flags: control flags
* @fmt: Error message format string
*
* Do some basic checking of register state at error time and
@ -3000,16 +3051,23 @@ static void i915_clear_error_registers(struct drm_i915_private *dev_priv)
*/
void i915_handle_error(struct drm_i915_private *dev_priv,
u32 engine_mask,
unsigned long flags,
const char *fmt, ...)
{
struct intel_engine_cs *engine;
unsigned int tmp;
va_list args;
char error_msg[80];
char *msg = NULL;
va_start(args, fmt);
vscnprintf(error_msg, sizeof(error_msg), fmt, args);
va_end(args);
if (fmt) {
va_list args;
va_start(args, fmt);
vscnprintf(error_msg, sizeof(error_msg), fmt, args);
va_end(args);
msg = error_msg;
}
/*
* In most cases it's guaranteed that we get here with an RPM
@ -3020,8 +3078,12 @@ void i915_handle_error(struct drm_i915_private *dev_priv,
*/
intel_runtime_pm_get(dev_priv);
i915_capture_error_state(dev_priv, engine_mask, error_msg);
i915_clear_error_registers(dev_priv);
engine_mask &= INTEL_INFO(dev_priv)->ring_mask;
if (flags & I915_ERROR_CAPTURE) {
i915_capture_error_state(dev_priv, engine_mask, msg);
i915_clear_error_registers(dev_priv);
}
/*
* Try engine reset when available. We fall back to full reset if
@ -3034,7 +3096,7 @@ void i915_handle_error(struct drm_i915_private *dev_priv,
&dev_priv->gpu_error.flags))
continue;
if (i915_reset_engine(engine, 0) == 0)
if (i915_reset_engine(engine, msg) == 0)
engine_mask &= ~intel_engine_flag(engine);
clear_bit(I915_RESET_ENGINE + engine->id,
@ -3064,7 +3126,7 @@ void i915_handle_error(struct drm_i915_private *dev_priv,
TASK_UNINTERRUPTIBLE);
}
i915_reset_device(dev_priv);
i915_reset_device(dev_priv, engine_mask, msg);
for_each_engine(engine, dev_priv, tmp) {
clear_bit(I915_RESET_ENGINE + engine->id,
@ -3349,6 +3411,9 @@ static void gen11_gt_irq_reset(struct drm_i915_private *dev_priv)
I915_WRITE(GEN11_VCS0_VCS1_INTR_MASK, ~0);
I915_WRITE(GEN11_VCS2_VCS3_INTR_MASK, ~0);
I915_WRITE(GEN11_VECS0_VECS1_INTR_MASK, ~0);
I915_WRITE(GEN11_GPM_WGBOXPERF_INTR_ENABLE, 0);
I915_WRITE(GEN11_GPM_WGBOXPERF_INTR_MASK, ~0);
}
static void gen11_irq_reset(struct drm_device *dev)
@ -3887,7 +3952,14 @@ static void gen11_gt_irq_postinstall(struct drm_i915_private *dev_priv)
I915_WRITE(GEN11_VCS2_VCS3_INTR_MASK, ~(irqs | irqs << 16));
I915_WRITE(GEN11_VECS0_VECS1_INTR_MASK, ~(irqs | irqs << 16));
dev_priv->pm_imr = 0xffffffff; /* TODO */
/*
* RPS interrupts will get enabled/disabled on demand when RPS itself
* is enabled/disabled.
*/
dev_priv->pm_ier = 0x0;
dev_priv->pm_imr = ~dev_priv->pm_ier;
I915_WRITE(GEN11_GPM_WGBOXPERF_INTR_ENABLE, 0);
I915_WRITE(GEN11_GPM_WGBOXPERF_INTR_MASK, ~0);
}
static int gen11_irq_postinstall(struct drm_device *dev)

118
drivers/gpu/drm/i915/i915_oa_icl.c 100644
View File

@ -0,0 +1,118 @@
/*
* Autogenerated file by GPU Top : https://github.com/rib/gputop
* DO NOT EDIT manually!
*
*
* Copyright (c) 2015 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*
*/
#include <linux/sysfs.h>
#include "i915_drv.h"
#include "i915_oa_icl.h"
static const struct i915_oa_reg b_counter_config_test_oa[] = {
{ _MMIO(0x2740), 0x00000000 },
{ _MMIO(0x2710), 0x00000000 },
{ _MMIO(0x2714), 0xf0800000 },
{ _MMIO(0x2720), 0x00000000 },
{ _MMIO(0x2724), 0xf0800000 },
{ _MMIO(0x2770), 0x00000004 },
{ _MMIO(0x2774), 0x0000ffff },
{ _MMIO(0x2778), 0x00000003 },
{ _MMIO(0x277c), 0x0000ffff },
{ _MMIO(0x2780), 0x00000007 },
{ _MMIO(0x2784), 0x0000ffff },
{ _MMIO(0x2788), 0x00100002 },
{ _MMIO(0x278c), 0x0000fff7 },
{ _MMIO(0x2790), 0x00100002 },
{ _MMIO(0x2794), 0x0000ffcf },
{ _MMIO(0x2798), 0x00100082 },
{ _MMIO(0x279c), 0x0000ffef },
{ _MMIO(0x27a0), 0x001000c2 },
{ _MMIO(0x27a4), 0x0000ffe7 },
{ _MMIO(0x27a8), 0x00100001 },
{ _MMIO(0x27ac), 0x0000ffe7 },
};
static const struct i915_oa_reg flex_eu_config_test_oa[] = {
};
static const struct i915_oa_reg mux_config_test_oa[] = {
{ _MMIO(0xd04), 0x00000200 },
{ _MMIO(0x9840), 0x00000000 },
{ _MMIO(0x9884), 0x00000000 },
{ _MMIO(0x9888), 0x10060000 },
{ _MMIO(0x9888), 0x22060000 },
{ _MMIO(0x9888), 0x16060000 },
{ _MMIO(0x9888), 0x24060000 },
{ _MMIO(0x9888), 0x18060000 },
{ _MMIO(0x9888), 0x1a060000 },
{ _MMIO(0x9888), 0x12060000 },
{ _MMIO(0x9888), 0x14060000 },
{ _MMIO(0x9888), 0x10060000 },
{ _MMIO(0x9888), 0x22060000 },
{ _MMIO(0x9884), 0x00000003 },
{ _MMIO(0x9888), 0x16130000 },
{ _MMIO(0x9888), 0x24000001 },
{ _MMIO(0x9888), 0x0e130056 },
{ _MMIO(0x9888), 0x10130000 },
{ _MMIO(0x9888), 0x1a130000 },
{ _MMIO(0x9888), 0x541f0001 },
{ _MMIO(0x9888), 0x181f0000 },
{ _MMIO(0x9888), 0x4c1f0000 },
{ _MMIO(0x9888), 0x301f0000 },
};
static ssize_t
show_test_oa_id(struct device *kdev, struct device_attribute *attr, char *buf)
{
return sprintf(buf, "1\n");
}
void
i915_perf_load_test_config_icl(struct drm_i915_private *dev_priv)
{
strlcpy(dev_priv->perf.oa.test_config.uuid,
"a291665e-244b-4b76-9b9a-01de9d3c8068",
sizeof(dev_priv->perf.oa.test_config.uuid));
dev_priv->perf.oa.test_config.id = 1;
dev_priv->perf.oa.test_config.mux_regs = mux_config_test_oa;
dev_priv->perf.oa.test_config.mux_regs_len = ARRAY_SIZE(mux_config_test_oa);
dev_priv->perf.oa.test_config.b_counter_regs = b_counter_config_test_oa;
dev_priv->perf.oa.test_config.b_counter_regs_len = ARRAY_SIZE(b_counter_config_test_oa);
dev_priv->perf.oa.test_config.flex_regs = flex_eu_config_test_oa;
dev_priv->perf.oa.test_config.flex_regs_len = ARRAY_SIZE(flex_eu_config_test_oa);
dev_priv->perf.oa.test_config.sysfs_metric.name = "a291665e-244b-4b76-9b9a-01de9d3c8068";
dev_priv->perf.oa.test_config.sysfs_metric.attrs = dev_priv->perf.oa.test_config.attrs;
dev_priv->perf.oa.test_config.attrs[0] = &dev_priv->perf.oa.test_config.sysfs_metric_id.attr;
dev_priv->perf.oa.test_config.sysfs_metric_id.attr.name = "id";
dev_priv->perf.oa.test_config.sysfs_metric_id.attr.mode = 0444;
dev_priv->perf.oa.test_config.sysfs_metric_id.show = show_test_oa_id;
}

34
drivers/gpu/drm/i915/i915_oa_icl.h 100644
View File

@ -0,0 +1,34 @@
/*
* Autogenerated file by GPU Top : https://github.com/rib/gputop
* DO NOT EDIT manually!
*
*
* Copyright (c) 2015 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*
*/
#ifndef __I915_OA_ICL_H__
#define __I915_OA_ICL_H__
extern void i915_perf_load_test_config_icl(struct drm_i915_private *dev_priv);
#endif

2
drivers/gpu/drm/i915/i915_params.h
View File

@ -48,7 +48,7 @@ struct drm_printer;
param(int, enable_ips, 1) \
param(int, invert_brightness, 0) \
param(int, enable_guc, 0) \
param(int, guc_log_level, 0) \
param(int, guc_log_level, -1) \
param(char *, guc_firmware_path, NULL) \
param(char *, huc_firmware_path, NULL) \
param(int, mmio_debug, 0) \

1
drivers/gpu/drm/i915/i915_pci.c
View File

@ -602,6 +602,7 @@ static const struct intel_device_info intel_icelake_11_info = {
PLATFORM(INTEL_ICELAKE),
.is_alpha_support = 1,
.has_resource_streamer = 0,
.ring_mask = RENDER_RING | BLT_RING | VEBOX_RING | BSD_RING | BSD3_RING,
};
#undef GEN

69
drivers/gpu/drm/i915/i915_perf.c
View File

@ -209,6 +209,7 @@
#include "i915_oa_cflgt2.h"
#include "i915_oa_cflgt3.h"
#include "i915_oa_cnl.h"
#include "i915_oa_icl.h"
/* HW requires this to be a power of two, between 128k and 16M, though driver
* is currently generally designed assuming the largest 16M size is used such
@ -1042,7 +1043,7 @@ static int gen7_append_oa_reports(struct i915_perf_stream *stream,
I915_WRITE(GEN7_OASTATUS2,
((head & GEN7_OASTATUS2_HEAD_MASK) |
OA_MEM_SELECT_GGTT));
GEN7_OASTATUS2_MEM_SELECT_GGTT));
dev_priv->perf.oa.oa_buffer.head = head;
spin_unlock_irqrestore(&dev_priv->perf.oa.oa_buffer.ptr_lock, flags);
@ -1332,7 +1333,8 @@ static void gen7_init_oa_buffer(struct drm_i915_private *dev_priv)
/* Pre-DevBDW: OABUFFER must be set with counters off,
* before OASTATUS1, but after OASTATUS2
*/
I915_WRITE(GEN7_OASTATUS2, gtt_offset | OA_MEM_SELECT_GGTT); /* head */
I915_WRITE(GEN7_OASTATUS2,
gtt_offset | GEN7_OASTATUS2_MEM_SELECT_GGTT); /* head */
dev_priv->perf.oa.oa_buffer.head = gtt_offset;
I915_WRITE(GEN7_OABUFFER, gtt_offset);
@ -1392,7 +1394,7 @@ static void gen8_init_oa_buffer(struct drm_i915_private *dev_priv)
* bit."
*/
I915_WRITE(GEN8_OABUFFER, gtt_offset |
OABUFFER_SIZE_16M | OA_MEM_SELECT_GGTT);
OABUFFER_SIZE_16M | GEN8_OABUFFER_MEM_SELECT_GGTT);
I915_WRITE(GEN8_OATAILPTR, gtt_offset & GEN8_OATAILPTR_MASK);
/* Mark that we need updated tail pointers to read from... */
@ -1840,7 +1842,7 @@ static int gen8_enable_metric_set(struct drm_i915_private *dev_priv,
* be read back from automatically triggered reports, as part of the
* RPT_ID field.
*/
if (IS_GEN9(dev_priv) || IS_GEN10(dev_priv)) {
if (IS_GEN(dev_priv, 9, 11)) {
I915_WRITE(GEN8_OA_DEBUG,
_MASKED_BIT_ENABLE(GEN9_OA_DEBUG_DISABLE_CLK_RATIO_REPORTS |
GEN9_OA_DEBUG_INCLUDE_CLK_RATIO));
@ -1870,7 +1872,6 @@ static void gen8_disable_metric_set(struct drm_i915_private *dev_priv)
I915_WRITE(GDT_CHICKEN_BITS, (I915_READ(GDT_CHICKEN_BITS) &
~GT_NOA_ENABLE));
}
static void gen10_disable_metric_set(struct drm_i915_private *dev_priv)
@ -1885,6 +1886,13 @@ static void gen10_disable_metric_set(struct drm_i915_private *dev_priv)
static void gen7_oa_enable(struct drm_i915_private *dev_priv)
{
struct i915_gem_context *ctx =
dev_priv->perf.oa.exclusive_stream->ctx;
u32 ctx_id = dev_priv->perf.oa.specific_ctx_id;
bool periodic = dev_priv->perf.oa.periodic;
u32 period_exponent = dev_priv->perf.oa.period_exponent;
u32 report_format = dev_priv->perf.oa.oa_buffer.format;
/*
* Reset buf pointers so we don't forward reports from before now.
*
@ -1896,25 +1904,14 @@ static void gen7_oa_enable(struct drm_i915_private *dev_priv)
*/
gen7_init_oa_buffer(dev_priv);
if (dev_priv->perf.oa.exclusive_stream->enabled) {
struct i915_gem_context *ctx =
dev_priv->perf.oa.exclusive_stream->ctx;
u32 ctx_id = dev_priv->perf.oa.specific_ctx_id;
bool periodic = dev_priv->perf.oa.periodic;
u32 period_exponent = dev_priv->perf.oa.period_exponent;
u32 report_format = dev_priv->perf.oa.oa_buffer.format;
I915_WRITE(GEN7_OACONTROL,
(ctx_id & GEN7_OACONTROL_CTX_MASK) |
(period_exponent <<
GEN7_OACONTROL_TIMER_PERIOD_SHIFT) |
(periodic ? GEN7_OACONTROL_TIMER_ENABLE : 0) |
(report_format << GEN7_OACONTROL_FORMAT_SHIFT) |
(ctx ? GEN7_OACONTROL_PER_CTX_ENABLE : 0) |
GEN7_OACONTROL_ENABLE);
} else
I915_WRITE(GEN7_OACONTROL, 0);
I915_WRITE(GEN7_OACONTROL,
(ctx_id & GEN7_OACONTROL_CTX_MASK) |
(period_exponent <<
GEN7_OACONTROL_TIMER_PERIOD_SHIFT) |
(periodic ? GEN7_OACONTROL_TIMER_ENABLE : 0) |
(report_format << GEN7_OACONTROL_FORMAT_SHIFT) |
(ctx ? GEN7_OACONTROL_PER_CTX_ENABLE : 0) |
GEN7_OACONTROL_ENABLE);
}
static void gen8_oa_enable(struct drm_i915_private *dev_priv)
@ -2099,13 +2096,17 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
if (stream->ctx) {
ret = oa_get_render_ctx_id(stream);
if (ret)
if (ret) {
DRM_DEBUG("Invalid context id to filter with\n");
return ret;
}
}
ret = get_oa_config(dev_priv, props->metrics_set, &stream->oa_config);
if (ret)
if (ret) {
DRM_DEBUG("Invalid OA config id=%i\n", props->metrics_set);
goto err_config;
}
/* PRM - observability performance counters:
*
@ -2132,8 +2133,10 @@ static int i915_oa_stream_init(struct i915_perf_stream *stream,
ret = dev_priv->perf.oa.ops.enable_metric_set(dev_priv,
stream->oa_config);
if (ret)
if (ret) {
DRM_DEBUG("Unable to enable metric set\n");
goto err_enable;
}
stream->ops = &i915_oa_stream_ops;
@ -2745,7 +2748,8 @@ static int read_properties_unlocked(struct drm_i915_private *dev_priv,
props->ctx_handle = value;
break;
case DRM_I915_PERF_PROP_SAMPLE_OA:
props->sample_flags |= SAMPLE_OA_REPORT;
if (value)
props->sample_flags |= SAMPLE_OA_REPORT;
break;
case DRM_I915_PERF_PROP_OA_METRICS_SET:
if (value == 0) {
@ -2935,6 +2939,8 @@ void i915_perf_register(struct drm_i915_private *dev_priv)
i915_perf_load_test_config_cflgt3(dev_priv);
} else if (IS_CANNONLAKE(dev_priv)) {
i915_perf_load_test_config_cnl(dev_priv);
} else if (IS_ICELAKE(dev_priv)) {
i915_perf_load_test_config_icl(dev_priv);
}
if (dev_priv->perf.oa.test_config.id == 0)
@ -3292,6 +3298,8 @@ int i915_perf_add_config_ioctl(struct drm_device *dev, void *data,
mutex_unlock(&dev_priv->perf.metrics_lock);
DRM_DEBUG("Added config %s id=%i\n", oa_config->uuid, oa_config->id);
return oa_config->id;
sysfs_err:
@ -3348,6 +3356,9 @@ int i915_perf_remove_config_ioctl(struct drm_device *dev, void *data,
&oa_config->sysfs_metric);
idr_remove(&dev_priv->perf.metrics_idr, *arg);
DRM_DEBUG("Removed config %s id=%i\n", oa_config->uuid, oa_config->id);
put_oa_config(dev_priv, oa_config);
config_err:
@ -3467,7 +3478,7 @@ void i915_perf_init(struct drm_i915_private *dev_priv)
dev_priv->perf.oa.gen8_valid_ctx_bit = (1<<16);
}
} else if (IS_GEN10(dev_priv)) {
} else if (IS_GEN(dev_priv, 10, 11)) {
dev_priv->perf.oa.ops.is_valid_b_counter_reg =
gen7_is_valid_b_counter_addr;
dev_priv->perf.oa.ops.is_valid_mux_reg =

27
drivers/gpu/drm/i915/i915_pmu.c
View File

@ -1,33 +1,12 @@
/*
* Copyright © 2017 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
* SPDX-License-Identifier: MIT
*
* Copyright © 2017-2018 Intel Corporation
*/
#include <linux/perf_event.h>
#include <linux/pm_runtime.h>
#include "i915_drv.h"
#include "i915_pmu.h"
#include "intel_ringbuffer.h"
#include "i915_drv.h"
/* Frequency for the sampling timer for events which need it. */
#define FREQUENCY 200

30
drivers/gpu/drm/i915/i915_pmu.h
View File

@ -1,29 +1,19 @@
/*
* Copyright © 2017 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
* SPDX-License-Identifier: MIT
*
* Copyright © 2017-2018 Intel Corporation
*/
#ifndef __I915_PMU_H__
#define __I915_PMU_H__
#include <linux/hrtimer.h>
#include <linux/perf_event.h>
#include <linux/spinlock_types.h>
#include <drm/i915_drm.h>
struct drm_i915_private;
enum {
__I915_SAMPLE_FREQ_ACT = 0,
__I915_SAMPLE_FREQ_REQ,

644
drivers/gpu/drm/i915/i915_reg.h
View File

@ -153,9 +153,6 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define _MMIO_PORT3(pipe, a, b, c) _MMIO(_PICK(pipe, a, b, c))
#define _PLL(pll, a, b) ((a) + (pll)*((b)-(a)))
#define _MMIO_PLL(pll, a, b) _MMIO(_PLL(pll, a, b))
#define _MMIO_PORT6(port, a, b, c, d, e, f) _MMIO(_PICK(port, a, b, c, d, e, f))
#define _MMIO_PORT6_LN(port, ln, a0, a1, b, c, d, e, f) \
_MMIO(_PICK(port, a0, b, c, d, e, f) + (ln * (a1 - a0)))
#define _PHY3(phy, ...) _PICK(phy, __VA_ARGS__)
#define _MMIO_PHY3(phy, a, b, c) _MMIO(_PHY3(phy, a, b, c))
@ -191,6 +188,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define OTHER_CLASS 4
#define MAX_ENGINE_CLASS 4
#define OTHER_GTPM_INSTANCE 1
#define MAX_ENGINE_INSTANCE 3
/* PCI config space */
@ -304,6 +302,17 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN6_GRDOM_VECS (1 << 4)
#define GEN9_GRDOM_GUC (1 << 5)
#define GEN8_GRDOM_MEDIA2 (1 << 7)
/* GEN11 changed all bit defs except for FULL & RENDER */
#define GEN11_GRDOM_FULL GEN6_GRDOM_FULL
#define GEN11_GRDOM_RENDER GEN6_GRDOM_RENDER
#define GEN11_GRDOM_BLT (1 << 2)
#define GEN11_GRDOM_GUC (1 << 3)
#define GEN11_GRDOM_MEDIA (1 << 5)
#define GEN11_GRDOM_MEDIA2 (1 << 6)
#define GEN11_GRDOM_MEDIA3 (1 << 7)
#define GEN11_GRDOM_MEDIA4 (1 << 8)
#define GEN11_GRDOM_VECS (1 << 13)
#define GEN11_GRDOM_VECS2 (1 << 14)
#define RING_PP_DIR_BASE(engine) _MMIO((engine)->mmio_base+0x228)
#define RING_PP_DIR_BASE_READ(engine) _MMIO((engine)->mmio_base+0x518)
@ -430,145 +439,6 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define VGA_CR_INDEX_CGA 0x3d4
#define VGA_CR_DATA_CGA 0x3d5
/*
* Instruction field definitions used by the command parser
*/
#define INSTR_CLIENT_SHIFT 29
#define INSTR_MI_CLIENT 0x0
#define INSTR_BC_CLIENT 0x2
#define INSTR_RC_CLIENT 0x3
#define INSTR_SUBCLIENT_SHIFT 27
#define INSTR_SUBCLIENT_MASK 0x18000000
#define INSTR_MEDIA_SUBCLIENT 0x2
#define INSTR_26_TO_24_MASK 0x7000000
#define INSTR_26_TO_24_SHIFT 24
/*
* Memory interface instructions used by the kernel
*/
#define MI_INSTR(opcode, flags) (((opcode) << 23) | (flags))
/* Many MI commands use bit 22 of the header dword for GGTT vs PPGTT */
#define MI_GLOBAL_GTT (1<<22)
#define MI_NOOP MI_INSTR(0, 0)
#define MI_USER_INTERRUPT MI_INSTR(0x02, 0)
#define MI_WAIT_FOR_EVENT MI_INSTR(0x03, 0)
#define MI_WAIT_FOR_OVERLAY_FLIP (1<<16)
#define MI_WAIT_FOR_PLANE_B_FLIP (1<<6)
#define MI_WAIT_FOR_PLANE_A_FLIP (1<<2)
#define MI_WAIT_FOR_PLANE_A_SCANLINES (1<<1)
#define MI_FLUSH MI_INSTR(0x04, 0)
#define MI_READ_FLUSH (1 << 0)
#define MI_EXE_FLUSH (1 << 1)
#define MI_NO_WRITE_FLUSH (1 << 2)
#define MI_SCENE_COUNT (1 << 3) /* just increment scene count */
#define MI_END_SCENE (1 << 4) /* flush binner and incr scene count */
#define MI_INVALIDATE_ISP (1 << 5) /* invalidate indirect state pointers */
#define MI_REPORT_HEAD MI_INSTR(0x07, 0)
#define MI_ARB_ON_OFF MI_INSTR(0x08, 0)
#define MI_ARB_ENABLE (1<<0)
#define MI_ARB_DISABLE (0<<0)
#define MI_BATCH_BUFFER_END MI_INSTR(0x0a, 0)
#define MI_SUSPEND_FLUSH MI_INSTR(0x0b, 0)
#define MI_SUSPEND_FLUSH_EN (1<<0)
#define MI_SET_APPID MI_INSTR(0x0e, 0)
#define MI_OVERLAY_FLIP MI_INSTR(0x11, 0)
#define MI_OVERLAY_CONTINUE (0x0<<21)
#define MI_OVERLAY_ON (0x1<<21)
#define MI_OVERLAY_OFF (0x2<<21)
#define MI_LOAD_SCAN_LINES_INCL MI_INSTR(0x12, 0)
#define MI_DISPLAY_FLIP MI_INSTR(0x14, 2)
#define MI_DISPLAY_FLIP_I915 MI_INSTR(0x14, 1)
#define MI_DISPLAY_FLIP_PLANE(n) ((n) << 20)
/* IVB has funny definitions for which plane to flip. */
#define MI_DISPLAY_FLIP_IVB_PLANE_A (0 << 19)
#define MI_DISPLAY_FLIP_IVB_PLANE_B (1 << 19)
#define MI_DISPLAY_FLIP_IVB_SPRITE_A (2 << 19)
#define MI_DISPLAY_FLIP_IVB_SPRITE_B (3 << 19)
#define MI_DISPLAY_FLIP_IVB_PLANE_C (4 << 19)
#define MI_DISPLAY_FLIP_IVB_SPRITE_C (5 << 19)
/* SKL ones */
#define MI_DISPLAY_FLIP_SKL_PLANE_1_A (0 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_1_B (1 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_1_C (2 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_2_A (4 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_2_B (5 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_2_C (6 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_3_A (7 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_3_B (8 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_3_C (9 << 8)
#define MI_SEMAPHORE_MBOX MI_INSTR(0x16, 1) /* gen6, gen7 */
#define MI_SEMAPHORE_GLOBAL_GTT (1<<22)
#define MI_SEMAPHORE_UPDATE (1<<21)
#define MI_SEMAPHORE_COMPARE (1<<20)
#define MI_SEMAPHORE_REGISTER (1<<18)
#define MI_SEMAPHORE_SYNC_VR (0<<16) /* RCS wait for VCS (RVSYNC) */
#define MI_SEMAPHORE_SYNC_VER (1<<16) /* RCS wait for VECS (RVESYNC) */
#define MI_SEMAPHORE_SYNC_BR (2<<16) /* RCS wait for BCS (RBSYNC) */
#define MI_SEMAPHORE_SYNC_BV (0<<16) /* VCS wait for BCS (VBSYNC) */
#define MI_SEMAPHORE_SYNC_VEV (1<<16) /* VCS wait for VECS (VVESYNC) */
#define MI_SEMAPHORE_SYNC_RV (2<<16) /* VCS wait for RCS (VRSYNC) */
#define MI_SEMAPHORE_SYNC_RB (0<<16) /* BCS wait for RCS (BRSYNC) */
#define MI_SEMAPHORE_SYNC_VEB (1<<16) /* BCS wait for VECS (BVESYNC) */
#define MI_SEMAPHORE_SYNC_VB (2<<16) /* BCS wait for VCS (BVSYNC) */
#define MI_SEMAPHORE_SYNC_BVE (0<<16) /* VECS wait for BCS (VEBSYNC) */
#define MI_SEMAPHORE_SYNC_VVE (1<<16) /* VECS wait for VCS (VEVSYNC) */
#define MI_SEMAPHORE_SYNC_RVE (2<<16) /* VECS wait for RCS (VERSYNC) */
#define MI_SEMAPHORE_SYNC_INVALID (3<<16)
#define MI_SEMAPHORE_SYNC_MASK (3<<16)
#define MI_SET_CONTEXT MI_INSTR(0x18, 0)
#define MI_MM_SPACE_GTT (1<<8)
#define MI_MM_SPACE_PHYSICAL (0<<8)
#define MI_SAVE_EXT_STATE_EN (1<<3)
#define MI_RESTORE_EXT_STATE_EN (1<<2)
#define MI_FORCE_RESTORE (1<<1)
#define MI_RESTORE_INHIBIT (1<<0)
#define HSW_MI_RS_SAVE_STATE_EN (1<<3)
#define HSW_MI_RS_RESTORE_STATE_EN (1<<2)
#define MI_SEMAPHORE_SIGNAL MI_INSTR(0x1b, 0) /* GEN8+ */
#define MI_SEMAPHORE_TARGET(engine) ((engine)<<15)
#define MI_SEMAPHORE_WAIT MI_INSTR(0x1c, 2) /* GEN8+ */
#define MI_SEMAPHORE_POLL (1<<15)
#define MI_SEMAPHORE_SAD_GTE_SDD (1<<12)
#define MI_STORE_DWORD_IMM MI_INSTR(0x20, 1)
#define MI_STORE_DWORD_IMM_GEN4 MI_INSTR(0x20, 2)
#define MI_MEM_VIRTUAL (1 << 22) /* 945,g33,965 */
#define MI_USE_GGTT (1 << 22) /* g4x+ */
#define MI_STORE_DWORD_INDEX MI_INSTR(0x21, 1)
#define MI_STORE_DWORD_INDEX_SHIFT 2
/* Official intel docs are somewhat sloppy concerning MI_LOAD_REGISTER_IMM:
* - Always issue a MI_NOOP _before_ the MI_LOAD_REGISTER_IMM - otherwise hw
* simply ignores the register load under certain conditions.
* - One can actually load arbitrary many arbitrary registers: Simply issue x
* address/value pairs. Don't overdue it, though, x <= 2^4 must hold!
*/
#define MI_LOAD_REGISTER_IMM(x) MI_INSTR(0x22, 2*(x)-1)
#define MI_LRI_FORCE_POSTED (1<<12)
#define MI_STORE_REGISTER_MEM MI_INSTR(0x24, 1)
#define MI_STORE_REGISTER_MEM_GEN8 MI_INSTR(0x24, 2)
#define MI_SRM_LRM_GLOBAL_GTT (1<<22)
#define MI_FLUSH_DW MI_INSTR(0x26, 1) /* for GEN6 */
#define MI_FLUSH_DW_STORE_INDEX (1<<21)
#define MI_INVALIDATE_TLB (1<<18)
#define MI_FLUSH_DW_OP_STOREDW (1<<14)
#define MI_FLUSH_DW_OP_MASK (3<<14)
#define MI_FLUSH_DW_NOTIFY (1<<8)
#define MI_INVALIDATE_BSD (1<<7)
#define MI_FLUSH_DW_USE_GTT (1<<2)
#define MI_FLUSH_DW_USE_PPGTT (0<<2)
#define MI_LOAD_REGISTER_MEM MI_INSTR(0x29, 1)
#define MI_LOAD_REGISTER_MEM_GEN8 MI_INSTR(0x29, 2)
#define MI_BATCH_BUFFER MI_INSTR(0x30, 1)
#define MI_BATCH_NON_SECURE (1)
/* for snb/ivb/vlv this also means "batch in ppgtt" when ppgtt is enabled. */
#define MI_BATCH_NON_SECURE_I965 (1<<8)
#define MI_BATCH_PPGTT_HSW (1<<8)
#define MI_BATCH_NON_SECURE_HSW (1<<13)
#define MI_BATCH_BUFFER_START MI_INSTR(0x31, 0)
#define MI_BATCH_GTT (2<<6) /* aliased with (1<<7) on gen4 */
#define MI_BATCH_BUFFER_START_GEN8 MI_INSTR(0x31, 1)
#define MI_BATCH_RESOURCE_STREAMER (1<<10)
#define MI_PREDICATE_SRC0 _MMIO(0x2400)
#define MI_PREDICATE_SRC0_UDW _MMIO(0x2400 + 4)
#define MI_PREDICATE_SRC1 _MMIO(0x2408)
@ -578,130 +448,6 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define LOWER_SLICE_ENABLED (1<<0)
#define LOWER_SLICE_DISABLED (0<<0)
/*
* 3D instructions used by the kernel
*/
#define GFX_INSTR(opcode, flags) ((0x3 << 29) | ((opcode) << 24) | (flags))
#define GEN9_MEDIA_POOL_STATE ((0x3 << 29) | (0x2 << 27) | (0x5 << 16) | 4)
#define GEN9_MEDIA_POOL_ENABLE (1 << 31)
#define GFX_OP_RASTER_RULES ((0x3<<29)|(0x7<<24))
#define GFX_OP_SCISSOR ((0x3<<29)|(0x1c<<24)|(0x10<<19))
#define SC_UPDATE_SCISSOR (0x1<<1)
#define SC_ENABLE_MASK (0x1<<0)
#define SC_ENABLE (0x1<<0)
#define GFX_OP_LOAD_INDIRECT ((0x3<<29)|(0x1d<<24)|(0x7<<16))
#define GFX_OP_SCISSOR_INFO ((0x3<<29)|(0x1d<<24)|(0x81<<16)|(0x1))
#define SCI_YMIN_MASK (0xffff<<16)
#define SCI_XMIN_MASK (0xffff<<0)
#define SCI_YMAX_MASK (0xffff<<16)
#define SCI_XMAX_MASK (0xffff<<0)
#define GFX_OP_SCISSOR_ENABLE ((0x3<<29)|(0x1c<<24)|(0x10<<19))
#define GFX_OP_SCISSOR_RECT ((0x3<<29)|(0x1d<<24)|(0x81<<16)|1)
#define GFX_OP_COLOR_FACTOR ((0x3<<29)|(0x1d<<24)|(0x1<<16)|0x0)
#define GFX_OP_STIPPLE ((0x3<<29)|(0x1d<<24)|(0x83<<16))
#define GFX_OP_MAP_INFO ((0x3<<29)|(0x1d<<24)|0x4)
#define GFX_OP_DESTBUFFER_VARS ((0x3<<29)|(0x1d<<24)|(0x85<<16)|0x0)
#define GFX_OP_DESTBUFFER_INFO ((0x3<<29)|(0x1d<<24)|(0x8e<<16)|1)
#define GFX_OP_DRAWRECT_INFO ((0x3<<29)|(0x1d<<24)|(0x80<<16)|(0x3))
#define GFX_OP_DRAWRECT_INFO_I965 ((0x7900<<16)|0x2)
#define COLOR_BLT_CMD (2<<29 | 0x40<<22 | (5-2))
#define SRC_COPY_BLT_CMD ((2<<29)|(0x43<<22)|4)
#define XY_SRC_COPY_BLT_CMD ((2<<29)|(0x53<<22)|6)
#define XY_MONO_SRC_COPY_IMM_BLT ((2<<29)|(0x71<<22)|5)
#define BLT_WRITE_A (2<<20)
#define BLT_WRITE_RGB (1<<20)
#define BLT_WRITE_RGBA (BLT_WRITE_RGB | BLT_WRITE_A)
#define BLT_DEPTH_8 (0<<24)
#define BLT_DEPTH_16_565 (1<<24)
#define BLT_DEPTH_16_1555 (2<<24)
#define BLT_DEPTH_32 (3<<24)
#define BLT_ROP_SRC_COPY (0xcc<<16)
#define BLT_ROP_COLOR_COPY (0xf0<<16)
#define XY_SRC_COPY_BLT_SRC_TILED (1<<15) /* 965+ only */
#define XY_SRC_COPY_BLT_DST_TILED (1<<11) /* 965+ only */
#define CMD_OP_DISPLAYBUFFER_INFO ((0x0<<29)|(0x14<<23)|2)
#define ASYNC_FLIP (1<<22)
#define DISPLAY_PLANE_A (0<<20)
#define DISPLAY_PLANE_B (1<<20)
#define GFX_OP_PIPE_CONTROL(len) ((0x3<<29)|(0x3<<27)|(0x2<<24)|((len)-2))
#define PIPE_CONTROL_FLUSH_L3 (1<<27)
#define PIPE_CONTROL_GLOBAL_GTT_IVB (1<<24) /* gen7+ */
#define PIPE_CONTROL_MMIO_WRITE (1<<23)
#define PIPE_CONTROL_STORE_DATA_INDEX (1<<21)
#define PIPE_CONTROL_CS_STALL (1<<20)
#define PIPE_CONTROL_TLB_INVALIDATE (1<<18)
#define PIPE_CONTROL_MEDIA_STATE_CLEAR (1<<16)
#define PIPE_CONTROL_QW_WRITE (1<<14)
#define PIPE_CONTROL_POST_SYNC_OP_MASK (3<<14)
#define PIPE_CONTROL_DEPTH_STALL (1<<13)
#define PIPE_CONTROL_WRITE_FLUSH (1<<12)
#define PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH (1<<12) /* gen6+ */
#define PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE (1<<11) /* MBZ on Ironlake */
#define PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE (1<<10) /* GM45+ only */
#define PIPE_CONTROL_INDIRECT_STATE_DISABLE (1<<9)
#define PIPE_CONTROL_NOTIFY (1<<8)
#define PIPE_CONTROL_FLUSH_ENABLE (1<<7) /* gen7+ */
#define PIPE_CONTROL_DC_FLUSH_ENABLE (1<<5)
#define PIPE_CONTROL_VF_CACHE_INVALIDATE (1<<4)
#define PIPE_CONTROL_CONST_CACHE_INVALIDATE (1<<3)
#define PIPE_CONTROL_STATE_CACHE_INVALIDATE (1<<2)
#define PIPE_CONTROL_STALL_AT_SCOREBOARD (1<<1)
#define PIPE_CONTROL_DEPTH_CACHE_FLUSH (1<<0)
#define PIPE_CONTROL_GLOBAL_GTT (1<<2) /* in addr dword */
/*
* Commands used only by the command parser
*/
#define MI_SET_PREDICATE MI_INSTR(0x01, 0)
#define MI_ARB_CHECK MI_INSTR(0x05, 0)
#define MI_RS_CONTROL MI_INSTR(0x06, 0)
#define MI_URB_ATOMIC_ALLOC MI_INSTR(0x09, 0)
#define MI_PREDICATE MI_INSTR(0x0C, 0)
#define MI_RS_CONTEXT MI_INSTR(0x0F, 0)
#define MI_TOPOLOGY_FILTER MI_INSTR(0x0D, 0)
#define MI_LOAD_SCAN_LINES_EXCL MI_INSTR(0x13, 0)
#define MI_URB_CLEAR MI_INSTR(0x19, 0)
#define MI_UPDATE_GTT MI_INSTR(0x23, 0)
#define MI_CLFLUSH MI_INSTR(0x27, 0)
#define MI_REPORT_PERF_COUNT MI_INSTR(0x28, 0)
#define MI_REPORT_PERF_COUNT_GGTT (1<<0)
#define MI_LOAD_REGISTER_REG MI_INSTR(0x2A, 0)
#define MI_RS_STORE_DATA_IMM MI_INSTR(0x2B, 0)
#define MI_LOAD_URB_MEM MI_INSTR(0x2C, 0)
#define MI_STORE_URB_MEM MI_INSTR(0x2D, 0)
#define MI_CONDITIONAL_BATCH_BUFFER_END MI_INSTR(0x36, 0)
#define PIPELINE_SELECT ((0x3<<29)|(0x1<<27)|(0x1<<24)|(0x4<<16))
#define GFX_OP_3DSTATE_VF_STATISTICS ((0x3<<29)|(0x1<<27)|(0x0<<24)|(0xB<<16))
#define MEDIA_VFE_STATE ((0x3<<29)|(0x2<<27)|(0x0<<24)|(0x0<<16))
#define MEDIA_VFE_STATE_MMIO_ACCESS_MASK (0x18)
#define GPGPU_OBJECT ((0x3<<29)|(0x2<<27)|(0x1<<24)|(0x4<<16))
#define GPGPU_WALKER ((0x3<<29)|(0x2<<27)|(0x1<<24)|(0x5<<16))
#define GFX_OP_3DSTATE_DX9_CONSTANTF_VS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x39<<16))
#define GFX_OP_3DSTATE_DX9_CONSTANTF_PS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x3A<<16))
#define GFX_OP_3DSTATE_SO_DECL_LIST \
((0x3<<29)|(0x3<<27)|(0x1<<24)|(0x17<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_VS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x43<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_GS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x44<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_HS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x45<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_DS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x46<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_PS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x47<<16))
#define MFX_WAIT ((0x3<<29)|(0x1<<27)|(0x0<<16))
#define COLOR_BLT ((0x2<<29)|(0x40<<22))
#define SRC_COPY_BLT ((0x2<<29)|(0x43<<22))
/*
* Registers used only by the command parser
*/
@ -802,6 +548,7 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN8_OABUFFER_UDW _MMIO(0x23b4)
#define GEN8_OABUFFER _MMIO(0x2b14)
#define GEN8_OABUFFER_MEM_SELECT_GGTT (1 << 0) /* 0: PPGTT, 1: GGTT */
#define GEN7_OASTATUS1 _MMIO(0x2364)
#define GEN7_OASTATUS1_TAIL_MASK 0xffffffc0
@ -810,7 +557,8 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN7_OASTATUS1_REPORT_LOST (1<<0)
#define GEN7_OASTATUS2 _MMIO(0x2368)
#define GEN7_OASTATUS2_HEAD_MASK 0xffffffc0
#define GEN7_OASTATUS2_HEAD_MASK 0xffffffc0
#define GEN7_OASTATUS2_MEM_SELECT_GGTT (1 << 0) /* 0: PPGTT, 1: GGTT */
#define GEN8_OASTATUS _MMIO(0x2b08)
#define GEN8_OASTATUS_OVERRUN_STATUS (1<<3)
@ -832,8 +580,6 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define OABUFFER_SIZE_8M (6<<3)
#define OABUFFER_SIZE_16M (7<<3)
#define OA_MEM_SELECT_GGTT (1<<0)
/*
* Flexible, Aggregate EU Counter Registers.
* Note: these aren't contiguous
@ -1127,6 +873,12 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK (1 << GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT)
#define GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_19_2_MHZ 0
#define GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_24_MHZ 1
#define GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT 3
#define GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK (0x7 << GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT)
#define GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_24_MHZ 0
#define GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_19_2_MHZ 1
#define GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_38_4_MHZ 2
#define GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_25_MHZ 3
#define GEN10_RPM_CONFIG0_CTC_SHIFT_PARAMETER_SHIFT 1
#define GEN10_RPM_CONFIG0_CTC_SHIFT_PARAMETER_MASK (0x3 << GEN10_RPM_CONFIG0_CTC_SHIFT_PARAMETER_SHIFT)
@ -1948,79 +1700,100 @@ enum i915_power_well_id {
#define _CNL_PORT_PCS_DW1_LN0_C 0x162C04
#define _CNL_PORT_PCS_DW1_LN0_D 0x162E04
#define _CNL_PORT_PCS_DW1_LN0_F 0x162804
#define CNL_PORT_PCS_DW1_GRP(port) _MMIO_PORT6(port, \
#define CNL_PORT_PCS_DW1_GRP(port) _MMIO(_PICK(port, \
_CNL_PORT_PCS_DW1_GRP_AE, \
_CNL_PORT_PCS_DW1_GRP_B, \
_CNL_PORT_PCS_DW1_GRP_C, \
_CNL_PORT_PCS_DW1_GRP_D, \
_CNL_PORT_PCS_DW1_GRP_AE, \
_CNL_PORT_PCS_DW1_GRP_F)
#define CNL_PORT_PCS_DW1_LN0(port) _MMIO_PORT6(port, \
_CNL_PORT_PCS_DW1_GRP_F))
#define CNL_PORT_PCS_DW1_LN0(port) _MMIO(_PICK(port, \
_CNL_PORT_PCS_DW1_LN0_AE, \
_CNL_PORT_PCS_DW1_LN0_B, \
_CNL_PORT_PCS_DW1_LN0_C, \
_CNL_PORT_PCS_DW1_LN0_D, \
_CNL_PORT_PCS_DW1_LN0_AE, \
_CNL_PORT_PCS_DW1_LN0_F)
_CNL_PORT_PCS_DW1_LN0_F))
#define _ICL_PORT_PCS_DW1_GRP_A 0x162604
#define _ICL_PORT_PCS_DW1_GRP_B 0x6C604
#define _ICL_PORT_PCS_DW1_LN0_A 0x162804
#define _ICL_PORT_PCS_DW1_LN0_B 0x6C804
#define ICL_PORT_PCS_DW1_GRP(port) _MMIO_PORT(port,\
_ICL_PORT_PCS_DW1_GRP_A, \
_ICL_PORT_PCS_DW1_GRP_B)
#define ICL_PORT_PCS_DW1_LN0(port) _MMIO_PORT(port, \
_ICL_PORT_PCS_DW1_LN0_A, \
_ICL_PORT_PCS_DW1_LN0_B)
#define COMMON_KEEPER_EN (1 << 26)
#define _CNL_PORT_TX_DW2_GRP_AE 0x162348
#define _CNL_PORT_TX_DW2_GRP_B 0x1623C8
#define _CNL_PORT_TX_DW2_GRP_C 0x162B48
#define _CNL_PORT_TX_DW2_GRP_D 0x162BC8
#define _CNL_PORT_TX_DW2_GRP_F 0x162A48
#define _CNL_PORT_TX_DW2_LN0_AE 0x162448
#define _CNL_PORT_TX_DW2_LN0_B 0x162648
#define _CNL_PORT_TX_DW2_LN0_C 0x162C48
#define _CNL_PORT_TX_DW2_LN0_D 0x162E48
#define _CNL_PORT_TX_DW2_LN0_F 0x162848
#define CNL_PORT_TX_DW2_GRP(port) _MMIO_PORT6(port, \
_CNL_PORT_TX_DW2_GRP_AE, \
_CNL_PORT_TX_DW2_GRP_B, \
_CNL_PORT_TX_DW2_GRP_C, \
_CNL_PORT_TX_DW2_GRP_D, \
_CNL_PORT_TX_DW2_GRP_AE, \
_CNL_PORT_TX_DW2_GRP_F)
#define CNL_PORT_TX_DW2_LN0(port) _MMIO_PORT6(port, \
_CNL_PORT_TX_DW2_LN0_AE, \
_CNL_PORT_TX_DW2_LN0_B, \
_CNL_PORT_TX_DW2_LN0_C, \
_CNL_PORT_TX_DW2_LN0_D, \
_CNL_PORT_TX_DW2_LN0_AE, \
_CNL_PORT_TX_DW2_LN0_F)
#define SWING_SEL_UPPER(x) ((x >> 3) << 15)
/* CNL Port TX registers */
#define _CNL_PORT_TX_AE_GRP_OFFSET 0x162340
#define _CNL_PORT_TX_B_GRP_OFFSET 0x1623C0
#define _CNL_PORT_TX_C_GRP_OFFSET 0x162B40
#define _CNL_PORT_TX_D_GRP_OFFSET 0x162BC0
#define _CNL_PORT_TX_F_GRP_OFFSET 0x162A40
#define _CNL_PORT_TX_AE_LN0_OFFSET 0x162440
#define _CNL_PORT_TX_B_LN0_OFFSET 0x162640
#define _CNL_PORT_TX_C_LN0_OFFSET 0x162C40
#define _CNL_PORT_TX_D_LN0_OFFSET 0x162E40
#define _CNL_PORT_TX_F_LN0_OFFSET 0x162840
#define _CNL_PORT_TX_DW_GRP(port, dw) (_PICK((port), \
_CNL_PORT_TX_AE_GRP_OFFSET, \
_CNL_PORT_TX_B_GRP_OFFSET, \
_CNL_PORT_TX_B_GRP_OFFSET, \
_CNL_PORT_TX_D_GRP_OFFSET, \
_CNL_PORT_TX_AE_GRP_OFFSET, \
_CNL_PORT_TX_F_GRP_OFFSET) + \
4*(dw))
#define _CNL_PORT_TX_DW_LN0(port, dw) (_PICK((port), \
_CNL_PORT_TX_AE_LN0_OFFSET, \
_CNL_PORT_TX_B_LN0_OFFSET, \
_CNL_PORT_TX_B_LN0_OFFSET, \
_CNL_PORT_TX_D_LN0_OFFSET, \
_CNL_PORT_TX_AE_LN0_OFFSET, \
_CNL_PORT_TX_F_LN0_OFFSET) + \
4*(dw))
#define CNL_PORT_TX_DW2_GRP(port) _MMIO(_CNL_PORT_TX_DW_GRP((port), 2))
#define CNL_PORT_TX_DW2_LN0(port) _MMIO(_CNL_PORT_TX_DW_LN0((port), 2))
#define _ICL_PORT_TX_DW2_GRP_A 0x162688
#define _ICL_PORT_TX_DW2_GRP_B 0x6C688
#define _ICL_PORT_TX_DW2_LN0_A 0x162888
#define _ICL_PORT_TX_DW2_LN0_B 0x6C888
#define ICL_PORT_TX_DW2_GRP(port) _MMIO_PORT(port, \
_ICL_PORT_TX_DW2_GRP_A, \
_ICL_PORT_TX_DW2_GRP_B)
#define ICL_PORT_TX_DW2_LN0(port) _MMIO_PORT(port, \
_ICL_PORT_TX_DW2_LN0_A, \
_ICL_PORT_TX_DW2_LN0_B)
#define SWING_SEL_UPPER(x) (((x) >> 3) << 15)
#define SWING_SEL_UPPER_MASK (1 << 15)
#define SWING_SEL_LOWER(x) ((x & 0x7) << 11)
#define SWING_SEL_LOWER(x) (((x) & 0x7) << 11)
#define SWING_SEL_LOWER_MASK (0x7 << 11)
#define RCOMP_SCALAR(x) ((x) << 0)
#define RCOMP_SCALAR_MASK (0xFF << 0)
#define _CNL_PORT_TX_DW4_GRP_AE 0x162350
#define _CNL_PORT_TX_DW4_GRP_B 0x1623D0
#define _CNL_PORT_TX_DW4_GRP_C 0x162B50
#define _CNL_PORT_TX_DW4_GRP_D 0x162BD0
#define _CNL_PORT_TX_DW4_GRP_F 0x162A50
#define _CNL_PORT_TX_DW4_LN0_AE 0x162450
#define _CNL_PORT_TX_DW4_LN1_AE 0x1624D0
#define _CNL_PORT_TX_DW4_LN0_B 0x162650
#define _CNL_PORT_TX_DW4_LN0_C 0x162C50
#define _CNL_PORT_TX_DW4_LN0_D 0x162E50
#define _CNL_PORT_TX_DW4_LN0_F 0x162850
#define CNL_PORT_TX_DW4_GRP(port) _MMIO_PORT6(port, \
_CNL_PORT_TX_DW4_GRP_AE, \
_CNL_PORT_TX_DW4_GRP_B, \
_CNL_PORT_TX_DW4_GRP_C, \
_CNL_PORT_TX_DW4_GRP_D, \
_CNL_PORT_TX_DW4_GRP_AE, \
_CNL_PORT_TX_DW4_GRP_F)
#define CNL_PORT_TX_DW4_LN(port, ln) _MMIO_PORT6_LN(port, ln, \
_CNL_PORT_TX_DW4_LN0_AE, \
_CNL_PORT_TX_DW4_LN1_AE, \
_CNL_PORT_TX_DW4_LN0_B, \
_CNL_PORT_TX_DW4_LN0_C, \
_CNL_PORT_TX_DW4_LN0_D, \
_CNL_PORT_TX_DW4_LN0_AE, \
_CNL_PORT_TX_DW4_LN0_F)
#define CNL_PORT_TX_DW4_GRP(port) _MMIO(_CNL_PORT_TX_DW_GRP((port), 4))
#define CNL_PORT_TX_DW4_LN0(port) _MMIO(_CNL_PORT_TX_DW_LN0((port), 4))
#define CNL_PORT_TX_DW4_LN(port, ln) _MMIO(_CNL_PORT_TX_DW_LN0((port), 4) + \
(ln * (_CNL_PORT_TX_DW4_LN1_AE - \
_CNL_PORT_TX_DW4_LN0_AE)))
#define _ICL_PORT_TX_DW4_GRP_A 0x162690
#define _ICL_PORT_TX_DW4_GRP_B 0x6C690
#define _ICL_PORT_TX_DW4_LN0_A 0x162890
#define _ICL_PORT_TX_DW4_LN1_A 0x162990
#define _ICL_PORT_TX_DW4_LN0_B 0x6C890
#define ICL_PORT_TX_DW4_GRP(port) _MMIO_PORT(port, \
_ICL_PORT_TX_DW4_GRP_A, \
_ICL_PORT_TX_DW4_GRP_B)
#define ICL_PORT_TX_DW4_LN(port, ln) _MMIO(_PORT(port, \
_ICL_PORT_TX_DW4_LN0_A, \
_ICL_PORT_TX_DW4_LN0_B) + \
(ln * (_ICL_PORT_TX_DW4_LN1_A - \
_ICL_PORT_TX_DW4_LN0_A)))
#define LOADGEN_SELECT (1 << 31)
#define POST_CURSOR_1(x) ((x) << 12)
#define POST_CURSOR_1_MASK (0x3F << 12)
@ -2029,64 +1802,147 @@ enum i915_power_well_id {
#define CURSOR_COEFF(x) ((x) << 0)
#define CURSOR_COEFF_MASK (0x3F << 0)
#define _CNL_PORT_TX_DW5_GRP_AE 0x162354
#define _CNL_PORT_TX_DW5_GRP_B 0x1623D4
#define _CNL_PORT_TX_DW5_GRP_C 0x162B54
#define _CNL_PORT_TX_DW5_GRP_D 0x162BD4
#define _CNL_PORT_TX_DW5_GRP_F 0x162A54
#define _CNL_PORT_TX_DW5_LN0_AE 0x162454
#define _CNL_PORT_TX_DW5_LN0_B 0x162654
#define _CNL_PORT_TX_DW5_LN0_C 0x162C54
#define _CNL_PORT_TX_DW5_LN0_D 0x162E54
#define _CNL_PORT_TX_DW5_LN0_F 0x162854
#define CNL_PORT_TX_DW5_GRP(port) _MMIO_PORT6(port, \
_CNL_PORT_TX_DW5_GRP_AE, \
_CNL_PORT_TX_DW5_GRP_B, \
_CNL_PORT_TX_DW5_GRP_C, \
_CNL_PORT_TX_DW5_GRP_D, \
_CNL_PORT_TX_DW5_GRP_AE, \
_CNL_PORT_TX_DW5_GRP_F)
#define CNL_PORT_TX_DW5_LN0(port) _MMIO_PORT6(port, \
_CNL_PORT_TX_DW5_LN0_AE, \
_CNL_PORT_TX_DW5_LN0_B, \
_CNL_PORT_TX_DW5_LN0_C, \
_CNL_PORT_TX_DW5_LN0_D, \
_CNL_PORT_TX_DW5_LN0_AE, \
_CNL_PORT_TX_DW5_LN0_F)
#define CNL_PORT_TX_DW5_GRP(port) _MMIO(_CNL_PORT_TX_DW_GRP((port), 5))
#define CNL_PORT_TX_DW5_LN0(port) _MMIO(_CNL_PORT_TX_DW_LN0((port), 5))
#define _ICL_PORT_TX_DW5_GRP_A 0x162694
#define _ICL_PORT_TX_DW5_GRP_B 0x6C694
#define _ICL_PORT_TX_DW5_LN0_A 0x162894
#define _ICL_PORT_TX_DW5_LN0_B 0x6C894
#define ICL_PORT_TX_DW5_GRP(port) _MMIO_PORT(port, \
_ICL_PORT_TX_DW5_GRP_A, \
_ICL_PORT_TX_DW5_GRP_B)
#define ICL_PORT_TX_DW5_LN0(port) _MMIO_PORT(port, \
_ICL_PORT_TX_DW5_LN0_A, \
_ICL_PORT_TX_DW5_LN0_B)
#define TX_TRAINING_EN (1 << 31)
#define TAP2_DISABLE (1 << 30)
#define TAP3_DISABLE (1 << 29)
#define SCALING_MODE_SEL(x) ((x) << 18)
#define SCALING_MODE_SEL_MASK (0x7 << 18)
#define RTERM_SELECT(x) ((x) << 3)
#define RTERM_SELECT_MASK (0x7 << 3)
#define _CNL_PORT_TX_DW7_GRP_AE 0x16235C
#define _CNL_PORT_TX_DW7_GRP_B 0x1623DC
#define _CNL_PORT_TX_DW7_GRP_C 0x162B5C
#define _CNL_PORT_TX_DW7_GRP_D 0x162BDC
#define _CNL_PORT_TX_DW7_GRP_F 0x162A5C
#define _CNL_PORT_TX_DW7_LN0_AE 0x16245C
#define _CNL_PORT_TX_DW7_LN0_B 0x16265C
#define _CNL_PORT_TX_DW7_LN0_C 0x162C5C
#define _CNL_PORT_TX_DW7_LN0_D 0x162E5C
#define _CNL_PORT_TX_DW7_LN0_F 0x16285C
#define CNL_PORT_TX_DW7_GRP(port) _MMIO_PORT6(port, \
_CNL_PORT_TX_DW7_GRP_AE, \
_CNL_PORT_TX_DW7_GRP_B, \
_CNL_PORT_TX_DW7_GRP_C, \
_CNL_PORT_TX_DW7_GRP_D, \
_CNL_PORT_TX_DW7_GRP_AE, \
_CNL_PORT_TX_DW7_GRP_F)
#define CNL_PORT_TX_DW7_LN0(port) _MMIO_PORT6(port, \
_CNL_PORT_TX_DW7_LN0_AE, \
_CNL_PORT_TX_DW7_LN0_B, \
_CNL_PORT_TX_DW7_LN0_C, \
_CNL_PORT_TX_DW7_LN0_D, \
_CNL_PORT_TX_DW7_LN0_AE, \
_CNL_PORT_TX_DW7_LN0_F)
#define CNL_PORT_TX_DW7_GRP(port) _MMIO(_CNL_PORT_TX_DW_GRP((port), 7))
#define CNL_PORT_TX_DW7_LN0(port) _MMIO(_CNL_PORT_TX_DW_LN0((port), 7))
#define N_SCALAR(x) ((x) << 24)
#define N_SCALAR_MASK (0x7F << 24)
#define _ICL_MG_PHY_PORT_LN(port, ln, ln0p1, ln0p2, ln1p1) \
_MMIO(_PORT((port) - PORT_C, ln0p1, ln0p2) + (ln) * ((ln1p1) - (ln0p1)))
#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT1 0x16812C
#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT1 0x16852C
#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT2 0x16912C
#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT2 0x16952C
#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT3 0x16A12C
#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT3 0x16A52C
#define _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT4 0x16B12C
#define _ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT4 0x16B52C
#define ICL_PORT_MG_TX1_LINK_PARAMS(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT1, \
_ICL_MG_TX_LINK_PARAMS_TX1LN0_PORT2, \
_ICL_MG_TX_LINK_PARAMS_TX1LN1_PORT1)
#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT1 0x1680AC
#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT1 0x1684AC
#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT2 0x1690AC
#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT2 0x1694AC
#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT3 0x16A0AC
#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT3 0x16A4AC
#define _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT4 0x16B0AC
#define _ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT4 0x16B4AC
#define ICL_PORT_MG_TX2_LINK_PARAMS(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT1, \
_ICL_MG_TX_LINK_PARAMS_TX2LN0_PORT2, \
_ICL_MG_TX_LINK_PARAMS_TX2LN1_PORT1)
#define CRI_USE_FS32 (1 << 5)
#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT1 0x16814C
#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT1 0x16854C
#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT2 0x16914C
#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT2 0x16954C
#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT3 0x16A14C
#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT3 0x16A54C
#define _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT4 0x16B14C
#define _ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT4 0x16B54C
#define ICL_PORT_MG_TX1_PISO_READLOAD(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT1, \
_ICL_MG_TX_PISO_READLOAD_TX1LN0_PORT2, \
_ICL_MG_TX_PISO_READLOAD_TX1LN1_PORT1)
#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT1 0x1680CC
#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT1 0x1684CC
#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT2 0x1690CC
#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT2 0x1694CC
#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT3 0x16A0CC
#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT3 0x16A4CC
#define _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT4 0x16B0CC
#define _ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT4 0x16B4CC
#define ICL_PORT_MG_TX2_PISO_READLOAD(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT1, \
_ICL_MG_TX_PISO_READLOAD_TX2LN0_PORT2, \
_ICL_MG_TX_PISO_READLOAD_TX2LN1_PORT1)
#define CRI_CALCINIT (1 << 1)
#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT1 0x168148
#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT1 0x168548
#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT2 0x169148
#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT2 0x169548
#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT3 0x16A148
#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT3 0x16A548
#define _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT4 0x16B148
#define _ICL_MG_TX_SWINGCTRL_TX1LN1_PORT4 0x16B548
#define ICL_PORT_MG_TX1_SWINGCTRL(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_SWINGCTRL_TX1LN0_PORT1, \
_ICL_MG_TX_SWINGCTRL_TX1LN0_PORT2, \
_ICL_MG_TX_SWINGCTRL_TX1LN1_PORT1)
#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT1 0x1680C8
#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT1 0x1684C8
#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT2 0x1690C8
#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT2 0x1694C8
#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT3 0x16A0C8
#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT3 0x16A4C8
#define _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT4 0x16B0C8
#define _ICL_MG_TX_SWINGCTRL_TX2LN1_PORT4 0x16B4C8
#define ICL_PORT_MG_TX2_SWINGCTRL(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_SWINGCTRL_TX2LN0_PORT1, \
_ICL_MG_TX_SWINGCTRL_TX2LN0_PORT2, \
_ICL_MG_TX_SWINGCTRL_TX2LN1_PORT1)
#define CRI_TXDEEMPH_OVERRIDE_17_12(x) ((x) << 0)
#define CRI_TXDEEMPH_OVERRIDE_17_12_MASK (0x3F << 0)
#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT1 0x168144
#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT1 0x168544
#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT2 0x169144
#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT2 0x169544
#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT3 0x16A144
#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT3 0x16A544
#define _ICL_MG_TX_DRVCTRL_TX1LN0_PORT4 0x16B144
#define _ICL_MG_TX_DRVCTRL_TX1LN1_PORT4 0x16B544
#define ICL_PORT_MG_TX1_DRVCTRL(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_DRVCTRL_TX1LN0_PORT1, \
_ICL_MG_TX_DRVCTRL_TX1LN0_PORT2, \
_ICL_MG_TX_DRVCTRL_TX1LN1_PORT1)
#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT1 0x1680C4
#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT1 0x1684C4
#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT2 0x1690C4
#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT2 0x1694C4
#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT3 0x16A0C4
#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT3 0x16A4C4
#define _ICL_MG_TX_DRVCTRL_TX2LN0_PORT4 0x16B0C4
#define _ICL_MG_TX_DRVCTRL_TX2LN1_PORT4 0x16B4C4
#define ICL_PORT_MG_TX2_DRVCTRL(port, ln) \
_ICL_MG_PHY_PORT_LN(port, ln, _ICL_MG_TX_DRVCTRL_TX2LN0_PORT1, \
_ICL_MG_TX_DRVCTRL_TX2LN0_PORT2, \
_ICL_MG_TX_DRVCTRL_TX2LN1_PORT1)
#define CRI_TXDEEMPH_OVERRIDE_11_6(x) ((x) << 24)
#define CRI_TXDEEMPH_OVERRIDE_11_6_MASK (0x3F << 24)
#define CRI_TXDEEMPH_OVERRIDE_EN (1 << 22)
#define CRI_TXDEEMPH_OVERRIDE_5_0(x) ((x) << 16)
#define CRI_TXDEEMPH_OVERRIDE_5_0_MASK (0x3F << 16)
/* The spec defines this only for BXT PHY0, but lets assume that this
* would exist for PHY1 too if it had a second channel.
*/
@ -2473,6 +2329,10 @@ enum i915_power_well_id {
#define GEN8_MCR_SLICE_MASK GEN8_MCR_SLICE(3)
#define GEN8_MCR_SUBSLICE(subslice) (((subslice) & 3) << 24)
#define GEN8_MCR_SUBSLICE_MASK GEN8_MCR_SUBSLICE(3)
#define GEN11_MCR_SLICE(slice) (((slice) & 0xf) << 27)
#define GEN11_MCR_SLICE_MASK GEN11_MCR_SLICE(0xf)
#define GEN11_MCR_SUBSLICE(subslice) (((subslice) & 0x7) << 24)
#define GEN11_MCR_SUBSLICE_MASK GEN11_MCR_SUBSLICE(0x7)
#define RING_IPEIR(base) _MMIO((base)+0x64)
#define RING_IPEHR(base) _MMIO((base)+0x68)
/*
@ -2867,6 +2727,19 @@ enum i915_power_well_id {
#define GEN10_EU_DISABLE3 _MMIO(0x9140)
#define GEN10_EU_DIS_SS_MASK 0xff
#define GEN11_GT_VEBOX_VDBOX_DISABLE _MMIO(0x9140)
#define GEN11_GT_VDBOX_DISABLE_MASK 0xff
#define GEN11_GT_VEBOX_DISABLE_SHIFT 16
#define GEN11_GT_VEBOX_DISABLE_MASK (0xff << GEN11_GT_VEBOX_DISABLE_SHIFT)
#define GEN11_EU_DISABLE _MMIO(0x9134)
#define GEN11_EU_DIS_MASK 0xFF
#define GEN11_GT_SLICE_ENABLE _MMIO(0x9138)
#define GEN11_GT_S_ENA_MASK 0xFF
#define GEN11_GT_SUBSLICE_DISABLE _MMIO(0x913C)
#define GEN6_BSD_SLEEP_PSMI_CONTROL _MMIO(0x12050)
#define GEN6_BSD_SLEEP_MSG_DISABLE (1 << 0)
#define GEN6_BSD_SLEEP_FLUSH_DISABLE (1 << 2)
@ -3951,6 +3824,9 @@ enum {
#define _CLKGATE_DIS_PSL_A 0x46520
#define _CLKGATE_DIS_PSL_B 0x46524
#define _CLKGATE_DIS_PSL_C 0x46528
#define DUPS1_GATING_DIS (1 << 15)
#define DUPS2_GATING_DIS (1 << 19)
#define DUPS3_GATING_DIS (1 << 23)
#define DPF_GATING_DIS (1 << 10)
#define DPF_RAM_GATING_DIS (1 << 9)
#define DPFR_GATING_DIS (1 << 8)
@ -4151,6 +4027,12 @@ enum {
#define EDP_PSR_IDLE_FRAME_SHIFT 0
#define EDP_PSR_AUX_CTL _MMIO(dev_priv->psr_mmio_base + 0x10)
#define EDP_PSR_AUX_CTL_TIME_OUT_MASK (3 << 26)
#define EDP_PSR_AUX_CTL_MESSAGE_SIZE_MASK (0x1f << 20)
#define EDP_PSR_AUX_CTL_PRECHARGE_2US_MASK (0xf << 16)
#define EDP_PSR_AUX_CTL_ERROR_INTERRUPT (1 << 11)
#define EDP_PSR_AUX_CTL_BIT_CLOCK_2X_MASK (0x7ff)
#define EDP_PSR_AUX_DATA(i) _MMIO(dev_priv->psr_mmio_base + 0x14 + (i) * 4) /* 5 registers */
#define EDP_PSR_STATUS _MMIO(dev_priv->psr_mmio_base + 0x40)
@ -4180,17 +4062,19 @@ enum {
#define EDP_PSR_PERF_CNT _MMIO(dev_priv->psr_mmio_base + 0x44)
#define EDP_PSR_PERF_CNT_MASK 0xffffff
#define EDP_PSR_DEBUG _MMIO(dev_priv->psr_mmio_base + 0x60)
#define EDP_PSR_DEBUG _MMIO(dev_priv->psr_mmio_base + 0x60) /* PSR_MASK on SKL+ */
#define EDP_PSR_DEBUG_MASK_MAX_SLEEP (1<<28)
#define EDP_PSR_DEBUG_MASK_LPSP (1<<27)
#define EDP_PSR_DEBUG_MASK_MEMUP (1<<26)
#define EDP_PSR_DEBUG_MASK_HPD (1<<25)
#define EDP_PSR_DEBUG_MASK_DISP_REG_WRITE (1<<16)
#define EDP_PSR_DEBUG_EXIT_ON_PIXEL_UNDERRUN (1<<15)
#define EDP_PSR_DEBUG_EXIT_ON_PIXEL_UNDERRUN (1<<15) /* SKL+ */
#define EDP_PSR2_CTL _MMIO(0x6f900)
#define EDP_PSR2_ENABLE (1<<31)
#define EDP_SU_TRACK_ENABLE (1<<30)
#define EDP_Y_COORDINATE_VALID (1<<26) /* GLK and CNL+ */
#define EDP_Y_COORDINATE_ENABLE (1<<25) /* GLK and CNL+ */
#define EDP_MAX_SU_DISABLE_TIME(t) ((t)<<20)
#define EDP_MAX_SU_DISABLE_TIME_MASK (0x1f<<20)
#define EDP_PSR2_TP2_TIME_500 (0<<8)
@ -4200,8 +4084,9 @@ enum {
#define EDP_PSR2_TP2_TIME_MASK (3<<8)
#define EDP_PSR2_FRAME_BEFORE_SU_SHIFT 4
#define EDP_PSR2_FRAME_BEFORE_SU_MASK (0xf<<4)
#define EDP_PSR2_IDLE_MASK 0xf
#define EDP_PSR2_FRAME_BEFORE_SU(a) ((a)<<4)
#define EDP_PSR2_IDLE_FRAME_MASK 0xf
#define EDP_PSR2_IDLE_FRAME_SHIFT 0
#define EDP_PSR2_STATUS _MMIO(0x6f940)
#define EDP_PSR2_STATUS_STATE_MASK (0xf<<28)
@ -5265,8 +5150,6 @@ enum {
#define DP_LINK_TRAIN_OFF (3 << 28)
#define DP_LINK_TRAIN_MASK (3 << 28)
#define DP_LINK_TRAIN_SHIFT 28
#define DP_LINK_TRAIN_PAT_3_CHV (1 << 14)
#define DP_LINK_TRAIN_MASK_CHV ((3 << 28)|(1<<14))
/* CPT Link training mode */
#define DP_LINK_TRAIN_PAT_1_CPT (0 << 8)
@ -6009,6 +5892,7 @@ enum {
#define CURSIZE _MMIO(0x700a0) /* 845/865 */
#define _CUR_FBC_CTL_A 0x700a0 /* ivb+ */
#define CUR_FBC_CTL_EN (1 << 31)
#define _CURASURFLIVE 0x700ac /* g4x+ */
#define _CURBCNTR 0x700c0
#define _CURBBASE 0x700c4
#define _CURBPOS 0x700c8
@ -6025,6 +5909,7 @@ enum {
#define CURBASE(pipe) _CURSOR2(pipe, _CURABASE)
#define CURPOS(pipe) _CURSOR2(pipe, _CURAPOS)
#define CUR_FBC_CTL(pipe) _CURSOR2(pipe, _CUR_FBC_CTL_A)
#define CURSURFLIVE(pipe) _CURSOR2(pipe, _CURASURFLIVE)
#define CURSOR_A_OFFSET 0x70080
#define CURSOR_B_OFFSET 0x700c0
@ -6779,6 +6664,8 @@ enum {
#define PS_SCALER_MODE_MASK (3 << 28)
#define PS_SCALER_MODE_DYN (0 << 28)
#define PS_SCALER_MODE_HQ (1 << 28)
#define SKL_PS_SCALER_MODE_NV12 (2 << 28)
#define PS_SCALER_MODE_PLANAR (1 << 29)
#define PS_PLANE_SEL_MASK (7 << 25)
#define PS_PLANE_SEL(plane) (((plane) + 1) << 25)
#define PS_FILTER_MASK (3 << 23)
@ -7117,7 +7004,9 @@ enum {
#define GEN11_INTR_IDENTITY_REG0 _MMIO(0x190060)
#define GEN11_INTR_IDENTITY_REG1 _MMIO(0x190064)
#define GEN11_INTR_DATA_VALID (1 << 31)
#define GEN11_INTR_ENGINE_MASK (0xffff)
#define GEN11_INTR_ENGINE_CLASS(x) (((x) & GENMASK(18, 16)) >> 16)
#define GEN11_INTR_ENGINE_INSTANCE(x) (((x) & GENMASK(25, 20)) >> 20)
#define GEN11_INTR_ENGINE_INTR(x) ((x) & 0xffff)
#define GEN11_INTR_IDENTITY_REG(x) _MMIO(0x190060 + (x * 4))
@ -7197,6 +7086,7 @@ enum {
#define CHICKEN_TRANS_A 0x420c0
#define CHICKEN_TRANS_B 0x420c4
#define CHICKEN_TRANS(trans) _MMIO_TRANS(trans, CHICKEN_TRANS_A, CHICKEN_TRANS_B)
#define VSC_DATA_SEL_SOFTWARE_CONTROL (1<<25) /* GLK and CNL+ */
#define DDI_TRAINING_OVERRIDE_ENABLE (1<<19)
#define DDI_TRAINING_OVERRIDE_VALUE (1<<18)
#define DDIE_TRAINING_OVERRIDE_ENABLE (1<<17) /* CHICKEN_TRANS_A only */

125
drivers/gpu/drm/i915/i915_request.c
View File

@ -59,11 +59,7 @@ static bool i915_fence_signaled(struct dma_fence *fence)
static bool i915_fence_enable_signaling(struct dma_fence *fence)
{
if (i915_fence_signaled(fence))
return false;
intel_engine_enable_signaling(to_request(fence), true);
return !i915_fence_signaled(fence);
return intel_engine_enable_signaling(to_request(fence), true);
}
static signed long i915_fence_wait(struct dma_fence *fence,
@ -211,11 +207,19 @@ static int reset_all_global_seqno(struct drm_i915_private *i915, u32 seqno)
if (ret)
return ret;
GEM_BUG_ON(i915->gt.active_requests);
/* If the seqno wraps around, we need to clear the breadcrumb rbtree */
for_each_engine(engine, i915, id) {
struct i915_gem_timeline *timeline;
struct intel_timeline *tl = engine->timeline;
GEM_TRACE("%s seqno %d (current %d) -> %d\n",
engine->name,
tl->seqno,
intel_engine_get_seqno(engine),
seqno);
if (!i915_seqno_passed(seqno, tl->seqno)) {
/* Flush any waiters before we reuse the seqno */
intel_engine_disarm_breadcrumbs(engine);
@ -251,47 +255,6 @@ int i915_gem_set_global_seqno(struct drm_device *dev, u32 seqno)
return reset_all_global_seqno(i915, seqno - 1);
}
static void mark_busy(struct drm_i915_private *i915)
{
if (i915->gt.awake)
return;
GEM_BUG_ON(!i915->gt.active_requests);
intel_runtime_pm_get_noresume(i915);
/*
* It seems that the DMC likes to transition between the DC states a lot
* when there are no connected displays (no active power domains) during
* command submission.
*
* This activity has negative impact on the performance of the chip with
* huge latencies observed in the interrupt handler and elsewhere.
*
* Work around it by grabbing a GT IRQ power domain whilst there is any
* GT activity, preventing any DC state transitions.
*/
intel_display_power_get(i915, POWER_DOMAIN_GT_IRQ);
i915->gt.awake = true;
if (unlikely(++i915->gt.epoch == 0)) /* keep 0 as invalid */
i915->gt.epoch = 1;
intel_enable_gt_powersave(i915);
i915_update_gfx_val(i915);
if (INTEL_GEN(i915) >= 6)
gen6_rps_busy(i915);
i915_pmu_gt_unparked(i915);
intel_engines_unpark(i915);
i915_queue_hangcheck(i915);
queue_delayed_work(i915->wq,
&i915->gt.retire_work,
round_jiffies_up_relative(HZ));
}
static int reserve_engine(struct intel_engine_cs *engine)
{
struct drm_i915_private *i915 = engine->i915;
@ -309,7 +272,7 @@ static int reserve_engine(struct intel_engine_cs *engine)
}
if (!i915->gt.active_requests++)
mark_busy(i915);
i915_gem_unpark(i915);
return 0;
}
@ -318,13 +281,8 @@ static void unreserve_engine(struct intel_engine_cs *engine)
{
struct drm_i915_private *i915 = engine->i915;
if (!--i915->gt.active_requests) {
/* Cancel the mark_busy() from our reserve_engine() */
GEM_BUG_ON(!i915->gt.awake);
mod_delayed_work(i915->wq,
&i915->gt.idle_work,
msecs_to_jiffies(100));
}
if (!--i915->gt.active_requests)
i915_gem_park(i915);
GEM_BUG_ON(!engine->timeline->inflight_seqnos);
engine->timeline->inflight_seqnos--;
@ -358,7 +316,7 @@ static void advance_ring(struct i915_request *request)
* is just about to be. Either works, if we miss the last two
* noops - they are safe to be replayed on a reset.
*/
tail = READ_ONCE(request->ring->tail);
tail = READ_ONCE(request->tail);
} else {
tail = request->postfix;
}
@ -385,6 +343,12 @@ static void i915_request_retire(struct i915_request *request)
struct intel_engine_cs *engine = request->engine;
struct i915_gem_active *active, *next;
GEM_TRACE("%s fence %llx:%d, global=%d, current %d\n",
engine->name,
request->fence.context, request->fence.seqno,
request->global_seqno,
intel_engine_get_seqno(engine));
lockdep_assert_held(&request->i915->drm.struct_mutex);
GEM_BUG_ON(!i915_sw_fence_signaled(&request->submit));
GEM_BUG_ON(!i915_request_completed(request));
@ -486,21 +450,34 @@ static u32 timeline_get_seqno(struct intel_timeline *tl)
return ++tl->seqno;
}
static void move_to_timeline(struct i915_request *request,
struct intel_timeline *timeline)
{
GEM_BUG_ON(request->timeline == request->engine->timeline);
lockdep_assert_held(&request->engine->timeline->lock);
spin_lock(&request->timeline->lock);
list_move_tail(&request->link, &timeline->requests);
spin_unlock(&request->timeline->lock);
}
void __i915_request_submit(struct i915_request *request)
{
struct intel_engine_cs *engine = request->engine;
struct intel_timeline *timeline;
u32 seqno;
GEM_TRACE("%s fence %llx:%d -> global=%d, current %d\n",
engine->name,
request->fence.context, request->fence.seqno,
engine->timeline->seqno + 1,
intel_engine_get_seqno(engine));
GEM_BUG_ON(!irqs_disabled());
lockdep_assert_held(&engine->timeline->lock);
/* Transfer from per-context onto the global per-engine timeline */
timeline = engine->timeline;
GEM_BUG_ON(timeline == request->timeline);
GEM_BUG_ON(request->global_seqno);
seqno = timeline_get_seqno(timeline);
seqno = timeline_get_seqno(engine->timeline);
GEM_BUG_ON(!seqno);
GEM_BUG_ON(i915_seqno_passed(intel_engine_get_seqno(engine), seqno));
@ -514,9 +491,8 @@ void __i915_request_submit(struct i915_request *request)
engine->emit_breadcrumb(request,
request->ring->vaddr + request->postfix);
spin_lock(&request->timeline->lock);
list_move_tail(&request->link, &timeline->requests);
spin_unlock(&request->timeline->lock);
/* Transfer from per-context onto the global per-engine timeline */
move_to_timeline(request, engine->timeline);
trace_i915_request_execute(request);
@ -539,7 +515,12 @@ void i915_request_submit(struct i915_request *request)
void __i915_request_unsubmit(struct i915_request *request)
{
struct intel_engine_cs *engine = request->engine;
struct intel_timeline *timeline;
GEM_TRACE("%s fence %llx:%d <- global=%d, current %d\n",
engine->name,
request->fence.context, request->fence.seqno,
request->global_seqno,
intel_engine_get_seqno(engine));
GEM_BUG_ON(!irqs_disabled());
lockdep_assert_held(&engine->timeline->lock);
@ -562,12 +543,7 @@ void __i915_request_unsubmit(struct i915_request *request)
spin_unlock(&request->lock);
/* Transfer back from the global per-engine timeline to per-context */
timeline = request->timeline;
GEM_BUG_ON(timeline == engine->timeline);
spin_lock(&timeline->lock);
list_move(&request->link, &timeline->requests);
spin_unlock(&timeline->lock);
move_to_timeline(request, request->timeline);
/*
* We don't need to wake_up any waiters on request->execute, they
@ -1000,6 +976,9 @@ void __i915_request_add(struct i915_request *request, bool flush_caches)
u32 *cs;
int err;
GEM_TRACE("%s fence %llx:%d\n",
engine->name, request->fence.context, request->fence.seqno);
lockdep_assert_held(&request->i915->drm.struct_mutex);
trace_i915_request_add(request);
@ -1206,11 +1185,13 @@ static bool __i915_spin_request(const struct i915_request *rq,
static bool __i915_wait_request_check_and_reset(struct i915_request *request)
{
if (likely(!i915_reset_handoff(&request->i915->gpu_error)))
struct i915_gpu_error *error = &request->i915->gpu_error;
if (likely(!i915_reset_handoff(error)))
return false;
__set_current_state(TASK_RUNNING);
i915_reset(request->i915, 0);
i915_reset(request->i915, error->stalled_mask, error->reason);
return true;
}

4
drivers/gpu/drm/i915/i915_utils.h
View File

@ -40,8 +40,8 @@
#undef WARN_ON_ONCE
#define WARN_ON_ONCE(x) WARN_ONCE((x), "%s", "WARN_ON_ONCE(" __stringify(x) ")")
#define MISSING_CASE(x) WARN(1, "Missing switch case (%lu) in %s\n", \
(long)(x), __func__)
#define MISSING_CASE(x) WARN(1, "Missing case (%s == %ld)\n", \
__stringify(x), (long)(x))
#if GCC_VERSION >= 70000
#define add_overflows(A, B) \

19
drivers/gpu/drm/i915/intel_atomic.c
View File

@ -227,6 +227,7 @@ int intel_atomic_setup_scalers(struct drm_i915_private *dev_priv,
struct intel_crtc_scaler_state *scaler_state =
&crtc_state->scaler_state;
struct drm_atomic_state *drm_state = crtc_state->base.state;
struct intel_atomic_state *intel_state = to_intel_atomic_state(drm_state);
int num_scalers_need;
int i, j;
@ -304,8 +305,8 @@ int intel_atomic_setup_scalers(struct drm_i915_private *dev_priv,
continue;
}
plane_state = intel_atomic_get_existing_plane_state(drm_state,
intel_plane);
plane_state = intel_atomic_get_new_plane_state(intel_state,
intel_plane);
scaler_id = &plane_state->scaler_id;
}
@ -328,8 +329,18 @@ int intel_atomic_setup_scalers(struct drm_i915_private *dev_priv,
}
/* set scaler mode */
if (IS_GEMINILAKE(dev_priv) || IS_CANNONLAKE(dev_priv)) {
scaler_state->scalers[*scaler_id].mode = 0;
if ((INTEL_GEN(dev_priv) >= 9) &&
plane_state && plane_state->base.fb &&
plane_state->base.fb->format->format ==
DRM_FORMAT_NV12) {
if (INTEL_GEN(dev_priv) == 9 &&
!IS_GEMINILAKE(dev_priv) &&
!IS_SKYLAKE(dev_priv))
scaler_state->scalers[*scaler_id].mode =
SKL_PS_SCALER_MODE_NV12;
else
scaler_state->scalers[*scaler_id].mode =
PS_SCALER_MODE_PLANAR;
} else if (num_scalers_need == 1 && intel_crtc->pipe != PIPE_C) {
/*
* when only 1 scaler is in use on either pipe A or B,

17
drivers/gpu/drm/i915/intel_bios.c
View File

@ -1215,10 +1215,8 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
{
struct child_device_config *it, *child = NULL;
struct ddi_vbt_port_info *info = &dev_priv->vbt.ddi_port_info[port];
uint8_t hdmi_level_shift;
int i, j;
bool is_dvi, is_hdmi, is_dp, is_edp, is_crt;
uint8_t aux_channel, ddc_pin;
/* Each DDI port can have more than one value on the "DVO Port" field,
* so look for all the possible values for each port.
*/
@ -1255,8 +1253,6 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
if (!child)
return;
aux_channel = child->aux_channel;
is_dvi = child->device_type & DEVICE_TYPE_TMDS_DVI_SIGNALING;
is_dp = child->device_type & DEVICE_TYPE_DISPLAYPORT_OUTPUT;
is_crt = child->device_type & DEVICE_TYPE_ANALOG_OUTPUT;
@ -1270,13 +1266,6 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
is_hdmi = false;
}
if (port == PORT_A && is_dvi) {
DRM_DEBUG_KMS("VBT claims port A supports DVI%s, ignoring\n",
is_hdmi ? "/HDMI" : "");
is_dvi = false;
is_hdmi = false;
}
info->supports_dvi = is_dvi;
info->supports_hdmi = is_hdmi;
info->supports_dp = is_dp;
@ -1302,6 +1291,8 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
DRM_DEBUG_KMS("Port %c is internal DP\n", port_name(port));
if (is_dvi) {
u8 ddc_pin;
ddc_pin = map_ddc_pin(dev_priv, child->ddc_pin);
if (intel_gmbus_is_valid_pin(dev_priv, ddc_pin)) {
info->alternate_ddc_pin = ddc_pin;
@ -1314,14 +1305,14 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
}
if (is_dp) {
info->alternate_aux_channel = aux_channel;
info->alternate_aux_channel = child->aux_channel;
sanitize_aux_ch(dev_priv, port);
}
if (bdb_version >= 158) {
/* The VBT HDMI level shift values match the table we have. */
hdmi_level_shift = child->hdmi_level_shifter_value;
u8 hdmi_level_shift = child->hdmi_level_shifter_value;
DRM_DEBUG_KMS("VBT HDMI level shift for port %c: %d\n",
port_name(port),
hdmi_level_shift);

21
drivers/gpu/drm/i915/intel_breadcrumbs.c
View File

@ -730,10 +730,11 @@ static void insert_signal(struct intel_breadcrumbs *b,
list_add(&request->signaling.link, &iter->signaling.link);
}
void intel_engine_enable_signaling(struct i915_request *request, bool wakeup)
bool intel_engine_enable_signaling(struct i915_request *request, bool wakeup)
{
struct intel_engine_cs *engine = request->engine;
struct intel_breadcrumbs *b = &engine->breadcrumbs;
struct intel_wait *wait = &request->signaling.wait;
u32 seqno;
/*
@ -750,12 +751,12 @@ void intel_engine_enable_signaling(struct i915_request *request, bool wakeup)
seqno = i915_request_global_seqno(request);
if (!seqno) /* will be enabled later upon execution */
return;
return true;
GEM_BUG_ON(request->signaling.wait.seqno);
request->signaling.wait.tsk = b->signaler;
request->signaling.wait.request = request;
request->signaling.wait.seqno = seqno;
GEM_BUG_ON(wait->seqno);
wait->tsk = b->signaler;
wait->request = request;
wait->seqno = seqno;
/*
* Add ourselves into the list of waiters, but registering our
@ -768,11 +769,15 @@ void intel_engine_enable_signaling(struct i915_request *request, bool wakeup)
*/
spin_lock(&b->rb_lock);
insert_signal(b, request, seqno);
wakeup &= __intel_engine_add_wait(engine, &request->signaling.wait);
wakeup &= __intel_engine_add_wait(engine, wait);
spin_unlock(&b->rb_lock);
if (wakeup)
if (wakeup) {
wake_up_process(b->signaler);
return !intel_wait_complete(wait);
}
return true;
}
void intel_engine_cancel_signaling(struct i915_request *request)

154
drivers/gpu/drm/i915/intel_ddi.c
View File

@ -493,6 +493,125 @@ static const struct cnl_ddi_buf_trans cnl_ddi_translations_edp_1_05V[] = {
{ 0x2, 0x7F, 0x3F, 0x00, 0x00 }, /* 400 400 0.0 */
};
struct icl_combo_phy_ddi_buf_trans {
u32 dw2_swing_select;
u32 dw2_swing_scalar;
u32 dw4_scaling;
};
/* Voltage Swing Programming for VccIO 0.85V for DP */
static const struct icl_combo_phy_ddi_buf_trans icl_combo_phy_ddi_translations_dp_hdmi_0_85V[] = {
/* Voltage mV db */
{ 0x2, 0x98, 0x0018 }, /* 400 0.0 */
{ 0x2, 0x98, 0x3015 }, /* 400 3.5 */
{ 0x2, 0x98, 0x6012 }, /* 400 6.0 */
{ 0x2, 0x98, 0x900F }, /* 400 9.5 */
{ 0xB, 0x70, 0x0018 }, /* 600 0.0 */
{ 0xB, 0x70, 0x3015 }, /* 600 3.5 */
{ 0xB, 0x70, 0x6012 }, /* 600 6.0 */
{ 0x5, 0x00, 0x0018 }, /* 800 0.0 */
{ 0x5, 0x00, 0x3015 }, /* 800 3.5 */
{ 0x6, 0x98, 0x0018 }, /* 1200 0.0 */
};
/* FIXME - After table is updated in Bspec */
/* Voltage Swing Programming for VccIO 0.85V for eDP */
static const struct icl_combo_phy_ddi_buf_trans icl_combo_phy_ddi_translations_edp_0_85V[] = {
/* Voltage mV db */
{ 0x0, 0x00, 0x00 }, /* 200 0.0 */
{ 0x0, 0x00, 0x00 }, /* 200 1.5 */
{ 0x0, 0x00, 0x00 }, /* 200 4.0 */
{ 0x0, 0x00, 0x00 }, /* 200 6.0 */
{ 0x0, 0x00, 0x00 }, /* 250 0.0 */
{ 0x0, 0x00, 0x00 }, /* 250 1.5 */
{ 0x0, 0x00, 0x00 }, /* 250 4.0 */
{ 0x0, 0x00, 0x00 }, /* 300 0.0 */
{ 0x0, 0x00, 0x00 }, /* 300 1.5 */
{ 0x0, 0x00, 0x00 }, /* 350 0.0 */
};
/* Voltage Swing Programming for VccIO 0.95V for DP */
static const struct icl_combo_phy_ddi_buf_trans icl_combo_phy_ddi_translations_dp_hdmi_0_95V[] = {
/* Voltage mV db */
{ 0x2, 0x98, 0x0018 }, /* 400 0.0 */
{ 0x2, 0x98, 0x3015 }, /* 400 3.5 */
{ 0x2, 0x98, 0x6012 }, /* 400 6.0 */
{ 0x2, 0x98, 0x900F }, /* 400 9.5 */
{ 0x4, 0x98, 0x0018 }, /* 600 0.0 */
{ 0x4, 0x98, 0x3015 }, /* 600 3.5 */
{ 0x4, 0x98, 0x6012 }, /* 600 6.0 */
{ 0x5, 0x76, 0x0018 }, /* 800 0.0 */
{ 0x5, 0x76, 0x3015 }, /* 800 3.5 */
{ 0x6, 0x98, 0x0018 }, /* 1200 0.0 */
};
/* FIXME - After table is updated in Bspec */
/* Voltage Swing Programming for VccIO 0.95V for eDP */
static const struct icl_combo_phy_ddi_buf_trans icl_combo_phy_ddi_translations_edp_0_95V[] = {
/* Voltage mV db */
{ 0x0, 0x00, 0x00 }, /* 200 0.0 */
{ 0x0, 0x00, 0x00 }, /* 200 1.5 */
{ 0x0, 0x00, 0x00 }, /* 200 4.0 */
{ 0x0, 0x00, 0x00 }, /* 200 6.0 */
{ 0x0, 0x00, 0x00 }, /* 250 0.0 */
{ 0x0, 0x00, 0x00 }, /* 250 1.5 */
{ 0x0, 0x00, 0x00 }, /* 250 4.0 */
{ 0x0, 0x00, 0x00 }, /* 300 0.0 */
{ 0x0, 0x00, 0x00 }, /* 300 1.5 */
{ 0x0, 0x00, 0x00 }, /* 350 0.0 */
};
/* Voltage Swing Programming for VccIO 1.05V for DP */
static const struct icl_combo_phy_ddi_buf_trans icl_combo_phy_ddi_translations_dp_hdmi_1_05V[] = {
/* Voltage mV db */
{ 0x2, 0x98, 0x0018 }, /* 400 0.0 */
{ 0x2, 0x98, 0x3015 }, /* 400 3.5 */
{ 0x2, 0x98, 0x6012 }, /* 400 6.0 */
{ 0x2, 0x98, 0x900F }, /* 400 9.5 */
{ 0x4, 0x98, 0x0018 }, /* 600 0.0 */
{ 0x4, 0x98, 0x3015 }, /* 600 3.5 */
{ 0x4, 0x98, 0x6012 }, /* 600 6.0 */
{ 0x5, 0x71, 0x0018 }, /* 800 0.0 */
{ 0x5, 0x71, 0x3015 }, /* 800 3.5 */
{ 0x6, 0x98, 0x0018 }, /* 1200 0.0 */
};
/* FIXME - After table is updated in Bspec */
/* Voltage Swing Programming for VccIO 1.05V for eDP */
static const struct icl_combo_phy_ddi_buf_trans icl_combo_phy_ddi_translations_edp_1_05V[] = {
/* Voltage mV db */
{ 0x0, 0x00, 0x00 }, /* 200 0.0 */
{ 0x0, 0x00, 0x00 }, /* 200 1.5 */
{ 0x0, 0x00, 0x00 }, /* 200 4.0 */
{ 0x0, 0x00, 0x00 }, /* 200 6.0 */
{ 0x0, 0x00, 0x00 }, /* 250 0.0 */
{ 0x0, 0x00, 0x00 }, /* 250 1.5 */
{ 0x0, 0x00, 0x00 }, /* 250 4.0 */
{ 0x0, 0x00, 0x00 }, /* 300 0.0 */
{ 0x0, 0x00, 0x00 }, /* 300 1.5 */
{ 0x0, 0x00, 0x00 }, /* 350 0.0 */
};
struct icl_mg_phy_ddi_buf_trans {
u32 cri_txdeemph_override_5_0;
u32 cri_txdeemph_override_11_6;
u32 cri_txdeemph_override_17_12;
};
static const struct icl_mg_phy_ddi_buf_trans icl_mg_phy_ddi_translations[] = {
/* Voltage swing pre-emphasis */
{ 0x0, 0x1B, 0x00 }, /* 0 0 */
{ 0x0, 0x23, 0x08 }, /* 0 1 */
{ 0x0, 0x2D, 0x12 }, /* 0 2 */
{ 0x0, 0x00, 0x00 }, /* 0 3 */
{ 0x0, 0x23, 0x00 }, /* 1 0 */
{ 0x0, 0x2B, 0x09 }, /* 1 1 */
{ 0x0, 0x2E, 0x11 }, /* 1 2 */
{ 0x0, 0x2F, 0x00 }, /* 2 0 */
{ 0x0, 0x33, 0x0C }, /* 2 1 */
{ 0x0, 0x00, 0x00 }, /* 3 0 */
};
static const struct ddi_buf_trans *
bdw_get_buf_trans_edp(struct drm_i915_private *dev_priv, int *n_entries)
{
@ -875,7 +994,7 @@ static void intel_wait_ddi_buf_idle(struct drm_i915_private *dev_priv,
static uint32_t hsw_pll_to_ddi_pll_sel(const struct intel_shared_dpll *pll)
{
switch (pll->id) {
switch (pll->info->id) {
case DPLL_ID_WRPLL1:
return PORT_CLK_SEL_WRPLL1;
case DPLL_ID_WRPLL2:
@ -889,7 +1008,7 @@ static uint32_t hsw_pll_to_ddi_pll_sel(const struct intel_shared_dpll *pll)
case DPLL_ID_LCPLL_2700:
return PORT_CLK_SEL_LCPLL_2700;
default:
MISSING_CASE(pll->id);
MISSING_CASE(pll->info->id);
return PORT_CLK_SEL_NONE;
}
}
@ -2131,7 +2250,7 @@ static void intel_ddi_clk_select(struct intel_encoder *encoder,
/* Configure DPCLKA_CFGCR0 to map the DPLL to the DDI. */
val = I915_READ(DPCLKA_CFGCR0);
val &= ~DPCLKA_CFGCR0_DDI_CLK_SEL_MASK(port);
val |= DPCLKA_CFGCR0_DDI_CLK_SEL(pll->id, port);
val |= DPCLKA_CFGCR0_DDI_CLK_SEL(pll->info->id, port);
I915_WRITE(DPCLKA_CFGCR0, val);
/*
@ -2148,7 +2267,7 @@ static void intel_ddi_clk_select(struct intel_encoder *encoder,
val &= ~(DPLL_CTRL2_DDI_CLK_OFF(port) |
DPLL_CTRL2_DDI_CLK_SEL_MASK(port));
val |= (DPLL_CTRL2_DDI_CLK_SEL(pll->id, port) |
val |= (DPLL_CTRL2_DDI_CLK_SEL(pll->info->id, port) |
DPLL_CTRL2_DDI_SEL_OVERRIDE(port));
I915_WRITE(DPLL_CTRL2, val);
@ -2205,7 +2324,8 @@ static void intel_ddi_pre_enable_dp(struct intel_encoder *encoder,
intel_prepare_dp_ddi_buffers(encoder, crtc_state);
intel_ddi_init_dp_buf_reg(encoder);
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_ON);
if (!is_mst)
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_ON);
intel_dp_start_link_train(intel_dp);
if (port != PORT_A || INTEL_GEN(dev_priv) >= 9)
intel_dp_stop_link_train(intel_dp);
@ -2303,12 +2423,15 @@ static void intel_ddi_post_disable_dp(struct intel_encoder *encoder,
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
struct intel_dp *intel_dp = &dig_port->dp;
bool is_mst = intel_crtc_has_type(old_crtc_state,
INTEL_OUTPUT_DP_MST);
/*
* Power down sink before disabling the port, otherwise we end
* up getting interrupts from the sink on detecting link loss.
*/
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);
if (!is_mst)
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);
intel_disable_ddi_buf(encoder);
@ -2424,12 +2547,14 @@ static void intel_enable_ddi_hdmi(struct intel_encoder *encoder,
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
struct drm_connector *connector = conn_state->connector;
enum port port = encoder->port;
intel_hdmi_handle_sink_scrambling(encoder,
conn_state->connector,
crtc_state->hdmi_high_tmds_clock_ratio,
crtc_state->hdmi_scrambling);
if (!intel_hdmi_handle_sink_scrambling(encoder, connector,
crtc_state->hdmi_high_tmds_clock_ratio,
crtc_state->hdmi_scrambling))
DRM_ERROR("[CONNECTOR:%d:%s] Failed to configure sink scrambling/TMDS bit clock ratio\n",
connector->base.id, connector->name);
/* Display WA #1143: skl,kbl,cfl */
if (IS_GEN9_BC(dev_priv)) {
@ -2520,13 +2645,16 @@ static void intel_disable_ddi_hdmi(struct intel_encoder *encoder,
const struct intel_crtc_state *old_crtc_state,
const struct drm_connector_state *old_conn_state)
{
struct drm_connector *connector = old_conn_state->connector;
if (old_crtc_state->has_audio)
intel_audio_codec_disable(encoder,
old_crtc_state, old_conn_state);
intel_hdmi_handle_sink_scrambling(encoder,
old_conn_state->connector,
false, false);
if (!intel_hdmi_handle_sink_scrambling(encoder, connector,
false, false))
DRM_DEBUG_KMS("[CONNECTOR:%d:%s] Failed to reset sink scrambling/TMDS bit clock ratio\n",
connector->base.id, connector->name);
}
static void intel_disable_ddi(struct intel_encoder *encoder,

167
drivers/gpu/drm/i915/intel_device_info.c
View File

@ -83,11 +83,11 @@ static void sseu_dump(const struct sseu_dev_info *sseu, struct drm_printer *p)
{
int s;
drm_printf(p, "slice mask: %04x\n", sseu->slice_mask);
drm_printf(p, "slice total: %u\n", hweight8(sseu->slice_mask));
drm_printf(p, "slice total: %u, mask=%04x\n",
hweight8(sseu->slice_mask), sseu->slice_mask);
drm_printf(p, "subslice total: %u\n", sseu_subslice_total(sseu));
for (s = 0; s < ARRAY_SIZE(sseu->subslice_mask); s++) {
drm_printf(p, "slice%d %u subslices mask=%04x\n",
for (s = 0; s < sseu->max_slices; s++) {
drm_printf(p, "slice%d: %u subslices, mask=%04x\n",
s, hweight8(sseu->subslice_mask[s]),
sseu->subslice_mask[s]);
}
@ -158,6 +158,45 @@ static u16 compute_eu_total(const struct sseu_dev_info *sseu)
return total;
}
static void gen11_sseu_info_init(struct drm_i915_private *dev_priv)
{
struct sseu_dev_info *sseu = &mkwrite_device_info(dev_priv)->sseu;
u8 s_en;
u32 ss_en, ss_en_mask;
u8 eu_en;
int s;
sseu->max_slices = 1;
sseu->max_subslices = 8;
sseu->max_eus_per_subslice = 8;
s_en = I915_READ(GEN11_GT_SLICE_ENABLE) & GEN11_GT_S_ENA_MASK;
ss_en = ~I915_READ(GEN11_GT_SUBSLICE_DISABLE);
ss_en_mask = BIT(sseu->max_subslices) - 1;
eu_en = ~(I915_READ(GEN11_EU_DISABLE) & GEN11_EU_DIS_MASK);
for (s = 0; s < sseu->max_slices; s++) {
if (s_en & BIT(s)) {
int ss_idx = sseu->max_subslices * s;
int ss;
sseu->slice_mask |= BIT(s);
sseu->subslice_mask[s] = (ss_en >> ss_idx) & ss_en_mask;
for (ss = 0; ss < sseu->max_subslices; ss++) {
if (sseu->subslice_mask[s] & BIT(ss))
sseu_set_eus(sseu, s, ss, eu_en);
}
}
}
sseu->eu_per_subslice = hweight8(eu_en);
sseu->eu_total = compute_eu_total(sseu);
/* ICL has no power gating restrictions. */
sseu->has_slice_pg = 1;
sseu->has_subslice_pg = 1;
sseu->has_eu_pg = 1;
}
static void gen10_sseu_info_init(struct drm_i915_private *dev_priv)
{
struct sseu_dev_info *sseu = &mkwrite_device_info(dev_priv)->sseu;
@ -557,6 +596,52 @@ static u32 read_reference_ts_freq(struct drm_i915_private *dev_priv)
return base_freq + frac_freq;
}
static u32 gen10_get_crystal_clock_freq(struct drm_i915_private *dev_priv,
u32 rpm_config_reg)
{
u32 f19_2_mhz = 19200;
u32 f24_mhz = 24000;
u32 crystal_clock = (rpm_config_reg &
GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>
GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;
switch (crystal_clock) {
case GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_19_2_MHZ:
return f19_2_mhz;
case GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_24_MHZ:
return f24_mhz;
default:
MISSING_CASE(crystal_clock);
return 0;
}
}
static u32 gen11_get_crystal_clock_freq(struct drm_i915_private *dev_priv,
u32 rpm_config_reg)
{
u32 f19_2_mhz = 19200;
u32 f24_mhz = 24000;
u32 f25_mhz = 25000;
u32 f38_4_mhz = 38400;
u32 crystal_clock = (rpm_config_reg &
GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>
GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;
switch (crystal_clock) {
case GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_24_MHZ:
return f24_mhz;
case GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_19_2_MHZ:
return f19_2_mhz;
case GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_38_4_MHZ:
return f38_4_mhz;
case GEN11_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_25_MHZ:
return f25_mhz;
default:
MISSING_CASE(crystal_clock);
return 0;
}
}
static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv)
{
u32 f12_5_mhz = 12500;
@ -597,10 +682,9 @@ static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv)
}
return freq;
} else if (INTEL_GEN(dev_priv) <= 10) {
} else if (INTEL_GEN(dev_priv) <= 11) {
u32 ctc_reg = I915_READ(CTC_MODE);
u32 freq = 0;
u32 rpm_config_reg = 0;
/* First figure out the reference frequency. There are 2 ways
* we can compute the frequency, either through the
@ -610,20 +694,14 @@ static u32 read_timestamp_frequency(struct drm_i915_private *dev_priv)
if ((ctc_reg & CTC_SOURCE_PARAMETER_MASK) == CTC_SOURCE_DIVIDE_LOGIC) {
freq = read_reference_ts_freq(dev_priv);
} else {
u32 crystal_clock;
u32 rpm_config_reg = I915_READ(RPM_CONFIG0);
rpm_config_reg = I915_READ(RPM_CONFIG0);
crystal_clock = (rpm_config_reg &
GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_MASK) >>
GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_SHIFT;
switch (crystal_clock) {
case GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_19_2_MHZ:
freq = f19_2_mhz;
break;
case GEN9_RPM_CONFIG0_CRYSTAL_CLOCK_FREQ_24_MHZ:
freq = f24_mhz;
break;
}
if (INTEL_GEN(dev_priv) <= 10)
freq = gen10_get_crystal_clock_freq(dev_priv,
rpm_config_reg);
else
freq = gen11_get_crystal_clock_freq(dev_priv,
rpm_config_reg);
/* Now figure out how the command stream's timestamp
* register increments from this frequency (it might
@ -768,8 +846,10 @@ void intel_device_info_runtime_init(struct intel_device_info *info)
broadwell_sseu_info_init(dev_priv);
else if (INTEL_GEN(dev_priv) == 9)
gen9_sseu_info_init(dev_priv);
else if (INTEL_GEN(dev_priv) >= 10)
else if (INTEL_GEN(dev_priv) == 10)
gen10_sseu_info_init(dev_priv);
else if (INTEL_INFO(dev_priv)->gen >= 11)
gen11_sseu_info_init(dev_priv);
/* Initialize command stream timestamp frequency */
info->cs_timestamp_frequency_khz = read_timestamp_frequency(dev_priv);
@ -780,3 +860,50 @@ void intel_driver_caps_print(const struct intel_driver_caps *caps,
{
drm_printf(p, "scheduler: %x\n", caps->scheduler);
}
/*
* Determine which engines are fused off in our particular hardware. Since the
* fuse register is in the blitter powerwell, we need forcewake to be ready at
* this point (but later we need to prune the forcewake domains for engines that
* are indeed fused off).
*/
void intel_device_info_init_mmio(struct drm_i915_private *dev_priv)
{
struct intel_device_info *info = mkwrite_device_info(dev_priv);
u8 vdbox_disable, vebox_disable;
u32 media_fuse;
int i;
if (INTEL_GEN(dev_priv) < 11)
return;
media_fuse = I915_READ(GEN11_GT_VEBOX_VDBOX_DISABLE);
vdbox_disable = media_fuse & GEN11_GT_VDBOX_DISABLE_MASK;
vebox_disable = (media_fuse & GEN11_GT_VEBOX_DISABLE_MASK) >>
GEN11_GT_VEBOX_DISABLE_SHIFT;
DRM_DEBUG_DRIVER("vdbox disable: %04x\n", vdbox_disable);
for (i = 0; i < I915_MAX_VCS; i++) {
if (!HAS_ENGINE(dev_priv, _VCS(i)))
continue;
if (!(BIT(i) & vdbox_disable))
continue;
info->ring_mask &= ~ENGINE_MASK(_VCS(i));
DRM_DEBUG_DRIVER("vcs%u fused off\n", i);
}
DRM_DEBUG_DRIVER("vebox disable: %04x\n", vebox_disable);
for (i = 0; i < I915_MAX_VECS; i++) {
if (!HAS_ENGINE(dev_priv, _VECS(i)))
continue;
if (!(BIT(i) & vebox_disable))
continue;
info->ring_mask &= ~ENGINE_MASK(_VECS(i));
DRM_DEBUG_DRIVER("vecs%u fused off\n", i);
}
}

4
drivers/gpu/drm/i915/intel_device_info.h
View File

@ -114,7 +114,7 @@ enum intel_platform {
func(has_ipc);
#define GEN_MAX_SLICES (6) /* CNL upper bound */
#define GEN_MAX_SUBSLICES (7)
#define GEN_MAX_SUBSLICES (8) /* ICL upper bound */
struct sseu_dev_info {
u8 slice_mask;
@ -247,6 +247,8 @@ void intel_device_info_dump_runtime(const struct intel_device_info *info,
void intel_device_info_dump_topology(const struct sseu_dev_info *sseu,
struct drm_printer *p);
void intel_device_info_init_mmio(struct drm_i915_private *dev_priv);
void intel_driver_caps_print(const struct intel_driver_caps *caps,
struct drm_printer *p);

251
drivers/gpu/drm/i915/intel_display.c
View File

@ -488,6 +488,21 @@ static const struct intel_limit intel_limits_bxt = {
.p2 = { .p2_slow = 1, .p2_fast = 20 },
};
static void
skl_wa_clkgate(struct drm_i915_private *dev_priv, int pipe, bool enable)
{
if (IS_SKYLAKE(dev_priv))
return;
if (enable)
I915_WRITE(CLKGATE_DIS_PSL(pipe),
DUPS1_GATING_DIS | DUPS2_GATING_DIS);
else
I915_WRITE(CLKGATE_DIS_PSL(pipe),
I915_READ(CLKGATE_DIS_PSL(pipe)) &
~(DUPS1_GATING_DIS | DUPS2_GATING_DIS));
}
static bool
needs_modeset(const struct drm_crtc_state *state)
{
@ -2657,11 +2672,13 @@ static int i9xx_format_to_fourcc(int format)
}
}
static int skl_format_to_fourcc(int format, bool rgb_order, bool alpha)
int skl_format_to_fourcc(int format, bool rgb_order, bool alpha)
{
switch (format) {
case PLANE_CTL_FORMAT_RGB_565:
return DRM_FORMAT_RGB565;
case PLANE_CTL_FORMAT_NV12:
return DRM_FORMAT_NV12;
default:
case PLANE_CTL_FORMAT_XRGB_8888:
if (rgb_order) {
@ -2858,6 +2875,9 @@ valid_fb:
return;
}
obj = intel_fb_obj(fb);
intel_fb_obj_flush(obj, ORIGIN_DIRTYFB);
plane_state->src_x = 0;
plane_state->src_y = 0;
plane_state->src_w = fb->width << 16;
@ -2871,7 +2891,6 @@ valid_fb:
intel_state->base.src = drm_plane_state_src(plane_state);
intel_state->base.dst = drm_plane_state_dest(plane_state);
obj = intel_fb_obj(fb);
if (i915_gem_object_is_tiled(obj))
dev_priv->preserve_bios_swizzle = true;
@ -3464,6 +3483,8 @@ static u32 skl_plane_ctl_format(uint32_t pixel_format)
return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_UYVY;
case DRM_FORMAT_VYUY:
return PLANE_CTL_FORMAT_YUV422 | PLANE_CTL_YUV422_VYUY;
case DRM_FORMAT_NV12:
return PLANE_CTL_FORMAT_NV12;
default:
MISSING_CASE(pixel_format);
}
@ -3611,6 +3632,11 @@ u32 glk_plane_color_ctl(const struct intel_crtc_state *crtc_state,
plane_color_ctl |= glk_plane_color_ctl_alpha(fb->format->format);
if (intel_format_is_yuv(fb->format->format)) {
if (fb->format->format == DRM_FORMAT_NV12) {
plane_color_ctl |=
PLANE_COLOR_CSC_MODE_YUV709_TO_RGB709;
goto out;
}
if (plane_state->base.color_encoding == DRM_COLOR_YCBCR_BT709)
plane_color_ctl |= PLANE_COLOR_CSC_MODE_YUV709_TO_RGB709;
else
@ -3619,7 +3645,7 @@ u32 glk_plane_color_ctl(const struct intel_crtc_state *crtc_state,
if (plane_state->base.color_range == DRM_COLOR_YCBCR_FULL_RANGE)
plane_color_ctl |= PLANE_COLOR_YUV_RANGE_CORRECTION_DISABLE;
}
out:
return plane_color_ctl;
}
@ -3675,7 +3701,6 @@ void intel_prepare_reset(struct drm_i915_private *dev_priv)
struct drm_atomic_state *state;
int ret;
/* reset doesn't touch the display */
if (!i915_modparams.force_reset_modeset_test &&
!gpu_reset_clobbers_display(dev_priv))
@ -3729,19 +3754,17 @@ void intel_finish_reset(struct drm_i915_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
struct drm_modeset_acquire_ctx *ctx = &dev_priv->reset_ctx;
struct drm_atomic_state *state = dev_priv->modeset_restore_state;
struct drm_atomic_state *state;
int ret;
/* reset doesn't touch the display */
if (!i915_modparams.force_reset_modeset_test &&
!gpu_reset_clobbers_display(dev_priv))
if (!test_bit(I915_RESET_MODESET, &dev_priv->gpu_error.flags))
return;
state = fetch_and_zero(&dev_priv->modeset_restore_state);
if (!state)
goto unlock;
dev_priv->modeset_restore_state = NULL;
/* reset doesn't touch the display */
if (!gpu_reset_clobbers_display(dev_priv)) {
/* for testing only restore the display */
@ -4703,7 +4726,9 @@ static void cpt_verify_modeset(struct drm_device *dev, int pipe)
static int
skl_update_scaler(struct intel_crtc_state *crtc_state, bool force_detach,
unsigned int scaler_user, int *scaler_id,
int src_w, int src_h, int dst_w, int dst_h)
int src_w, int src_h, int dst_w, int dst_h,
bool plane_scaler_check,
uint32_t pixel_format)
{
struct intel_crtc_scaler_state *scaler_state =
&crtc_state->scaler_state;
@ -4721,6 +4746,10 @@ skl_update_scaler(struct intel_crtc_state *crtc_state, bool force_detach,
*/
need_scaling = src_w != dst_w || src_h != dst_h;
if (plane_scaler_check)
if (pixel_format == DRM_FORMAT_NV12)
need_scaling = true;
if (crtc_state->ycbcr420 && scaler_user == SKL_CRTC_INDEX)
need_scaling = true;
@ -4760,12 +4789,22 @@ skl_update_scaler(struct intel_crtc_state *crtc_state, bool force_detach,
return 0;
}
if (plane_scaler_check && pixel_format == DRM_FORMAT_NV12 &&
(src_h < SKL_MIN_YUV_420_SRC_H || (src_w % 4) != 0 ||
(src_h % 4) != 0)) {
DRM_DEBUG_KMS("NV12: src dimensions not met\n");
return -EINVAL;
}
/* range checks */
if (src_w < SKL_MIN_SRC_W || src_h < SKL_MIN_SRC_H ||
dst_w < SKL_MIN_DST_W || dst_h < SKL_MIN_DST_H ||
src_w > SKL_MAX_SRC_W || src_h > SKL_MAX_SRC_H ||
dst_w > SKL_MAX_DST_W || dst_h > SKL_MAX_DST_H) {
dst_w < SKL_MIN_DST_W || dst_h < SKL_MIN_DST_H ||
(IS_GEN11(dev_priv) &&
(src_w > ICL_MAX_SRC_W || src_h > ICL_MAX_SRC_H ||
dst_w > ICL_MAX_DST_W || dst_h > ICL_MAX_DST_H)) ||
(!IS_GEN11(dev_priv) &&
(src_w > SKL_MAX_SRC_W || src_h > SKL_MAX_SRC_H ||
dst_w > SKL_MAX_DST_W || dst_h > SKL_MAX_DST_H))) {
DRM_DEBUG_KMS("scaler_user index %u.%u: src %ux%u dst %ux%u "
"size is out of scaler range\n",
intel_crtc->pipe, scaler_user, src_w, src_h, dst_w, dst_h);
@ -4796,9 +4835,10 @@ int skl_update_scaler_crtc(struct intel_crtc_state *state)
const struct drm_display_mode *adjusted_mode = &state->base.adjusted_mode;
return skl_update_scaler(state, !state->base.active, SKL_CRTC_INDEX,
&state->scaler_state.scaler_id,
state->pipe_src_w, state->pipe_src_h,
adjusted_mode->crtc_hdisplay, adjusted_mode->crtc_vdisplay);
&state->scaler_state.scaler_id,
state->pipe_src_w, state->pipe_src_h,
adjusted_mode->crtc_hdisplay,
adjusted_mode->crtc_vdisplay, false, 0);
}
/**
@ -4827,7 +4867,8 @@ static int skl_update_scaler_plane(struct intel_crtc_state *crtc_state,
drm_rect_width(&plane_state->base.src) >> 16,
drm_rect_height(&plane_state->base.src) >> 16,
drm_rect_width(&plane_state->base.dst),
drm_rect_height(&plane_state->base.dst));
drm_rect_height(&plane_state->base.dst),
fb ? true : false, fb ? fb->format->format : 0);
if (ret || plane_state->scaler_id < 0)
return ret;
@ -4853,6 +4894,7 @@ static int skl_update_scaler_plane(struct intel_crtc_state *crtc_state,
case DRM_FORMAT_YVYU:
case DRM_FORMAT_UYVY:
case DRM_FORMAT_VYUY:
case DRM_FORMAT_NV12:
break;
default:
DRM_DEBUG_KMS("[PLANE:%d:%s] FB:%d unsupported scaling format 0x%x\n",
@ -5099,13 +5141,15 @@ static bool hsw_post_update_enable_ips(const struct intel_crtc_state *old_crtc_s
static void intel_post_plane_update(struct intel_crtc_state *old_crtc_state)
{
struct intel_crtc *crtc = to_intel_crtc(old_crtc_state->base.crtc);
struct drm_device *dev = crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_atomic_state *old_state = old_crtc_state->base.state;
struct intel_crtc_state *pipe_config =
intel_atomic_get_new_crtc_state(to_intel_atomic_state(old_state),
crtc);
struct drm_plane *primary = crtc->base.primary;
struct drm_plane_state *old_pri_state =
drm_atomic_get_existing_plane_state(old_state, primary);
struct drm_plane_state *old_primary_state =
drm_atomic_get_old_plane_state(old_state, primary);
intel_frontbuffer_flip(to_i915(crtc->base.dev), pipe_config->fb_bits);
@ -5115,19 +5159,25 @@ static void intel_post_plane_update(struct intel_crtc_state *old_crtc_state)
if (hsw_post_update_enable_ips(old_crtc_state, pipe_config))
hsw_enable_ips(pipe_config);
if (old_pri_state) {
struct intel_plane_state *primary_state =
intel_atomic_get_new_plane_state(to_intel_atomic_state(old_state),
to_intel_plane(primary));
struct intel_plane_state *old_primary_state =
to_intel_plane_state(old_pri_state);
if (old_primary_state) {
struct drm_plane_state *new_primary_state =
drm_atomic_get_new_plane_state(old_state, primary);
struct drm_framebuffer *fb = new_primary_state->fb;
intel_fbc_post_update(crtc);
if (primary_state->base.visible &&
if (new_primary_state->visible &&
(needs_modeset(&pipe_config->base) ||
!old_primary_state->base.visible))
!old_primary_state->visible))
intel_post_enable_primary(&crtc->base, pipe_config);
/* Display WA 827 */
if ((INTEL_GEN(dev_priv) == 9 && !IS_GEMINILAKE(dev_priv)) ||
IS_CANNONLAKE(dev_priv)) {
if (fb && fb->format->format == DRM_FORMAT_NV12)
skl_wa_clkgate(dev_priv, crtc->pipe, false);
}
}
}
@ -5139,8 +5189,8 @@ static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state,
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_atomic_state *old_state = old_crtc_state->base.state;
struct drm_plane *primary = crtc->base.primary;
struct drm_plane_state *old_pri_state =
drm_atomic_get_existing_plane_state(old_state, primary);
struct drm_plane_state *old_primary_state =
drm_atomic_get_old_plane_state(old_state, primary);
bool modeset = needs_modeset(&pipe_config->base);
struct intel_atomic_state *old_intel_state =
to_intel_atomic_state(old_state);
@ -5148,20 +5198,26 @@ static void intel_pre_plane_update(struct intel_crtc_state *old_crtc_state,
if (hsw_pre_update_disable_ips(old_crtc_state, pipe_config))
hsw_disable_ips(old_crtc_state);
if (old_pri_state) {
struct intel_plane_state *primary_state =
if (old_primary_state) {
struct intel_plane_state *new_primary_state =
intel_atomic_get_new_plane_state(old_intel_state,
to_intel_plane(primary));
struct intel_plane_state *old_primary_state =
to_intel_plane_state(old_pri_state);
struct drm_framebuffer *fb = new_primary_state->base.fb;
intel_fbc_pre_update(crtc, pipe_config, primary_state);
/* Display WA 827 */
if ((INTEL_GEN(dev_priv) == 9 && !IS_GEMINILAKE(dev_priv)) ||
IS_CANNONLAKE(dev_priv)) {
if (fb && fb->format->format == DRM_FORMAT_NV12)
skl_wa_clkgate(dev_priv, crtc->pipe, true);
}
intel_fbc_pre_update(crtc, pipe_config, new_primary_state);
/*
* Gen2 reports pipe underruns whenever all planes are disabled.
* So disable underrun reporting before all the planes get disabled.
*/
if (IS_GEN2(dev_priv) && old_primary_state->base.visible &&
(modeset || !primary_state->base.visible))
if (IS_GEN2(dev_priv) && old_primary_state->visible &&
(modeset || !new_primary_state->base.visible))
intel_set_cpu_fifo_underrun_reporting(dev_priv, crtc->pipe, false);
}
@ -8766,8 +8822,8 @@ static bool ironlake_get_pipe_config(struct intel_crtc *crtc,
intel_get_shared_dpll_by_id(dev_priv, pll_id);
pll = pipe_config->shared_dpll;
WARN_ON(!pll->funcs.get_hw_state(dev_priv, pll,
&pipe_config->dpll_hw_state));
WARN_ON(!pll->info->funcs->get_hw_state(dev_priv, pll,
&pipe_config->dpll_hw_state));
tmp = pipe_config->dpll_hw_state.dpll;
pipe_config->pixel_multiplier =
@ -9243,8 +9299,8 @@ static void haswell_get_ddi_port_state(struct intel_crtc *crtc,
pll = pipe_config->shared_dpll;
if (pll) {
WARN_ON(!pll->funcs.get_hw_state(dev_priv, pll,
&pipe_config->dpll_hw_state));
WARN_ON(!pll->info->funcs->get_hw_state(dev_priv, pll,
&pipe_config->dpll_hw_state));
}
/*
@ -10775,7 +10831,7 @@ static bool check_digital_port_conflicts(struct drm_atomic_state *state)
struct drm_connector_state *connector_state;
struct intel_encoder *encoder;
connector_state = drm_atomic_get_existing_connector_state(state, connector);
connector_state = drm_atomic_get_new_connector_state(state, connector);
if (!connector_state)
connector_state = connector->state;
@ -11645,11 +11701,11 @@ verify_single_dpll_state(struct drm_i915_private *dev_priv,
memset(&dpll_hw_state, 0, sizeof(dpll_hw_state));
DRM_DEBUG_KMS("%s\n", pll->name);
DRM_DEBUG_KMS("%s\n", pll->info->name);
active = pll->funcs.get_hw_state(dev_priv, pll, &dpll_hw_state);
active = pll->info->funcs->get_hw_state(dev_priv, pll, &dpll_hw_state);
if (!(pll->flags & INTEL_DPLL_ALWAYS_ON)) {
if (!(pll->info->flags & INTEL_DPLL_ALWAYS_ON)) {
I915_STATE_WARN(!pll->on && pll->active_mask,
"pll in active use but not on in sw tracking\n");
I915_STATE_WARN(pll->on && !pll->active_mask,
@ -12138,20 +12194,23 @@ static void intel_update_crtc(struct drm_crtc *crtc,
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_crtc_state *pipe_config = to_intel_crtc_state(new_crtc_state);
bool modeset = needs_modeset(new_crtc_state);
struct intel_plane_state *new_plane_state =
intel_atomic_get_new_plane_state(to_intel_atomic_state(state),
to_intel_plane(crtc->primary));
if (modeset) {
update_scanline_offset(intel_crtc);
dev_priv->display.crtc_enable(pipe_config, state);
/* vblanks work again, re-enable pipe CRC. */
intel_crtc_enable_pipe_crc(intel_crtc);
} else {
intel_pre_plane_update(to_intel_crtc_state(old_crtc_state),
pipe_config);
}
if (drm_atomic_get_existing_plane_state(state, crtc->primary)) {
intel_fbc_enable(
intel_crtc, pipe_config,
to_intel_plane_state(crtc->primary->state));
}
if (new_plane_state)
intel_fbc_enable(intel_crtc, pipe_config, new_plane_state);
drm_atomic_helper_commit_planes_on_crtc(old_crtc_state);
}
@ -12322,6 +12381,13 @@ static void intel_atomic_commit_tail(struct drm_atomic_state *state)
if (old_crtc_state->active) {
intel_crtc_disable_planes(crtc, old_crtc_state->plane_mask);
/*
* We need to disable pipe CRC before disabling the pipe,
* or we race against vblank off.
*/
intel_crtc_disable_pipe_crc(intel_crtc);
dev_priv->display.crtc_disable(to_intel_crtc_state(old_crtc_state), state);
intel_crtc->active = false;
intel_fbc_disable(intel_crtc);
@ -12725,8 +12791,8 @@ intel_prepare_plane_fb(struct drm_plane *plane,
if (old_obj) {
struct drm_crtc_state *crtc_state =
drm_atomic_get_existing_crtc_state(new_state->state,
plane->state->crtc);
drm_atomic_get_new_crtc_state(new_state->state,
plane->state->crtc);
/* Big Hammer, we also need to ensure that any pending
* MI_WAIT_FOR_EVENT inside a user batch buffer on the
@ -12780,6 +12846,8 @@ intel_prepare_plane_fb(struct drm_plane *plane,
if (ret)
return ret;
intel_fb_obj_flush(obj, ORIGIN_DIRTYFB);
if (!new_state->fence) { /* implicit fencing */
struct dma_fence *fence;
@ -12824,11 +12892,13 @@ intel_cleanup_plane_fb(struct drm_plane *plane,
}
int
skl_max_scale(struct intel_crtc *intel_crtc, struct intel_crtc_state *crtc_state)
skl_max_scale(struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state,
uint32_t pixel_format)
{
struct drm_i915_private *dev_priv;
int max_scale;
int crtc_clock, max_dotclk;
int max_scale, mult;
int crtc_clock, max_dotclk, tmpclk1, tmpclk2;
if (!intel_crtc || !crtc_state->base.enable)
return DRM_PLANE_HELPER_NO_SCALING;
@ -12850,8 +12920,10 @@ skl_max_scale(struct intel_crtc *intel_crtc, struct intel_crtc_state *crtc_state
* or
* cdclk/crtc_clock
*/
max_scale = min((1 << 16) * 3 - 1,
(1 << 8) * ((max_dotclk << 8) / crtc_clock));
mult = pixel_format == DRM_FORMAT_NV12 ? 2 : 3;
tmpclk1 = (1 << 16) * mult - 1;
tmpclk2 = (1 << 8) * ((max_dotclk << 8) / crtc_clock);
max_scale = min(tmpclk1, tmpclk2);
return max_scale;
}
@ -12867,12 +12939,16 @@ intel_check_primary_plane(struct intel_plane *plane,
int max_scale = DRM_PLANE_HELPER_NO_SCALING;
bool can_position = false;
int ret;
uint32_t pixel_format = 0;
if (INTEL_GEN(dev_priv) >= 9) {
/* use scaler when colorkey is not required */
if (!state->ckey.flags) {
min_scale = 1;
max_scale = skl_max_scale(to_intel_crtc(crtc), crtc_state);
if (state->base.fb)
pixel_format = state->base.fb->format->format;
max_scale = skl_max_scale(to_intel_crtc(crtc),
crtc_state, pixel_format);
}
can_position = true;
}
@ -12945,10 +13021,25 @@ out:
intel_cstate);
}
void intel_crtc_arm_fifo_underrun(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(crtc->base.dev);
if (!IS_GEN2(dev_priv))
intel_set_cpu_fifo_underrun_reporting(dev_priv, crtc->pipe, true);
if (crtc_state->has_pch_encoder) {
enum pipe pch_transcoder =
intel_crtc_pch_transcoder(crtc);
intel_set_pch_fifo_underrun_reporting(dev_priv, pch_transcoder, true);
}
}
static void intel_finish_crtc_commit(struct drm_crtc *crtc,
struct drm_crtc_state *old_crtc_state)
{
struct drm_i915_private *dev_priv = to_i915(crtc->dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
struct intel_atomic_state *old_intel_state =
to_intel_atomic_state(old_crtc_state->state);
@ -12959,17 +13050,8 @@ static void intel_finish_crtc_commit(struct drm_crtc *crtc,
if (new_crtc_state->update_pipe &&
!needs_modeset(&new_crtc_state->base) &&
old_crtc_state->mode.private_flags & I915_MODE_FLAG_INHERITED) {
if (!IS_GEN2(dev_priv))
intel_set_cpu_fifo_underrun_reporting(dev_priv, intel_crtc->pipe, true);
if (new_crtc_state->has_pch_encoder) {
enum pipe pch_transcoder =
intel_crtc_pch_transcoder(intel_crtc);
intel_set_pch_fifo_underrun_reporting(dev_priv, pch_transcoder, true);
}
}
old_crtc_state->mode.private_flags & I915_MODE_FLAG_INHERITED)
intel_crtc_arm_fifo_underrun(intel_crtc, new_crtc_state);
}
/**
@ -13167,8 +13249,9 @@ intel_legacy_cursor_update(struct drm_plane *plane,
if (ret)
goto out_unlock;
old_fb = old_plane_state->fb;
intel_fb_obj_flush(intel_fb_obj(fb), ORIGIN_FLIP);
old_fb = old_plane_state->fb;
i915_gem_track_fb(intel_fb_obj(old_fb), intel_fb_obj(fb),
intel_plane->frontbuffer_bit);
@ -13555,10 +13638,17 @@ static int intel_crtc_init(struct drm_i915_private *dev_priv, enum pipe pipe)
/* initialize shared scalers */
intel_crtc_init_scalers(intel_crtc, crtc_state);
BUG_ON(pipe >= ARRAY_SIZE(dev_priv->plane_to_crtc_mapping) ||
dev_priv->plane_to_crtc_mapping[primary->i9xx_plane] != NULL);
dev_priv->plane_to_crtc_mapping[primary->i9xx_plane] = intel_crtc;
dev_priv->pipe_to_crtc_mapping[intel_crtc->pipe] = intel_crtc;
BUG_ON(pipe >= ARRAY_SIZE(dev_priv->pipe_to_crtc_mapping) ||
dev_priv->pipe_to_crtc_mapping[pipe] != NULL);
dev_priv->pipe_to_crtc_mapping[pipe] = intel_crtc;
if (INTEL_GEN(dev_priv) < 9) {
enum i9xx_plane_id i9xx_plane = primary->i9xx_plane;
BUG_ON(i9xx_plane >= ARRAY_SIZE(dev_priv->plane_to_crtc_mapping) ||
dev_priv->plane_to_crtc_mapping[i9xx_plane] != NULL);
dev_priv->plane_to_crtc_mapping[i9xx_plane] = intel_crtc;
}
drm_crtc_helper_add(&intel_crtc->base, &intel_helper_funcs);
@ -15103,8 +15193,8 @@ static void intel_modeset_readout_hw_state(struct drm_device *dev)
for (i = 0; i < dev_priv->num_shared_dpll; i++) {
struct intel_shared_dpll *pll = &dev_priv->shared_dplls[i];
pll->on = pll->funcs.get_hw_state(dev_priv, pll,
&pll->state.hw_state);
pll->on = pll->info->funcs->get_hw_state(dev_priv, pll,
&pll->state.hw_state);
pll->state.crtc_mask = 0;
for_each_intel_crtc(dev, crtc) {
struct intel_crtc_state *crtc_state =
@ -15117,7 +15207,7 @@ static void intel_modeset_readout_hw_state(struct drm_device *dev)
pll->active_mask = pll->state.crtc_mask;
DRM_DEBUG_KMS("%s hw state readout: crtc_mask 0x%08x, on %i\n",
pll->name, pll->state.crtc_mask, pll->on);
pll->info->name, pll->state.crtc_mask, pll->on);
}
for_each_intel_encoder(dev, encoder) {
@ -15291,9 +15381,10 @@ intel_modeset_setup_hw_state(struct drm_device *dev,
if (!pll->on || pll->active_mask)
continue;
DRM_DEBUG_KMS("%s enabled but not in use, disabling\n", pll->name);
DRM_DEBUG_KMS("%s enabled but not in use, disabling\n",
pll->info->name);
pll->funcs.disable(dev_priv, pll);
pll->info->funcs->disable(dev_priv, pll);
pll->on = false;
}

21
drivers/gpu/drm/i915/intel_dp.c
View File

@ -43,7 +43,6 @@
#include <drm/i915_drm.h>
#include "i915_drv.h"
#define DP_LINK_CHECK_TIMEOUT (10 * 1000)
#define DP_DPRX_ESI_LEN 14
/* Compliance test status bits */
@ -92,8 +91,6 @@ static const struct dp_link_dpll chv_dpll[] = {
{ .p1 = 4, .p2 = 2, .n = 1, .m1 = 2, .m2 = 0x819999a } },
{ 270000, /* m2_int = 27, m2_fraction = 0 */
{ .p1 = 4, .p2 = 1, .n = 1, .m1 = 2, .m2 = 0x6c00000 } },
{ 540000, /* m2_int = 27, m2_fraction = 0 */
{ .p1 = 2, .p2 = 1, .n = 1, .m1 = 2, .m2 = 0x6c00000 } }
};
/**
@ -2901,10 +2898,7 @@ _intel_dp_set_link_train(struct intel_dp *intel_dp,
}
} else {
if (IS_CHERRYVIEW(dev_priv))
*DP &= ~DP_LINK_TRAIN_MASK_CHV;
else
*DP &= ~DP_LINK_TRAIN_MASK;
*DP &= ~DP_LINK_TRAIN_MASK;
switch (dp_train_pat & DP_TRAINING_PATTERN_MASK) {
case DP_TRAINING_PATTERN_DISABLE:
@ -2917,12 +2911,8 @@ _intel_dp_set_link_train(struct intel_dp *intel_dp,
*DP |= DP_LINK_TRAIN_PAT_2;
break;
case DP_TRAINING_PATTERN_3:
if (IS_CHERRYVIEW(dev_priv)) {
*DP |= DP_LINK_TRAIN_PAT_3_CHV;
} else {
DRM_DEBUG_KMS("TPS3 not supported, using TPS2 instead\n");
*DP |= DP_LINK_TRAIN_PAT_2;
}
DRM_DEBUG_KMS("TPS3 not supported, using TPS2 instead\n");
*DP |= DP_LINK_TRAIN_PAT_2;
break;
}
}
@ -3661,10 +3651,7 @@ intel_dp_link_down(struct intel_encoder *encoder,
DP &= ~DP_LINK_TRAIN_MASK_CPT;
DP |= DP_LINK_TRAIN_PAT_IDLE_CPT;
} else {
if (IS_CHERRYVIEW(dev_priv))
DP &= ~DP_LINK_TRAIN_MASK_CHV;
else
DP &= ~DP_LINK_TRAIN_MASK;
DP &= ~DP_LINK_TRAIN_MASK;
DP |= DP_LINK_TRAIN_PAT_IDLE;
}
I915_WRITE(intel_dp->output_reg, DP);

8
drivers/gpu/drm/i915/intel_dp_mst.c
View File

@ -180,9 +180,11 @@ static void intel_mst_post_disable_dp(struct intel_encoder *encoder,
intel_dp->active_mst_links--;
intel_mst->connector = NULL;
if (intel_dp->active_mst_links == 0)
if (intel_dp->active_mst_links == 0) {
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);
intel_dig_port->base.post_disable(&intel_dig_port->base,
old_crtc_state, NULL);
}
DRM_DEBUG_KMS("active links %d\n", intel_dp->active_mst_links);
}
@ -223,7 +225,11 @@ static void intel_mst_pre_enable_dp(struct intel_encoder *encoder,
DRM_DEBUG_KMS("active links %d\n", intel_dp->active_mst_links);
if (intel_dp->active_mst_links == 0)
intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_ON);
drm_dp_send_power_updown_phy(&intel_dp->mst_mgr, connector->port, true);
if (intel_dp->active_mst_links == 0)
intel_dig_port->base.pre_enable(&intel_dig_port->base,
pipe_config, NULL);

11
drivers/gpu/drm/i915/intel_dpio_phy.c
View File

@ -380,13 +380,14 @@ static void _bxt_ddi_phy_init(struct drm_i915_private *dev_priv,
* all 1s. Eventually they become accessible as they power up, then
* the reserved bit will give the default 0. Poll on the reserved bit
* becoming 0 to find when the PHY is accessible.
* HW team confirmed that the time to reach phypowergood status is
* anywhere between 50 us and 100us.
* The flag should get set in 100us according to the HW team, but
* use 1ms due to occasional timeouts observed with that.
*/
if (wait_for_us(((I915_READ(BXT_PORT_CL1CM_DW0(phy)) &
(PHY_RESERVED | PHY_POWER_GOOD)) == PHY_POWER_GOOD), 100)) {
if (intel_wait_for_register_fw(dev_priv, BXT_PORT_CL1CM_DW0(phy),
PHY_RESERVED | PHY_POWER_GOOD,
PHY_POWER_GOOD,
1))
DRM_ERROR("timeout during PHY%d power on\n", phy);
}
/* Program PLL Rcomp code offset */
val = I915_READ(BXT_PORT_CL1CM_DW9(phy));

253
drivers/gpu/drm/i915/intel_dpll_mgr.c
View File

@ -118,10 +118,10 @@ void assert_shared_dpll(struct drm_i915_private *dev_priv,
if (WARN(!pll, "asserting DPLL %s with no DPLL\n", onoff(state)))
return;
cur_state = pll->funcs.get_hw_state(dev_priv, pll, &hw_state);
cur_state = pll->info->funcs->get_hw_state(dev_priv, pll, &hw_state);
I915_STATE_WARN(cur_state != state,
"%s assertion failure (expected %s, current %s)\n",
pll->name, onoff(state), onoff(cur_state));
pll->info->name, onoff(state), onoff(cur_state));
}
/**
@ -143,11 +143,11 @@ void intel_prepare_shared_dpll(struct intel_crtc *crtc)
mutex_lock(&dev_priv->dpll_lock);
WARN_ON(!pll->state.crtc_mask);
if (!pll->active_mask) {
DRM_DEBUG_DRIVER("setting up %s\n", pll->name);
DRM_DEBUG_DRIVER("setting up %s\n", pll->info->name);
WARN_ON(pll->on);
assert_shared_dpll_disabled(dev_priv, pll);
pll->funcs.prepare(dev_priv, pll);
pll->info->funcs->prepare(dev_priv, pll);
}
mutex_unlock(&dev_priv->dpll_lock);
}
@ -179,7 +179,7 @@ void intel_enable_shared_dpll(struct intel_crtc *crtc)
pll->active_mask |= crtc_mask;
DRM_DEBUG_KMS("enable %s (active %x, on? %d) for crtc %d\n",
pll->name, pll->active_mask, pll->on,
pll->info->name, pll->active_mask, pll->on,
crtc->base.base.id);
if (old_mask) {
@ -189,8 +189,8 @@ void intel_enable_shared_dpll(struct intel_crtc *crtc)
}
WARN_ON(pll->on);
DRM_DEBUG_KMS("enabling %s\n", pll->name);
pll->funcs.enable(dev_priv, pll);
DRM_DEBUG_KMS("enabling %s\n", pll->info->name);
pll->info->funcs->enable(dev_priv, pll);
pll->on = true;
out:
@ -221,7 +221,7 @@ void intel_disable_shared_dpll(struct intel_crtc *crtc)
goto out;
DRM_DEBUG_KMS("disable %s (active %x, on? %d) for crtc %d\n",
pll->name, pll->active_mask, pll->on,
pll->info->name, pll->active_mask, pll->on,
crtc->base.base.id);
assert_shared_dpll_enabled(dev_priv, pll);
@ -231,8 +231,8 @@ void intel_disable_shared_dpll(struct intel_crtc *crtc)
if (pll->active_mask)
goto out;
DRM_DEBUG_KMS("disabling %s\n", pll->name);
pll->funcs.disable(dev_priv, pll);
DRM_DEBUG_KMS("disabling %s\n", pll->info->name);
pll->info->funcs->disable(dev_priv, pll);
pll->on = false;
out:
@ -263,7 +263,8 @@ intel_find_shared_dpll(struct intel_crtc *crtc,
&shared_dpll[i].hw_state,
sizeof(crtc_state->dpll_hw_state)) == 0) {
DRM_DEBUG_KMS("[CRTC:%d:%s] sharing existing %s (crtc mask 0x%08x, active %x)\n",
crtc->base.base.id, crtc->base.name, pll->name,
crtc->base.base.id, crtc->base.name,
pll->info->name,
shared_dpll[i].crtc_mask,
pll->active_mask);
return pll;
@ -275,7 +276,8 @@ intel_find_shared_dpll(struct intel_crtc *crtc,
pll = &dev_priv->shared_dplls[i];
if (shared_dpll[i].crtc_mask == 0) {
DRM_DEBUG_KMS("[CRTC:%d:%s] allocated %s\n",
crtc->base.base.id, crtc->base.name, pll->name);
crtc->base.base.id, crtc->base.name,
pll->info->name);
return pll;
}
}
@ -289,19 +291,19 @@ intel_reference_shared_dpll(struct intel_shared_dpll *pll,
{
struct intel_shared_dpll_state *shared_dpll;
struct intel_crtc *crtc = to_intel_crtc(crtc_state->base.crtc);
enum intel_dpll_id i = pll->id;
const enum intel_dpll_id id = pll->info->id;
shared_dpll = intel_atomic_get_shared_dpll_state(crtc_state->base.state);
if (shared_dpll[i].crtc_mask == 0)
shared_dpll[i].hw_state =
if (shared_dpll[id].crtc_mask == 0)
shared_dpll[id].hw_state =
crtc_state->dpll_hw_state;
crtc_state->shared_dpll = pll;
DRM_DEBUG_DRIVER("using %s for pipe %c\n", pll->name,
DRM_DEBUG_DRIVER("using %s for pipe %c\n", pll->info->name,
pipe_name(crtc->pipe));
shared_dpll[pll->id].crtc_mask |= 1 << crtc->pipe;
shared_dpll[id].crtc_mask |= 1 << crtc->pipe;
}
/**
@ -341,15 +343,16 @@ static bool ibx_pch_dpll_get_hw_state(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll,
struct intel_dpll_hw_state *hw_state)
{
const enum intel_dpll_id id = pll->info->id;
uint32_t val;
if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
return false;
val = I915_READ(PCH_DPLL(pll->id));
val = I915_READ(PCH_DPLL(id));
hw_state->dpll = val;
hw_state->fp0 = I915_READ(PCH_FP0(pll->id));
hw_state->fp1 = I915_READ(PCH_FP1(pll->id));
hw_state->fp0 = I915_READ(PCH_FP0(id));
hw_state->fp1 = I915_READ(PCH_FP1(id));
intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
@ -359,8 +362,10 @@ static bool ibx_pch_dpll_get_hw_state(struct drm_i915_private *dev_priv,
static void ibx_pch_dpll_prepare(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
I915_WRITE(PCH_FP0(pll->id), pll->state.hw_state.fp0);
I915_WRITE(PCH_FP1(pll->id), pll->state.hw_state.fp1);
const enum intel_dpll_id id = pll->info->id;
I915_WRITE(PCH_FP0(id), pll->state.hw_state.fp0);
I915_WRITE(PCH_FP1(id), pll->state.hw_state.fp1);
}
static void ibx_assert_pch_refclk_enabled(struct drm_i915_private *dev_priv)
@ -379,13 +384,15 @@ static void ibx_assert_pch_refclk_enabled(struct drm_i915_private *dev_priv)
static void ibx_pch_dpll_enable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const enum intel_dpll_id id = pll->info->id;
/* PCH refclock must be enabled first */
ibx_assert_pch_refclk_enabled(dev_priv);
I915_WRITE(PCH_DPLL(pll->id), pll->state.hw_state.dpll);
I915_WRITE(PCH_DPLL(id), pll->state.hw_state.dpll);
/* Wait for the clocks to stabilize. */
POSTING_READ(PCH_DPLL(pll->id));
POSTING_READ(PCH_DPLL(id));
udelay(150);
/* The pixel multiplier can only be updated once the
@ -393,14 +400,15 @@ static void ibx_pch_dpll_enable(struct drm_i915_private *dev_priv,
*
* So write it again.
*/
I915_WRITE(PCH_DPLL(pll->id), pll->state.hw_state.dpll);
POSTING_READ(PCH_DPLL(pll->id));
I915_WRITE(PCH_DPLL(id), pll->state.hw_state.dpll);
POSTING_READ(PCH_DPLL(id));
udelay(200);
}
static void ibx_pch_dpll_disable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const enum intel_dpll_id id = pll->info->id;
struct drm_device *dev = &dev_priv->drm;
struct intel_crtc *crtc;
@ -410,8 +418,8 @@ static void ibx_pch_dpll_disable(struct drm_i915_private *dev_priv,
assert_pch_transcoder_disabled(dev_priv, crtc->pipe);
}
I915_WRITE(PCH_DPLL(pll->id), 0);
POSTING_READ(PCH_DPLL(pll->id));
I915_WRITE(PCH_DPLL(id), 0);
POSTING_READ(PCH_DPLL(id));
udelay(200);
}
@ -429,7 +437,8 @@ ibx_get_dpll(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
pll = &dev_priv->shared_dplls[i];
DRM_DEBUG_KMS("[CRTC:%d:%s] using pre-allocated %s\n",
crtc->base.base.id, crtc->base.name, pll->name);
crtc->base.base.id, crtc->base.name,
pll->info->name);
} else {
pll = intel_find_shared_dpll(crtc, crtc_state,
DPLL_ID_PCH_PLL_A,
@ -466,8 +475,10 @@ static const struct intel_shared_dpll_funcs ibx_pch_dpll_funcs = {
static void hsw_ddi_wrpll_enable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
I915_WRITE(WRPLL_CTL(pll->id), pll->state.hw_state.wrpll);
POSTING_READ(WRPLL_CTL(pll->id));
const enum intel_dpll_id id = pll->info->id;
I915_WRITE(WRPLL_CTL(id), pll->state.hw_state.wrpll);
POSTING_READ(WRPLL_CTL(id));
udelay(20);
}
@ -482,11 +493,12 @@ static void hsw_ddi_spll_enable(struct drm_i915_private *dev_priv,
static void hsw_ddi_wrpll_disable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const enum intel_dpll_id id = pll->info->id;
uint32_t val;
val = I915_READ(WRPLL_CTL(pll->id));
I915_WRITE(WRPLL_CTL(pll->id), val & ~WRPLL_PLL_ENABLE);
POSTING_READ(WRPLL_CTL(pll->id));
val = I915_READ(WRPLL_CTL(id));
I915_WRITE(WRPLL_CTL(id), val & ~WRPLL_PLL_ENABLE);
POSTING_READ(WRPLL_CTL(id));
}
static void hsw_ddi_spll_disable(struct drm_i915_private *dev_priv,
@ -503,12 +515,13 @@ static bool hsw_ddi_wrpll_get_hw_state(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll,
struct intel_dpll_hw_state *hw_state)
{
const enum intel_dpll_id id = pll->info->id;
uint32_t val;
if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
return false;
val = I915_READ(WRPLL_CTL(pll->id));
val = I915_READ(WRPLL_CTL(id));
hw_state->wrpll = val;
intel_display_power_put(dev_priv, POWER_DOMAIN_PLLS);
@ -914,13 +927,15 @@ static const struct skl_dpll_regs skl_dpll_regs[4] = {
static void skl_ddi_pll_write_ctrl1(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const enum intel_dpll_id id = pll->info->id;
uint32_t val;
val = I915_READ(DPLL_CTRL1);
val &= ~(DPLL_CTRL1_HDMI_MODE(pll->id) | DPLL_CTRL1_SSC(pll->id) |
DPLL_CTRL1_LINK_RATE_MASK(pll->id));
val |= pll->state.hw_state.ctrl1 << (pll->id * 6);
val &= ~(DPLL_CTRL1_HDMI_MODE(id) |
DPLL_CTRL1_SSC(id) |
DPLL_CTRL1_LINK_RATE_MASK(id));
val |= pll->state.hw_state.ctrl1 << (id * 6);
I915_WRITE(DPLL_CTRL1, val);
POSTING_READ(DPLL_CTRL1);
@ -930,24 +945,25 @@ static void skl_ddi_pll_enable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const struct skl_dpll_regs *regs = skl_dpll_regs;
const enum intel_dpll_id id = pll->info->id;
skl_ddi_pll_write_ctrl1(dev_priv, pll);
I915_WRITE(regs[pll->id].cfgcr1, pll->state.hw_state.cfgcr1);
I915_WRITE(regs[pll->id].cfgcr2, pll->state.hw_state.cfgcr2);
POSTING_READ(regs[pll->id].cfgcr1);
POSTING_READ(regs[pll->id].cfgcr2);
I915_WRITE(regs[id].cfgcr1, pll->state.hw_state.cfgcr1);
I915_WRITE(regs[id].cfgcr2, pll->state.hw_state.cfgcr2);
POSTING_READ(regs[id].cfgcr1);
POSTING_READ(regs[id].cfgcr2);
/* the enable bit is always bit 31 */
I915_WRITE(regs[pll->id].ctl,
I915_READ(regs[pll->id].ctl) | LCPLL_PLL_ENABLE);
I915_WRITE(regs[id].ctl,
I915_READ(regs[id].ctl) | LCPLL_PLL_ENABLE);
if (intel_wait_for_register(dev_priv,
DPLL_STATUS,
DPLL_LOCK(pll->id),
DPLL_LOCK(pll->id),
DPLL_LOCK(id),
DPLL_LOCK(id),
5))
DRM_ERROR("DPLL %d not locked\n", pll->id);
DRM_ERROR("DPLL %d not locked\n", id);
}
static void skl_ddi_dpll0_enable(struct drm_i915_private *dev_priv,
@ -960,11 +976,12 @@ static void skl_ddi_pll_disable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const struct skl_dpll_regs *regs = skl_dpll_regs;
const enum intel_dpll_id id = pll->info->id;
/* the enable bit is always bit 31 */
I915_WRITE(regs[pll->id].ctl,
I915_READ(regs[pll->id].ctl) & ~LCPLL_PLL_ENABLE);
POSTING_READ(regs[pll->id].ctl);
I915_WRITE(regs[id].ctl,
I915_READ(regs[id].ctl) & ~LCPLL_PLL_ENABLE);
POSTING_READ(regs[id].ctl);
}
static void skl_ddi_dpll0_disable(struct drm_i915_private *dev_priv,
@ -978,6 +995,7 @@ static bool skl_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
{
uint32_t val;
const struct skl_dpll_regs *regs = skl_dpll_regs;
const enum intel_dpll_id id = pll->info->id;
bool ret;
if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
@ -985,17 +1003,17 @@ static bool skl_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
ret = false;
val = I915_READ(regs[pll->id].ctl);
val = I915_READ(regs[id].ctl);
if (!(val & LCPLL_PLL_ENABLE))
goto out;
val = I915_READ(DPLL_CTRL1);
hw_state->ctrl1 = (val >> (pll->id * 6)) & 0x3f;
hw_state->ctrl1 = (val >> (id * 6)) & 0x3f;
/* avoid reading back stale values if HDMI mode is not enabled */
if (val & DPLL_CTRL1_HDMI_MODE(pll->id)) {
hw_state->cfgcr1 = I915_READ(regs[pll->id].cfgcr1);
hw_state->cfgcr2 = I915_READ(regs[pll->id].cfgcr2);
if (val & DPLL_CTRL1_HDMI_MODE(id)) {
hw_state->cfgcr1 = I915_READ(regs[id].cfgcr1);
hw_state->cfgcr2 = I915_READ(regs[id].cfgcr2);
}
ret = true;
@ -1011,6 +1029,7 @@ static bool skl_ddi_dpll0_get_hw_state(struct drm_i915_private *dev_priv,
{
uint32_t val;
const struct skl_dpll_regs *regs = skl_dpll_regs;
const enum intel_dpll_id id = pll->info->id;
bool ret;
if (!intel_display_power_get_if_enabled(dev_priv, POWER_DOMAIN_PLLS))
@ -1019,12 +1038,12 @@ static bool skl_ddi_dpll0_get_hw_state(struct drm_i915_private *dev_priv,
ret = false;
/* DPLL0 is always enabled since it drives CDCLK */
val = I915_READ(regs[pll->id].ctl);
val = I915_READ(regs[id].ctl);
if (WARN_ON(!(val & LCPLL_PLL_ENABLE)))
goto out;
val = I915_READ(DPLL_CTRL1);
hw_state->ctrl1 = (val >> (pll->id * 6)) & 0x3f;
hw_state->ctrl1 = (val >> (id * 6)) & 0x3f;
ret = true;
@ -1424,7 +1443,7 @@ static void bxt_ddi_pll_enable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
uint32_t temp;
enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
enum port port = (enum port)pll->info->id; /* 1:1 port->PLL mapping */
enum dpio_phy phy;
enum dpio_channel ch;
@ -1543,7 +1562,7 @@ static void bxt_ddi_pll_enable(struct drm_i915_private *dev_priv,
static void bxt_ddi_pll_disable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
enum port port = (enum port)pll->info->id; /* 1:1 port->PLL mapping */
uint32_t temp;
temp = I915_READ(BXT_PORT_PLL_ENABLE(port));
@ -1566,7 +1585,7 @@ static bool bxt_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll,
struct intel_dpll_hw_state *hw_state)
{
enum port port = (enum port)pll->id; /* 1:1 port->PLL mapping */
enum port port = (enum port)pll->info->id; /* 1:1 port->PLL mapping */
uint32_t val;
bool ret;
enum dpio_phy phy;
@ -1824,7 +1843,7 @@ bxt_get_dpll(struct intel_crtc *crtc,
pll = intel_get_shared_dpll_by_id(dev_priv, i);
DRM_DEBUG_KMS("[CRTC:%d:%s] using pre-allocated %s\n",
crtc->base.base.id, crtc->base.name, pll->name);
crtc->base.base.id, crtc->base.name, pll->info->name);
intel_reference_shared_dpll(pll, crtc_state);
@ -1877,13 +1896,6 @@ static void intel_ddi_pll_init(struct drm_device *dev)
}
}
struct dpll_info {
const char *name;
const int id;
const struct intel_shared_dpll_funcs *funcs;
uint32_t flags;
};
struct intel_dpll_mgr {
const struct dpll_info *dpll_info;
@ -1896,9 +1908,9 @@ struct intel_dpll_mgr {
};
static const struct dpll_info pch_plls[] = {
{ "PCH DPLL A", DPLL_ID_PCH_PLL_A, &ibx_pch_dpll_funcs, 0 },
{ "PCH DPLL B", DPLL_ID_PCH_PLL_B, &ibx_pch_dpll_funcs, 0 },
{ NULL, -1, NULL, 0 },
{ "PCH DPLL A", &ibx_pch_dpll_funcs, DPLL_ID_PCH_PLL_A, 0 },
{ "PCH DPLL B", &ibx_pch_dpll_funcs, DPLL_ID_PCH_PLL_B, 0 },
{ },
};
static const struct intel_dpll_mgr pch_pll_mgr = {
@ -1908,13 +1920,13 @@ static const struct intel_dpll_mgr pch_pll_mgr = {
};
static const struct dpll_info hsw_plls[] = {
{ "WRPLL 1", DPLL_ID_WRPLL1, &hsw_ddi_wrpll_funcs, 0 },
{ "WRPLL 2", DPLL_ID_WRPLL2, &hsw_ddi_wrpll_funcs, 0 },
{ "SPLL", DPLL_ID_SPLL, &hsw_ddi_spll_funcs, 0 },
{ "LCPLL 810", DPLL_ID_LCPLL_810, &hsw_ddi_lcpll_funcs, INTEL_DPLL_ALWAYS_ON },
{ "LCPLL 1350", DPLL_ID_LCPLL_1350, &hsw_ddi_lcpll_funcs, INTEL_DPLL_ALWAYS_ON },
{ "LCPLL 2700", DPLL_ID_LCPLL_2700, &hsw_ddi_lcpll_funcs, INTEL_DPLL_ALWAYS_ON },
{ NULL, -1, NULL, },
{ "WRPLL 1", &hsw_ddi_wrpll_funcs, DPLL_ID_WRPLL1, 0 },
{ "WRPLL 2", &hsw_ddi_wrpll_funcs, DPLL_ID_WRPLL2, 0 },
{ "SPLL", &hsw_ddi_spll_funcs, DPLL_ID_SPLL, 0 },
{ "LCPLL 810", &hsw_ddi_lcpll_funcs, DPLL_ID_LCPLL_810, INTEL_DPLL_ALWAYS_ON },
{ "LCPLL 1350", &hsw_ddi_lcpll_funcs, DPLL_ID_LCPLL_1350, INTEL_DPLL_ALWAYS_ON },
{ "LCPLL 2700", &hsw_ddi_lcpll_funcs, DPLL_ID_LCPLL_2700, INTEL_DPLL_ALWAYS_ON },
{ },
};
static const struct intel_dpll_mgr hsw_pll_mgr = {
@ -1924,11 +1936,11 @@ static const struct intel_dpll_mgr hsw_pll_mgr = {
};
static const struct dpll_info skl_plls[] = {
{ "DPLL 0", DPLL_ID_SKL_DPLL0, &skl_ddi_dpll0_funcs, INTEL_DPLL_ALWAYS_ON },
{ "DPLL 1", DPLL_ID_SKL_DPLL1, &skl_ddi_pll_funcs, 0 },
{ "DPLL 2", DPLL_ID_SKL_DPLL2, &skl_ddi_pll_funcs, 0 },
{ "DPLL 3", DPLL_ID_SKL_DPLL3, &skl_ddi_pll_funcs, 0 },
{ NULL, -1, NULL, },
{ "DPLL 0", &skl_ddi_dpll0_funcs, DPLL_ID_SKL_DPLL0, INTEL_DPLL_ALWAYS_ON },
{ "DPLL 1", &skl_ddi_pll_funcs, DPLL_ID_SKL_DPLL1, 0 },
{ "DPLL 2", &skl_ddi_pll_funcs, DPLL_ID_SKL_DPLL2, 0 },
{ "DPLL 3", &skl_ddi_pll_funcs, DPLL_ID_SKL_DPLL3, 0 },
{ },
};
static const struct intel_dpll_mgr skl_pll_mgr = {
@ -1938,10 +1950,10 @@ static const struct intel_dpll_mgr skl_pll_mgr = {
};
static const struct dpll_info bxt_plls[] = {
{ "PORT PLL A", DPLL_ID_SKL_DPLL0, &bxt_ddi_pll_funcs, 0 },
{ "PORT PLL B", DPLL_ID_SKL_DPLL1, &bxt_ddi_pll_funcs, 0 },
{ "PORT PLL C", DPLL_ID_SKL_DPLL2, &bxt_ddi_pll_funcs, 0 },
{ NULL, -1, NULL, },
{ "PORT PLL A", &bxt_ddi_pll_funcs, DPLL_ID_SKL_DPLL0, 0 },
{ "PORT PLL B", &bxt_ddi_pll_funcs, DPLL_ID_SKL_DPLL1, 0 },
{ "PORT PLL C", &bxt_ddi_pll_funcs, DPLL_ID_SKL_DPLL2, 0 },
{ },
};
static const struct intel_dpll_mgr bxt_pll_mgr = {
@ -1953,38 +1965,39 @@ static const struct intel_dpll_mgr bxt_pll_mgr = {
static void cnl_ddi_pll_enable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const enum intel_dpll_id id = pll->info->id;
uint32_t val;
/* 1. Enable DPLL power in DPLL_ENABLE. */
val = I915_READ(CNL_DPLL_ENABLE(pll->id));
val = I915_READ(CNL_DPLL_ENABLE(id));
val |= PLL_POWER_ENABLE;
I915_WRITE(CNL_DPLL_ENABLE(pll->id), val);
I915_WRITE(CNL_DPLL_ENABLE(id), val);
/* 2. Wait for DPLL power state enabled in DPLL_ENABLE. */
if (intel_wait_for_register(dev_priv,
CNL_DPLL_ENABLE(pll->id),
CNL_DPLL_ENABLE(id),
PLL_POWER_STATE,
PLL_POWER_STATE,
5))
DRM_ERROR("PLL %d Power not enabled\n", pll->id);
DRM_ERROR("PLL %d Power not enabled\n", id);
/*
* 3. Configure DPLL_CFGCR0 to set SSC enable/disable,
* select DP mode, and set DP link rate.
*/
val = pll->state.hw_state.cfgcr0;
I915_WRITE(CNL_DPLL_CFGCR0(pll->id), val);
I915_WRITE(CNL_DPLL_CFGCR0(id), val);
/* 4. Reab back to ensure writes completed */
POSTING_READ(CNL_DPLL_CFGCR0(pll->id));
POSTING_READ(CNL_DPLL_CFGCR0(id));
/* 3. Configure DPLL_CFGCR0 */
/* Avoid touch CFGCR1 if HDMI mode is not enabled */
if (pll->state.hw_state.cfgcr0 & DPLL_CFGCR0_HDMI_MODE) {
val = pll->state.hw_state.cfgcr1;
I915_WRITE(CNL_DPLL_CFGCR1(pll->id), val);
I915_WRITE(CNL_DPLL_CFGCR1(id), val);
/* 4. Reab back to ensure writes completed */
POSTING_READ(CNL_DPLL_CFGCR1(pll->id));
POSTING_READ(CNL_DPLL_CFGCR1(id));
}
/*
@ -1997,17 +2010,17 @@ static void cnl_ddi_pll_enable(struct drm_i915_private *dev_priv,
*/
/* 6. Enable DPLL in DPLL_ENABLE. */
val = I915_READ(CNL_DPLL_ENABLE(pll->id));
val = I915_READ(CNL_DPLL_ENABLE(id));
val |= PLL_ENABLE;
I915_WRITE(CNL_DPLL_ENABLE(pll->id), val);
I915_WRITE(CNL_DPLL_ENABLE(id), val);
/* 7. Wait for PLL lock status in DPLL_ENABLE. */
if (intel_wait_for_register(dev_priv,
CNL_DPLL_ENABLE(pll->id),
CNL_DPLL_ENABLE(id),
PLL_LOCK,
PLL_LOCK,
5))
DRM_ERROR("PLL %d not locked\n", pll->id);
DRM_ERROR("PLL %d not locked\n", id);
/*
* 8. If the frequency will result in a change to the voltage
@ -2027,6 +2040,7 @@ static void cnl_ddi_pll_enable(struct drm_i915_private *dev_priv,
static void cnl_ddi_pll_disable(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll)
{
const enum intel_dpll_id id = pll->info->id;
uint32_t val;
/*
@ -2044,17 +2058,17 @@ static void cnl_ddi_pll_disable(struct drm_i915_private *dev_priv,
*/
/* 3. Disable DPLL through DPLL_ENABLE. */
val = I915_READ(CNL_DPLL_ENABLE(pll->id));
val = I915_READ(CNL_DPLL_ENABLE(id));
val &= ~PLL_ENABLE;
I915_WRITE(CNL_DPLL_ENABLE(pll->id), val);
I915_WRITE(CNL_DPLL_ENABLE(id), val);
/* 4. Wait for PLL not locked status in DPLL_ENABLE. */
if (intel_wait_for_register(dev_priv,
CNL_DPLL_ENABLE(pll->id),
CNL_DPLL_ENABLE(id),
PLL_LOCK,
0,
5))
DRM_ERROR("PLL %d locked\n", pll->id);
DRM_ERROR("PLL %d locked\n", id);
/*
* 5. If the frequency will result in a change to the voltage
@ -2066,23 +2080,24 @@ static void cnl_ddi_pll_disable(struct drm_i915_private *dev_priv,
*/
/* 6. Disable DPLL power in DPLL_ENABLE. */
val = I915_READ(CNL_DPLL_ENABLE(pll->id));
val = I915_READ(CNL_DPLL_ENABLE(id));
val &= ~PLL_POWER_ENABLE;
I915_WRITE(CNL_DPLL_ENABLE(pll->id), val);
I915_WRITE(CNL_DPLL_ENABLE(id), val);
/* 7. Wait for DPLL power state disabled in DPLL_ENABLE. */
if (intel_wait_for_register(dev_priv,
CNL_DPLL_ENABLE(pll->id),
CNL_DPLL_ENABLE(id),
PLL_POWER_STATE,
0,
5))
DRM_ERROR("PLL %d Power not disabled\n", pll->id);
DRM_ERROR("PLL %d Power not disabled\n", id);
}
static bool cnl_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
struct intel_shared_dpll *pll,
struct intel_dpll_hw_state *hw_state)
{
const enum intel_dpll_id id = pll->info->id;
uint32_t val;
bool ret;
@ -2091,16 +2106,16 @@ static bool cnl_ddi_pll_get_hw_state(struct drm_i915_private *dev_priv,
ret = false;
val = I915_READ(CNL_DPLL_ENABLE(pll->id));
val = I915_READ(CNL_DPLL_ENABLE(id));
if (!(val & PLL_ENABLE))
goto out;
val = I915_READ(CNL_DPLL_CFGCR0(pll->id));
val = I915_READ(CNL_DPLL_CFGCR0(id));
hw_state->cfgcr0 = val;
/* avoid reading back stale values if HDMI mode is not enabled */
if (val & DPLL_CFGCR0_HDMI_MODE) {
hw_state->cfgcr1 = I915_READ(CNL_DPLL_CFGCR1(pll->id));
hw_state->cfgcr1 = I915_READ(CNL_DPLL_CFGCR1(id));
}
ret = true;
@ -2372,10 +2387,10 @@ static const struct intel_shared_dpll_funcs cnl_ddi_pll_funcs = {
};
static const struct dpll_info cnl_plls[] = {
{ "DPLL 0", DPLL_ID_SKL_DPLL0, &cnl_ddi_pll_funcs, 0 },
{ "DPLL 1", DPLL_ID_SKL_DPLL1, &cnl_ddi_pll_funcs, 0 },
{ "DPLL 2", DPLL_ID_SKL_DPLL2, &cnl_ddi_pll_funcs, 0 },
{ NULL, -1, NULL, },
{ "DPLL 0", &cnl_ddi_pll_funcs, DPLL_ID_SKL_DPLL0, 0 },
{ "DPLL 1", &cnl_ddi_pll_funcs, DPLL_ID_SKL_DPLL1, 0 },
{ "DPLL 2", &cnl_ddi_pll_funcs, DPLL_ID_SKL_DPLL2, 0 },
{ },
};
static const struct intel_dpll_mgr cnl_pll_mgr = {
@ -2415,13 +2430,9 @@ void intel_shared_dpll_init(struct drm_device *dev)
dpll_info = dpll_mgr->dpll_info;
for (i = 0; dpll_info[i].id >= 0; i++) {
for (i = 0; dpll_info[i].name; i++) {
WARN_ON(i != dpll_info[i].id);
dev_priv->shared_dplls[i].id = dpll_info[i].id;
dev_priv->shared_dplls[i].name = dpll_info[i].name;
dev_priv->shared_dplls[i].funcs = *dpll_info[i].funcs;
dev_priv->shared_dplls[i].flags = dpll_info[i].flags;
dev_priv->shared_dplls[i].info = &dpll_info[i];
}
dev_priv->dpll_mgr = dpll_mgr;
@ -2481,7 +2492,7 @@ void intel_release_shared_dpll(struct intel_shared_dpll *dpll,
struct intel_shared_dpll_state *shared_dpll_state;
shared_dpll_state = intel_atomic_get_shared_dpll_state(state);
shared_dpll_state[dpll->id].crtc_mask &= ~(1 << crtc->pipe);
shared_dpll_state[dpll->info->id].crtc_mask &= ~(1 << crtc->pipe);
}
/**

56
drivers/gpu/drm/i915/intel_dpll_mgr.h
View File

@ -205,6 +205,37 @@ struct intel_shared_dpll_funcs {
struct intel_dpll_hw_state *hw_state);
};
/**
* struct dpll_info - display PLL platform specific info
*/
struct dpll_info {
/**
* @name: DPLL name; used for logging
*/
const char *name;
/**
* @funcs: platform specific hooks
*/
const struct intel_shared_dpll_funcs *funcs;
/**
* @id: unique indentifier for this DPLL; should match the index in the
* dev_priv->shared_dplls array
*/
enum intel_dpll_id id;
#define INTEL_DPLL_ALWAYS_ON (1 << 0)
/**
* @flags:
*
* INTEL_DPLL_ALWAYS_ON
* Inform the state checker that the DPLL is kept enabled even if
* not in use by any CRTC.
*/
uint32_t flags;
};
/**
* struct intel_shared_dpll - display PLL with tracked state and users
*/
@ -228,30 +259,9 @@ struct intel_shared_dpll {
bool on;
/**
* @name: DPLL name; used for logging
* @info: platform specific info
*/
const char *name;
/**
* @id: unique indentifier for this DPLL; should match the index in the
* dev_priv->shared_dplls array
*/
enum intel_dpll_id id;
/**
* @funcs: platform specific hooks
*/
struct intel_shared_dpll_funcs funcs;
#define INTEL_DPLL_ALWAYS_ON (1 << 0)
/**
* @flags:
*
* INTEL_DPLL_ALWAYS_ON
* Inform the state checker that the DPLL is kept enabled even if
* not in use by any CRTC.
*/
uint32_t flags;
const struct dpll_info *info;
};
#define SKL_DPLL0 0

58
drivers/gpu/drm/i915/intel_drv.h
View File

@ -482,7 +482,7 @@ struct intel_atomic_state {
bool skip_intermediate_wm;
/* Gen9+ only */
struct skl_wm_values wm_results;
struct skl_ddb_values wm_results;
struct i915_sw_fence commit_ready;
@ -548,6 +548,12 @@ struct intel_initial_plane_config {
#define SKL_MAX_DST_W 4096
#define SKL_MIN_DST_H 8
#define SKL_MAX_DST_H 4096
#define ICL_MAX_SRC_W 5120
#define ICL_MAX_SRC_H 4096
#define ICL_MAX_DST_W 5120
#define ICL_MAX_DST_H 4096
#define SKL_MIN_YUV_420_SRC_W 16
#define SKL_MIN_YUV_420_SRC_H 16
struct intel_scaler {
int in_use;
@ -598,7 +604,9 @@ struct intel_pipe_wm {
struct skl_plane_wm {
struct skl_wm_level wm[8];
struct skl_wm_level uv_wm[8];
struct skl_wm_level trans_wm;
bool is_planar;
};
struct skl_pipe_wm {
@ -1325,6 +1333,7 @@ void gen5_enable_gt_irq(struct drm_i915_private *dev_priv, uint32_t mask);
void gen5_disable_gt_irq(struct drm_i915_private *dev_priv, uint32_t mask);
void gen6_mask_pm_irq(struct drm_i915_private *dev_priv, u32 mask);
void gen6_unmask_pm_irq(struct drm_i915_private *dev_priv, u32 mask);
void gen11_reset_rps_interrupts(struct drm_i915_private *dev_priv);
void gen6_reset_rps_interrupts(struct drm_i915_private *dev_priv);
void gen6_enable_rps_interrupts(struct drm_i915_private *dev_priv);
void gen6_disable_rps_interrupts(struct drm_i915_private *dev_priv);
@ -1588,9 +1597,12 @@ void hsw_disable_ips(const struct intel_crtc_state *crtc_state);
enum intel_display_power_domain intel_port_to_power_domain(enum port port);
void intel_mode_from_pipe_config(struct drm_display_mode *mode,
struct intel_crtc_state *pipe_config);
void intel_crtc_arm_fifo_underrun(struct intel_crtc *crtc,
struct intel_crtc_state *crtc_state);
int skl_update_scaler_crtc(struct intel_crtc_state *crtc_state);
int skl_max_scale(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state);
int skl_max_scale(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
uint32_t pixel_format);
static inline u32 intel_plane_ggtt_offset(const struct intel_plane_state *state)
{
@ -1607,6 +1619,7 @@ u32 skl_plane_stride(const struct drm_framebuffer *fb, int plane,
int skl_check_plane_surface(const struct intel_crtc_state *crtc_state,
struct intel_plane_state *plane_state);
int i9xx_check_plane_surface(struct intel_plane_state *plane_state);
int skl_format_to_fourcc(int format, bool rgb_order, bool alpha);
/* intel_csr.c */
void intel_csr_ucode_init(struct drm_i915_private *);
@ -1773,6 +1786,7 @@ void intel_fbc_flush(struct drm_i915_private *dev_priv,
unsigned int frontbuffer_bits, enum fb_op_origin origin);
void intel_fbc_cleanup_cfb(struct drm_i915_private *dev_priv);
void intel_fbc_handle_fifo_underrun_irq(struct drm_i915_private *dev_priv);
int intel_fbc_reset_underrun(struct drm_i915_private *dev_priv);
/* intel_hdmi.c */
void intel_hdmi_init(struct drm_i915_private *dev_priv, i915_reg_t hdmi_reg,
@ -1783,7 +1797,7 @@ struct intel_hdmi *enc_to_intel_hdmi(struct drm_encoder *encoder);
bool intel_hdmi_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state);
void intel_hdmi_handle_sink_scrambling(struct intel_encoder *intel_encoder,
bool intel_hdmi_handle_sink_scrambling(struct intel_encoder *encoder,
struct drm_connector *connector,
bool high_tmds_clock_ratio,
bool scrambling);
@ -1877,7 +1891,8 @@ void intel_psr_enable(struct intel_dp *intel_dp,
void intel_psr_disable(struct intel_dp *intel_dp,
const struct intel_crtc_state *old_crtc_state);
void intel_psr_invalidate(struct drm_i915_private *dev_priv,
unsigned frontbuffer_bits);
unsigned frontbuffer_bits,
enum fb_op_origin origin);
void intel_psr_flush(struct drm_i915_private *dev_priv,
unsigned frontbuffer_bits,
enum fb_op_origin origin);
@ -2046,6 +2061,7 @@ void skl_disable_plane(struct intel_plane *plane, struct intel_crtc *crtc);
bool skl_plane_get_hw_state(struct intel_plane *plane);
bool skl_plane_has_ccs(struct drm_i915_private *dev_priv,
enum pipe pipe, enum plane_id plane_id);
bool intel_format_is_yuv(uint32_t format);
/* intel_tv.c */
void intel_tv_init(struct drm_i915_private *dev_priv);
@ -2082,31 +2098,6 @@ intel_atomic_get_crtc_state(struct drm_atomic_state *state,
return to_intel_crtc_state(crtc_state);
}
static inline struct intel_crtc_state *
intel_atomic_get_existing_crtc_state(struct drm_atomic_state *state,
struct intel_crtc *crtc)
{
struct drm_crtc_state *crtc_state;
crtc_state = drm_atomic_get_existing_crtc_state(state, &crtc->base);
if (crtc_state)
return to_intel_crtc_state(crtc_state);
else
return NULL;
}
static inline struct intel_plane_state *
intel_atomic_get_existing_plane_state(struct drm_atomic_state *state,
struct intel_plane *plane)
{
struct drm_plane_state *plane_state;
plane_state = drm_atomic_get_existing_plane_state(state, &plane->base);
return to_intel_plane_state(plane_state);
}
int intel_atomic_setup_scalers(struct drm_i915_private *dev_priv,
struct intel_crtc *intel_crtc,
struct intel_crtc_state *crtc_state);
@ -2138,8 +2129,17 @@ int intel_pipe_crc_create(struct drm_minor *minor);
#ifdef CONFIG_DEBUG_FS
int intel_crtc_set_crc_source(struct drm_crtc *crtc, const char *source_name,
size_t *values_cnt);
void intel_crtc_disable_pipe_crc(struct intel_crtc *crtc);
void intel_crtc_enable_pipe_crc(struct intel_crtc *crtc);
#else
#define intel_crtc_set_crc_source NULL
static inline void intel_crtc_disable_pipe_crc(struct intel_crtc *crtc)
{
}
static inline void intel_crtc_enable_pipe_crc(struct intel_crtc *crtc)
{
}
#endif
extern const struct file_operations i915_display_crc_ctl_fops;
#endif /* __INTEL_DRV_H__ */

796
drivers/gpu/drm/i915/intel_engine_cs.c
View File

@ -81,13 +81,17 @@ static const struct engine_class_info intel_engine_classes[] = {
},
};
#define MAX_MMIO_BASES 3
struct engine_info {
unsigned int hw_id;
unsigned int uabi_id;
u8 class;
u8 instance;
u32 mmio_base;
unsigned irq_shift;
/* mmio bases table *must* be sorted in reverse gen order */
struct engine_mmio_base {
u32 gen : 8;
u32 base : 24;
} mmio_bases[MAX_MMIO_BASES];
};
static const struct engine_info intel_engines[] = {
@ -96,64 +100,76 @@ static const struct engine_info intel_engines[] = {
.uabi_id = I915_EXEC_RENDER,
.class = RENDER_CLASS,
.instance = 0,
.mmio_base = RENDER_RING_BASE,
.irq_shift = GEN8_RCS_IRQ_SHIFT,
.mmio_bases = {
{ .gen = 1, .base = RENDER_RING_BASE }
},
},
[BCS] = {
.hw_id = BCS_HW,
.uabi_id = I915_EXEC_BLT,
.class = COPY_ENGINE_CLASS,
.instance = 0,
.mmio_base = BLT_RING_BASE,
.irq_shift = GEN8_BCS_IRQ_SHIFT,
.mmio_bases = {
{ .gen = 6, .base = BLT_RING_BASE }
},
},
[VCS] = {
.hw_id = VCS_HW,
.uabi_id = I915_EXEC_BSD,
.class = VIDEO_DECODE_CLASS,
.instance = 0,
.mmio_base = GEN6_BSD_RING_BASE,
.irq_shift = GEN8_VCS1_IRQ_SHIFT,
.mmio_bases = {
{ .gen = 11, .base = GEN11_BSD_RING_BASE },
{ .gen = 6, .base = GEN6_BSD_RING_BASE },
{ .gen = 4, .base = BSD_RING_BASE }
},
},
[VCS2] = {
.hw_id = VCS2_HW,
.uabi_id = I915_EXEC_BSD,
.class = VIDEO_DECODE_CLASS,
.instance = 1,
.mmio_base = GEN8_BSD2_RING_BASE,
.irq_shift = GEN8_VCS2_IRQ_SHIFT,
.mmio_bases = {
{ .gen = 11, .base = GEN11_BSD2_RING_BASE },
{ .gen = 8, .base = GEN8_BSD2_RING_BASE }
},
},
[VCS3] = {
.hw_id = VCS3_HW,
.uabi_id = I915_EXEC_BSD,
.class = VIDEO_DECODE_CLASS,
.instance = 2,
.mmio_base = GEN11_BSD3_RING_BASE,
.irq_shift = 0, /* not used */
.mmio_bases = {
{ .gen = 11, .base = GEN11_BSD3_RING_BASE }
},
},
[VCS4] = {
.hw_id = VCS4_HW,
.uabi_id = I915_EXEC_BSD,
.class = VIDEO_DECODE_CLASS,
.instance = 3,
.mmio_base = GEN11_BSD4_RING_BASE,
.irq_shift = 0, /* not used */
.mmio_bases = {
{ .gen = 11, .base = GEN11_BSD4_RING_BASE }
},
},
[VECS] = {
.hw_id = VECS_HW,
.uabi_id = I915_EXEC_VEBOX,
.class = VIDEO_ENHANCEMENT_CLASS,
.instance = 0,
.mmio_base = VEBOX_RING_BASE,
.irq_shift = GEN8_VECS_IRQ_SHIFT,
.mmio_bases = {
{ .gen = 11, .base = GEN11_VEBOX_RING_BASE },
{ .gen = 7, .base = VEBOX_RING_BASE }
},
},
[VECS2] = {
.hw_id = VECS2_HW,
.uabi_id = I915_EXEC_VEBOX,
.class = VIDEO_ENHANCEMENT_CLASS,
.instance = 1,
.mmio_base = GEN11_VEBOX2_RING_BASE,
.irq_shift = 0, /* not used */
.mmio_bases = {
{ .gen = 11, .base = GEN11_VEBOX2_RING_BASE }
},
},
};
@ -223,16 +239,36 @@ __intel_engine_context_size(struct drm_i915_private *dev_priv, u8 class)
}
}
static u32 __engine_mmio_base(struct drm_i915_private *i915,
const struct engine_mmio_base *bases)
{
int i;
for (i = 0; i < MAX_MMIO_BASES; i++)
if (INTEL_GEN(i915) >= bases[i].gen)
break;
GEM_BUG_ON(i == MAX_MMIO_BASES);
GEM_BUG_ON(!bases[i].base);
return bases[i].base;
}
static void __sprint_engine_name(char *name, const struct engine_info *info)
{
WARN_ON(snprintf(name, INTEL_ENGINE_CS_MAX_NAME, "%s%u",
intel_engine_classes[info->class].name,
info->instance) >= INTEL_ENGINE_CS_MAX_NAME);
}
static int
intel_engine_setup(struct drm_i915_private *dev_priv,
enum intel_engine_id id)
{
const struct engine_info *info = &intel_engines[id];
const struct engine_class_info *class_info;
struct intel_engine_cs *engine;
GEM_BUG_ON(info->class >= ARRAY_SIZE(intel_engine_classes));
class_info = &intel_engine_classes[info->class];
BUILD_BUG_ON(MAX_ENGINE_CLASS >= BIT(GEN11_ENGINE_CLASS_WIDTH));
BUILD_BUG_ON(MAX_ENGINE_INSTANCE >= BIT(GEN11_ENGINE_INSTANCE_WIDTH));
@ -253,35 +289,14 @@ intel_engine_setup(struct drm_i915_private *dev_priv,
engine->id = id;
engine->i915 = dev_priv;
WARN_ON(snprintf(engine->name, sizeof(engine->name), "%s%u",
class_info->name, info->instance) >=
sizeof(engine->name));
__sprint_engine_name(engine->name, info);
engine->hw_id = engine->guc_id = info->hw_id;
if (INTEL_GEN(dev_priv) >= 11) {
switch (engine->id) {
case VCS:
engine->mmio_base = GEN11_BSD_RING_BASE;
break;
case VCS2:
engine->mmio_base = GEN11_BSD2_RING_BASE;
break;
case VECS:
engine->mmio_base = GEN11_VEBOX_RING_BASE;
break;
default:
/* take the original value for all other engines */
engine->mmio_base = info->mmio_base;
break;
}
} else {
engine->mmio_base = info->mmio_base;
}
engine->irq_shift = info->irq_shift;
engine->mmio_base = __engine_mmio_base(dev_priv, info->mmio_bases);
engine->class = info->class;
engine->instance = info->instance;
engine->uabi_id = info->uabi_id;
engine->uabi_class = class_info->uabi_class;
engine->uabi_class = intel_engine_classes[info->class].uabi_class;
engine->context_size = __intel_engine_context_size(dev_priv,
engine->class);
@ -441,6 +456,11 @@ static void intel_engine_init_timeline(struct intel_engine_cs *engine)
engine->timeline = &engine->i915->gt.global_timeline.engine[engine->id];
}
static void intel_engine_init_batch_pool(struct intel_engine_cs *engine)
{
i915_gem_batch_pool_init(&engine->batch_pool, engine);
}
static bool csb_force_mmio(struct drm_i915_private *i915)
{
/*
@ -455,6 +475,9 @@ static bool csb_force_mmio(struct drm_i915_private *i915)
if (intel_vgpu_active(i915) && !intel_vgpu_has_hwsp_emulation(i915))
return true;
if (IS_CANNONLAKE(i915))
return true;
return false;
}
@ -485,11 +508,9 @@ static void intel_engine_init_execlist(struct intel_engine_cs *engine)
void intel_engine_setup_common(struct intel_engine_cs *engine)
{
intel_engine_init_execlist(engine);
intel_engine_init_timeline(engine);
intel_engine_init_hangcheck(engine);
i915_gem_batch_pool_init(engine, &engine->batch_pool);
intel_engine_init_batch_pool(engine);
intel_engine_init_cmd_parser(engine);
}
@ -782,10 +803,24 @@ static inline uint32_t
read_subslice_reg(struct drm_i915_private *dev_priv, int slice,
int subslice, i915_reg_t reg)
{
uint32_t mcr_slice_subslice_mask;
uint32_t mcr_slice_subslice_select;
uint32_t mcr;
uint32_t ret;
enum forcewake_domains fw_domains;
if (INTEL_GEN(dev_priv) >= 11) {
mcr_slice_subslice_mask = GEN11_MCR_SLICE_MASK |
GEN11_MCR_SUBSLICE_MASK;
mcr_slice_subslice_select = GEN11_MCR_SLICE(slice) |
GEN11_MCR_SUBSLICE(subslice);
} else {
mcr_slice_subslice_mask = GEN8_MCR_SLICE_MASK |
GEN8_MCR_SUBSLICE_MASK;
mcr_slice_subslice_select = GEN8_MCR_SLICE(slice) |
GEN8_MCR_SUBSLICE(subslice);
}
fw_domains = intel_uncore_forcewake_for_reg(dev_priv, reg,
FW_REG_READ);
fw_domains |= intel_uncore_forcewake_for_reg(dev_priv,
@ -800,14 +835,14 @@ read_subslice_reg(struct drm_i915_private *dev_priv, int slice,
* The HW expects the slice and sublice selectors to be reset to 0
* after reading out the registers.
*/
WARN_ON_ONCE(mcr & (GEN8_MCR_SLICE_MASK | GEN8_MCR_SUBSLICE_MASK));
mcr &= ~(GEN8_MCR_SLICE_MASK | GEN8_MCR_SUBSLICE_MASK);
mcr |= GEN8_MCR_SLICE(slice) | GEN8_MCR_SUBSLICE(subslice);
WARN_ON_ONCE(mcr & mcr_slice_subslice_mask);
mcr &= ~mcr_slice_subslice_mask;
mcr |= mcr_slice_subslice_select;
I915_WRITE_FW(GEN8_MCR_SELECTOR, mcr);
ret = I915_READ_FW(reg);
mcr &= ~(GEN8_MCR_SLICE_MASK | GEN8_MCR_SUBSLICE_MASK);
mcr &= ~mcr_slice_subslice_mask;
I915_WRITE_FW(GEN8_MCR_SELECTOR, mcr);
intel_uncore_forcewake_put__locked(dev_priv, fw_domains);
@ -871,640 +906,6 @@ void intel_engine_get_instdone(struct intel_engine_cs *engine,
}
}
static int wa_add(struct drm_i915_private *dev_priv,
i915_reg_t addr,
const u32 mask, const u32 val)
{
const u32 idx = dev_priv->workarounds.count;
if (WARN_ON(idx >= I915_MAX_WA_REGS))
return -ENOSPC;
dev_priv->workarounds.reg[idx].addr = addr;
dev_priv->workarounds.reg[idx].value = val;
dev_priv->workarounds.reg[idx].mask = mask;
dev_priv->workarounds.count++;
return 0;
}
#define WA_REG(addr, mask, val) do { \
const int r = wa_add(dev_priv, (addr), (mask), (val)); \
if (r) \
return r; \
} while (0)
#define WA_SET_BIT_MASKED(addr, mask) \
WA_REG(addr, (mask), _MASKED_BIT_ENABLE(mask))
#define WA_CLR_BIT_MASKED(addr, mask) \
WA_REG(addr, (mask), _MASKED_BIT_DISABLE(mask))
#define WA_SET_FIELD_MASKED(addr, mask, value) \
WA_REG(addr, mask, _MASKED_FIELD(mask, value))
static int wa_ring_whitelist_reg(struct intel_engine_cs *engine,
i915_reg_t reg)
{
struct drm_i915_private *dev_priv = engine->i915;
struct i915_workarounds *wa = &dev_priv->workarounds;
const uint32_t index = wa->hw_whitelist_count[engine->id];
if (WARN_ON(index >= RING_MAX_NONPRIV_SLOTS))
return -EINVAL;
I915_WRITE(RING_FORCE_TO_NONPRIV(engine->mmio_base, index),
i915_mmio_reg_offset(reg));
wa->hw_whitelist_count[engine->id]++;
return 0;
}
static int gen8_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
WA_SET_BIT_MASKED(INSTPM, INSTPM_FORCE_ORDERING);
/* WaDisableAsyncFlipPerfMode:bdw,chv */
WA_SET_BIT_MASKED(MI_MODE, ASYNC_FLIP_PERF_DISABLE);
/* WaDisablePartialInstShootdown:bdw,chv */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
/* Use Force Non-Coherent whenever executing a 3D context. This is a
* workaround for for a possible hang in the unlikely event a TLB
* invalidation occurs during a PSD flush.
*/
/* WaForceEnableNonCoherent:bdw,chv */
/* WaHdcDisableFetchWhenMasked:bdw,chv */
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_DONOT_FETCH_MEM_WHEN_MASKED |
HDC_FORCE_NON_COHERENT);
/* From the Haswell PRM, Command Reference: Registers, CACHE_MODE_0:
* "The Hierarchical Z RAW Stall Optimization allows non-overlapping
* polygons in the same 8x4 pixel/sample area to be processed without
* stalling waiting for the earlier ones to write to Hierarchical Z
* buffer."
*
* This optimization is off by default for BDW and CHV; turn it on.
*/
WA_CLR_BIT_MASKED(CACHE_MODE_0_GEN7, HIZ_RAW_STALL_OPT_DISABLE);
/* Wa4x4STCOptimizationDisable:bdw,chv */
WA_SET_BIT_MASKED(CACHE_MODE_1, GEN8_4x4_STC_OPTIMIZATION_DISABLE);
/*
* BSpec recommends 8x4 when MSAA is used,
* however in practice 16x4 seems fastest.
*
* Note that PS/WM thread counts depend on the WIZ hashing
* disable bit, which we don't touch here, but it's good
* to keep in mind (see 3DSTATE_PS and 3DSTATE_WM).
*/
WA_SET_FIELD_MASKED(GEN7_GT_MODE,
GEN6_WIZ_HASHING_MASK,
GEN6_WIZ_HASHING_16x4);
return 0;
}
static int bdw_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
ret = gen8_init_workarounds(engine);
if (ret)
return ret;
/* WaDisableThreadStallDopClockGating:bdw (pre-production) */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
/* WaDisableDopClockGating:bdw
*
* Also see the related UCGTCL1 write in broadwell_init_clock_gating()
* to disable EUTC clock gating.
*/
WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2,
DOP_CLOCK_GATING_DISABLE);
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
GEN8_SAMPLER_POWER_BYPASS_DIS);
WA_SET_BIT_MASKED(HDC_CHICKEN0,
/* WaForceContextSaveRestoreNonCoherent:bdw */
HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
/* WaDisableFenceDestinationToSLM:bdw (pre-prod) */
(IS_BDW_GT3(dev_priv) ? HDC_FENCE_DEST_SLM_DISABLE : 0));
return 0;
}
static int chv_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
ret = gen8_init_workarounds(engine);
if (ret)
return ret;
/* WaDisableThreadStallDopClockGating:chv */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
/* Improve HiZ throughput on CHV. */
WA_SET_BIT_MASKED(HIZ_CHICKEN, CHV_HZ_8X8_MODE_IN_1X);
return 0;
}
static int gen9_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
/* WaConextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk,cfl */
I915_WRITE(GEN9_CSFE_CHICKEN1_RCS, _MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
/* WaEnableLbsSlaRetryTimerDecrement:skl,bxt,kbl,glk,cfl */
I915_WRITE(BDW_SCRATCH1, I915_READ(BDW_SCRATCH1) |
GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE);
/* WaDisableKillLogic:bxt,skl,kbl */
if (!IS_COFFEELAKE(dev_priv))
I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
ECOCHK_DIS_TLB);
if (HAS_LLC(dev_priv)) {
/* WaCompressedResourceSamplerPbeMediaNewHashMode:skl,kbl
*
* Must match Display Engine. See
* WaCompressedResourceDisplayNewHashMode.
*/
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN9_PBE_COMPRESSED_HASH_SELECTION);
WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
GEN9_SAMPLER_HASH_COMPRESSED_READ_ADDR);
I915_WRITE(MMCD_MISC_CTRL,
I915_READ(MMCD_MISC_CTRL) |
MMCD_PCLA |
MMCD_HOTSPOT_EN);
}
/* WaClearFlowControlGpgpuContextSave:skl,bxt,kbl,glk,cfl */
/* WaDisablePartialInstShootdown:skl,bxt,kbl,glk,cfl */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
FLOW_CONTROL_ENABLE |
PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
/* Syncing dependencies between camera and graphics:skl,bxt,kbl */
if (!IS_COFFEELAKE(dev_priv))
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
GEN9_DISABLE_OCL_OOB_SUPPRESS_LOGIC);
/* WaEnableYV12BugFixInHalfSliceChicken7:skl,bxt,kbl,glk,cfl */
/* WaEnableSamplerGPGPUPreemptionSupport:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
GEN9_ENABLE_YV12_BUGFIX |
GEN9_ENABLE_GPGPU_PREEMPTION);
/* Wa4x4STCOptimizationDisable:skl,bxt,kbl,glk,cfl */
/* WaDisablePartialResolveInVc:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(CACHE_MODE_1, (GEN8_4x4_STC_OPTIMIZATION_DISABLE |
GEN9_PARTIAL_RESOLVE_IN_VC_DISABLE));
/* WaCcsTlbPrefetchDisable:skl,bxt,kbl,glk,cfl */
WA_CLR_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN5,
GEN9_CCS_TLB_PREFETCH_ENABLE);
/* WaForceContextSaveRestoreNonCoherent:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
HDC_FORCE_CSR_NON_COHERENT_OVR_DISABLE);
/* WaForceEnableNonCoherent and WaDisableHDCInvalidation are
* both tied to WaForceContextSaveRestoreNonCoherent
* in some hsds for skl. We keep the tie for all gen9. The
* documentation is a bit hazy and so we want to get common behaviour,
* even though there is no clear evidence we would need both on kbl/bxt.
* This area has been source of system hangs so we play it safe
* and mimic the skl regardless of what bspec says.
*
* Use Force Non-Coherent whenever executing a 3D context. This
* is a workaround for a possible hang in the unlikely event
* a TLB invalidation occurs during a PSD flush.
*/
/* WaForceEnableNonCoherent:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FORCE_NON_COHERENT);
/* WaDisableHDCInvalidation:skl,bxt,kbl,cfl */
I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
BDW_DISABLE_HDC_INVALIDATION);
/* WaDisableSamplerPowerBypassForSOPingPong:skl,bxt,kbl,cfl */
if (IS_SKYLAKE(dev_priv) ||
IS_KABYLAKE(dev_priv) ||
IS_COFFEELAKE(dev_priv))
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
GEN8_SAMPLER_POWER_BYPASS_DIS);
/* WaDisableSTUnitPowerOptimization:skl,bxt,kbl,glk,cfl */
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN2, GEN8_ST_PO_DISABLE);
/* WaProgramL3SqcReg1DefaultForPerf:bxt,glk */
if (IS_GEN9_LP(dev_priv)) {
u32 val = I915_READ(GEN8_L3SQCREG1);
val &= ~L3_PRIO_CREDITS_MASK;
val |= L3_GENERAL_PRIO_CREDITS(62) | L3_HIGH_PRIO_CREDITS(2);
I915_WRITE(GEN8_L3SQCREG1, val);
}
/* WaOCLCoherentLineFlush:skl,bxt,kbl,cfl */
I915_WRITE(GEN8_L3SQCREG4, (I915_READ(GEN8_L3SQCREG4) |
GEN8_LQSC_FLUSH_COHERENT_LINES));
/*
* Supporting preemption with fine-granularity requires changes in the
* batch buffer programming. Since we can't break old userspace, we
* need to set our default preemption level to safe value. Userspace is
* still able to use more fine-grained preemption levels, since in
* WaEnablePreemptionGranularityControlByUMD we're whitelisting the
* per-ctx register. As such, WaDisable{3D,GPGPU}MidCmdPreemption are
* not real HW workarounds, but merely a way to start using preemption
* while maintaining old contract with userspace.
*/
/* WaDisable3DMidCmdPreemption:skl,bxt,glk,cfl,[cnl] */
WA_CLR_BIT_MASKED(GEN8_CS_CHICKEN1, GEN9_PREEMPT_3D_OBJECT_LEVEL);
/* WaDisableGPGPUMidCmdPreemption:skl,bxt,blk,cfl,[cnl] */
WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1, GEN9_PREEMPT_GPGPU_LEVEL_MASK,
GEN9_PREEMPT_GPGPU_COMMAND_LEVEL);
/* WaVFEStateAfterPipeControlwithMediaStateClear:skl,bxt,glk,cfl */
ret = wa_ring_whitelist_reg(engine, GEN9_CTX_PREEMPT_REG);
if (ret)
return ret;
/* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,[cnl] */
I915_WRITE(GEN7_FF_SLICE_CS_CHICKEN1,
_MASKED_BIT_ENABLE(GEN9_FFSC_PERCTX_PREEMPT_CTRL));
ret = wa_ring_whitelist_reg(engine, GEN8_CS_CHICKEN1);
if (ret)
return ret;
/* WaAllowUMDToModifyHDCChicken1:skl,bxt,kbl,glk,cfl */
ret = wa_ring_whitelist_reg(engine, GEN8_HDC_CHICKEN1);
if (ret)
return ret;
return 0;
}
static int skl_tune_iz_hashing(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
u8 vals[3] = { 0, 0, 0 };
unsigned int i;
for (i = 0; i < 3; i++) {
u8 ss;
/*
* Only consider slices where one, and only one, subslice has 7
* EUs
*/
if (!is_power_of_2(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]))
continue;
/*
* subslice_7eu[i] != 0 (because of the check above) and
* ss_max == 4 (maximum number of subslices possible per slice)
*
* -> 0 <= ss <= 3;
*/
ss = ffs(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]) - 1;
vals[i] = 3 - ss;
}
if (vals[0] == 0 && vals[1] == 0 && vals[2] == 0)
return 0;
/* Tune IZ hashing. See intel_device_info_runtime_init() */
WA_SET_FIELD_MASKED(GEN7_GT_MODE,
GEN9_IZ_HASHING_MASK(2) |
GEN9_IZ_HASHING_MASK(1) |
GEN9_IZ_HASHING_MASK(0),
GEN9_IZ_HASHING(2, vals[2]) |
GEN9_IZ_HASHING(1, vals[1]) |
GEN9_IZ_HASHING(0, vals[0]));
return 0;
}
static int skl_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
ret = gen9_init_workarounds(engine);
if (ret)
return ret;
/* WaEnableGapsTsvCreditFix:skl */
I915_WRITE(GEN8_GARBCNTL, (I915_READ(GEN8_GARBCNTL) |
GEN9_GAPS_TSV_CREDIT_DISABLE));
/* WaDisableGafsUnitClkGating:skl */
I915_WRITE(GEN7_UCGCTL4, (I915_READ(GEN7_UCGCTL4) |
GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE));
/* WaInPlaceDecompressionHang:skl */
if (IS_SKL_REVID(dev_priv, SKL_REVID_H0, REVID_FOREVER))
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
(I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS));
/* WaDisableLSQCROPERFforOCL:skl */
ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
if (ret)
return ret;
return skl_tune_iz_hashing(engine);
}
static int bxt_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
ret = gen9_init_workarounds(engine);
if (ret)
return ret;
/* WaDisableThreadStallDopClockGating:bxt */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
STALL_DOP_GATING_DISABLE);
/* WaDisablePooledEuLoadBalancingFix:bxt */
I915_WRITE(FF_SLICE_CS_CHICKEN2,
_MASKED_BIT_ENABLE(GEN9_POOLED_EU_LOAD_BALANCING_FIX_DISABLE));
/* WaToEnableHwFixForPushConstHWBug:bxt */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaInPlaceDecompressionHang:bxt */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
(I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS));
return 0;
}
static int cnl_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
/* WaDisableI2mCycleOnWRPort:cnl (pre-prod) */
if (IS_CNL_REVID(dev_priv, CNL_REVID_B0, CNL_REVID_B0))
I915_WRITE(GAMT_CHKN_BIT_REG,
(I915_READ(GAMT_CHKN_BIT_REG) |
GAMT_CHKN_DISABLE_I2M_CYCLE_ON_WR_PORT));
/* WaForceContextSaveRestoreNonCoherent:cnl */
WA_SET_BIT_MASKED(CNL_HDC_CHICKEN0,
HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT);
/* WaThrottleEUPerfToAvoidTDBackPressure:cnl(pre-prod) */
if (IS_CNL_REVID(dev_priv, CNL_REVID_B0, CNL_REVID_B0))
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, THROTTLE_12_5);
/* WaDisableReplayBufferBankArbitrationOptimization:cnl */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableEnhancedSBEVertexCaching:cnl (pre-prod) */
if (IS_CNL_REVID(dev_priv, 0, CNL_REVID_B0))
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_CSC2_SBE_VUE_CACHE_CONSERVATIVE);
/* WaInPlaceDecompressionHang:cnl */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
(I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS));
/* WaPushConstantDereferenceHoldDisable:cnl */
WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2, PUSH_CONSTANT_DEREF_DISABLE);
/* FtrEnableFastAnisoL1BankingFix: cnl */
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3, CNL_FAST_ANISO_L1_BANKING_FIX);
/* WaDisable3DMidCmdPreemption:cnl */
WA_CLR_BIT_MASKED(GEN8_CS_CHICKEN1, GEN9_PREEMPT_3D_OBJECT_LEVEL);
/* WaDisableGPGPUMidCmdPreemption:cnl */
WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1, GEN9_PREEMPT_GPGPU_LEVEL_MASK,
GEN9_PREEMPT_GPGPU_COMMAND_LEVEL);
/* WaEnablePreemptionGranularityControlByUMD:cnl */
I915_WRITE(GEN7_FF_SLICE_CS_CHICKEN1,
_MASKED_BIT_ENABLE(GEN9_FFSC_PERCTX_PREEMPT_CTRL));
ret= wa_ring_whitelist_reg(engine, GEN8_CS_CHICKEN1);
if (ret)
return ret;
/* WaDisableEarlyEOT:cnl */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, DISABLE_EARLY_EOT);
return 0;
}
static int kbl_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
ret = gen9_init_workarounds(engine);
if (ret)
return ret;
/* WaEnableGapsTsvCreditFix:kbl */
I915_WRITE(GEN8_GARBCNTL, (I915_READ(GEN8_GARBCNTL) |
GEN9_GAPS_TSV_CREDIT_DISABLE));
/* WaDisableDynamicCreditSharing:kbl */
if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_B0))
I915_WRITE(GAMT_CHKN_BIT_REG,
(I915_READ(GAMT_CHKN_BIT_REG) |
GAMT_CHKN_DISABLE_DYNAMIC_CREDIT_SHARING));
/* WaDisableFenceDestinationToSLM:kbl (pre-prod) */
if (IS_KBL_REVID(dev_priv, KBL_REVID_A0, KBL_REVID_A0))
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FENCE_DEST_SLM_DISABLE);
/* WaToEnableHwFixForPushConstHWBug:kbl */
if (IS_KBL_REVID(dev_priv, KBL_REVID_C0, REVID_FOREVER))
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableGafsUnitClkGating:kbl */
I915_WRITE(GEN7_UCGCTL4, (I915_READ(GEN7_UCGCTL4) |
GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE));
/* WaDisableSbeCacheDispatchPortSharing:kbl */
WA_SET_BIT_MASKED(
GEN7_HALF_SLICE_CHICKEN1,
GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
/* WaInPlaceDecompressionHang:kbl */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
(I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS));
/* WaDisableLSQCROPERFforOCL:kbl */
ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
if (ret)
return ret;
return 0;
}
static int glk_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
ret = gen9_init_workarounds(engine);
if (ret)
return ret;
/* WA #0862: Userspace has to set "Barrier Mode" to avoid hangs. */
ret = wa_ring_whitelist_reg(engine, GEN9_SLICE_COMMON_ECO_CHICKEN1);
if (ret)
return ret;
/* WaToEnableHwFixForPushConstHWBug:glk */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
return 0;
}
static int cfl_init_workarounds(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int ret;
ret = gen9_init_workarounds(engine);
if (ret)
return ret;
/* WaEnableGapsTsvCreditFix:cfl */
I915_WRITE(GEN8_GARBCNTL, (I915_READ(GEN8_GARBCNTL) |
GEN9_GAPS_TSV_CREDIT_DISABLE));
/* WaToEnableHwFixForPushConstHWBug:cfl */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableGafsUnitClkGating:cfl */
I915_WRITE(GEN7_UCGCTL4, (I915_READ(GEN7_UCGCTL4) |
GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE));
/* WaDisableSbeCacheDispatchPortSharing:cfl */
WA_SET_BIT_MASKED(
GEN7_HALF_SLICE_CHICKEN1,
GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
/* WaInPlaceDecompressionHang:cfl */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
(I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS));
return 0;
}
int init_workarounds_ring(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int err;
if (GEM_WARN_ON(engine->id != RCS))
return -EINVAL;
dev_priv->workarounds.count = 0;
dev_priv->workarounds.hw_whitelist_count[engine->id] = 0;
if (IS_BROADWELL(dev_priv))
err = bdw_init_workarounds(engine);
else if (IS_CHERRYVIEW(dev_priv))
err = chv_init_workarounds(engine);
else if (IS_SKYLAKE(dev_priv))
err = skl_init_workarounds(engine);
else if (IS_BROXTON(dev_priv))
err = bxt_init_workarounds(engine);
else if (IS_KABYLAKE(dev_priv))
err = kbl_init_workarounds(engine);
else if (IS_GEMINILAKE(dev_priv))
err = glk_init_workarounds(engine);
else if (IS_COFFEELAKE(dev_priv))
err = cfl_init_workarounds(engine);
else if (IS_CANNONLAKE(dev_priv))
err = cnl_init_workarounds(engine);
else
err = 0;
if (err)
return err;
DRM_DEBUG_DRIVER("%s: Number of context specific w/a: %d\n",
engine->name, dev_priv->workarounds.count);
return 0;
}
int intel_ring_workarounds_emit(struct i915_request *rq)
{
struct i915_workarounds *w = &rq->i915->workarounds;
u32 *cs;
int ret, i;
if (w->count == 0)
return 0;
ret = rq->engine->emit_flush(rq, EMIT_BARRIER);
if (ret)
return ret;
cs = intel_ring_begin(rq, w->count * 2 + 2);
if (IS_ERR(cs))
return PTR_ERR(cs);
*cs++ = MI_LOAD_REGISTER_IMM(w->count);
for (i = 0; i < w->count; i++) {
*cs++ = i915_mmio_reg_offset(w->reg[i].addr);
*cs++ = w->reg[i].value;
}
*cs++ = MI_NOOP;
intel_ring_advance(rq, cs);
ret = rq->engine->emit_flush(rq, EMIT_BARRIER);
if (ret)
return ret;
return 0;
}
static bool ring_is_idle(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
@ -1655,6 +1056,9 @@ void intel_engines_park(struct drm_i915_private *i915)
intel_engine_dump(engine, &p, NULL);
}
/* Must be reset upon idling, or we may miss the busy wakeup. */
GEM_BUG_ON(engine->execlists.queue_priority != INT_MIN);
if (engine->park)
engine->park(engine);
@ -1713,13 +1117,15 @@ static void print_request(struct drm_printer *m,
struct i915_request *rq,
const char *prefix)
{
const char *name = rq->fence.ops->get_timeline_name(&rq->fence);
drm_printf(m, "%s%x%s [%llx:%x] prio=%d @ %dms: %s\n", prefix,
rq->global_seqno,
i915_request_completed(rq) ? "!" : "",
rq->fence.context, rq->fence.seqno,
rq->priotree.priority,
jiffies_to_msecs(jiffies - rq->emitted_jiffies),
rq->timeline->common->name);
name);
}
static void hexdump(struct drm_printer *m, const void *buf, size_t len)
@ -1825,12 +1231,15 @@ static void intel_engine_print_registers(const struct intel_engine_cs *engine,
ptr = I915_READ(RING_CONTEXT_STATUS_PTR(engine));
read = GEN8_CSB_READ_PTR(ptr);
write = GEN8_CSB_WRITE_PTR(ptr);
drm_printf(m, "\tExeclist CSB read %d [%d cached], write %d [%d from hws], interrupt posted? %s\n",
drm_printf(m, "\tExeclist CSB read %d [%d cached], write %d [%d from hws], interrupt posted? %s, tasklet queued? %s (%s)\n",
read, execlists->csb_head,
write,
intel_read_status_page(engine, intel_hws_csb_write_index(engine->i915)),
yesno(test_bit(ENGINE_IRQ_EXECLIST,
&engine->irq_posted)));
&engine->irq_posted)),
yesno(test_bit(TASKLET_STATE_SCHED,
&engine->execlists.tasklet.state)),
enableddisabled(!atomic_read(&engine->execlists.tasklet.count)));
if (read >= GEN8_CSB_ENTRIES)
read = 0;
if (write >= GEN8_CSB_ENTRIES)
@ -1929,12 +1338,16 @@ void intel_engine_dump(struct intel_engine_cs *engine,
rq->head, rq->postfix, rq->tail,
rq->batch ? upper_32_bits(rq->batch->node.start) : ~0u,
rq->batch ? lower_32_bits(rq->batch->node.start) : ~0u);
drm_printf(m, "\t\tring->start: 0x%08x\n",
drm_printf(m, "\t\tring->start: 0x%08x\n",
i915_ggtt_offset(rq->ring->vma));
drm_printf(m, "\t\tring->head: 0x%08x\n",
drm_printf(m, "\t\tring->head: 0x%08x\n",
rq->ring->head);
drm_printf(m, "\t\tring->tail: 0x%08x\n",
drm_printf(m, "\t\tring->tail: 0x%08x\n",
rq->ring->tail);
drm_printf(m, "\t\tring->emit: 0x%08x\n",
rq->ring->emit);
drm_printf(m, "\t\tring->space: 0x%08x\n",
rq->ring->space);
}
rcu_read_unlock();
@ -2109,4 +1522,5 @@ void intel_disable_engine_stats(struct intel_engine_cs *engine)
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftests/mock_engine.c"
#include "selftests/intel_engine_cs.c"
#endif

28
drivers/gpu/drm/i915/intel_fbc.c
View File

@ -1272,6 +1272,34 @@ out:
mutex_unlock(&fbc->lock);
}
/*
* intel_fbc_reset_underrun - reset FBC fifo underrun status.
* @dev_priv: i915 device instance
*
* See intel_fbc_handle_fifo_underrun_irq(). For automated testing we
* want to re-enable FBC after an underrun to increase test coverage.
*/
int intel_fbc_reset_underrun(struct drm_i915_private *dev_priv)
{
int ret;
cancel_work_sync(&dev_priv->fbc.underrun_work);
ret = mutex_lock_interruptible(&dev_priv->fbc.lock);
if (ret)
return ret;
if (dev_priv->fbc.underrun_detected) {
DRM_DEBUG_KMS("Re-allowing FBC after fifo underrun\n");
dev_priv->fbc.no_fbc_reason = "FIFO underrun cleared";
}
dev_priv->fbc.underrun_detected = false;
mutex_unlock(&dev_priv->fbc.lock);
return 0;
}
/**
* intel_fbc_handle_fifo_underrun_irq - disable FBC when we get a FIFO underrun
* @dev_priv: i915 device instance

5
drivers/gpu/drm/i915/intel_fbdev.c
View File

@ -221,6 +221,9 @@ static int intelfb_create(struct drm_fb_helper *helper,
goto out_unlock;
}
fb = &ifbdev->fb->base;
intel_fb_obj_flush(intel_fb_obj(fb), ORIGIN_DIRTYFB);
info = drm_fb_helper_alloc_fbi(helper);
if (IS_ERR(info)) {
DRM_ERROR("Failed to allocate fb_info\n");
@ -230,8 +233,6 @@ static int intelfb_create(struct drm_fb_helper *helper,
info->par = helper;
fb = &ifbdev->fb->base;
ifbdev->helper.fb = fb;
strcpy(info->fix.id, "inteldrmfb");

2
drivers/gpu/drm/i915/intel_frontbuffer.c
View File

@ -80,7 +80,7 @@ void __intel_fb_obj_invalidate(struct drm_i915_gem_object *obj,
}
might_sleep();
intel_psr_invalidate(dev_priv, frontbuffer_bits);
intel_psr_invalidate(dev_priv, frontbuffer_bits, origin);
intel_edp_drrs_invalidate(dev_priv, frontbuffer_bits);
intel_fbc_invalidate(dev_priv, frontbuffer_bits, origin);
}

274
drivers/gpu/drm/i915/intel_gpu_commands.h 100644
View File

@ -0,0 +1,274 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright <EFBFBD> 2003-2018 Intel Corporation
*/
#ifndef _INTEL_GPU_COMMANDS_H_
#define _INTEL_GPU_COMMANDS_H_
/*
* Instruction field definitions used by the command parser
*/
#define INSTR_CLIENT_SHIFT 29
#define INSTR_MI_CLIENT 0x0
#define INSTR_BC_CLIENT 0x2
#define INSTR_RC_CLIENT 0x3
#define INSTR_SUBCLIENT_SHIFT 27
#define INSTR_SUBCLIENT_MASK 0x18000000
#define INSTR_MEDIA_SUBCLIENT 0x2
#define INSTR_26_TO_24_MASK 0x7000000
#define INSTR_26_TO_24_SHIFT 24
/*
* Memory interface instructions used by the kernel
*/
#define MI_INSTR(opcode, flags) (((opcode) << 23) | (flags))
/* Many MI commands use bit 22 of the header dword for GGTT vs PPGTT */
#define MI_GLOBAL_GTT (1<<22)
#define MI_NOOP MI_INSTR(0, 0)
#define MI_USER_INTERRUPT MI_INSTR(0x02, 0)
#define MI_WAIT_FOR_EVENT MI_INSTR(0x03, 0)
#define MI_WAIT_FOR_OVERLAY_FLIP (1<<16)
#define MI_WAIT_FOR_PLANE_B_FLIP (1<<6)
#define MI_WAIT_FOR_PLANE_A_FLIP (1<<2)
#define MI_WAIT_FOR_PLANE_A_SCANLINES (1<<1)
#define MI_FLUSH MI_INSTR(0x04, 0)
#define MI_READ_FLUSH (1 << 0)
#define MI_EXE_FLUSH (1 << 1)
#define MI_NO_WRITE_FLUSH (1 << 2)
#define MI_SCENE_COUNT (1 << 3) /* just increment scene count */
#define MI_END_SCENE (1 << 4) /* flush binner and incr scene count */
#define MI_INVALIDATE_ISP (1 << 5) /* invalidate indirect state pointers */
#define MI_REPORT_HEAD MI_INSTR(0x07, 0)
#define MI_ARB_ON_OFF MI_INSTR(0x08, 0)
#define MI_ARB_ENABLE (1<<0)
#define MI_ARB_DISABLE (0<<0)
#define MI_BATCH_BUFFER_END MI_INSTR(0x0a, 0)
#define MI_SUSPEND_FLUSH MI_INSTR(0x0b, 0)
#define MI_SUSPEND_FLUSH_EN (1<<0)
#define MI_SET_APPID MI_INSTR(0x0e, 0)
#define MI_OVERLAY_FLIP MI_INSTR(0x11, 0)
#define MI_OVERLAY_CONTINUE (0x0<<21)
#define MI_OVERLAY_ON (0x1<<21)
#define MI_OVERLAY_OFF (0x2<<21)
#define MI_LOAD_SCAN_LINES_INCL MI_INSTR(0x12, 0)
#define MI_DISPLAY_FLIP MI_INSTR(0x14, 2)
#define MI_DISPLAY_FLIP_I915 MI_INSTR(0x14, 1)
#define MI_DISPLAY_FLIP_PLANE(n) ((n) << 20)
/* IVB has funny definitions for which plane to flip. */
#define MI_DISPLAY_FLIP_IVB_PLANE_A (0 << 19)
#define MI_DISPLAY_FLIP_IVB_PLANE_B (1 << 19)
#define MI_DISPLAY_FLIP_IVB_SPRITE_A (2 << 19)
#define MI_DISPLAY_FLIP_IVB_SPRITE_B (3 << 19)
#define MI_DISPLAY_FLIP_IVB_PLANE_C (4 << 19)
#define MI_DISPLAY_FLIP_IVB_SPRITE_C (5 << 19)
/* SKL ones */
#define MI_DISPLAY_FLIP_SKL_PLANE_1_A (0 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_1_B (1 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_1_C (2 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_2_A (4 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_2_B (5 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_2_C (6 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_3_A (7 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_3_B (8 << 8)
#define MI_DISPLAY_FLIP_SKL_PLANE_3_C (9 << 8)
#define MI_SEMAPHORE_MBOX MI_INSTR(0x16, 1) /* gen6, gen7 */
#define MI_SEMAPHORE_GLOBAL_GTT (1<<22)
#define MI_SEMAPHORE_UPDATE (1<<21)
#define MI_SEMAPHORE_COMPARE (1<<20)
#define MI_SEMAPHORE_REGISTER (1<<18)
#define MI_SEMAPHORE_SYNC_VR (0<<16) /* RCS wait for VCS (RVSYNC) */
#define MI_SEMAPHORE_SYNC_VER (1<<16) /* RCS wait for VECS (RVESYNC) */
#define MI_SEMAPHORE_SYNC_BR (2<<16) /* RCS wait for BCS (RBSYNC) */
#define MI_SEMAPHORE_SYNC_BV (0<<16) /* VCS wait for BCS (VBSYNC) */
#define MI_SEMAPHORE_SYNC_VEV (1<<16) /* VCS wait for VECS (VVESYNC) */
#define MI_SEMAPHORE_SYNC_RV (2<<16) /* VCS wait for RCS (VRSYNC) */
#define MI_SEMAPHORE_SYNC_RB (0<<16) /* BCS wait for RCS (BRSYNC) */
#define MI_SEMAPHORE_SYNC_VEB (1<<16) /* BCS wait for VECS (BVESYNC) */
#define MI_SEMAPHORE_SYNC_VB (2<<16) /* BCS wait for VCS (BVSYNC) */
#define MI_SEMAPHORE_SYNC_BVE (0<<16) /* VECS wait for BCS (VEBSYNC) */
#define MI_SEMAPHORE_SYNC_VVE (1<<16) /* VECS wait for VCS (VEVSYNC) */
#define MI_SEMAPHORE_SYNC_RVE (2<<16) /* VECS wait for RCS (VERSYNC) */
#define MI_SEMAPHORE_SYNC_INVALID (3<<16)
#define MI_SEMAPHORE_SYNC_MASK (3<<16)
#define MI_SET_CONTEXT MI_INSTR(0x18, 0)
#define MI_MM_SPACE_GTT (1<<8)
#define MI_MM_SPACE_PHYSICAL (0<<8)
#define MI_SAVE_EXT_STATE_EN (1<<3)
#define MI_RESTORE_EXT_STATE_EN (1<<2)
#define MI_FORCE_RESTORE (1<<1)
#define MI_RESTORE_INHIBIT (1<<0)
#define HSW_MI_RS_SAVE_STATE_EN (1<<3)
#define HSW_MI_RS_RESTORE_STATE_EN (1<<2)
#define MI_SEMAPHORE_SIGNAL MI_INSTR(0x1b, 0) /* GEN8+ */
#define MI_SEMAPHORE_TARGET(engine) ((engine)<<15)
#define MI_SEMAPHORE_WAIT MI_INSTR(0x1c, 2) /* GEN8+ */
#define MI_SEMAPHORE_POLL (1<<15)
#define MI_SEMAPHORE_SAD_GTE_SDD (1<<12)
#define MI_STORE_DWORD_IMM MI_INSTR(0x20, 1)
#define MI_STORE_DWORD_IMM_GEN4 MI_INSTR(0x20, 2)
#define MI_MEM_VIRTUAL (1 << 22) /* 945,g33,965 */
#define MI_USE_GGTT (1 << 22) /* g4x+ */
#define MI_STORE_DWORD_INDEX MI_INSTR(0x21, 1)
#define MI_STORE_DWORD_INDEX_SHIFT 2
/*
* Official intel docs are somewhat sloppy concerning MI_LOAD_REGISTER_IMM:
* - Always issue a MI_NOOP _before_ the MI_LOAD_REGISTER_IMM - otherwise hw
* simply ignores the register load under certain conditions.
* - One can actually load arbitrary many arbitrary registers: Simply issue x
* address/value pairs. Don't overdue it, though, x <= 2^4 must hold!
*/
#define MI_LOAD_REGISTER_IMM(x) MI_INSTR(0x22, 2*(x)-1)
#define MI_LRI_FORCE_POSTED (1<<12)
#define MI_STORE_REGISTER_MEM MI_INSTR(0x24, 1)
#define MI_STORE_REGISTER_MEM_GEN8 MI_INSTR(0x24, 2)
#define MI_SRM_LRM_GLOBAL_GTT (1<<22)
#define MI_FLUSH_DW MI_INSTR(0x26, 1) /* for GEN6 */
#define MI_FLUSH_DW_STORE_INDEX (1<<21)
#define MI_INVALIDATE_TLB (1<<18)
#define MI_FLUSH_DW_OP_STOREDW (1<<14)
#define MI_FLUSH_DW_OP_MASK (3<<14)
#define MI_FLUSH_DW_NOTIFY (1<<8)
#define MI_INVALIDATE_BSD (1<<7)
#define MI_FLUSH_DW_USE_GTT (1<<2)
#define MI_FLUSH_DW_USE_PPGTT (0<<2)
#define MI_LOAD_REGISTER_MEM MI_INSTR(0x29, 1)
#define MI_LOAD_REGISTER_MEM_GEN8 MI_INSTR(0x29, 2)
#define MI_BATCH_BUFFER MI_INSTR(0x30, 1)
#define MI_BATCH_NON_SECURE (1)
/* for snb/ivb/vlv this also means "batch in ppgtt" when ppgtt is enabled. */
#define MI_BATCH_NON_SECURE_I965 (1<<8)
#define MI_BATCH_PPGTT_HSW (1<<8)
#define MI_BATCH_NON_SECURE_HSW (1<<13)
#define MI_BATCH_BUFFER_START MI_INSTR(0x31, 0)
#define MI_BATCH_GTT (2<<6) /* aliased with (1<<7) on gen4 */
#define MI_BATCH_BUFFER_START_GEN8 MI_INSTR(0x31, 1)
#define MI_BATCH_RESOURCE_STREAMER (1<<10)
/*
* 3D instructions used by the kernel
*/
#define GFX_INSTR(opcode, flags) ((0x3 << 29) | ((opcode) << 24) | (flags))
#define GEN9_MEDIA_POOL_STATE ((0x3 << 29) | (0x2 << 27) | (0x5 << 16) | 4)
#define GEN9_MEDIA_POOL_ENABLE (1 << 31)
#define GFX_OP_RASTER_RULES ((0x3<<29)|(0x7<<24))
#define GFX_OP_SCISSOR ((0x3<<29)|(0x1c<<24)|(0x10<<19))
#define SC_UPDATE_SCISSOR (0x1<<1)
#define SC_ENABLE_MASK (0x1<<0)
#define SC_ENABLE (0x1<<0)
#define GFX_OP_LOAD_INDIRECT ((0x3<<29)|(0x1d<<24)|(0x7<<16))
#define GFX_OP_SCISSOR_INFO ((0x3<<29)|(0x1d<<24)|(0x81<<16)|(0x1))
#define SCI_YMIN_MASK (0xffff<<16)
#define SCI_XMIN_MASK (0xffff<<0)
#define SCI_YMAX_MASK (0xffff<<16)
#define SCI_XMAX_MASK (0xffff<<0)
#define GFX_OP_SCISSOR_ENABLE ((0x3<<29)|(0x1c<<24)|(0x10<<19))
#define GFX_OP_SCISSOR_RECT ((0x3<<29)|(0x1d<<24)|(0x81<<16)|1)
#define GFX_OP_COLOR_FACTOR ((0x3<<29)|(0x1d<<24)|(0x1<<16)|0x0)
#define GFX_OP_STIPPLE ((0x3<<29)|(0x1d<<24)|(0x83<<16))
#define GFX_OP_MAP_INFO ((0x3<<29)|(0x1d<<24)|0x4)
#define GFX_OP_DESTBUFFER_VARS ((0x3<<29)|(0x1d<<24)|(0x85<<16)|0x0)
#define GFX_OP_DESTBUFFER_INFO ((0x3<<29)|(0x1d<<24)|(0x8e<<16)|1)
#define GFX_OP_DRAWRECT_INFO ((0x3<<29)|(0x1d<<24)|(0x80<<16)|(0x3))
#define GFX_OP_DRAWRECT_INFO_I965 ((0x7900<<16)|0x2)
#define COLOR_BLT_CMD (2<<29 | 0x40<<22 | (5-2))
#define SRC_COPY_BLT_CMD ((2<<29)|(0x43<<22)|4)
#define XY_SRC_COPY_BLT_CMD ((2<<29)|(0x53<<22)|6)
#define XY_MONO_SRC_COPY_IMM_BLT ((2<<29)|(0x71<<22)|5)
#define BLT_WRITE_A (2<<20)
#define BLT_WRITE_RGB (1<<20)
#define BLT_WRITE_RGBA (BLT_WRITE_RGB | BLT_WRITE_A)
#define BLT_DEPTH_8 (0<<24)
#define BLT_DEPTH_16_565 (1<<24)
#define BLT_DEPTH_16_1555 (2<<24)
#define BLT_DEPTH_32 (3<<24)
#define BLT_ROP_SRC_COPY (0xcc<<16)
#define BLT_ROP_COLOR_COPY (0xf0<<16)
#define XY_SRC_COPY_BLT_SRC_TILED (1<<15) /* 965+ only */
#define XY_SRC_COPY_BLT_DST_TILED (1<<11) /* 965+ only */
#define CMD_OP_DISPLAYBUFFER_INFO ((0x0<<29)|(0x14<<23)|2)
#define ASYNC_FLIP (1<<22)
#define DISPLAY_PLANE_A (0<<20)
#define DISPLAY_PLANE_B (1<<20)
#define GFX_OP_PIPE_CONTROL(len) ((0x3<<29)|(0x3<<27)|(0x2<<24)|((len)-2))
#define PIPE_CONTROL_FLUSH_L3 (1<<27)
#define PIPE_CONTROL_GLOBAL_GTT_IVB (1<<24) /* gen7+ */
#define PIPE_CONTROL_MMIO_WRITE (1<<23)
#define PIPE_CONTROL_STORE_DATA_INDEX (1<<21)
#define PIPE_CONTROL_CS_STALL (1<<20)
#define PIPE_CONTROL_TLB_INVALIDATE (1<<18)
#define PIPE_CONTROL_MEDIA_STATE_CLEAR (1<<16)
#define PIPE_CONTROL_QW_WRITE (1<<14)
#define PIPE_CONTROL_POST_SYNC_OP_MASK (3<<14)
#define PIPE_CONTROL_DEPTH_STALL (1<<13)
#define PIPE_CONTROL_WRITE_FLUSH (1<<12)
#define PIPE_CONTROL_RENDER_TARGET_CACHE_FLUSH (1<<12) /* gen6+ */
#define PIPE_CONTROL_INSTRUCTION_CACHE_INVALIDATE (1<<11) /* MBZ on ILK */
#define PIPE_CONTROL_TEXTURE_CACHE_INVALIDATE (1<<10) /* GM45+ only */
#define PIPE_CONTROL_INDIRECT_STATE_DISABLE (1<<9)
#define PIPE_CONTROL_NOTIFY (1<<8)
#define PIPE_CONTROL_FLUSH_ENABLE (1<<7) /* gen7+ */
#define PIPE_CONTROL_DC_FLUSH_ENABLE (1<<5)
#define PIPE_CONTROL_VF_CACHE_INVALIDATE (1<<4)
#define PIPE_CONTROL_CONST_CACHE_INVALIDATE (1<<3)
#define PIPE_CONTROL_STATE_CACHE_INVALIDATE (1<<2)
#define PIPE_CONTROL_STALL_AT_SCOREBOARD (1<<1)
#define PIPE_CONTROL_DEPTH_CACHE_FLUSH (1<<0)
#define PIPE_CONTROL_GLOBAL_GTT (1<<2) /* in addr dword */
/*
* Commands used only by the command parser
*/
#define MI_SET_PREDICATE MI_INSTR(0x01, 0)
#define MI_ARB_CHECK MI_INSTR(0x05, 0)
#define MI_RS_CONTROL MI_INSTR(0x06, 0)
#define MI_URB_ATOMIC_ALLOC MI_INSTR(0x09, 0)
#define MI_PREDICATE MI_INSTR(0x0C, 0)
#define MI_RS_CONTEXT MI_INSTR(0x0F, 0)
#define MI_TOPOLOGY_FILTER MI_INSTR(0x0D, 0)
#define MI_LOAD_SCAN_LINES_EXCL MI_INSTR(0x13, 0)
#define MI_URB_CLEAR MI_INSTR(0x19, 0)
#define MI_UPDATE_GTT MI_INSTR(0x23, 0)
#define MI_CLFLUSH MI_INSTR(0x27, 0)
#define MI_REPORT_PERF_COUNT MI_INSTR(0x28, 0)
#define MI_REPORT_PERF_COUNT_GGTT (1<<0)
#define MI_LOAD_REGISTER_REG MI_INSTR(0x2A, 0)
#define MI_RS_STORE_DATA_IMM MI_INSTR(0x2B, 0)
#define MI_LOAD_URB_MEM MI_INSTR(0x2C, 0)
#define MI_STORE_URB_MEM MI_INSTR(0x2D, 0)
#define MI_CONDITIONAL_BATCH_BUFFER_END MI_INSTR(0x36, 0)
#define PIPELINE_SELECT ((0x3<<29)|(0x1<<27)|(0x1<<24)|(0x4<<16))
#define GFX_OP_3DSTATE_VF_STATISTICS ((0x3<<29)|(0x1<<27)|(0x0<<24)|(0xB<<16))
#define MEDIA_VFE_STATE ((0x3<<29)|(0x2<<27)|(0x0<<24)|(0x0<<16))
#define MEDIA_VFE_STATE_MMIO_ACCESS_MASK (0x18)
#define GPGPU_OBJECT ((0x3<<29)|(0x2<<27)|(0x1<<24)|(0x4<<16))
#define GPGPU_WALKER ((0x3<<29)|(0x2<<27)|(0x1<<24)|(0x5<<16))
#define GFX_OP_3DSTATE_DX9_CONSTANTF_VS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x39<<16))
#define GFX_OP_3DSTATE_DX9_CONSTANTF_PS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x3A<<16))
#define GFX_OP_3DSTATE_SO_DECL_LIST \
((0x3<<29)|(0x3<<27)|(0x1<<24)|(0x17<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_VS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x43<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_GS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x44<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_HS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x45<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_DS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x46<<16))
#define GFX_OP_3DSTATE_BINDING_TABLE_EDIT_PS \
((0x3<<29)|(0x3<<27)|(0x0<<24)|(0x47<<16))
#define MFX_WAIT ((0x3<<29)|(0x1<<27)|(0x0<<16))
#define COLOR_BLT ((0x2<<29)|(0x40<<22))
#define SRC_COPY_BLT ((0x2<<29)|(0x43<<22))
#endif /* _INTEL_GPU_COMMANDS_H_ */

209
drivers/gpu/drm/i915/intel_guc.c
View File

@ -64,10 +64,12 @@ void intel_guc_init_early(struct intel_guc *guc)
{
intel_guc_fw_init_early(guc);
intel_guc_ct_init_early(&guc->ct);
intel_guc_log_init_early(guc);
intel_guc_log_init_early(&guc->log);
mutex_init(&guc->send_mutex);
spin_lock_init(&guc->irq_lock);
guc->send = intel_guc_send_nop;
guc->handler = intel_guc_to_host_event_handler_nop;
guc->notify = gen8_guc_raise_irq;
}
@ -86,9 +88,10 @@ int intel_guc_init_wq(struct intel_guc *guc)
* or scheduled later on resume. This way the handling of work
* item can be kept same between system suspend & rpm suspend.
*/
guc->log.runtime.flush_wq = alloc_ordered_workqueue("i915-guc_log",
WQ_HIGHPRI | WQ_FREEZABLE);
if (!guc->log.runtime.flush_wq) {
guc->log.relay.flush_wq =
alloc_ordered_workqueue("i915-guc_log",
WQ_HIGHPRI | WQ_FREEZABLE);
if (!guc->log.relay.flush_wq) {
DRM_ERROR("Couldn't allocate workqueue for GuC log\n");
return -ENOMEM;
}
@ -111,7 +114,7 @@ int intel_guc_init_wq(struct intel_guc *guc)
guc->preempt_wq = alloc_ordered_workqueue("i915-guc_preempt",
WQ_HIGHPRI);
if (!guc->preempt_wq) {
destroy_workqueue(guc->log.runtime.flush_wq);
destroy_workqueue(guc->log.relay.flush_wq);
DRM_ERROR("Couldn't allocate workqueue for GuC "
"preemption\n");
return -ENOMEM;
@ -129,7 +132,7 @@ void intel_guc_fini_wq(struct intel_guc *guc)
USES_GUC_SUBMISSION(dev_priv))
destroy_workqueue(guc->preempt_wq);
destroy_workqueue(guc->log.runtime.flush_wq);
destroy_workqueue(guc->log.relay.flush_wq);
}
static int guc_shared_data_create(struct intel_guc *guc)
@ -169,7 +172,7 @@ int intel_guc_init(struct intel_guc *guc)
return ret;
GEM_BUG_ON(!guc->shared_data);
ret = intel_guc_log_create(guc);
ret = intel_guc_log_create(&guc->log);
if (ret)
goto err_shared;
@ -184,7 +187,7 @@ int intel_guc_init(struct intel_guc *guc)
return 0;
err_log:
intel_guc_log_destroy(guc);
intel_guc_log_destroy(&guc->log);
err_shared:
guc_shared_data_destroy(guc);
return ret;
@ -196,7 +199,7 @@ void intel_guc_fini(struct intel_guc *guc)
i915_ggtt_disable_guc(dev_priv);
intel_guc_ads_destroy(guc);
intel_guc_log_destroy(guc);
intel_guc_log_destroy(&guc->log);
guc_shared_data_destroy(guc);
}
@ -220,17 +223,23 @@ static u32 get_core_family(struct drm_i915_private *dev_priv)
}
}
static u32 get_log_verbosity_flags(void)
static u32 get_log_control_flags(void)
{
if (i915_modparams.guc_log_level > 0) {
u32 verbosity = i915_modparams.guc_log_level - 1;
u32 level = i915_modparams.guc_log_level;
u32 flags = 0;
GEM_BUG_ON(verbosity > GUC_LOG_VERBOSITY_MAX);
return verbosity << GUC_LOG_VERBOSITY_SHIFT;
}
GEM_BUG_ON(level < 0);
GEM_BUG_ON(i915_modparams.enable_guc < 0);
return GUC_LOG_DISABLED;
if (!GUC_LOG_LEVEL_IS_ENABLED(level))
flags |= GUC_LOG_DEFAULT_DISABLED;
if (!GUC_LOG_LEVEL_IS_VERBOSE(level))
flags |= GUC_LOG_DISABLED;
else
flags |= GUC_LOG_LEVEL_TO_VERBOSITY(level) <<
GUC_LOG_VERBOSITY_SHIFT;
return flags;
}
/*
@ -265,12 +274,13 @@ void intel_guc_init_params(struct intel_guc *guc)
params[GUC_CTL_LOG_PARAMS] = guc->log.flags;
params[GUC_CTL_DEBUG] = get_log_verbosity_flags();
params[GUC_CTL_DEBUG] = get_log_control_flags();
/* If GuC submission is enabled, set up additional parameters here */
if (USES_GUC_SUBMISSION(dev_priv)) {
u32 ads = guc_ggtt_offset(guc->ads_vma) >> PAGE_SHIFT;
u32 pgs = guc_ggtt_offset(dev_priv->guc.stage_desc_pool);
u32 ads = intel_guc_ggtt_offset(guc,
guc->ads_vma) >> PAGE_SHIFT;
u32 pgs = intel_guc_ggtt_offset(guc, guc->stage_desc_pool);
u32 ctx_in_16 = GUC_MAX_STAGE_DESCRIPTORS / 16;
params[GUC_CTL_DEBUG] |= ads << GUC_ADS_ADDR_SHIFT;
@ -301,16 +311,23 @@ void intel_guc_init_params(struct intel_guc *guc)
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_BLITTER);
}
int intel_guc_send_nop(struct intel_guc *guc, const u32 *action, u32 len)
int intel_guc_send_nop(struct intel_guc *guc, const u32 *action, u32 len,
u32 *response_buf, u32 response_buf_size)
{
WARN(1, "Unexpected send: action=%#x\n", *action);
return -ENODEV;
}
void intel_guc_to_host_event_handler_nop(struct intel_guc *guc)
{
WARN(1, "Unexpected event: no suitable handler\n");
}
/*
* This function implements the MMIO based host to GuC interface.
*/
int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len)
int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len,
u32 *response_buf, u32 response_buf_size)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
u32 status;
@ -320,6 +337,9 @@ int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len)
GEM_BUG_ON(!len);
GEM_BUG_ON(len > guc->send_regs.count);
/* We expect only action code */
GEM_BUG_ON(*action & ~INTEL_GUC_MSG_CODE_MASK);
/* If CT is available, we expect to use MMIO only during init/fini */
GEM_BUG_ON(HAS_GUC_CT(dev_priv) &&
*action != INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER &&
@ -341,29 +361,74 @@ int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len)
*/
ret = __intel_wait_for_register_fw(dev_priv,
guc_send_reg(guc, 0),
INTEL_GUC_RECV_MASK,
INTEL_GUC_RECV_MASK,
INTEL_GUC_MSG_TYPE_MASK,
INTEL_GUC_MSG_TYPE_RESPONSE <<
INTEL_GUC_MSG_TYPE_SHIFT,
10, 10, &status);
if (status != INTEL_GUC_STATUS_SUCCESS) {
/*
* Either the GuC explicitly returned an error (which
* we convert to -EIO here) or no response at all was
* received within the timeout limit (-ETIMEDOUT)
*/
if (ret != -ETIMEDOUT)
ret = -EIO;
/* If GuC explicitly returned an error, convert it to -EIO */
if (!ret && !INTEL_GUC_MSG_IS_RESPONSE_SUCCESS(status))
ret = -EIO;
DRM_WARN("INTEL_GUC_SEND: Action 0x%X failed;"
" ret=%d status=0x%08X response=0x%08X\n",
action[0], ret, status, I915_READ(SOFT_SCRATCH(15)));
if (ret) {
DRM_DEBUG_DRIVER("INTEL_GUC_SEND: Action 0x%X failed;"
" ret=%d status=0x%08X response=0x%08X\n",
action[0], ret, status,
I915_READ(SOFT_SCRATCH(15)));
goto out;
}
if (response_buf) {
int count = min(response_buf_size, guc->send_regs.count - 1);
for (i = 0; i < count; i++)
response_buf[i] = I915_READ(guc_send_reg(guc, i + 1));
}
/* Use data from the GuC response as our return value */
ret = INTEL_GUC_MSG_TO_DATA(status);
out:
intel_uncore_forcewake_put(dev_priv, guc->send_regs.fw_domains);
mutex_unlock(&guc->send_mutex);
return ret;
}
void intel_guc_to_host_event_handler_mmio(struct intel_guc *guc)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
u32 msg, val;
/*
* Sample the log buffer flush related bits & clear them out now
* itself from the message identity register to minimize the
* probability of losing a flush interrupt, when there are back
* to back flush interrupts.
* There can be a new flush interrupt, for different log buffer
* type (like for ISR), whilst Host is handling one (for DPC).
* Since same bit is used in message register for ISR & DPC, it
* could happen that GuC sets the bit for 2nd interrupt but Host
* clears out the bit on handling the 1st interrupt.
*/
spin_lock(&guc->irq_lock);
val = I915_READ(SOFT_SCRATCH(15));
msg = val & guc->msg_enabled_mask;
I915_WRITE(SOFT_SCRATCH(15), val & ~msg);
spin_unlock(&guc->irq_lock);
intel_guc_to_host_process_recv_msg(guc, msg);
}
void intel_guc_to_host_process_recv_msg(struct intel_guc *guc, u32 msg)
{
/* Make sure to handle only enabled messages */
msg &= guc->msg_enabled_mask;
if (msg & (INTEL_GUC_RECV_MSG_FLUSH_LOG_BUFFER |
INTEL_GUC_RECV_MSG_CRASH_DUMP_POSTED))
intel_guc_log_handle_flush_event(&guc->log);
}
int intel_guc_sample_forcewake(struct intel_guc *guc)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
@ -410,7 +475,7 @@ int intel_guc_suspend(struct intel_guc *guc)
u32 data[] = {
INTEL_GUC_ACTION_ENTER_S_STATE,
GUC_POWER_D1, /* any value greater than GUC_POWER_D0 */
guc_ggtt_offset(guc->shared_data)
intel_guc_ggtt_offset(guc, guc->shared_data)
};
return intel_guc_send(guc, data, ARRAY_SIZE(data));
@ -434,7 +499,7 @@ int intel_guc_reset_engine(struct intel_guc *guc,
data[3] = 0;
data[4] = 0;
data[5] = guc->execbuf_client->stage_id;
data[6] = guc_ggtt_offset(guc->shared_data);
data[6] = intel_guc_ggtt_offset(guc, guc->shared_data);
return intel_guc_send(guc, data, ARRAY_SIZE(data));
}
@ -448,12 +513,65 @@ int intel_guc_resume(struct intel_guc *guc)
u32 data[] = {
INTEL_GUC_ACTION_EXIT_S_STATE,
GUC_POWER_D0,
guc_ggtt_offset(guc->shared_data)
intel_guc_ggtt_offset(guc, guc->shared_data)
};
return intel_guc_send(guc, data, ARRAY_SIZE(data));
}
/**
* DOC: GuC Address Space
*
* The layout of GuC address space is shown below:
*
* ::
*
* +==============> +====================+ <== GUC_GGTT_TOP
* ^ | |
* | | |
* | | DRAM |
* | | Memory |
* | | |
* GuC | |
* Address +========> +====================+ <== WOPCM Top
* Space ^ | HW contexts RSVD |
* | | | WOPCM |
* | | +==> +--------------------+ <== GuC WOPCM Top
* | GuC ^ | |
* | GGTT | | |
* | Pin GuC | GuC |
* | Bias WOPCM | WOPCM |
* | | Size | |
* | | | | |
* v v v | |
* +=====+=====+==> +====================+ <== GuC WOPCM Base
* | Non-GuC WOPCM |
* | (HuC/Reserved) |
* +====================+ <== WOPCM Base
*
* The lower part of GuC Address Space [0, ggtt_pin_bias) is mapped to WOPCM
* while upper part of GuC Address Space [ggtt_pin_bias, GUC_GGTT_TOP) is mapped
* to DRAM. The value of the GuC ggtt_pin_bias is determined by WOPCM size and
* actual GuC WOPCM size.
*/
/**
* intel_guc_init_ggtt_pin_bias() - Initialize the GuC ggtt_pin_bias value.
* @guc: intel_guc structure.
*
* This function will calculate and initialize the ggtt_pin_bias value based on
* overall WOPCM size and GuC WOPCM size.
*/
void intel_guc_init_ggtt_pin_bias(struct intel_guc *guc)
{
struct drm_i915_private *i915 = guc_to_i915(guc);
GEM_BUG_ON(!i915->wopcm.size);
GEM_BUG_ON(i915->wopcm.size < i915->wopcm.guc.base);
guc->ggtt_pin_bias = i915->wopcm.size - i915->wopcm.guc.base;
}
/**
* intel_guc_allocate_vma() - Allocate a GGTT VMA for GuC usage
* @guc: the guc
@ -462,7 +580,7 @@ int intel_guc_resume(struct intel_guc *guc)
* This is a wrapper to create an object for use with the GuC. In order to
* use it inside the GuC, an object needs to be pinned lifetime, so we allocate
* both some backing storage and a range inside the Global GTT. We must pin
* it in the GGTT somewhere other than than [0, GUC_WOPCM_TOP) because that
* it in the GGTT somewhere other than than [0, GUC ggtt_pin_bias) because that
* range is reserved inside GuC.
*
* Return: A i915_vma if successful, otherwise an ERR_PTR.
@ -483,7 +601,7 @@ struct i915_vma *intel_guc_allocate_vma(struct intel_guc *guc, u32 size)
goto err;
ret = i915_vma_pin(vma, 0, PAGE_SIZE,
PIN_GLOBAL | PIN_OFFSET_BIAS | GUC_WOPCM_TOP);
PIN_GLOBAL | PIN_OFFSET_BIAS | guc->ggtt_pin_bias);
if (ret) {
vma = ERR_PTR(ret);
goto err;
@ -495,14 +613,3 @@ err:
i915_gem_object_put(obj);
return vma;
}
u32 intel_guc_wopcm_size(struct drm_i915_private *dev_priv)
{
u32 wopcm_size = GUC_WOPCM_TOP;
/* On BXT, the top of WOPCM is reserved for RC6 context */
if (IS_GEN9_LP(dev_priv))
wopcm_size -= BXT_GUC_WOPCM_RC6_RESERVED;
return wopcm_size;
}

82
drivers/gpu/drm/i915/intel_guc.h
View File

@ -49,11 +49,16 @@ struct intel_guc {
struct intel_guc_log log;
struct intel_guc_ct ct;
/* Offset where Non-WOPCM memory starts. */
u32 ggtt_pin_bias;
/* Log snapshot if GuC errors during load */
struct drm_i915_gem_object *load_err_log;
/* intel_guc_recv interrupt related state */
spinlock_t irq_lock;
bool interrupts_enabled;
unsigned int msg_enabled_mask;
struct i915_vma *ads_vma;
struct i915_vma *stage_desc_pool;
@ -83,7 +88,11 @@ struct intel_guc {
struct mutex send_mutex;
/* GuC's FW specific send function */
int (*send)(struct intel_guc *guc, const u32 *data, u32 len);
int (*send)(struct intel_guc *guc, const u32 *data, u32 len,
u32 *response_buf, u32 response_buf_size);
/* GuC's FW specific event handler function */
void (*handler)(struct intel_guc *guc);
/* GuC's FW specific notify function */
void (*notify)(struct intel_guc *guc);
@ -92,7 +101,14 @@ struct intel_guc {
static
inline int intel_guc_send(struct intel_guc *guc, const u32 *action, u32 len)
{
return guc->send(guc, action, len);
return guc->send(guc, action, len, NULL, 0);
}
static inline int
intel_guc_send_and_receive(struct intel_guc *guc, const u32 *action, u32 len,
u32 *response_buf, u32 response_buf_size)
{
return guc->send(guc, action, len, response_buf, response_buf_size);
}
static inline void intel_guc_notify(struct intel_guc *guc)
@ -100,17 +116,33 @@ static inline void intel_guc_notify(struct intel_guc *guc)
guc->notify(guc);
}
/*
* GuC does not allow any gfx GGTT address that falls into range [0, WOPCM_TOP),
* which is reserved for Boot ROM, SRAM and WOPCM. Currently this top address is
* 512K. In order to exclude 0-512K address space from GGTT, all gfx objects
* used by GuC is pinned with PIN_OFFSET_BIAS along with size of WOPCM.
static inline void intel_guc_to_host_event_handler(struct intel_guc *guc)
{
guc->handler(guc);
}
/* GuC addresses above GUC_GGTT_TOP also don't map through the GTT */
#define GUC_GGTT_TOP 0xFEE00000
/**
* intel_guc_ggtt_offset() - Get and validate the GGTT offset of @vma
* @guc: intel_guc structure.
* @vma: i915 graphics virtual memory area.
*
* GuC does not allow any gfx GGTT address that falls into range
* [0, GuC ggtt_pin_bias), which is reserved for Boot ROM, SRAM and WOPCM.
* Currently, in order to exclude [0, GuC ggtt_pin_bias) address space from
* GGTT, all gfx objects used by GuC are allocated with intel_guc_allocate_vma()
* and pinned with PIN_OFFSET_BIAS along with the value of GuC ggtt_pin_bias.
*
* Return: GGTT offset of the @vma.
*/
static inline u32 guc_ggtt_offset(struct i915_vma *vma)
static inline u32 intel_guc_ggtt_offset(struct intel_guc *guc,
struct i915_vma *vma)
{
u32 offset = i915_ggtt_offset(vma);
GEM_BUG_ON(offset < GUC_WOPCM_TOP);
GEM_BUG_ON(offset < guc->ggtt_pin_bias);
GEM_BUG_ON(range_overflows_t(u64, offset, vma->size, GUC_GGTT_TOP));
return offset;
@ -119,17 +151,43 @@ static inline u32 guc_ggtt_offset(struct i915_vma *vma)
void intel_guc_init_early(struct intel_guc *guc);
void intel_guc_init_send_regs(struct intel_guc *guc);
void intel_guc_init_params(struct intel_guc *guc);
void intel_guc_init_ggtt_pin_bias(struct intel_guc *guc);
int intel_guc_init_wq(struct intel_guc *guc);
void intel_guc_fini_wq(struct intel_guc *guc);
int intel_guc_init(struct intel_guc *guc);
void intel_guc_fini(struct intel_guc *guc);
int intel_guc_send_nop(struct intel_guc *guc, const u32 *action, u32 len);
int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len);
int intel_guc_send_nop(struct intel_guc *guc, const u32 *action, u32 len,
u32 *response_buf, u32 response_buf_size);
int intel_guc_send_mmio(struct intel_guc *guc, const u32 *action, u32 len,
u32 *response_buf, u32 response_buf_size);
void intel_guc_to_host_event_handler(struct intel_guc *guc);
void intel_guc_to_host_event_handler_nop(struct intel_guc *guc);
void intel_guc_to_host_event_handler_mmio(struct intel_guc *guc);
void intel_guc_to_host_process_recv_msg(struct intel_guc *guc, u32 msg);
int intel_guc_sample_forcewake(struct intel_guc *guc);
int intel_guc_auth_huc(struct intel_guc *guc, u32 rsa_offset);
int intel_guc_suspend(struct intel_guc *guc);
int intel_guc_resume(struct intel_guc *guc);
struct i915_vma *intel_guc_allocate_vma(struct intel_guc *guc, u32 size);
u32 intel_guc_wopcm_size(struct drm_i915_private *dev_priv);
static inline int intel_guc_sanitize(struct intel_guc *guc)
{
intel_uc_fw_sanitize(&guc->fw);
return 0;
}
static inline void intel_guc_enable_msg(struct intel_guc *guc, u32 mask)
{
spin_lock_irq(&guc->irq_lock);
guc->msg_enabled_mask |= mask;
spin_unlock_irq(&guc->irq_lock);
}
static inline void intel_guc_disable_msg(struct intel_guc *guc, u32 mask)
{
spin_lock_irq(&guc->irq_lock);
guc->msg_enabled_mask &= ~mask;
spin_unlock_irq(&guc->irq_lock);
}
#endif

8
drivers/gpu/drm/i915/intel_guc_ads.c
View File

@ -75,7 +75,7 @@ static void guc_policies_init(struct guc_policies *policies)
int intel_guc_ads_create(struct intel_guc *guc)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct i915_vma *vma;
struct i915_vma *vma, *kernel_ctx_vma;
struct page *page;
/* The ads obj includes the struct itself and buffers passed to GuC */
struct {
@ -121,9 +121,9 @@ int intel_guc_ads_create(struct intel_guc *guc)
* to find it. Note that we have to skip our header (1 page),
* because our GuC shared data is there.
*/
kernel_ctx_vma = dev_priv->kernel_context->engine[RCS].state;
blob->ads.golden_context_lrca =
guc_ggtt_offset(dev_priv->kernel_context->engine[RCS].state) +
skipped_offset;
intel_guc_ggtt_offset(guc, kernel_ctx_vma) + skipped_offset;
/*
* The GuC expects us to exclude the portion of the context image that
@ -135,7 +135,7 @@ int intel_guc_ads_create(struct intel_guc *guc)
blob->ads.eng_state_size[engine->guc_id] =
engine->context_size - skipped_size;
base = guc_ggtt_offset(vma);
base = intel_guc_ggtt_offset(guc, vma);
blob->ads.scheduler_policies = base + ptr_offset(blob, policies);
blob->ads.reg_state_buffer = base + ptr_offset(blob, reg_state_buffer);
blob->ads.reg_state_addr = base + ptr_offset(blob, reg_state);

549
drivers/gpu/drm/i915/intel_guc_ct.c
View File

@ -24,14 +24,49 @@
#include "i915_drv.h"
#include "intel_guc_ct.h"
#ifdef CONFIG_DRM_I915_DEBUG_GUC
#define CT_DEBUG_DRIVER(...) DRM_DEBUG_DRIVER(__VA_ARGS__)
#else
#define CT_DEBUG_DRIVER(...) do { } while (0)
#endif
struct ct_request {
struct list_head link;
u32 fence;
u32 status;
u32 response_len;
u32 *response_buf;
};
struct ct_incoming_request {
struct list_head link;
u32 msg[];
};
enum { CTB_SEND = 0, CTB_RECV = 1 };
enum { CTB_OWNER_HOST = 0 };
static void ct_incoming_request_worker_func(struct work_struct *w);
/**
* intel_guc_ct_init_early - Initialize CT state without requiring device access
* @ct: pointer to CT struct
*/
void intel_guc_ct_init_early(struct intel_guc_ct *ct)
{
/* we're using static channel owners */
ct->host_channel.owner = CTB_OWNER_HOST;
spin_lock_init(&ct->lock);
INIT_LIST_HEAD(&ct->pending_requests);
INIT_LIST_HEAD(&ct->incoming_requests);
INIT_WORK(&ct->worker, ct_incoming_request_worker_func);
}
static inline struct intel_guc *ct_to_guc(struct intel_guc_ct *ct)
{
return container_of(ct, struct intel_guc, ct);
}
static inline const char *guc_ct_buffer_type_to_str(u32 type)
@ -49,8 +84,8 @@ static inline const char *guc_ct_buffer_type_to_str(u32 type)
static void guc_ct_buffer_desc_init(struct guc_ct_buffer_desc *desc,
u32 cmds_addr, u32 size, u32 owner)
{
DRM_DEBUG_DRIVER("CT: desc %p init addr=%#x size=%u owner=%u\n",
desc, cmds_addr, size, owner);
CT_DEBUG_DRIVER("CT: desc %p init addr=%#x size=%u owner=%u\n",
desc, cmds_addr, size, owner);
memset(desc, 0, sizeof(*desc));
desc->addr = cmds_addr;
desc->size = size;
@ -59,8 +94,8 @@ static void guc_ct_buffer_desc_init(struct guc_ct_buffer_desc *desc,
static void guc_ct_buffer_desc_reset(struct guc_ct_buffer_desc *desc)
{
DRM_DEBUG_DRIVER("CT: desc %p reset head=%u tail=%u\n",
desc, desc->head, desc->tail);
CT_DEBUG_DRIVER("CT: desc %p reset head=%u tail=%u\n",
desc, desc->head, desc->tail);
desc->head = 0;
desc->tail = 0;
desc->is_in_error = 0;
@ -79,7 +114,7 @@ static int guc_action_register_ct_buffer(struct intel_guc *guc,
int err;
/* Can't use generic send(), CT registration must go over MMIO */
err = intel_guc_send_mmio(guc, action, ARRAY_SIZE(action));
err = intel_guc_send_mmio(guc, action, ARRAY_SIZE(action), NULL, 0);
if (err)
DRM_ERROR("CT: register %s buffer failed; err=%d\n",
guc_ct_buffer_type_to_str(type), err);
@ -98,7 +133,7 @@ static int guc_action_deregister_ct_buffer(struct intel_guc *guc,
int err;
/* Can't use generic send(), CT deregistration must go over MMIO */
err = intel_guc_send_mmio(guc, action, ARRAY_SIZE(action));
err = intel_guc_send_mmio(guc, action, ARRAY_SIZE(action), NULL, 0);
if (err)
DRM_ERROR("CT: deregister %s buffer failed; owner=%d err=%d\n",
guc_ct_buffer_type_to_str(type), owner, err);
@ -156,7 +191,8 @@ static int ctch_init(struct intel_guc *guc,
err = PTR_ERR(blob);
goto err_vma;
}
DRM_DEBUG_DRIVER("CT: vma base=%#x\n", guc_ggtt_offset(ctch->vma));
CT_DEBUG_DRIVER("CT: vma base=%#x\n",
intel_guc_ggtt_offset(guc, ctch->vma));
/* store pointers to desc and cmds */
for (i = 0; i < ARRAY_SIZE(ctch->ctbs); i++) {
@ -170,8 +206,8 @@ static int ctch_init(struct intel_guc *guc,
err_vma:
i915_vma_unpin_and_release(&ctch->vma);
err_out:
DRM_DEBUG_DRIVER("CT: channel %d initialization failed; err=%d\n",
ctch->owner, err);
CT_DEBUG_DRIVER("CT: channel %d initialization failed; err=%d\n",
ctch->owner, err);
return err;
}
@ -191,8 +227,8 @@ static int ctch_open(struct intel_guc *guc,
int err;
int i;
DRM_DEBUG_DRIVER("CT: channel %d reopen=%s\n",
ctch->owner, yesno(ctch_is_open(ctch)));
CT_DEBUG_DRIVER("CT: channel %d reopen=%s\n",
ctch->owner, yesno(ctch_is_open(ctch)));
if (!ctch->vma) {
err = ctch_init(guc, ctch);
@ -202,7 +238,7 @@ static int ctch_open(struct intel_guc *guc,
}
/* vma should be already allocated and map'ed */
base = guc_ggtt_offset(ctch->vma);
base = intel_guc_ggtt_offset(guc, ctch->vma);
/* (re)initialize descriptors
* cmds buffers are in the second half of the blob page
@ -263,10 +299,29 @@ static u32 ctch_get_next_fence(struct intel_guc_ct_channel *ctch)
return ++ctch->next_fence;
}
/**
* DOC: CTB Host to GuC request
*
* Format of the CTB Host to GuC request message is as follows::
*
* +------------+---------+---------+---------+---------+
* | msg[0] | [1] | [2] | ... | [n-1] |
* +------------+---------+---------+---------+---------+
* | MESSAGE | MESSAGE PAYLOAD |
* + HEADER +---------+---------+---------+---------+
* | | 0 | 1 | ... | n |
* +============+=========+=========+=========+=========+
* | len >= 1 | FENCE | request specific data |
* +------+-----+---------+---------+---------+---------+
*
* ^-----------------len-------------------^
*/
static int ctb_write(struct intel_guc_ct_buffer *ctb,
const u32 *action,
u32 len /* in dwords */,
u32 fence)
u32 fence,
bool want_response)
{
struct guc_ct_buffer_desc *desc = ctb->desc;
u32 head = desc->head / 4; /* in dwords */
@ -295,15 +350,21 @@ static int ctb_write(struct intel_guc_ct_buffer *ctb,
if (unlikely(used + len + 1 >= size))
return -ENOSPC;
/* Write the message. The format is the following:
/*
* Write the message. The format is the following:
* DW0: header (including action code)
* DW1: fence
* DW2+: action data
*/
header = (len << GUC_CT_MSG_LEN_SHIFT) |
(GUC_CT_MSG_WRITE_FENCE_TO_DESC) |
(want_response ? GUC_CT_MSG_SEND_STATUS : 0) |
(action[0] << GUC_CT_MSG_ACTION_SHIFT);
CT_DEBUG_DRIVER("CT: writing %*ph %*ph %*ph\n",
4, &header, 4, &fence,
4 * (len - 1), &action[1]);
cmds[tail] = header;
tail = (tail + 1) % size;
@ -322,16 +383,25 @@ static int ctb_write(struct intel_guc_ct_buffer *ctb,
return 0;
}
/* Wait for the response from the GuC.
/**
* wait_for_ctb_desc_update - Wait for the CT buffer descriptor update.
* @desc: buffer descriptor
* @fence: response fence
* @status: placeholder for status
* return: 0 response received (status is valid)
* -ETIMEDOUT no response within hardcoded timeout
* -EPROTO no response, ct buffer was in error
*
* Guc will update CT buffer descriptor with new fence and status
* after processing the command identified by the fence. Wait for
* specified fence and then read from the descriptor status of the
* command.
*
* Return:
* * 0 response received (status is valid)
* * -ETIMEDOUT no response within hardcoded timeout
* * -EPROTO no response, CT buffer is in error
*/
static int wait_for_response(struct guc_ct_buffer_desc *desc,
u32 fence,
u32 *status)
static int wait_for_ctb_desc_update(struct guc_ct_buffer_desc *desc,
u32 fence,
u32 *status)
{
int err;
@ -363,71 +433,440 @@ static int wait_for_response(struct guc_ct_buffer_desc *desc,
return err;
}
static int ctch_send(struct intel_guc *guc,
/**
* wait_for_ct_request_update - Wait for CT request state update.
* @req: pointer to pending request
* @status: placeholder for status
*
* For each sent request, Guc shall send bac CT response message.
* Our message handler will update status of tracked request once
* response message with given fence is received. Wait here and
* check for valid response status value.
*
* Return:
* * 0 response received (status is valid)
* * -ETIMEDOUT no response within hardcoded timeout
*/
static int wait_for_ct_request_update(struct ct_request *req, u32 *status)
{
int err;
/*
* Fast commands should complete in less than 10us, so sample quickly
* up to that length of time, then switch to a slower sleep-wait loop.
* No GuC command should ever take longer than 10ms.
*/
#define done INTEL_GUC_MSG_IS_RESPONSE(READ_ONCE(req->status))
err = wait_for_us(done, 10);
if (err)
err = wait_for(done, 10);
#undef done
if (unlikely(err))
DRM_ERROR("CT: fence %u err %d\n", req->fence, err);
*status = req->status;
return err;
}
static int ctch_send(struct intel_guc_ct *ct,
struct intel_guc_ct_channel *ctch,
const u32 *action,
u32 len,
u32 *response_buf,
u32 response_buf_size,
u32 *status)
{
struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_SEND];
struct guc_ct_buffer_desc *desc = ctb->desc;
struct ct_request request;
unsigned long flags;
u32 fence;
int err;
GEM_BUG_ON(!ctch_is_open(ctch));
GEM_BUG_ON(!len);
GEM_BUG_ON(len & ~GUC_CT_MSG_LEN_MASK);
GEM_BUG_ON(!response_buf && response_buf_size);
fence = ctch_get_next_fence(ctch);
err = ctb_write(ctb, action, len, fence);
if (unlikely(err))
return err;
request.fence = fence;
request.status = 0;
request.response_len = response_buf_size;
request.response_buf = response_buf;
intel_guc_notify(guc);
spin_lock_irqsave(&ct->lock, flags);
list_add_tail(&request.link, &ct->pending_requests);
spin_unlock_irqrestore(&ct->lock, flags);
err = wait_for_response(desc, fence, status);
err = ctb_write(ctb, action, len, fence, !!response_buf);
if (unlikely(err))
return err;
if (*status != INTEL_GUC_STATUS_SUCCESS)
return -EIO;
return 0;
goto unlink;
intel_guc_notify(ct_to_guc(ct));
if (response_buf)
err = wait_for_ct_request_update(&request, status);
else
err = wait_for_ctb_desc_update(desc, fence, status);
if (unlikely(err))
goto unlink;
if (!INTEL_GUC_MSG_IS_RESPONSE_SUCCESS(*status)) {
err = -EIO;
goto unlink;
}
if (response_buf) {
/* There shall be no data in the status */
WARN_ON(INTEL_GUC_MSG_TO_DATA(request.status));
/* Return actual response len */
err = request.response_len;
} else {
/* There shall be no response payload */
WARN_ON(request.response_len);
/* Return data decoded from the status dword */
err = INTEL_GUC_MSG_TO_DATA(*status);
}
unlink:
spin_lock_irqsave(&ct->lock, flags);
list_del(&request.link);
spin_unlock_irqrestore(&ct->lock, flags);
return err;
}
/*
* Command Transport (CT) buffer based GuC send function.
*/
static int intel_guc_send_ct(struct intel_guc *guc, const u32 *action, u32 len)
static int intel_guc_send_ct(struct intel_guc *guc, const u32 *action, u32 len,
u32 *response_buf, u32 response_buf_size)
{
struct intel_guc_ct_channel *ctch = &guc->ct.host_channel;
struct intel_guc_ct *ct = &guc->ct;
struct intel_guc_ct_channel *ctch = &ct->host_channel;
u32 status = ~0; /* undefined */
int err;
int ret;
mutex_lock(&guc->send_mutex);
err = ctch_send(guc, ctch, action, len, &status);
if (unlikely(err)) {
ret = ctch_send(ct, ctch, action, len, response_buf, response_buf_size,
&status);
if (unlikely(ret < 0)) {
DRM_ERROR("CT: send action %#X failed; err=%d status=%#X\n",
action[0], err, status);
action[0], ret, status);
} else if (unlikely(ret)) {
CT_DEBUG_DRIVER("CT: send action %#x returned %d (%#x)\n",
action[0], ret, ret);
}
mutex_unlock(&guc->send_mutex);
return err;
return ret;
}
static inline unsigned int ct_header_get_len(u32 header)
{
return (header >> GUC_CT_MSG_LEN_SHIFT) & GUC_CT_MSG_LEN_MASK;
}
static inline unsigned int ct_header_get_action(u32 header)
{
return (header >> GUC_CT_MSG_ACTION_SHIFT) & GUC_CT_MSG_ACTION_MASK;
}
static inline bool ct_header_is_response(u32 header)
{
return ct_header_get_action(header) == INTEL_GUC_ACTION_DEFAULT;
}
static int ctb_read(struct intel_guc_ct_buffer *ctb, u32 *data)
{
struct guc_ct_buffer_desc *desc = ctb->desc;
u32 head = desc->head / 4; /* in dwords */
u32 tail = desc->tail / 4; /* in dwords */
u32 size = desc->size / 4; /* in dwords */
u32 *cmds = ctb->cmds;
s32 available; /* in dwords */
unsigned int len;
unsigned int i;
GEM_BUG_ON(desc->size % 4);
GEM_BUG_ON(desc->head % 4);
GEM_BUG_ON(desc->tail % 4);
GEM_BUG_ON(tail >= size);
GEM_BUG_ON(head >= size);
/* tail == head condition indicates empty */
available = tail - head;
if (unlikely(available == 0))
return -ENODATA;
/* beware of buffer wrap case */
if (unlikely(available < 0))
available += size;
CT_DEBUG_DRIVER("CT: available %d (%u:%u)\n", available, head, tail);
GEM_BUG_ON(available < 0);
data[0] = cmds[head];
head = (head + 1) % size;
/* message len with header */
len = ct_header_get_len(data[0]) + 1;
if (unlikely(len > (u32)available)) {
DRM_ERROR("CT: incomplete message %*ph %*ph %*ph\n",
4, data,
4 * (head + available - 1 > size ?
size - head : available - 1), &cmds[head],
4 * (head + available - 1 > size ?
available - 1 - size + head : 0), &cmds[0]);
return -EPROTO;
}
for (i = 1; i < len; i++) {
data[i] = cmds[head];
head = (head + 1) % size;
}
CT_DEBUG_DRIVER("CT: received %*ph\n", 4 * len, data);
desc->head = head * 4;
return 0;
}
/**
* Enable buffer based command transport
* Shall only be called for platforms with HAS_GUC_CT.
* @guc: the guc
* return: 0 on success
* non-zero on failure
* DOC: CTB GuC to Host response
*
* Format of the CTB GuC to Host response message is as follows::
*
* +------------+---------+---------+---------+---------+---------+
* | msg[0] | [1] | [2] | [3] | ... | [n-1] |
* +------------+---------+---------+---------+---------+---------+
* | MESSAGE | MESSAGE PAYLOAD |
* + HEADER +---------+---------+---------+---------+---------+
* | | 0 | 1 | 2 | ... | n |
* +============+=========+=========+=========+=========+=========+
* | len >= 2 | FENCE | STATUS | response specific data |
* +------+-----+---------+---------+---------+---------+---------+
*
* ^-----------------------len-----------------------^
*/
int intel_guc_enable_ct(struct intel_guc *guc)
static int ct_handle_response(struct intel_guc_ct *ct, const u32 *msg)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct intel_guc_ct_channel *ctch = &guc->ct.host_channel;
u32 header = msg[0];
u32 len = ct_header_get_len(header);
u32 msglen = len + 1; /* total message length including header */
u32 fence;
u32 status;
u32 datalen;
struct ct_request *req;
bool found = false;
GEM_BUG_ON(!ct_header_is_response(header));
GEM_BUG_ON(!in_irq());
/* Response payload shall at least include fence and status */
if (unlikely(len < 2)) {
DRM_ERROR("CT: corrupted response %*ph\n", 4 * msglen, msg);
return -EPROTO;
}
fence = msg[1];
status = msg[2];
datalen = len - 2;
/* Format of the status follows RESPONSE message */
if (unlikely(!INTEL_GUC_MSG_IS_RESPONSE(status))) {
DRM_ERROR("CT: corrupted response %*ph\n", 4 * msglen, msg);
return -EPROTO;
}
CT_DEBUG_DRIVER("CT: response fence %u status %#x\n", fence, status);
spin_lock(&ct->lock);
list_for_each_entry(req, &ct->pending_requests, link) {
if (unlikely(fence != req->fence)) {
CT_DEBUG_DRIVER("CT: request %u awaits response\n",
req->fence);
continue;
}
if (unlikely(datalen > req->response_len)) {
DRM_ERROR("CT: response %u too long %*ph\n",
req->fence, 4 * msglen, msg);
datalen = 0;
}
if (datalen)
memcpy(req->response_buf, msg + 3, 4 * datalen);
req->response_len = datalen;
WRITE_ONCE(req->status, status);
found = true;
break;
}
spin_unlock(&ct->lock);
if (!found)
DRM_ERROR("CT: unsolicited response %*ph\n", 4 * msglen, msg);
return 0;
}
static void ct_process_request(struct intel_guc_ct *ct,
u32 action, u32 len, const u32 *payload)
{
struct intel_guc *guc = ct_to_guc(ct);
CT_DEBUG_DRIVER("CT: request %x %*ph\n", action, 4 * len, payload);
switch (action) {
case INTEL_GUC_ACTION_DEFAULT:
if (unlikely(len < 1))
goto fail_unexpected;
intel_guc_to_host_process_recv_msg(guc, *payload);
break;
default:
fail_unexpected:
DRM_ERROR("CT: unexpected request %x %*ph\n",
action, 4 * len, payload);
break;
}
}
static bool ct_process_incoming_requests(struct intel_guc_ct *ct)
{
unsigned long flags;
struct ct_incoming_request *request;
u32 header;
u32 *payload;
bool done;
spin_lock_irqsave(&ct->lock, flags);
request = list_first_entry_or_null(&ct->incoming_requests,
struct ct_incoming_request, link);
if (request)
list_del(&request->link);
done = !!list_empty(&ct->incoming_requests);
spin_unlock_irqrestore(&ct->lock, flags);
if (!request)
return true;
header = request->msg[0];
payload = &request->msg[1];
ct_process_request(ct,
ct_header_get_action(header),
ct_header_get_len(header),
payload);
kfree(request);
return done;
}
static void ct_incoming_request_worker_func(struct work_struct *w)
{
struct intel_guc_ct *ct = container_of(w, struct intel_guc_ct, worker);
bool done;
done = ct_process_incoming_requests(ct);
if (!done)
queue_work(system_unbound_wq, &ct->worker);
}
/**
* DOC: CTB GuC to Host request
*
* Format of the CTB GuC to Host request message is as follows::
*
* +------------+---------+---------+---------+---------+---------+
* | msg[0] | [1] | [2] | [3] | ... | [n-1] |
* +------------+---------+---------+---------+---------+---------+
* | MESSAGE | MESSAGE PAYLOAD |
* + HEADER +---------+---------+---------+---------+---------+
* | | 0 | 1 | 2 | ... | n |
* +============+=========+=========+=========+=========+=========+
* | len | request specific data |
* +------+-----+---------+---------+---------+---------+---------+
*
* ^-----------------------len-----------------------^
*/
static int ct_handle_request(struct intel_guc_ct *ct, const u32 *msg)
{
u32 header = msg[0];
u32 len = ct_header_get_len(header);
u32 msglen = len + 1; /* total message length including header */
struct ct_incoming_request *request;
unsigned long flags;
GEM_BUG_ON(ct_header_is_response(header));
request = kmalloc(sizeof(*request) + 4 * msglen, GFP_ATOMIC);
if (unlikely(!request)) {
DRM_ERROR("CT: dropping request %*ph\n", 4 * msglen, msg);
return 0; /* XXX: -ENOMEM ? */
}
memcpy(request->msg, msg, 4 * msglen);
spin_lock_irqsave(&ct->lock, flags);
list_add_tail(&request->link, &ct->incoming_requests);
spin_unlock_irqrestore(&ct->lock, flags);
queue_work(system_unbound_wq, &ct->worker);
return 0;
}
static void ct_process_host_channel(struct intel_guc_ct *ct)
{
struct intel_guc_ct_channel *ctch = &ct->host_channel;
struct intel_guc_ct_buffer *ctb = &ctch->ctbs[CTB_RECV];
u32 msg[GUC_CT_MSG_LEN_MASK + 1]; /* one extra dw for the header */
int err = 0;
if (!ctch_is_open(ctch))
return;
do {
err = ctb_read(ctb, msg);
if (err)
break;
if (ct_header_is_response(msg[0]))
err = ct_handle_response(ct, msg);
else
err = ct_handle_request(ct, msg);
} while (!err);
if (GEM_WARN_ON(err == -EPROTO)) {
DRM_ERROR("CT: corrupted message detected!\n");
ctb->desc->is_in_error = 1;
}
}
/*
* When we're communicating with the GuC over CT, GuC uses events
* to notify us about new messages being posted on the RECV buffer.
*/
static void intel_guc_to_host_event_handler_ct(struct intel_guc *guc)
{
struct intel_guc_ct *ct = &guc->ct;
ct_process_host_channel(ct);
}
/**
* intel_guc_ct_enable - Enable buffer based command transport.
* @ct: pointer to CT struct
*
* Shall only be called for platforms with HAS_GUC_CT.
*
* Return: 0 on success, a negative errno code on failure.
*/
int intel_guc_ct_enable(struct intel_guc_ct *ct)
{
struct intel_guc *guc = ct_to_guc(ct);
struct drm_i915_private *i915 = guc_to_i915(guc);
struct intel_guc_ct_channel *ctch = &ct->host_channel;
int err;
GEM_BUG_ON(!HAS_GUC_CT(dev_priv));
GEM_BUG_ON(!HAS_GUC_CT(i915));
err = ctch_open(guc, ctch);
if (unlikely(err))
@ -435,21 +874,24 @@ int intel_guc_enable_ct(struct intel_guc *guc)
/* Switch into cmd transport buffer based send() */
guc->send = intel_guc_send_ct;
guc->handler = intel_guc_to_host_event_handler_ct;
DRM_INFO("CT: %s\n", enableddisabled(true));
return 0;
}
/**
* Disable buffer based command transport.
* intel_guc_ct_disable - Disable buffer based command transport.
* @ct: pointer to CT struct
*
* Shall only be called for platforms with HAS_GUC_CT.
* @guc: the guc
*/
void intel_guc_disable_ct(struct intel_guc *guc)
void intel_guc_ct_disable(struct intel_guc_ct *ct)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct intel_guc_ct_channel *ctch = &guc->ct.host_channel;
struct intel_guc *guc = ct_to_guc(ct);
struct drm_i915_private *i915 = guc_to_i915(guc);
struct intel_guc_ct_channel *ctch = &ct->host_channel;
GEM_BUG_ON(!HAS_GUC_CT(dev_priv));
GEM_BUG_ON(!HAS_GUC_CT(i915));
if (!ctch_is_open(ctch))
return;
@ -458,5 +900,6 @@ void intel_guc_disable_ct(struct intel_guc *guc)
/* Disable send */
guc->send = intel_guc_send_nop;
guc->handler = intel_guc_to_host_event_handler_nop;
DRM_INFO("CT: %s\n", enableddisabled(false));
}

18
drivers/gpu/drm/i915/intel_guc_ct.h
View File

@ -75,12 +75,22 @@ struct intel_guc_ct_channel {
struct intel_guc_ct {
struct intel_guc_ct_channel host_channel;
/* other channels are tbd */
/** @lock: protects pending requests list */
spinlock_t lock;
/** @pending_requests: list of requests waiting for response */
struct list_head pending_requests;
/** @incoming_requests: list of incoming requests */
struct list_head incoming_requests;
/** @worker: worker for handling incoming requests */
struct work_struct worker;
};
void intel_guc_ct_init_early(struct intel_guc_ct *ct);
/* XXX: move to intel_uc.h ? don't fit there either */
int intel_guc_enable_ct(struct intel_guc *guc);
void intel_guc_disable_ct(struct intel_guc *guc);
int intel_guc_ct_enable(struct intel_guc_ct *ct);
void intel_guc_ct_disable(struct intel_guc_ct *ct);
#endif /* _INTEL_GUC_CT_H_ */

7
drivers/gpu/drm/i915/intel_guc_fw.c
View File

@ -165,7 +165,7 @@ static int guc_xfer_ucode(struct intel_guc *guc, struct i915_vma *vma)
I915_WRITE(DMA_COPY_SIZE, guc_fw->header_size + guc_fw->ucode_size);
/* Set the source address for the new blob */
offset = guc_ggtt_offset(vma) + guc_fw->header_offset;
offset = intel_guc_ggtt_offset(guc, vma) + guc_fw->header_offset;
I915_WRITE(DMA_ADDR_0_LOW, lower_32_bits(offset));
I915_WRITE(DMA_ADDR_0_HIGH, upper_32_bits(offset) & 0xFFFF);
@ -275,9 +275,8 @@ static int guc_fw_xfer(struct intel_uc_fw *guc_fw, struct i915_vma *vma)
* Called from intel_uc_init_hw() during driver load, resume from sleep and
* after a GPU reset.
*
* The firmware image should have already been fetched into memory by the
* earlier call to intel_uc_init_fw(), so here we need to only check that
* fetch succeeded, and then transfer the image to the h/w.
* The firmware image should have already been fetched into memory, so only
* check that fetch succeeded, and then transfer the image to the h/w.
*
* Return: non-zero code on error
*/

155
drivers/gpu/drm/i915/intel_guc_fwif.h
View File

@ -127,7 +127,7 @@
#define GUC_PROFILE_ENABLED (1 << 7)
#define GUC_WQ_TRACK_ENABLED (1 << 8)
#define GUC_ADS_ENABLED (1 << 9)
#define GUC_DEBUG_RESERVED (1 << 10)
#define GUC_LOG_DEFAULT_DISABLED (1 << 10)
#define GUC_ADS_ADDR_SHIFT 11
#define GUC_ADS_ADDR_MASK 0xfffff800
@ -327,6 +327,58 @@ struct guc_stage_desc {
u64 desc_private;
} __packed;
/**
* DOC: CTB based communication
*
* The CTB (command transport buffer) communication between Host and GuC
* is based on u32 data stream written to the shared buffer. One buffer can
* be used to transmit data only in one direction (one-directional channel).
*
* Current status of the each buffer is stored in the buffer descriptor.
* Buffer descriptor holds tail and head fields that represents active data
* stream. The tail field is updated by the data producer (sender), and head
* field is updated by the data consumer (receiver)::
*
* +------------+
* | DESCRIPTOR | +=================+============+========+
* +============+ | | MESSAGE(s) | |
* | address |--------->+=================+============+========+
* +------------+
* | head | ^-----head--------^
* +------------+
* | tail | ^---------tail-----------------^
* +------------+
* | size | ^---------------size--------------------^
* +------------+
*
* Each message in data stream starts with the single u32 treated as a header,
* followed by optional set of u32 data that makes message specific payload::
*
* +------------+---------+---------+---------+
* | MESSAGE |
* +------------+---------+---------+---------+
* | msg[0] | [1] | ... | [n-1] |
* +------------+---------+---------+---------+
* | MESSAGE | MESSAGE PAYLOAD |
* + HEADER +---------+---------+---------+
* | | 0 | ... | n |
* +======+=====+=========+=========+=========+
* | 31:16| code| | | |
* +------+-----+ | | |
* | 15:5|flags| | | |
* +------+-----+ | | |
* | 4:0| len| | | |
* +------+-----+---------+---------+---------+
*
* ^-------------len-------------^
*
* The message header consists of:
*
* - **len**, indicates length of the message payload (in u32)
* - **code**, indicates message code
* - **flags**, holds various bits to control message handling
*/
/*
* Describes single command transport buffer.
* Used by both guc-master and clients.
@ -534,16 +586,6 @@ struct guc_log_buffer_state {
u32 version;
} __packed;
union guc_log_control {
struct {
u32 logging_enabled:1;
u32 reserved1:3;
u32 verbosity:4;
u32 reserved2:24;
};
u32 value;
} __packed;
struct guc_ctx_report {
u32 report_return_status;
u32 reserved1[64];
@ -570,7 +612,68 @@ struct guc_shared_ctx_data {
struct guc_ctx_report preempt_ctx_report[GUC_MAX_ENGINES_NUM];
} __packed;
/* This Action will be programmed in C180 - SOFT_SCRATCH_O_REG */
/**
* DOC: MMIO based communication
*
* The MMIO based communication between Host and GuC uses software scratch
* registers, where first register holds data treated as message header,
* and other registers are used to hold message payload.
*
* For Gen9+, GuC uses software scratch registers 0xC180-0xC1B8
*
* +-----------+---------+---------+---------+
* | MMIO[0] | MMIO[1] | ... | MMIO[n] |
* +-----------+---------+---------+---------+
* | header | optional payload |
* +======+====+=========+=========+=========+
* | 31:28|type| | | |
* +------+----+ | | |
* | 27:16|data| | | |
* +------+----+ | | |
* | 15:0|code| | | |
* +------+----+---------+---------+---------+
*
* The message header consists of:
*
* - **type**, indicates message type
* - **code**, indicates message code, is specific for **type**
* - **data**, indicates message data, optional, depends on **code**
*
* The following message **types** are supported:
*
* - **REQUEST**, indicates Host-to-GuC request, requested GuC action code
* must be priovided in **code** field. Optional action specific parameters
* can be provided in remaining payload registers or **data** field.
*
* - **RESPONSE**, indicates GuC-to-Host response from earlier GuC request,
* action response status will be provided in **code** field. Optional
* response data can be returned in remaining payload registers or **data**
* field.
*/
#define INTEL_GUC_MSG_TYPE_SHIFT 28
#define INTEL_GUC_MSG_TYPE_MASK (0xF << INTEL_GUC_MSG_TYPE_SHIFT)
#define INTEL_GUC_MSG_DATA_SHIFT 16
#define INTEL_GUC_MSG_DATA_MASK (0xFFF << INTEL_GUC_MSG_DATA_SHIFT)
#define INTEL_GUC_MSG_CODE_SHIFT 0
#define INTEL_GUC_MSG_CODE_MASK (0xFFFF << INTEL_GUC_MSG_CODE_SHIFT)
#define __INTEL_GUC_MSG_GET(T, m) \
(((m) & INTEL_GUC_MSG_ ## T ## _MASK) >> INTEL_GUC_MSG_ ## T ## _SHIFT)
#define INTEL_GUC_MSG_TO_TYPE(m) __INTEL_GUC_MSG_GET(TYPE, m)
#define INTEL_GUC_MSG_TO_DATA(m) __INTEL_GUC_MSG_GET(DATA, m)
#define INTEL_GUC_MSG_TO_CODE(m) __INTEL_GUC_MSG_GET(CODE, m)
enum intel_guc_msg_type {
INTEL_GUC_MSG_TYPE_REQUEST = 0x0,
INTEL_GUC_MSG_TYPE_RESPONSE = 0xF,
};
#define __INTEL_GUC_MSG_TYPE_IS(T, m) \
(INTEL_GUC_MSG_TO_TYPE(m) == INTEL_GUC_MSG_TYPE_ ## T)
#define INTEL_GUC_MSG_IS_REQUEST(m) __INTEL_GUC_MSG_TYPE_IS(REQUEST, m)
#define INTEL_GUC_MSG_IS_RESPONSE(m) __INTEL_GUC_MSG_TYPE_IS(RESPONSE, m)
enum intel_guc_action {
INTEL_GUC_ACTION_DEFAULT = 0x0,
INTEL_GUC_ACTION_REQUEST_PREEMPTION = 0x2,
@ -602,24 +705,22 @@ enum intel_guc_report_status {
INTEL_GUC_REPORT_STATUS_COMPLETE = 0x4,
};
/*
* The GuC sends its response to a command by overwriting the
* command in SS0. The response is distinguishable from a command
* by the fact that all the MASK bits are set. The remaining bits
* give more detail.
*/
#define INTEL_GUC_RECV_MASK ((u32)0xF0000000)
#define INTEL_GUC_RECV_IS_RESPONSE(x) ((u32)(x) >= INTEL_GUC_RECV_MASK)
#define INTEL_GUC_RECV_STATUS(x) (INTEL_GUC_RECV_MASK | (x))
#define GUC_LOG_CONTROL_LOGGING_ENABLED (1 << 0)
#define GUC_LOG_CONTROL_VERBOSITY_SHIFT 4
#define GUC_LOG_CONTROL_VERBOSITY_MASK (0xF << GUC_LOG_CONTROL_VERBOSITY_SHIFT)
#define GUC_LOG_CONTROL_DEFAULT_LOGGING (1 << 8)
/* GUC will return status back to SOFT_SCRATCH_O_REG */
enum intel_guc_status {
INTEL_GUC_STATUS_SUCCESS = INTEL_GUC_RECV_STATUS(0x0),
INTEL_GUC_STATUS_ALLOCATE_DOORBELL_FAIL = INTEL_GUC_RECV_STATUS(0x10),
INTEL_GUC_STATUS_DEALLOCATE_DOORBELL_FAIL = INTEL_GUC_RECV_STATUS(0x20),
INTEL_GUC_STATUS_GENERIC_FAIL = INTEL_GUC_RECV_STATUS(0x0000F000)
enum intel_guc_response_status {
INTEL_GUC_RESPONSE_STATUS_SUCCESS = 0x0,
INTEL_GUC_RESPONSE_STATUS_GENERIC_FAIL = 0xF000,
};
#define INTEL_GUC_MSG_IS_RESPONSE_SUCCESS(m) \
(typecheck(u32, (m)) && \
((m) & (INTEL_GUC_MSG_TYPE_MASK | INTEL_GUC_MSG_CODE_MASK)) == \
((INTEL_GUC_MSG_TYPE_RESPONSE << INTEL_GUC_MSG_TYPE_SHIFT) | \
(INTEL_GUC_RESPONSE_STATUS_SUCCESS << INTEL_GUC_MSG_CODE_SHIFT)))
/* This action will be programmed in C1BC - SOFT_SCRATCH_15_REG */
enum intel_guc_recv_message {
INTEL_GUC_RECV_MSG_CRASH_DUMP_POSTED = BIT(1),

552
drivers/gpu/drm/i915/intel_guc_log.c
View File

@ -23,12 +23,11 @@
*/
#include <linux/debugfs.h>
#include <linux/relay.h>
#include "intel_guc_log.h"
#include "i915_drv.h"
static void guc_log_capture_logs(struct intel_guc *guc);
static void guc_log_capture_logs(struct intel_guc_log *log);
/**
* DOC: GuC firmware log
@ -39,7 +38,7 @@ static void guc_log_capture_logs(struct intel_guc *guc);
* registers value.
*/
static int guc_log_flush_complete(struct intel_guc *guc)
static int guc_action_flush_log_complete(struct intel_guc *guc)
{
u32 action[] = {
INTEL_GUC_ACTION_LOG_BUFFER_FILE_FLUSH_COMPLETE
@ -48,7 +47,7 @@ static int guc_log_flush_complete(struct intel_guc *guc)
return intel_guc_send(guc, action, ARRAY_SIZE(action));
}
static int guc_log_flush(struct intel_guc *guc)
static int guc_action_flush_log(struct intel_guc *guc)
{
u32 action[] = {
INTEL_GUC_ACTION_FORCE_LOG_BUFFER_FLUSH,
@ -58,22 +57,40 @@ static int guc_log_flush(struct intel_guc *guc)
return intel_guc_send(guc, action, ARRAY_SIZE(action));
}
static int guc_log_control(struct intel_guc *guc, bool enable, u32 verbosity)
static int guc_action_control_log(struct intel_guc *guc, bool enable,
bool default_logging, u32 verbosity)
{
union guc_log_control control_val = {
{
.logging_enabled = enable,
.verbosity = verbosity,
},
};
u32 action[] = {
INTEL_GUC_ACTION_UK_LOG_ENABLE_LOGGING,
control_val.value
(enable ? GUC_LOG_CONTROL_LOGGING_ENABLED : 0) |
(verbosity << GUC_LOG_CONTROL_VERBOSITY_SHIFT) |
(default_logging ? GUC_LOG_CONTROL_DEFAULT_LOGGING : 0)
};
GEM_BUG_ON(verbosity > GUC_LOG_VERBOSITY_MAX);
return intel_guc_send(guc, action, ARRAY_SIZE(action));
}
static inline struct intel_guc *log_to_guc(struct intel_guc_log *log)
{
return container_of(log, struct intel_guc, log);
}
static void guc_log_enable_flush_events(struct intel_guc_log *log)
{
intel_guc_enable_msg(log_to_guc(log),
INTEL_GUC_RECV_MSG_FLUSH_LOG_BUFFER |
INTEL_GUC_RECV_MSG_CRASH_DUMP_POSTED);
}
static void guc_log_disable_flush_events(struct intel_guc_log *log)
{
intel_guc_disable_msg(log_to_guc(log),
INTEL_GUC_RECV_MSG_FLUSH_LOG_BUFFER |
INTEL_GUC_RECV_MSG_CRASH_DUMP_POSTED);
}
/*
* Sub buffer switch callback. Called whenever relay has to switch to a new
* sub buffer, relay stays on the same sub buffer if 0 is returned.
@ -121,14 +138,7 @@ static struct dentry *create_buf_file_callback(const char *filename,
if (!parent)
return NULL;
/*
* Not using the channel filename passed as an argument, since for each
* channel relay appends the corresponding CPU number to the filename
* passed in relay_open(). This should be fine as relay just needs a
* dentry of the file associated with the channel buffer and that file's
* name need not be same as the filename passed as an argument.
*/
buf_file = debugfs_create_file("guc_log", mode,
buf_file = debugfs_create_file(filename, mode,
parent, buf, &relay_file_operations);
return buf_file;
}
@ -149,59 +159,7 @@ static struct rchan_callbacks relay_callbacks = {
.remove_buf_file = remove_buf_file_callback,
};
static int guc_log_relay_file_create(struct intel_guc *guc)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct dentry *log_dir;
int ret;
if (!i915_modparams.guc_log_level)
return 0;
mutex_lock(&guc->log.runtime.relay_lock);
/* For now create the log file in /sys/kernel/debug/dri/0 dir */
log_dir = dev_priv->drm.primary->debugfs_root;
/*
* If /sys/kernel/debug/dri/0 location do not exist, then debugfs is
* not mounted and so can't create the relay file.
* The relay API seems to fit well with debugfs only, for availing relay
* there are 3 requirements which can be met for debugfs file only in a
* straightforward/clean manner :-
* i) Need the associated dentry pointer of the file, while opening the
* relay channel.
* ii) Should be able to use 'relay_file_operations' fops for the file.
* iii) Set the 'i_private' field of file's inode to the pointer of
* relay channel buffer.
*/
if (!log_dir) {
DRM_ERROR("Debugfs dir not available yet for GuC log file\n");
ret = -ENODEV;
goto out_unlock;
}
ret = relay_late_setup_files(guc->log.runtime.relay_chan, "guc_log", log_dir);
if (ret < 0 && ret != -EEXIST) {
DRM_ERROR("Couldn't associate relay chan with file %d\n", ret);
goto out_unlock;
}
ret = 0;
out_unlock:
mutex_unlock(&guc->log.runtime.relay_lock);
return ret;
}
static bool guc_log_has_relay(struct intel_guc *guc)
{
lockdep_assert_held(&guc->log.runtime.relay_lock);
return guc->log.runtime.relay_chan != NULL;
}
static void guc_move_to_next_buf(struct intel_guc *guc)
static void guc_move_to_next_buf(struct intel_guc_log *log)
{
/*
* Make sure the updates made in the sub buffer are visible when
@ -209,21 +167,15 @@ static void guc_move_to_next_buf(struct intel_guc *guc)
*/
smp_wmb();
if (!guc_log_has_relay(guc))
return;
/* All data has been written, so now move the offset of sub buffer. */
relay_reserve(guc->log.runtime.relay_chan, guc->log.vma->obj->base.size);
relay_reserve(log->relay.channel, log->vma->obj->base.size);
/* Switch to the next sub buffer */
relay_flush(guc->log.runtime.relay_chan);
relay_flush(log->relay.channel);
}
static void *guc_get_write_buffer(struct intel_guc *guc)
static void *guc_get_write_buffer(struct intel_guc_log *log)
{
if (!guc_log_has_relay(guc))
return NULL;
/*
* Just get the base address of a new sub buffer and copy data into it
* ourselves. NULL will be returned in no-overwrite mode, if all sub
@ -233,25 +185,25 @@ static void *guc_get_write_buffer(struct intel_guc *guc)
* done without using relay_reserve() along with relay_write(). So its
* better to use relay_reserve() alone.
*/
return relay_reserve(guc->log.runtime.relay_chan, 0);
return relay_reserve(log->relay.channel, 0);
}
static bool guc_check_log_buf_overflow(struct intel_guc *guc,
static bool guc_check_log_buf_overflow(struct intel_guc_log *log,
enum guc_log_buffer_type type,
unsigned int full_cnt)
{
unsigned int prev_full_cnt = guc->log.prev_overflow_count[type];
unsigned int prev_full_cnt = log->stats[type].sampled_overflow;
bool overflow = false;
if (full_cnt != prev_full_cnt) {
overflow = true;
guc->log.prev_overflow_count[type] = full_cnt;
guc->log.total_overflow_count[type] += full_cnt - prev_full_cnt;
log->stats[type].overflow = full_cnt;
log->stats[type].sampled_overflow += full_cnt - prev_full_cnt;
if (full_cnt < prev_full_cnt) {
/* buffer_full_cnt is a 4 bit counter */
guc->log.total_overflow_count[type] += 16;
log->stats[type].sampled_overflow += 16;
}
DRM_ERROR_RATELIMITED("GuC log buffer overflow\n");
}
@ -275,7 +227,7 @@ static unsigned int guc_get_log_buffer_size(enum guc_log_buffer_type type)
return 0;
}
static void guc_read_update_log_buffer(struct intel_guc *guc)
static void guc_read_update_log_buffer(struct intel_guc_log *log)
{
unsigned int buffer_size, read_offset, write_offset, bytes_to_copy, full_cnt;
struct guc_log_buffer_state *log_buf_state, *log_buf_snapshot_state;
@ -284,16 +236,16 @@ static void guc_read_update_log_buffer(struct intel_guc *guc)
void *src_data, *dst_data;
bool new_overflow;
if (WARN_ON(!guc->log.runtime.buf_addr))
return;
mutex_lock(&log->relay.lock);
if (WARN_ON(!intel_guc_log_relay_enabled(log)))
goto out_unlock;
/* Get the pointer to shared GuC log buffer */
log_buf_state = src_data = guc->log.runtime.buf_addr;
mutex_lock(&guc->log.runtime.relay_lock);
log_buf_state = src_data = log->relay.buf_addr;
/* Get the pointer to local buffer to store the logs */
log_buf_snapshot_state = dst_data = guc_get_write_buffer(guc);
log_buf_snapshot_state = dst_data = guc_get_write_buffer(log);
if (unlikely(!log_buf_snapshot_state)) {
/*
@ -301,10 +253,9 @@ static void guc_read_update_log_buffer(struct intel_guc *guc)
* getting consumed by User at a slow rate.
*/
DRM_ERROR_RATELIMITED("no sub-buffer to capture logs\n");
guc->log.capture_miss_count++;
mutex_unlock(&guc->log.runtime.relay_lock);
log->relay.full_count++;
return;
goto out_unlock;
}
/* Actual logs are present from the 2nd page */
@ -325,8 +276,8 @@ static void guc_read_update_log_buffer(struct intel_guc *guc)
full_cnt = log_buf_state_local.buffer_full_cnt;
/* Bookkeeping stuff */
guc->log.flush_count[type] += log_buf_state_local.flush_to_file;
new_overflow = guc_check_log_buf_overflow(guc, type, full_cnt);
log->stats[type].flush += log_buf_state_local.flush_to_file;
new_overflow = guc_check_log_buf_overflow(log, type, full_cnt);
/* Update the state of shared log buffer */
log_buf_state->read_ptr = write_offset;
@ -373,38 +324,35 @@ static void guc_read_update_log_buffer(struct intel_guc *guc)
dst_data += buffer_size;
}
guc_move_to_next_buf(guc);
guc_move_to_next_buf(log);
mutex_unlock(&guc->log.runtime.relay_lock);
out_unlock:
mutex_unlock(&log->relay.lock);
}
static void capture_logs_work(struct work_struct *work)
{
struct intel_guc *guc =
container_of(work, struct intel_guc, log.runtime.flush_work);
struct intel_guc_log *log =
container_of(work, struct intel_guc_log, relay.flush_work);
guc_log_capture_logs(guc);
guc_log_capture_logs(log);
}
static bool guc_log_has_runtime(struct intel_guc *guc)
{
return guc->log.runtime.buf_addr != NULL;
}
static int guc_log_runtime_create(struct intel_guc *guc)
static int guc_log_map(struct intel_guc_log *log)
{
struct intel_guc *guc = log_to_guc(log);
struct drm_i915_private *dev_priv = guc_to_i915(guc);
void *vaddr;
int ret;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
lockdep_assert_held(&log->relay.lock);
if (!guc->log.vma)
if (!log->vma)
return -ENODEV;
GEM_BUG_ON(guc_log_has_runtime(guc));
ret = i915_gem_object_set_to_wc_domain(guc->log.vma->obj, true);
mutex_lock(&dev_priv->drm.struct_mutex);
ret = i915_gem_object_set_to_wc_domain(log->vma->obj, true);
mutex_unlock(&dev_priv->drm.struct_mutex);
if (ret)
return ret;
@ -413,49 +361,40 @@ static int guc_log_runtime_create(struct intel_guc *guc)
* buffer pages, so that we can directly get the data
* (up-to-date) from memory.
*/
vaddr = i915_gem_object_pin_map(guc->log.vma->obj, I915_MAP_WC);
vaddr = i915_gem_object_pin_map(log->vma->obj, I915_MAP_WC);
if (IS_ERR(vaddr)) {
DRM_ERROR("Couldn't map log buffer pages %d\n", ret);
return PTR_ERR(vaddr);
}
guc->log.runtime.buf_addr = vaddr;
log->relay.buf_addr = vaddr;
return 0;
}
static void guc_log_runtime_destroy(struct intel_guc *guc)
static void guc_log_unmap(struct intel_guc_log *log)
{
/*
* It's possible that the runtime stuff was never allocated because
* GuC log was disabled at the boot time.
*/
if (!guc_log_has_runtime(guc))
return;
lockdep_assert_held(&log->relay.lock);
i915_gem_object_unpin_map(guc->log.vma->obj);
guc->log.runtime.buf_addr = NULL;
i915_gem_object_unpin_map(log->vma->obj);
log->relay.buf_addr = NULL;
}
void intel_guc_log_init_early(struct intel_guc *guc)
void intel_guc_log_init_early(struct intel_guc_log *log)
{
mutex_init(&guc->log.runtime.relay_lock);
INIT_WORK(&guc->log.runtime.flush_work, capture_logs_work);
mutex_init(&log->relay.lock);
INIT_WORK(&log->relay.flush_work, capture_logs_work);
}
int intel_guc_log_relay_create(struct intel_guc *guc)
static int guc_log_relay_create(struct intel_guc_log *log)
{
struct intel_guc *guc = log_to_guc(log);
struct drm_i915_private *dev_priv = guc_to_i915(guc);
struct rchan *guc_log_relay_chan;
size_t n_subbufs, subbuf_size;
int ret;
if (!i915_modparams.guc_log_level)
return 0;
mutex_lock(&guc->log.runtime.relay_lock);
GEM_BUG_ON(guc_log_has_relay(guc));
lockdep_assert_held(&log->relay.lock);
/* Keep the size of sub buffers same as shared log buffer */
subbuf_size = GUC_LOG_SIZE;
@ -468,157 +407,56 @@ int intel_guc_log_relay_create(struct intel_guc *guc)
*/
n_subbufs = 8;
/*
* Create a relay channel, so that we have buffers for storing
* the GuC firmware logs, the channel will be linked with a file
* later on when debugfs is registered.
*/
guc_log_relay_chan = relay_open(NULL, NULL, subbuf_size,
n_subbufs, &relay_callbacks, dev_priv);
guc_log_relay_chan = relay_open("guc_log",
dev_priv->drm.primary->debugfs_root,
subbuf_size, n_subbufs,
&relay_callbacks, dev_priv);
if (!guc_log_relay_chan) {
DRM_ERROR("Couldn't create relay chan for GuC logging\n");
ret = -ENOMEM;
goto err;
return ret;
}
GEM_BUG_ON(guc_log_relay_chan->subbuf_size < subbuf_size);
guc->log.runtime.relay_chan = guc_log_relay_chan;
mutex_unlock(&guc->log.runtime.relay_lock);
log->relay.channel = guc_log_relay_chan;
return 0;
err:
mutex_unlock(&guc->log.runtime.relay_lock);
/* logging will be off */
i915_modparams.guc_log_level = 0;
return ret;
}
void intel_guc_log_relay_destroy(struct intel_guc *guc)
static void guc_log_relay_destroy(struct intel_guc_log *log)
{
mutex_lock(&guc->log.runtime.relay_lock);
lockdep_assert_held(&log->relay.lock);
/*
* It's possible that the relay was never allocated because
* GuC log was disabled at the boot time.
*/
if (!guc_log_has_relay(guc))
goto out_unlock;
relay_close(guc->log.runtime.relay_chan);
guc->log.runtime.relay_chan = NULL;
out_unlock:
mutex_unlock(&guc->log.runtime.relay_lock);
relay_close(log->relay.channel);
log->relay.channel = NULL;
}
static int guc_log_late_setup(struct intel_guc *guc)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
int ret;
if (!guc_log_has_runtime(guc)) {
/*
* If log was disabled at boot time, then setup needed to handle
* log buffer flush interrupts would not have been done yet, so
* do that now.
*/
ret = intel_guc_log_relay_create(guc);
if (ret)
goto err;
mutex_lock(&dev_priv->drm.struct_mutex);
intel_runtime_pm_get(dev_priv);
ret = guc_log_runtime_create(guc);
intel_runtime_pm_put(dev_priv);
mutex_unlock(&dev_priv->drm.struct_mutex);
if (ret)
goto err_relay;
}
ret = guc_log_relay_file_create(guc);
if (ret)
goto err_runtime;
return 0;
err_runtime:
mutex_lock(&dev_priv->drm.struct_mutex);
guc_log_runtime_destroy(guc);
mutex_unlock(&dev_priv->drm.struct_mutex);
err_relay:
intel_guc_log_relay_destroy(guc);
err:
/* logging will remain off */
i915_modparams.guc_log_level = 0;
return ret;
}
static void guc_log_capture_logs(struct intel_guc *guc)
static void guc_log_capture_logs(struct intel_guc_log *log)
{
struct intel_guc *guc = log_to_guc(log);
struct drm_i915_private *dev_priv = guc_to_i915(guc);
guc_read_update_log_buffer(guc);
guc_read_update_log_buffer(log);
/*
* Generally device is expected to be active only at this
* time, so get/put should be really quick.
*/
intel_runtime_pm_get(dev_priv);
guc_log_flush_complete(guc);
guc_action_flush_log_complete(guc);
intel_runtime_pm_put(dev_priv);
}
static void guc_flush_logs(struct intel_guc *guc)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
if (!USES_GUC_SUBMISSION(dev_priv) || !i915_modparams.guc_log_level)
return;
/* First disable the interrupts, will be renabled afterwards */
mutex_lock(&dev_priv->drm.struct_mutex);
intel_runtime_pm_get(dev_priv);
gen9_disable_guc_interrupts(dev_priv);
intel_runtime_pm_put(dev_priv);
mutex_unlock(&dev_priv->drm.struct_mutex);
/*
* Before initiating the forceful flush, wait for any pending/ongoing
* flush to complete otherwise forceful flush may not actually happen.
*/
flush_work(&guc->log.runtime.flush_work);
/* Ask GuC to update the log buffer state */
intel_runtime_pm_get(dev_priv);
guc_log_flush(guc);
intel_runtime_pm_put(dev_priv);
/* GuC would have updated log buffer by now, so capture it */
guc_log_capture_logs(guc);
}
int intel_guc_log_create(struct intel_guc *guc)
int intel_guc_log_create(struct intel_guc_log *log)
{
struct intel_guc *guc = log_to_guc(log);
struct i915_vma *vma;
unsigned long offset;
u32 flags;
int ret;
GEM_BUG_ON(guc->log.vma);
/*
* We require SSE 4.1 for fast reads from the GuC log buffer and
* it should be present on the chipsets supporting GuC based
* submisssions.
*/
if (WARN_ON(!i915_has_memcpy_from_wc())) {
ret = -EINVAL;
goto err;
}
GEM_BUG_ON(log->vma);
vma = intel_guc_allocate_vma(guc, GUC_LOG_SIZE);
if (IS_ERR(vma)) {
@ -626,13 +464,7 @@ int intel_guc_log_create(struct intel_guc *guc)
goto err;
}
guc->log.vma = vma;
if (i915_modparams.guc_log_level) {
ret = guc_log_runtime_create(guc);
if (ret < 0)
goto err_vma;
}
log->vma = vma;
/* each allocated unit is a page */
flags = GUC_LOG_VALID | GUC_LOG_NOTIFY_ON_HALF_FULL |
@ -640,117 +472,159 @@ int intel_guc_log_create(struct intel_guc *guc)
(GUC_LOG_ISR_PAGES << GUC_LOG_ISR_SHIFT) |
(GUC_LOG_CRASH_PAGES << GUC_LOG_CRASH_SHIFT);
offset = guc_ggtt_offset(vma) >> PAGE_SHIFT; /* in pages */
guc->log.flags = (offset << GUC_LOG_BUF_ADDR_SHIFT) | flags;
offset = intel_guc_ggtt_offset(guc, vma) >> PAGE_SHIFT;
log->flags = (offset << GUC_LOG_BUF_ADDR_SHIFT) | flags;
return 0;
err_vma:
i915_vma_unpin_and_release(&guc->log.vma);
err:
/* logging will be off */
i915_modparams.guc_log_level = 0;
return ret;
}
void intel_guc_log_destroy(struct intel_guc *guc)
void intel_guc_log_destroy(struct intel_guc_log *log)
{
guc_log_runtime_destroy(guc);
i915_vma_unpin_and_release(&guc->log.vma);
i915_vma_unpin_and_release(&log->vma);
}
int intel_guc_log_control(struct intel_guc *guc, u64 control_val)
int intel_guc_log_level_get(struct intel_guc_log *log)
{
GEM_BUG_ON(!log->vma);
GEM_BUG_ON(i915_modparams.guc_log_level < 0);
return i915_modparams.guc_log_level;
}
int intel_guc_log_level_set(struct intel_guc_log *log, u64 val)
{
struct intel_guc *guc = log_to_guc(log);
struct drm_i915_private *dev_priv = guc_to_i915(guc);
bool enable_logging = control_val > 0;
u32 verbosity;
int ret;
if (!guc->log.vma)
return -ENODEV;
BUILD_BUG_ON(GUC_LOG_VERBOSITY_MIN != 0);
GEM_BUG_ON(!log->vma);
GEM_BUG_ON(i915_modparams.guc_log_level < 0);
BUILD_BUG_ON(GUC_LOG_VERBOSITY_MIN);
if (control_val > 1 + GUC_LOG_VERBOSITY_MAX)
/*
* GuC is recognizing log levels starting from 0 to max, we're using 0
* as indication that logging should be disabled.
*/
if (val < GUC_LOG_LEVEL_DISABLED || val > GUC_LOG_LEVEL_MAX)
return -EINVAL;
/* This combination doesn't make sense & won't have any effect */
if (!enable_logging && !i915_modparams.guc_log_level)
return 0;
mutex_lock(&dev_priv->drm.struct_mutex);
verbosity = enable_logging ? control_val - 1 : 0;
if (i915_modparams.guc_log_level == val) {
ret = 0;
goto out_unlock;
}
ret = mutex_lock_interruptible(&dev_priv->drm.struct_mutex);
if (ret)
return ret;
intel_runtime_pm_get(dev_priv);
ret = guc_log_control(guc, enable_logging, verbosity);
ret = guc_action_control_log(guc, GUC_LOG_LEVEL_IS_VERBOSE(val),
GUC_LOG_LEVEL_IS_ENABLED(val),
GUC_LOG_LEVEL_TO_VERBOSITY(val));
intel_runtime_pm_put(dev_priv);
if (ret) {
DRM_DEBUG_DRIVER("guc_log_control action failed %d\n", ret);
goto out_unlock;
}
i915_modparams.guc_log_level = val;
out_unlock:
mutex_unlock(&dev_priv->drm.struct_mutex);
if (ret < 0) {
DRM_DEBUG_DRIVER("guc_logging_control action failed %d\n", ret);
return ret;
}
if (enable_logging) {
i915_modparams.guc_log_level = 1 + verbosity;
/*
* If log was disabled at boot time, then the relay channel file
* wouldn't have been created by now and interrupts also would
* not have been enabled. Try again now, just in case.
*/
ret = guc_log_late_setup(guc);
if (ret < 0) {
DRM_DEBUG_DRIVER("GuC log late setup failed %d\n", ret);
return ret;
}
/* GuC logging is currently the only user of Guc2Host interrupts */
mutex_lock(&dev_priv->drm.struct_mutex);
intel_runtime_pm_get(dev_priv);
gen9_enable_guc_interrupts(dev_priv);
intel_runtime_pm_put(dev_priv);
mutex_unlock(&dev_priv->drm.struct_mutex);
} else {
/*
* Once logging is disabled, GuC won't generate logs & send an
* interrupt. But there could be some data in the log buffer
* which is yet to be captured. So request GuC to update the log
* buffer state and then collect the left over logs.
*/
guc_flush_logs(guc);
/* As logging is disabled, update log level to reflect that */
i915_modparams.guc_log_level = 0;
}
return ret;
}
void i915_guc_log_register(struct drm_i915_private *dev_priv)
bool intel_guc_log_relay_enabled(const struct intel_guc_log *log)
{
if (!USES_GUC_SUBMISSION(dev_priv) || !i915_modparams.guc_log_level)
return;
guc_log_late_setup(&dev_priv->guc);
return log->relay.buf_addr;
}
void i915_guc_log_unregister(struct drm_i915_private *dev_priv)
int intel_guc_log_relay_open(struct intel_guc_log *log)
{
struct intel_guc *guc = &dev_priv->guc;
int ret;
if (!USES_GUC_SUBMISSION(dev_priv))
return;
mutex_lock(&log->relay.lock);
mutex_lock(&dev_priv->drm.struct_mutex);
/* GuC logging is currently the only user of Guc2Host interrupts */
intel_runtime_pm_get(dev_priv);
gen9_disable_guc_interrupts(dev_priv);
intel_runtime_pm_put(dev_priv);
if (intel_guc_log_relay_enabled(log)) {
ret = -EEXIST;
goto out_unlock;
}
guc_log_runtime_destroy(guc);
mutex_unlock(&dev_priv->drm.struct_mutex);
/*
* We require SSE 4.1 for fast reads from the GuC log buffer and
* it should be present on the chipsets supporting GuC based
* submisssions.
*/
if (!i915_has_memcpy_from_wc()) {
ret = -ENXIO;
goto out_unlock;
}
intel_guc_log_relay_destroy(guc);
ret = guc_log_relay_create(log);
if (ret)
goto out_unlock;
ret = guc_log_map(log);
if (ret)
goto out_relay;
mutex_unlock(&log->relay.lock);
guc_log_enable_flush_events(log);
/*
* When GuC is logging without us relaying to userspace, we're ignoring
* the flush notification. This means that we need to unconditionally
* flush on relay enabling, since GuC only notifies us once.
*/
queue_work(log->relay.flush_wq, &log->relay.flush_work);
return 0;
out_relay:
guc_log_relay_destroy(log);
out_unlock:
mutex_unlock(&log->relay.lock);
return ret;
}
void intel_guc_log_relay_flush(struct intel_guc_log *log)
{
struct intel_guc *guc = log_to_guc(log);
struct drm_i915_private *i915 = guc_to_i915(guc);
/*
* Before initiating the forceful flush, wait for any pending/ongoing
* flush to complete otherwise forceful flush may not actually happen.
*/
flush_work(&log->relay.flush_work);
intel_runtime_pm_get(i915);
guc_action_flush_log(guc);
intel_runtime_pm_put(i915);
/* GuC would have updated log buffer by now, so capture it */
guc_log_capture_logs(log);
}
void intel_guc_log_relay_close(struct intel_guc_log *log)
{
guc_log_disable_flush_events(log);
flush_work(&log->relay.flush_work);
mutex_lock(&log->relay.lock);
GEM_BUG_ON(!intel_guc_log_relay_enabled(log));
guc_log_unmap(log);
guc_log_relay_destroy(log);
mutex_unlock(&log->relay.lock);
}
void intel_guc_log_handle_flush_event(struct intel_guc_log *log)
{
queue_work(log->relay.flush_wq, &log->relay.flush_work);
}

59
drivers/gpu/drm/i915/intel_guc_log.h
View File

@ -25,11 +25,12 @@
#ifndef _INTEL_GUC_LOG_H_
#define _INTEL_GUC_LOG_H_
#include <linux/mutex.h>
#include <linux/relay.h>
#include <linux/workqueue.h>
#include "intel_guc_fwif.h"
struct drm_i915_private;
struct intel_guc;
/*
@ -39,33 +40,53 @@ struct intel_guc;
#define GUC_LOG_SIZE ((1 + GUC_LOG_DPC_PAGES + 1 + GUC_LOG_ISR_PAGES + \
1 + GUC_LOG_CRASH_PAGES + 1) << PAGE_SHIFT)
/*
* While we're using plain log level in i915, GuC controls are much more...
* "elaborate"? We have a couple of bits for verbosity, separate bit for actual
* log enabling, and separate bit for default logging - which "conveniently"
* ignores the enable bit.
*/
#define GUC_LOG_LEVEL_DISABLED 0
#define GUC_LOG_LEVEL_NON_VERBOSE 1
#define GUC_LOG_LEVEL_IS_ENABLED(x) ((x) > GUC_LOG_LEVEL_DISABLED)
#define GUC_LOG_LEVEL_IS_VERBOSE(x) ((x) > GUC_LOG_LEVEL_NON_VERBOSE)
#define GUC_LOG_LEVEL_TO_VERBOSITY(x) ({ \
typeof(x) _x = (x); \
GUC_LOG_LEVEL_IS_VERBOSE(_x) ? _x - 2 : 0; \
})
#define GUC_VERBOSITY_TO_LOG_LEVEL(x) ((x) + 2)
#define GUC_LOG_LEVEL_MAX GUC_VERBOSITY_TO_LOG_LEVEL(GUC_LOG_VERBOSITY_MAX)
struct intel_guc_log {
u32 flags;
struct i915_vma *vma;
/* The runtime stuff gets created only when GuC logging gets enabled */
struct {
void *buf_addr;
struct workqueue_struct *flush_wq;
struct work_struct flush_work;
struct rchan *relay_chan;
/* To serialize the access to relay_chan */
struct mutex relay_lock;
} runtime;
struct rchan *channel;
struct mutex lock;
u32 full_count;
} relay;
/* logging related stats */
u32 capture_miss_count;
u32 flush_interrupt_count;
u32 prev_overflow_count[GUC_MAX_LOG_BUFFER];
u32 total_overflow_count[GUC_MAX_LOG_BUFFER];
u32 flush_count[GUC_MAX_LOG_BUFFER];
struct {
u32 sampled_overflow;
u32 overflow;
u32 flush;
} stats[GUC_MAX_LOG_BUFFER];
};
int intel_guc_log_create(struct intel_guc *guc);
void intel_guc_log_destroy(struct intel_guc *guc);
void intel_guc_log_init_early(struct intel_guc *guc);
int intel_guc_log_relay_create(struct intel_guc *guc);
void intel_guc_log_relay_destroy(struct intel_guc *guc);
int intel_guc_log_control(struct intel_guc *guc, u64 control_val);
void i915_guc_log_register(struct drm_i915_private *dev_priv);
void i915_guc_log_unregister(struct drm_i915_private *dev_priv);
void intel_guc_log_init_early(struct intel_guc_log *log);
int intel_guc_log_create(struct intel_guc_log *log);
void intel_guc_log_destroy(struct intel_guc_log *log);
int intel_guc_log_level_get(struct intel_guc_log *log);
int intel_guc_log_level_set(struct intel_guc_log *log, u64 control_val);
bool intel_guc_log_relay_enabled(const struct intel_guc_log *log);
int intel_guc_log_relay_open(struct intel_guc_log *log);
void intel_guc_log_relay_flush(struct intel_guc_log *log);
void intel_guc_log_relay_close(struct intel_guc_log *log);
void intel_guc_log_handle_flush_event(struct intel_guc_log *log);
#endif

14
drivers/gpu/drm/i915/intel_guc_reg.h
View File

@ -66,22 +66,20 @@
#define UOS_MOVE (1<<4)
#define START_DMA (1<<0)
#define DMA_GUC_WOPCM_OFFSET _MMIO(0xc340)
#define GUC_WOPCM_OFFSET_VALID (1<<0)
#define HUC_LOADING_AGENT_VCR (0<<1)
#define HUC_LOADING_AGENT_GUC (1<<1)
#define GUC_WOPCM_OFFSET_VALUE 0x80000 /* 512KB */
#define GUC_WOPCM_OFFSET_SHIFT 14
#define GUC_WOPCM_OFFSET_MASK (0x3ffff << GUC_WOPCM_OFFSET_SHIFT)
#define GUC_MAX_IDLE_COUNT _MMIO(0xC3E4)
#define HUC_STATUS2 _MMIO(0xD3B0)
#define HUC_FW_VERIFIED (1<<7)
/* Defines WOPCM space available to GuC firmware */
#define GUC_WOPCM_SIZE _MMIO(0xc050)
/* GuC addresses below GUC_WOPCM_TOP don't map through the GTT */
#define GUC_WOPCM_TOP (0x80 << 12) /* 512KB */
#define BXT_GUC_WOPCM_RC6_RESERVED (0x10 << 12) /* 64KB */
/* GuC addresses above GUC_GGTT_TOP also don't map through the GTT */
#define GUC_GGTT_TOP 0xFEE00000
#define GUC_WOPCM_SIZE_LOCKED (1<<0)
#define GUC_WOPCM_SIZE_SHIFT 12
#define GUC_WOPCM_SIZE_MASK (0xfffff << GUC_WOPCM_SIZE_SHIFT)
#define GEN8_GT_PM_CONFIG _MMIO(0x138140)
#define GEN9LP_GT_PM_CONFIG _MMIO(0x138140)

47
drivers/gpu/drm/i915/intel_guc_submission.c
View File

@ -231,8 +231,8 @@ static int create_doorbell(struct intel_guc_client *client)
if (ret) {
__destroy_doorbell(client);
__update_doorbell_desc(client, GUC_DOORBELL_INVALID);
DRM_ERROR("Couldn't create client %u doorbell: %d\n",
client->stage_id, ret);
DRM_DEBUG_DRIVER("Couldn't create client %u doorbell: %d\n",
client->stage_id, ret);
return ret;
}
@ -386,8 +386,8 @@ static void guc_stage_desc_init(struct intel_guc *guc,
lrc->context_desc = lower_32_bits(ce->lrc_desc);
/* The state page is after PPHWSP */
lrc->ring_lrca =
guc_ggtt_offset(ce->state) + LRC_STATE_PN * PAGE_SIZE;
lrc->ring_lrca = intel_guc_ggtt_offset(guc, ce->state) +
LRC_STATE_PN * PAGE_SIZE;
/* XXX: In direct submission, the GuC wants the HW context id
* here. In proxy submission, it wants the stage id
@ -395,7 +395,7 @@ static void guc_stage_desc_init(struct intel_guc *guc,
lrc->context_id = (client->stage_id << GUC_ELC_CTXID_OFFSET) |
(guc_engine_id << GUC_ELC_ENGINE_OFFSET);
lrc->ring_begin = guc_ggtt_offset(ce->ring->vma);
lrc->ring_begin = intel_guc_ggtt_offset(guc, ce->ring->vma);
lrc->ring_end = lrc->ring_begin + ce->ring->size - 1;
lrc->ring_next_free_location = lrc->ring_begin;
lrc->ring_current_tail_pointer_value = 0;
@ -411,7 +411,7 @@ static void guc_stage_desc_init(struct intel_guc *guc,
* The doorbell, process descriptor, and workqueue are all parts
* of the client object, which the GuC will reference via the GGTT
*/
gfx_addr = guc_ggtt_offset(client->vma);
gfx_addr = intel_guc_ggtt_offset(guc, client->vma);
desc->db_trigger_phy = sg_dma_address(client->vma->pages->sgl) +
client->doorbell_offset;
desc->db_trigger_cpu = ptr_to_u64(__get_doorbell(client));
@ -584,7 +584,7 @@ static void inject_preempt_context(struct work_struct *work)
data[3] = engine->guc_id;
data[4] = guc->execbuf_client->priority;
data[5] = guc->execbuf_client->stage_id;
data[6] = guc_ggtt_offset(guc->shared_data);
data[6] = intel_guc_ggtt_offset(guc, guc->shared_data);
if (WARN_ON(intel_guc_send(guc, data, ARRAY_SIZE(data)))) {
execlists_clear_active(&engine->execlists,
@ -657,6 +657,16 @@ static void port_assign(struct execlist_port *port, struct i915_request *rq)
port_set(port, i915_request_get(rq));
}
static inline int rq_prio(const struct i915_request *rq)
{
return rq->priotree.priority;
}
static inline int port_prio(const struct execlist_port *port)
{
return rq_prio(port_request(port));
}
static void guc_dequeue(struct intel_engine_cs *engine)
{
struct intel_engine_execlists * const execlists = &engine->execlists;
@ -672,12 +682,12 @@ static void guc_dequeue(struct intel_engine_cs *engine)
GEM_BUG_ON(rb_first(&execlists->queue) != rb);
if (port_isset(port)) {
if (engine->i915->preempt_context) {
if (intel_engine_has_preemption(engine)) {
struct guc_preempt_work *preempt_work =
&engine->i915->guc.preempt_work[engine->id];
int prio = execlists->queue_priority;
if (execlists->queue_priority >
max(port_request(port)->priotree.priority, 0)) {
if (__execlists_need_preempt(prio, port_prio(port))) {
execlists_set_active(execlists,
EXECLISTS_ACTIVE_PREEMPT);
queue_work(engine->i915->guc.preempt_wq,
@ -728,7 +738,7 @@ done:
execlists->first = rb;
if (submit) {
port_assign(port, last);
execlists_set_active(execlists, EXECLISTS_ACTIVE_USER);
execlists_user_begin(execlists, execlists->port);
guc_submit(engine);
}
@ -748,17 +758,20 @@ static void guc_submission_tasklet(unsigned long data)
struct execlist_port *port = execlists->port;
struct i915_request *rq;
rq = port_request(&port[0]);
rq = port_request(port);
while (rq && i915_request_completed(rq)) {
trace_i915_request_out(rq);
i915_request_put(rq);
execlists_port_complete(execlists, port);
rq = port_request(&port[0]);
port = execlists_port_complete(execlists, port);
if (port_isset(port)) {
execlists_user_begin(execlists, port);
rq = port_request(port);
} else {
execlists_user_end(execlists);
rq = NULL;
}
}
if (!rq)
execlists_clear_active(execlists, EXECLISTS_ACTIVE_USER);
if (execlists_is_active(execlists, EXECLISTS_ACTIVE_PREEMPT) &&
intel_read_status_page(engine, I915_GEM_HWS_PREEMPT_INDEX) ==

13
drivers/gpu/drm/i915/intel_hangcheck.c
View File

@ -246,9 +246,8 @@ engine_stuck(struct intel_engine_cs *engine, u64 acthd)
*/
tmp = I915_READ_CTL(engine);
if (tmp & RING_WAIT) {
i915_handle_error(dev_priv, BIT(engine->id),
"Kicking stuck wait on %s",
engine->name);
i915_handle_error(dev_priv, BIT(engine->id), 0,
"stuck wait on %s", engine->name);
I915_WRITE_CTL(engine, tmp);
return ENGINE_WAIT_KICK;
}
@ -258,8 +257,8 @@ engine_stuck(struct intel_engine_cs *engine, u64 acthd)
default:
return ENGINE_DEAD;
case 1:
i915_handle_error(dev_priv, ALL_ENGINES,
"Kicking stuck semaphore on %s",
i915_handle_error(dev_priv, ALL_ENGINES, 0,
"stuck semaphore on %s",
engine->name);
I915_WRITE_CTL(engine, tmp);
return ENGINE_WAIT_KICK;
@ -386,13 +385,13 @@ static void hangcheck_declare_hang(struct drm_i915_private *i915,
if (stuck != hung)
hung &= ~stuck;
len = scnprintf(msg, sizeof(msg),
"%s on ", stuck == hung ? "No progress" : "Hang");
"%s on ", stuck == hung ? "no progress" : "hang");
for_each_engine_masked(engine, i915, hung, tmp)
len += scnprintf(msg + len, sizeof(msg) - len,
"%s, ", engine->name);
msg[len-2] = '\0';
return i915_handle_error(i915, hung, "%s", msg);
return i915_handle_error(i915, hung, I915_ERROR_CAPTURE, "%s", msg);
}
/*

185
drivers/gpu/drm/i915/intel_hdcp.c
View File

@ -37,6 +37,43 @@ static int intel_hdcp_poll_ksv_fifo(struct intel_digital_port *intel_dig_port,
return 0;
}
static bool hdcp_key_loadable(struct drm_i915_private *dev_priv)
{
struct i915_power_domains *power_domains = &dev_priv->power_domains;
struct i915_power_well *power_well;
enum i915_power_well_id id;
bool enabled = false;
/*
* On HSW and BDW, Display HW loads the Key as soon as Display resumes.
* On all BXT+, SW can load the keys only when the PW#1 is turned on.
*/
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
id = HSW_DISP_PW_GLOBAL;
else
id = SKL_DISP_PW_1;
mutex_lock(&power_domains->lock);
/* PG1 (power well #1) needs to be enabled */
for_each_power_well(dev_priv, power_well) {
if (power_well->id == id) {
enabled = power_well->ops->is_enabled(dev_priv,
power_well);
break;
}
}
mutex_unlock(&power_domains->lock);
/*
* Another req for hdcp key loadability is enabled state of pll for
* cdclk. Without active crtc we wont land here. So we are assuming that
* cdclk is already on.
*/
return enabled;
}
static void intel_hdcp_clear_keys(struct drm_i915_private *dev_priv)
{
I915_WRITE(HDCP_KEY_CONF, HDCP_CLEAR_KEYS_TRIGGER);
@ -142,53 +179,17 @@ bool intel_hdcp_is_ksv_valid(u8 *ksv)
return true;
}
/* Implements Part 2 of the HDCP authorization procedure */
static
int intel_hdcp_auth_downstream(struct intel_digital_port *intel_dig_port,
const struct intel_hdcp_shim *shim)
int intel_hdcp_validate_v_prime(struct intel_digital_port *intel_dig_port,
const struct intel_hdcp_shim *shim,
u8 *ksv_fifo, u8 num_downstream, u8 *bstatus)
{
struct drm_i915_private *dev_priv;
u32 vprime, sha_text, sha_leftovers, rep_ctl;
u8 bstatus[2], num_downstream, *ksv_fifo;
int ret, i, j, sha_idx;
dev_priv = intel_dig_port->base.base.dev->dev_private;
ret = intel_hdcp_poll_ksv_fifo(intel_dig_port, shim);
if (ret) {
DRM_ERROR("KSV list failed to become ready (%d)\n", ret);
return ret;
}
ret = shim->read_bstatus(intel_dig_port, bstatus);
if (ret)
return ret;
if (DRM_HDCP_MAX_DEVICE_EXCEEDED(bstatus[0]) ||
DRM_HDCP_MAX_CASCADE_EXCEEDED(bstatus[1])) {
DRM_ERROR("Max Topology Limit Exceeded\n");
return -EPERM;
}
/*
* When repeater reports 0 device count, HDCP1.4 spec allows disabling
* the HDCP encryption. That implies that repeater can't have its own
* display. As there is no consumption of encrypted content in the
* repeater with 0 downstream devices, we are failing the
* authentication.
*/
num_downstream = DRM_HDCP_NUM_DOWNSTREAM(bstatus[0]);
if (num_downstream == 0)
return -EINVAL;
ksv_fifo = kzalloc(num_downstream * DRM_HDCP_KSV_LEN, GFP_KERNEL);
if (!ksv_fifo)
return -ENOMEM;
ret = shim->read_ksv_fifo(intel_dig_port, num_downstream, ksv_fifo);
if (ret)
return ret;
/* Process V' values from the receiver */
for (i = 0; i < DRM_HDCP_V_PRIME_NUM_PARTS; i++) {
ret = shim->read_v_prime_part(intel_dig_port, i, &vprime);
@ -353,7 +354,8 @@ int intel_hdcp_auth_downstream(struct intel_digital_port *intel_dig_port,
return ret;
sha_idx += sizeof(sha_text);
} else {
DRM_ERROR("Invalid number of leftovers %d\n", sha_leftovers);
DRM_DEBUG_KMS("Invalid number of leftovers %d\n",
sha_leftovers);
return -EINVAL;
}
@ -381,17 +383,83 @@ int intel_hdcp_auth_downstream(struct intel_digital_port *intel_dig_port,
if (intel_wait_for_register(dev_priv, HDCP_REP_CTL,
HDCP_SHA1_COMPLETE,
HDCP_SHA1_COMPLETE, 1)) {
DRM_ERROR("Timed out waiting for SHA1 complete\n");
DRM_DEBUG_KMS("Timed out waiting for SHA1 complete\n");
return -ETIMEDOUT;
}
if (!(I915_READ(HDCP_REP_CTL) & HDCP_SHA1_V_MATCH)) {
DRM_ERROR("SHA-1 mismatch, HDCP failed\n");
DRM_DEBUG_KMS("SHA-1 mismatch, HDCP failed\n");
return -ENXIO;
}
return 0;
}
/* Implements Part 2 of the HDCP authorization procedure */
static
int intel_hdcp_auth_downstream(struct intel_digital_port *intel_dig_port,
const struct intel_hdcp_shim *shim)
{
u8 bstatus[2], num_downstream, *ksv_fifo;
int ret, i, tries = 3;
ret = intel_hdcp_poll_ksv_fifo(intel_dig_port, shim);
if (ret) {
DRM_ERROR("KSV list failed to become ready (%d)\n", ret);
return ret;
}
ret = shim->read_bstatus(intel_dig_port, bstatus);
if (ret)
return ret;
if (DRM_HDCP_MAX_DEVICE_EXCEEDED(bstatus[0]) ||
DRM_HDCP_MAX_CASCADE_EXCEEDED(bstatus[1])) {
DRM_ERROR("Max Topology Limit Exceeded\n");
return -EPERM;
}
/*
* When repeater reports 0 device count, HDCP1.4 spec allows disabling
* the HDCP encryption. That implies that repeater can't have its own
* display. As there is no consumption of encrypted content in the
* repeater with 0 downstream devices, we are failing the
* authentication.
*/
num_downstream = DRM_HDCP_NUM_DOWNSTREAM(bstatus[0]);
if (num_downstream == 0)
return -EINVAL;
ksv_fifo = kzalloc(num_downstream * DRM_HDCP_KSV_LEN, GFP_KERNEL);
if (!ksv_fifo)
return -ENOMEM;
ret = shim->read_ksv_fifo(intel_dig_port, num_downstream, ksv_fifo);
if (ret)
goto err;
/*
* When V prime mismatches, DP Spec mandates re-read of
* V prime atleast twice.
*/
for (i = 0; i < tries; i++) {
ret = intel_hdcp_validate_v_prime(intel_dig_port, shim,
ksv_fifo, num_downstream,
bstatus);
if (!ret)
break;
}
if (i == tries) {
DRM_ERROR("V Prime validation failed.(%d)\n", ret);
goto err;
}
DRM_DEBUG_KMS("HDCP is enabled (%d downstream devices)\n",
num_downstream);
return 0;
ret = 0;
err:
kfree(ksv_fifo);
return ret;
}
/* Implements Part 1 of the HDCP authorization procedure */
@ -506,15 +574,26 @@ static int intel_hdcp_auth(struct intel_digital_port *intel_dig_port,
*/
wait_remaining_ms_from_jiffies(r0_prime_gen_start, 300);
ri.reg = 0;
ret = shim->read_ri_prime(intel_dig_port, ri.shim);
if (ret)
return ret;
I915_WRITE(PORT_HDCP_RPRIME(port), ri.reg);
tries = 3;
/* Wait for Ri prime match */
if (wait_for(I915_READ(PORT_HDCP_STATUS(port)) &
(HDCP_STATUS_RI_MATCH | HDCP_STATUS_ENC), 1)) {
/*
* DP HDCP Spec mandates the two more reattempt to read R0, incase
* of R0 mismatch.
*/
for (i = 0; i < tries; i++) {
ri.reg = 0;
ret = shim->read_ri_prime(intel_dig_port, ri.shim);
if (ret)
return ret;
I915_WRITE(PORT_HDCP_RPRIME(port), ri.reg);
/* Wait for Ri prime match */
if (!wait_for(I915_READ(PORT_HDCP_STATUS(port)) &
(HDCP_STATUS_RI_MATCH | HDCP_STATUS_ENC), 1))
break;
}
if (i == tries) {
DRM_ERROR("Timed out waiting for Ri prime match (%x)\n",
I915_READ(PORT_HDCP_STATUS(port)));
return -ETIMEDOUT;
@ -580,8 +659,8 @@ static int _intel_hdcp_enable(struct intel_connector *connector)
DRM_DEBUG_KMS("[%s:%d] HDCP is being enabled...\n",
connector->base.name, connector->base.base.id);
if (!(I915_READ(SKL_FUSE_STATUS) & SKL_FUSE_PG_DIST_STATUS(1))) {
DRM_ERROR("PG1 is disabled, cannot load keys\n");
if (!hdcp_key_loadable(dev_priv)) {
DRM_ERROR("HDCP key Load is not possible\n");
return -ENXIO;
}

40
drivers/gpu/drm/i915/intel_hdmi.c
View File

@ -2082,41 +2082,33 @@ intel_hdmi_add_properties(struct intel_hdmi *intel_hdmi, struct drm_connector *c
* it enables scrambling. This should be called before enabling the HDMI
* 2.0 port, as the sink can choose to disable the scrambling if it doesn't
* detect a scrambled clock within 100 ms.
*
* Returns:
* True on success, false on failure.
*/
void intel_hdmi_handle_sink_scrambling(struct intel_encoder *encoder,
bool intel_hdmi_handle_sink_scrambling(struct intel_encoder *encoder,
struct drm_connector *connector,
bool high_tmds_clock_ratio,
bool scrambling)
{
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_hdmi *intel_hdmi = enc_to_intel_hdmi(&encoder->base);
struct drm_i915_private *dev_priv = connector->dev->dev_private;
struct drm_scrambling *sink_scrambling =
&connector->display_info.hdmi.scdc.scrambling;
struct i2c_adapter *adptr = intel_gmbus_get_adapter(dev_priv,
intel_hdmi->ddc_bus);
bool ret;
&connector->display_info.hdmi.scdc.scrambling;
struct i2c_adapter *adapter =
intel_gmbus_get_adapter(dev_priv, intel_hdmi->ddc_bus);
if (!sink_scrambling->supported)
return;
return true;
DRM_DEBUG_KMS("Setting sink scrambling for enc:%s connector:%s\n",
encoder->base.name, connector->name);
DRM_DEBUG_KMS("[CONNECTOR:%d:%s] scrambling=%s, TMDS bit clock ratio=1/%d\n",
connector->base.id, connector->name,
yesno(scrambling), high_tmds_clock_ratio ? 40 : 10);
/* Set TMDS bit clock ratio to 1/40 or 1/10 */
ret = drm_scdc_set_high_tmds_clock_ratio(adptr, high_tmds_clock_ratio);
if (!ret) {
DRM_ERROR("Set TMDS ratio failed\n");
return;
}
/* Enable/disable sink scrambling */
ret = drm_scdc_set_scrambling(adptr, scrambling);
if (!ret) {
DRM_ERROR("Set sink scrambling failed\n");
return;
}
DRM_DEBUG_KMS("sink scrambling handled\n");
/* Set TMDS bit clock ratio to 1/40 or 1/10, and enable/disable scrambling */
return drm_scdc_set_high_tmds_clock_ratio(adapter,
high_tmds_clock_ratio) &&
drm_scdc_set_scrambling(adapter, scrambling);
}
static u8 chv_port_to_ddc_pin(struct drm_i915_private *dev_priv, enum port port)

3
drivers/gpu/drm/i915/intel_hotplug.c
View File

@ -100,6 +100,8 @@ enum port intel_hpd_pin_to_port(struct drm_i915_private *dev_priv,
if (IS_CNL_WITH_PORT_F(dev_priv))
return PORT_F;
return PORT_E;
case HPD_PORT_F:
return PORT_F;
default:
return PORT_NONE; /* no port for this pin */
}
@ -132,6 +134,7 @@ enum hpd_pin intel_hpd_pin_default(struct drm_i915_private *dev_priv,
case PORT_F:
if (IS_CNL_WITH_PORT_F(dev_priv))
return HPD_PORT_E;
return HPD_PORT_F;
default:
MISSING_CASE(port);
return HPD_NONE;

30
drivers/gpu/drm/i915/intel_huc.c
View File

@ -55,7 +55,7 @@ int intel_huc_auth(struct intel_huc *huc)
return -ENOEXEC;
vma = i915_gem_object_ggtt_pin(huc->fw.obj, NULL, 0, 0,
PIN_OFFSET_BIAS | GUC_WOPCM_TOP);
PIN_OFFSET_BIAS | guc->ggtt_pin_bias);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
DRM_ERROR("HuC: Failed to pin huc fw object %d\n", ret);
@ -63,7 +63,8 @@ int intel_huc_auth(struct intel_huc *huc)
}
ret = intel_guc_auth_huc(guc,
guc_ggtt_offset(vma) + huc->fw.rsa_offset);
intel_guc_ggtt_offset(guc, vma) +
huc->fw.rsa_offset);
if (ret) {
DRM_ERROR("HuC: GuC did not ack Auth request %d\n", ret);
goto fail_unpin;
@ -91,3 +92,28 @@ fail:
DRM_ERROR("HuC: Authentication failed %d\n", ret);
return ret;
}
/**
* intel_huc_check_status() - check HuC status
* @huc: intel_huc structure
*
* This function reads status register to verify if HuC
* firmware was successfully loaded.
*
* Returns positive value if HuC firmware is loaded and verified
* and -ENODEV if HuC is not present.
*/
int intel_huc_check_status(struct intel_huc *huc)
{
struct drm_i915_private *dev_priv = huc_to_i915(huc);
u32 status;
if (!HAS_HUC(dev_priv))
return -ENODEV;
intel_runtime_pm_get(dev_priv);
status = I915_READ(HUC_STATUS2) & HUC_FW_VERIFIED;
intel_runtime_pm_put(dev_priv);
return status;
}

7
drivers/gpu/drm/i915/intel_huc.h
View File

@ -37,5 +37,12 @@ struct intel_huc {
void intel_huc_init_early(struct intel_huc *huc);
int intel_huc_auth(struct intel_huc *huc);
int intel_huc_check_status(struct intel_huc *huc);
static inline int intel_huc_sanitize(struct intel_huc *huc)
{
intel_uc_fw_sanitize(&huc->fw);
return 0;
}
#endif

8
drivers/gpu/drm/i915/intel_huc_fw.c
View File

@ -118,7 +118,8 @@ static int huc_fw_xfer(struct intel_uc_fw *huc_fw, struct i915_vma *vma)
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
/* Set the source address for the uCode */
offset = guc_ggtt_offset(vma) + huc_fw->header_offset;
offset = intel_guc_ggtt_offset(&dev_priv->guc, vma) +
huc_fw->header_offset;
I915_WRITE(DMA_ADDR_0_LOW, lower_32_bits(offset));
I915_WRITE(DMA_ADDR_0_HIGH, upper_32_bits(offset) & 0xFFFF);
@ -154,9 +155,8 @@ static int huc_fw_xfer(struct intel_uc_fw *huc_fw, struct i915_vma *vma)
* Called from intel_uc_init_hw() during driver load, resume from sleep and
* after a GPU reset. Note that HuC must be loaded before GuC.
*
* The firmware image should have already been fetched into memory by the
* earlier call to intel_uc_init_fw(), so here we need to only check that
* fetch succeeded, and then transfer the image to the h/w.
* The firmware image should have already been fetched into memory, so only
* check that fetch succeeded, and then transfer the image to the h/w.
*
* Return: non-zero code on error
*/

254
drivers/gpu/drm/i915/intel_lrc.c
View File

@ -139,6 +139,7 @@
#include "i915_gem_render_state.h"
#include "intel_lrc_reg.h"
#include "intel_mocs.h"
#include "intel_workarounds.h"
#define RING_EXECLIST_QFULL (1 << 0x2)
#define RING_EXECLIST1_VALID (1 << 0x3)
@ -183,7 +184,8 @@ static inline bool need_preempt(const struct intel_engine_cs *engine,
const struct i915_request *last,
int prio)
{
return engine->i915->preempt_context && prio > max(rq_prio(last), 0);
return (intel_engine_has_preemption(engine) &&
__execlists_need_preempt(prio, rq_prio(last)));
}
/**
@ -374,6 +376,19 @@ execlists_context_status_change(struct i915_request *rq, unsigned long status)
status, rq);
}
inline void
execlists_user_begin(struct intel_engine_execlists *execlists,
const struct execlist_port *port)
{
execlists_set_active_once(execlists, EXECLISTS_ACTIVE_USER);
}
inline void
execlists_user_end(struct intel_engine_execlists *execlists)
{
execlists_clear_active(execlists, EXECLISTS_ACTIVE_USER);
}
static inline void
execlists_context_schedule_in(struct i915_request *rq)
{
@ -454,10 +469,12 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
desc = execlists_update_context(rq);
GEM_DEBUG_EXEC(port[n].context_id = upper_32_bits(desc));
GEM_TRACE("%s in[%d]: ctx=%d.%d, seqno=%x, prio=%d\n",
GEM_TRACE("%s in[%d]: ctx=%d.%d, global=%d (fence %llx:%d) (current %d), prio=%d\n",
engine->name, n,
port[n].context_id, count,
rq->global_seqno,
rq->fence.context, rq->fence.seqno,
intel_engine_get_seqno(engine),
rq_prio(rq));
} else {
GEM_BUG_ON(!n);
@ -697,8 +714,27 @@ static void execlists_dequeue(struct intel_engine_cs *engine)
if (p->priority != I915_PRIORITY_NORMAL)
kmem_cache_free(engine->i915->priorities, p);
}
done:
execlists->queue_priority = rb ? to_priolist(rb)->priority : INT_MIN;
/*
* Here be a bit of magic! Or sleight-of-hand, whichever you prefer.
*
* We choose queue_priority such that if we add a request of greater
* priority than this, we kick the submission tasklet to decide on
* the right order of submitting the requests to hardware. We must
* also be prepared to reorder requests as they are in-flight on the
* HW. We derive the queue_priority then as the first "hole" in
* the HW submission ports and if there are no available slots,
* the priority of the lowest executing request, i.e. last.
*
* When we do receive a higher priority request ready to run from the
* user, see queue_request(), the queue_priority is bumped to that
* request triggering preemption on the next dequeue (or subsequent
* interrupt for secondary ports).
*/
execlists->queue_priority =
port != execlists->port ? rq_prio(last) : INT_MIN;
execlists->first = rb;
if (submit)
port_assign(port, last);
@ -710,7 +746,7 @@ unlock:
spin_unlock_irq(&engine->timeline->lock);
if (submit) {
execlists_set_active(execlists, EXECLISTS_ACTIVE_USER);
execlists_user_begin(execlists, execlists->port);
execlists_submit_ports(engine);
}
@ -727,6 +763,13 @@ execlists_cancel_port_requests(struct intel_engine_execlists * const execlists)
while (num_ports-- && port_isset(port)) {
struct i915_request *rq = port_request(port);
GEM_TRACE("%s:port%u global=%d (fence %llx:%d), (current %d)\n",
rq->engine->name,
(unsigned int)(port - execlists->port),
rq->global_seqno,
rq->fence.context, rq->fence.seqno,
intel_engine_get_seqno(rq->engine));
GEM_BUG_ON(!execlists->active);
intel_engine_context_out(rq->engine);
@ -741,7 +784,58 @@ execlists_cancel_port_requests(struct intel_engine_execlists * const execlists)
port++;
}
execlists_clear_active(execlists, EXECLISTS_ACTIVE_USER);
execlists_user_end(execlists);
}
static void clear_gtiir(struct intel_engine_cs *engine)
{
static const u8 gtiir[] = {
[RCS] = 0,
[BCS] = 0,
[VCS] = 1,
[VCS2] = 1,
[VECS] = 3,
};
struct drm_i915_private *dev_priv = engine->i915;
int i;
/* TODO: correctly reset irqs for gen11 */
if (WARN_ON_ONCE(INTEL_GEN(engine->i915) >= 11))
return;
GEM_BUG_ON(engine->id >= ARRAY_SIZE(gtiir));
/*
* Clear any pending interrupt state.
*
* We do it twice out of paranoia that some of the IIR are
* double buffered, and so if we only reset it once there may
* still be an interrupt pending.
*/
for (i = 0; i < 2; i++) {
I915_WRITE(GEN8_GT_IIR(gtiir[engine->id]),
engine->irq_keep_mask);
POSTING_READ(GEN8_GT_IIR(gtiir[engine->id]));
}
GEM_BUG_ON(I915_READ(GEN8_GT_IIR(gtiir[engine->id])) &
engine->irq_keep_mask);
}
static void reset_irq(struct intel_engine_cs *engine)
{
/* Mark all CS interrupts as complete */
smp_store_mb(engine->execlists.active, 0);
synchronize_hardirq(engine->i915->drm.irq);
clear_gtiir(engine);
/*
* The port is checked prior to scheduling a tasklet, but
* just in case we have suspended the tasklet to do the
* wedging make sure that when it wakes, it decides there
* is no work to do by clearing the irq_posted bit.
*/
clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
}
static void execlists_cancel_requests(struct intel_engine_cs *engine)
@ -751,7 +845,8 @@ static void execlists_cancel_requests(struct intel_engine_cs *engine)
struct rb_node *rb;
unsigned long flags;
GEM_TRACE("%s\n", engine->name);
GEM_TRACE("%s current %d\n",
engine->name, intel_engine_get_seqno(engine));
/*
* Before we call engine->cancel_requests(), we should have exclusive
@ -771,6 +866,7 @@ static void execlists_cancel_requests(struct intel_engine_cs *engine)
/* Cancel the requests on the HW and clear the ELSP tracker. */
execlists_cancel_port_requests(execlists);
reset_irq(engine);
spin_lock(&engine->timeline->lock);
@ -809,17 +905,6 @@ static void execlists_cancel_requests(struct intel_engine_cs *engine)
spin_unlock(&engine->timeline->lock);
/*
* The port is checked prior to scheduling a tasklet, but
* just in case we have suspended the tasklet to do the
* wedging make sure that when it wakes, it decides there
* is no work to do by clearing the irq_posted bit.
*/
clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
/* Mark all CS interrupts as complete */
execlists->active = 0;
local_irq_restore(flags);
}
@ -831,7 +916,7 @@ static void execlists_submission_tasklet(unsigned long data)
{
struct intel_engine_cs * const engine = (struct intel_engine_cs *)data;
struct intel_engine_execlists * const execlists = &engine->execlists;
struct execlist_port * const port = execlists->port;
struct execlist_port *port = execlists->port;
struct drm_i915_private *dev_priv = engine->i915;
bool fw = false;
@ -958,10 +1043,13 @@ static void execlists_submission_tasklet(unsigned long data)
EXECLISTS_ACTIVE_USER));
rq = port_unpack(port, &count);
GEM_TRACE("%s out[0]: ctx=%d.%d, seqno=%x, prio=%d\n",
GEM_TRACE("%s out[0]: ctx=%d.%d, global=%d (fence %llx:%d) (current %d), prio=%d\n",
engine->name,
port->context_id, count,
rq ? rq->global_seqno : 0,
rq ? rq->fence.context : 0,
rq ? rq->fence.seqno : 0,
intel_engine_get_seqno(engine),
rq ? rq_prio(rq) : 0);
/* Check the context/desc id for this event matches */
@ -969,10 +1057,28 @@ static void execlists_submission_tasklet(unsigned long data)
GEM_BUG_ON(count == 0);
if (--count == 0) {
/*
* On the final event corresponding to the
* submission of this context, we expect either
* an element-switch event or a completion
* event (and on completion, the active-idle
* marker). No more preemptions, lite-restore
* or otherwise.
*/
GEM_BUG_ON(status & GEN8_CTX_STATUS_PREEMPTED);
GEM_BUG_ON(port_isset(&port[1]) &&
!(status & GEN8_CTX_STATUS_ELEMENT_SWITCH));
GEM_BUG_ON(!port_isset(&port[1]) &&
!(status & GEN8_CTX_STATUS_ACTIVE_IDLE));
/*
* We rely on the hardware being strongly
* ordered, that the breadcrumb write is
* coherent (visible from the CPU) before the
* user interrupt and CSB is processed.
*/
GEM_BUG_ON(!i915_request_completed(rq));
execlists_context_schedule_out(rq);
trace_i915_request_out(rq);
i915_request_put(rq);
@ -980,17 +1086,14 @@ static void execlists_submission_tasklet(unsigned long data)
GEM_TRACE("%s completed ctx=%d\n",
engine->name, port->context_id);
execlists_port_complete(execlists, port);
port = execlists_port_complete(execlists, port);
if (port_isset(port))
execlists_user_begin(execlists, port);
else
execlists_user_end(execlists);
} else {
port_set(port, port_pack(rq, count));
}
/* After the final element, the hw should be idle */
GEM_BUG_ON(port_count(port) == 0 &&
!(status & GEN8_CTX_STATUS_ACTIVE_IDLE));
if (port_count(port) == 0)
execlists_clear_active(execlists,
EXECLISTS_ACTIVE_USER);
}
if (head != execlists->csb_head) {
@ -1019,12 +1122,16 @@ static void queue_request(struct intel_engine_cs *engine,
list_add_tail(&pt->link, &lookup_priolist(engine, pt, prio)->requests);
}
static void __submit_queue(struct intel_engine_cs *engine, int prio)
{
engine->execlists.queue_priority = prio;
tasklet_hi_schedule(&engine->execlists.tasklet);
}
static void submit_queue(struct intel_engine_cs *engine, int prio)
{
if (prio > engine->execlists.queue_priority) {
engine->execlists.queue_priority = prio;
tasklet_hi_schedule(&engine->execlists.tasklet);
}
if (prio > engine->execlists.queue_priority)
__submit_queue(engine, prio);
}
static void execlists_submit_request(struct i915_request *request)
@ -1157,7 +1264,10 @@ static void execlists_schedule(struct i915_request *request, int prio)
__list_del_entry(&pt->link);
queue_request(engine, pt, prio);
}
submit_queue(engine, prio);
if (prio > engine->execlists.queue_priority &&
i915_sw_fence_done(&pt_to_request(pt)->submit))
__submit_queue(engine, prio);
}
spin_unlock_irq(&engine->timeline->lock);
@ -1224,6 +1334,7 @@ execlists_context_pin(struct intel_engine_cs *engine,
ce->lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
ce->lrc_reg_state[CTX_RING_BUFFER_START+1] =
i915_ggtt_offset(ce->ring->vma);
ce->lrc_reg_state[CTX_RING_HEAD+1] = ce->ring->head;
ce->state->obj->pin_global++;
i915_gem_context_get(ctx);
@ -1574,14 +1685,6 @@ static int intel_init_workaround_bb(struct intel_engine_cs *engine)
return ret;
}
static u8 gtiir[] = {
[RCS] = 0,
[BCS] = 0,
[VCS] = 1,
[VCS2] = 1,
[VECS] = 3,
};
static void enable_execlists(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
@ -1641,6 +1744,10 @@ static int gen8_init_render_ring(struct intel_engine_cs *engine)
if (ret)
return ret;
ret = intel_whitelist_workarounds_apply(engine);
if (ret)
return ret;
/* We need to disable the AsyncFlip performance optimisations in order
* to use MI_WAIT_FOR_EVENT within the CS. It should already be
* programmed to '1' on all products.
@ -1651,7 +1758,7 @@ static int gen8_init_render_ring(struct intel_engine_cs *engine)
I915_WRITE(INSTPM, _MASKED_BIT_ENABLE(INSTPM_FORCE_ORDERING));
return init_workarounds_ring(engine);
return 0;
}
static int gen9_init_render_ring(struct intel_engine_cs *engine)
@ -1662,32 +1769,11 @@ static int gen9_init_render_ring(struct intel_engine_cs *engine)
if (ret)
return ret;
return init_workarounds_ring(engine);
}
ret = intel_whitelist_workarounds_apply(engine);
if (ret)
return ret;
static void reset_irq(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int i;
GEM_BUG_ON(engine->id >= ARRAY_SIZE(gtiir));
/*
* Clear any pending interrupt state.
*
* We do it twice out of paranoia that some of the IIR are double
* buffered, and if we only reset it once there may still be
* an interrupt pending.
*/
for (i = 0; i < 2; i++) {
I915_WRITE(GEN8_GT_IIR(gtiir[engine->id]),
GT_CONTEXT_SWITCH_INTERRUPT << engine->irq_shift);
POSTING_READ(GEN8_GT_IIR(gtiir[engine->id]));
}
GEM_BUG_ON(I915_READ(GEN8_GT_IIR(gtiir[engine->id])) &
(GT_CONTEXT_SWITCH_INTERRUPT << engine->irq_shift));
clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
return 0;
}
static void reset_common_ring(struct intel_engine_cs *engine,
@ -1697,14 +1783,13 @@ static void reset_common_ring(struct intel_engine_cs *engine,
struct intel_context *ce;
unsigned long flags;
GEM_TRACE("%s seqno=%x\n",
engine->name, request ? request->global_seqno : 0);
GEM_TRACE("%s request global=%x, current=%d\n",
engine->name, request ? request->global_seqno : 0,
intel_engine_get_seqno(engine));
/* See execlists_cancel_requests() for the irq/spinlock split. */
local_irq_save(flags);
reset_irq(engine);
/*
* Catch up with any missed context-switch interrupts.
*
@ -1715,15 +1800,13 @@ static void reset_common_ring(struct intel_engine_cs *engine,
* requests were completed.
*/
execlists_cancel_port_requests(execlists);
reset_irq(engine);
/* Push back any incomplete requests for replay after the reset. */
spin_lock(&engine->timeline->lock);
__unwind_incomplete_requests(engine);
spin_unlock(&engine->timeline->lock);
/* Mark all CS interrupts as complete */
execlists->active = 0;
local_irq_restore(flags);
/*
@ -2015,7 +2098,7 @@ static int gen8_init_rcs_context(struct i915_request *rq)
{
int ret;
ret = intel_ring_workarounds_emit(rq);
ret = intel_ctx_workarounds_emit(rq);
if (ret)
return ret;
@ -2075,11 +2158,13 @@ static void execlists_set_default_submission(struct intel_engine_cs *engine)
engine->unpark = NULL;
engine->flags |= I915_ENGINE_SUPPORTS_STATS;
if (engine->i915->preempt_context)
engine->flags |= I915_ENGINE_HAS_PREEMPTION;
engine->i915->caps.scheduler =
I915_SCHEDULER_CAP_ENABLED |
I915_SCHEDULER_CAP_PRIORITY;
if (engine->i915->preempt_context)
if (intel_engine_has_preemption(engine))
engine->i915->caps.scheduler |= I915_SCHEDULER_CAP_PREEMPTION;
}
@ -2118,7 +2203,20 @@ logical_ring_default_vfuncs(struct intel_engine_cs *engine)
static inline void
logical_ring_default_irqs(struct intel_engine_cs *engine)
{
unsigned shift = engine->irq_shift;
unsigned int shift = 0;
if (INTEL_GEN(engine->i915) < 11) {
const u8 irq_shifts[] = {
[RCS] = GEN8_RCS_IRQ_SHIFT,
[BCS] = GEN8_BCS_IRQ_SHIFT,
[VCS] = GEN8_VCS1_IRQ_SHIFT,
[VCS2] = GEN8_VCS2_IRQ_SHIFT,
[VECS] = GEN8_VECS_IRQ_SHIFT,
};
shift = irq_shifts[engine->id];
}
engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT << shift;
engine->irq_keep_mask = GT_CONTEXT_SWITCH_INTERRUPT << shift;
}
@ -2551,3 +2649,7 @@ void intel_lr_context_resume(struct drm_i915_private *dev_priv)
}
}
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftests/intel_lrc.c"
#endif

1
drivers/gpu/drm/i915/intel_overlay.c
View File

@ -807,6 +807,7 @@ static int intel_overlay_do_put_image(struct intel_overlay *overlay,
ret = PTR_ERR(vma);
goto out_pin_section;
}
intel_fb_obj_flush(new_bo, ORIGIN_DIRTYFB);
ret = i915_vma_put_fence(vma);
if (ret)

53
drivers/gpu/drm/i915/intel_pipe_crc.c
View File

@ -569,7 +569,8 @@ unlock:
static int ivb_pipe_crc_ctl_reg(struct drm_i915_private *dev_priv,
enum pipe pipe,
enum intel_pipe_crc_source *source,
uint32_t *val)
uint32_t *val,
bool set_wa)
{
if (*source == INTEL_PIPE_CRC_SOURCE_AUTO)
*source = INTEL_PIPE_CRC_SOURCE_PF;
@ -582,7 +583,7 @@ static int ivb_pipe_crc_ctl_reg(struct drm_i915_private *dev_priv,
*val = PIPE_CRC_ENABLE | PIPE_CRC_SOURCE_SPRITE_IVB;
break;
case INTEL_PIPE_CRC_SOURCE_PF:
if ((IS_HASWELL(dev_priv) ||
if (set_wa && (IS_HASWELL(dev_priv) ||
IS_BROADWELL(dev_priv)) && pipe == PIPE_A)
hsw_pipe_A_crc_wa(dev_priv, true);
@ -600,7 +601,8 @@ static int ivb_pipe_crc_ctl_reg(struct drm_i915_private *dev_priv,
static int get_new_crc_ctl_reg(struct drm_i915_private *dev_priv,
enum pipe pipe,
enum intel_pipe_crc_source *source, u32 *val)
enum intel_pipe_crc_source *source, u32 *val,
bool set_wa)
{
if (IS_GEN2(dev_priv))
return i8xx_pipe_crc_ctl_reg(source, val);
@ -611,7 +613,7 @@ static int get_new_crc_ctl_reg(struct drm_i915_private *dev_priv,
else if (IS_GEN5(dev_priv) || IS_GEN6(dev_priv))
return ilk_pipe_crc_ctl_reg(source, val);
else
return ivb_pipe_crc_ctl_reg(dev_priv, pipe, source, val);
return ivb_pipe_crc_ctl_reg(dev_priv, pipe, source, val, set_wa);
}
static int pipe_crc_set_source(struct drm_i915_private *dev_priv,
@ -636,7 +638,7 @@ static int pipe_crc_set_source(struct drm_i915_private *dev_priv,
return -EIO;
}
ret = get_new_crc_ctl_reg(dev_priv, pipe, &source, &val);
ret = get_new_crc_ctl_reg(dev_priv, pipe, &source, &val, true);
if (ret != 0)
goto out;
@ -916,7 +918,7 @@ int intel_pipe_crc_create(struct drm_minor *minor)
int intel_crtc_set_crc_source(struct drm_crtc *crtc, const char *source_name,
size_t *values_cnt)
{
struct drm_i915_private *dev_priv = crtc->dev->dev_private;
struct drm_i915_private *dev_priv = to_i915(crtc->dev);
struct intel_pipe_crc *pipe_crc = &dev_priv->pipe_crc[crtc->index];
enum intel_display_power_domain power_domain;
enum intel_pipe_crc_source source;
@ -934,10 +936,11 @@ int intel_crtc_set_crc_source(struct drm_crtc *crtc, const char *source_name,
return -EIO;
}
ret = get_new_crc_ctl_reg(dev_priv, crtc->index, &source, &val);
ret = get_new_crc_ctl_reg(dev_priv, crtc->index, &source, &val, true);
if (ret != 0)
goto out;
pipe_crc->source = source;
I915_WRITE(PIPE_CRC_CTL(crtc->index), val);
POSTING_READ(PIPE_CRC_CTL(crtc->index));
@ -959,3 +962,39 @@ out:
return ret;
}
void intel_crtc_enable_pipe_crc(struct intel_crtc *intel_crtc)
{
struct drm_crtc *crtc = &intel_crtc->base;
struct drm_i915_private *dev_priv = to_i915(crtc->dev);
struct intel_pipe_crc *pipe_crc = &dev_priv->pipe_crc[crtc->index];
u32 val = 0;
if (!crtc->crc.opened)
return;
if (get_new_crc_ctl_reg(dev_priv, crtc->index, &pipe_crc->source, &val, false) < 0)
return;
/* Don't need pipe_crc->lock here, IRQs are not generated. */
pipe_crc->skipped = 0;
I915_WRITE(PIPE_CRC_CTL(crtc->index), val);
POSTING_READ(PIPE_CRC_CTL(crtc->index));
}
void intel_crtc_disable_pipe_crc(struct intel_crtc *intel_crtc)
{
struct drm_crtc *crtc = &intel_crtc->base;
struct drm_i915_private *dev_priv = to_i915(crtc->dev);
struct intel_pipe_crc *pipe_crc = &dev_priv->pipe_crc[crtc->index];
/* Swallow crc's until we stop generating them. */
spin_lock_irq(&pipe_crc->lock);
pipe_crc->skipped = INT_MIN;
spin_unlock_irq(&pipe_crc->lock);
I915_WRITE(PIPE_CRC_CTL(crtc->index), 0);
POSTING_READ(PIPE_CRC_CTL(crtc->index));
synchronize_irq(dev_priv->drm.irq);
}

496
drivers/gpu/drm/i915/intel_pm.c
View File

@ -3825,6 +3825,44 @@ static void skl_ddb_entry_init_from_hw(struct skl_ddb_entry *entry, u32 reg)
entry->end += 1;
}
static void
skl_ddb_get_hw_plane_state(struct drm_i915_private *dev_priv,
const enum pipe pipe,
const enum plane_id plane_id,
struct skl_ddb_allocation *ddb /* out */)
{
u32 val, val2 = 0;
int fourcc, pixel_format;
/* Cursor doesn't support NV12/planar, so no extra calculation needed */
if (plane_id == PLANE_CURSOR) {
val = I915_READ(CUR_BUF_CFG(pipe));
skl_ddb_entry_init_from_hw(&ddb->plane[pipe][plane_id], val);
return;
}
val = I915_READ(PLANE_CTL(pipe, plane_id));
/* No DDB allocated for disabled planes */
if (!(val & PLANE_CTL_ENABLE))
return;
pixel_format = val & PLANE_CTL_FORMAT_MASK;
fourcc = skl_format_to_fourcc(pixel_format,
val & PLANE_CTL_ORDER_RGBX,
val & PLANE_CTL_ALPHA_MASK);
val = I915_READ(PLANE_BUF_CFG(pipe, plane_id));
val2 = I915_READ(PLANE_NV12_BUF_CFG(pipe, plane_id));
if (fourcc == DRM_FORMAT_NV12) {
skl_ddb_entry_init_from_hw(&ddb->plane[pipe][plane_id], val2);
skl_ddb_entry_init_from_hw(&ddb->uv_plane[pipe][plane_id], val);
} else {
skl_ddb_entry_init_from_hw(&ddb->plane[pipe][plane_id], val);
}
}
void skl_ddb_get_hw_state(struct drm_i915_private *dev_priv,
struct skl_ddb_allocation *ddb /* out */)
{
@ -3841,16 +3879,9 @@ void skl_ddb_get_hw_state(struct drm_i915_private *dev_priv,
if (!intel_display_power_get_if_enabled(dev_priv, power_domain))
continue;
for_each_plane_id_on_crtc(crtc, plane_id) {
u32 val;
if (plane_id != PLANE_CURSOR)
val = I915_READ(PLANE_BUF_CFG(pipe, plane_id));
else
val = I915_READ(CUR_BUF_CFG(pipe));
skl_ddb_entry_init_from_hw(&ddb->plane[pipe][plane_id], val);
}
for_each_plane_id_on_crtc(crtc, plane_id)
skl_ddb_get_hw_plane_state(dev_priv, pipe,
plane_id, ddb);
intel_display_power_put(dev_priv, power_domain);
}
@ -4009,9 +4040,9 @@ int skl_check_pipe_max_pixel_rate(struct intel_crtc *intel_crtc,
static unsigned int
skl_plane_relative_data_rate(const struct intel_crtc_state *cstate,
const struct drm_plane_state *pstate,
int y)
const int plane)
{
struct intel_plane *plane = to_intel_plane(pstate->plane);
struct intel_plane *intel_plane = to_intel_plane(pstate->plane);
struct intel_plane_state *intel_pstate = to_intel_plane_state(pstate);
uint32_t data_rate;
uint32_t width = 0, height = 0;
@ -4025,9 +4056,9 @@ skl_plane_relative_data_rate(const struct intel_crtc_state *cstate,
fb = pstate->fb;
format = fb->format->format;
if (plane->id == PLANE_CURSOR)
if (intel_plane->id == PLANE_CURSOR)
return 0;
if (y && format != DRM_FORMAT_NV12)
if (plane == 1 && format != DRM_FORMAT_NV12)
return 0;
/*
@ -4038,19 +4069,14 @@ skl_plane_relative_data_rate(const struct intel_crtc_state *cstate,
width = drm_rect_width(&intel_pstate->base.src) >> 16;
height = drm_rect_height(&intel_pstate->base.src) >> 16;
/* for planar format */
if (format == DRM_FORMAT_NV12) {
if (y) /* y-plane data rate */
data_rate = width * height *
fb->format->cpp[0];
else /* uv-plane data rate */
data_rate = (width / 2) * (height / 2) *
fb->format->cpp[1];
} else {
/* for packed formats */
data_rate = width * height * fb->format->cpp[0];
/* UV plane does 1/2 pixel sub-sampling */
if (plane == 1 && format == DRM_FORMAT_NV12) {
width /= 2;
height /= 2;
}
data_rate = width * height * fb->format->cpp[plane];
down_scale_amount = skl_plane_downscale_amount(cstate, intel_pstate);
return mul_round_up_u32_fixed16(data_rate, down_scale_amount);
@ -4063,8 +4089,8 @@ skl_plane_relative_data_rate(const struct intel_crtc_state *cstate,
*/
static unsigned int
skl_get_total_relative_data_rate(struct intel_crtc_state *intel_cstate,
unsigned *plane_data_rate,
unsigned *plane_y_data_rate)
unsigned int *plane_data_rate,
unsigned int *uv_plane_data_rate)
{
struct drm_crtc_state *cstate = &intel_cstate->base;
struct drm_atomic_state *state = cstate->state;
@ -4080,17 +4106,17 @@ skl_get_total_relative_data_rate(struct intel_crtc_state *intel_cstate,
enum plane_id plane_id = to_intel_plane(plane)->id;
unsigned int rate;
/* packed/uv */
/* packed/y */
rate = skl_plane_relative_data_rate(intel_cstate,
pstate, 0);
plane_data_rate[plane_id] = rate;
total_data_rate += rate;
/* y-plane */
/* uv-plane */
rate = skl_plane_relative_data_rate(intel_cstate,
pstate, 1);
plane_y_data_rate[plane_id] = rate;
uv_plane_data_rate[plane_id] = rate;
total_data_rate += rate;
}
@ -4099,8 +4125,7 @@ skl_get_total_relative_data_rate(struct intel_crtc_state *intel_cstate,
}
static uint16_t
skl_ddb_min_alloc(const struct drm_plane_state *pstate,
const int y)
skl_ddb_min_alloc(const struct drm_plane_state *pstate, const int plane)
{
struct drm_framebuffer *fb = pstate->fb;
struct intel_plane_state *intel_pstate = to_intel_plane_state(pstate);
@ -4111,8 +4136,8 @@ skl_ddb_min_alloc(const struct drm_plane_state *pstate,
if (WARN_ON(!fb))
return 0;
/* For packed formats, no y-plane, return 0 */
if (y && fb->format->format != DRM_FORMAT_NV12)
/* For packed formats, and uv-plane, return 0 */
if (plane == 1 && fb->format->format != DRM_FORMAT_NV12)
return 0;
/* For Non Y-tile return 8-blocks */
@ -4131,15 +4156,12 @@ skl_ddb_min_alloc(const struct drm_plane_state *pstate,
src_h = drm_rect_height(&intel_pstate->base.src) >> 16;
/* Halve UV plane width and height for NV12 */
if (fb->format->format == DRM_FORMAT_NV12 && !y) {
if (plane == 1) {
src_w /= 2;
src_h /= 2;
}
if (fb->format->format == DRM_FORMAT_NV12 && !y)
plane_bpp = fb->format->cpp[1];
else
plane_bpp = fb->format->cpp[0];
plane_bpp = fb->format->cpp[plane];
if (drm_rotation_90_or_270(pstate->rotation)) {
switch (plane_bpp) {
@ -4167,7 +4189,7 @@ skl_ddb_min_alloc(const struct drm_plane_state *pstate,
static void
skl_ddb_calc_min(const struct intel_crtc_state *cstate, int num_active,
uint16_t *minimum, uint16_t *y_minimum)
uint16_t *minimum, uint16_t *uv_minimum)
{
const struct drm_plane_state *pstate;
struct drm_plane *plane;
@ -4182,7 +4204,7 @@ skl_ddb_calc_min(const struct intel_crtc_state *cstate, int num_active,
continue;
minimum[plane_id] = skl_ddb_min_alloc(pstate, 0);
y_minimum[plane_id] = skl_ddb_min_alloc(pstate, 1);
uv_minimum[plane_id] = skl_ddb_min_alloc(pstate, 1);
}
minimum[PLANE_CURSOR] = skl_cursor_allocation(num_active);
@ -4200,17 +4222,17 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *cstate,
struct skl_ddb_entry *alloc = &cstate->wm.skl.ddb;
uint16_t alloc_size, start;
uint16_t minimum[I915_MAX_PLANES] = {};
uint16_t y_minimum[I915_MAX_PLANES] = {};
uint16_t uv_minimum[I915_MAX_PLANES] = {};
unsigned int total_data_rate;
enum plane_id plane_id;
int num_active;
unsigned plane_data_rate[I915_MAX_PLANES] = {};
unsigned plane_y_data_rate[I915_MAX_PLANES] = {};
unsigned int plane_data_rate[I915_MAX_PLANES] = {};
unsigned int uv_plane_data_rate[I915_MAX_PLANES] = {};
uint16_t total_min_blocks = 0;
/* Clear the partitioning for disabled planes. */
memset(ddb->plane[pipe], 0, sizeof(ddb->plane[pipe]));
memset(ddb->y_plane[pipe], 0, sizeof(ddb->y_plane[pipe]));
memset(ddb->uv_plane[pipe], 0, sizeof(ddb->uv_plane[pipe]));
if (WARN_ON(!state))
return 0;
@ -4225,7 +4247,7 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *cstate,
if (alloc_size == 0)
return 0;
skl_ddb_calc_min(cstate, num_active, minimum, y_minimum);
skl_ddb_calc_min(cstate, num_active, minimum, uv_minimum);
/*
* 1. Allocate the mininum required blocks for each active plane
@ -4235,7 +4257,7 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *cstate,
for_each_plane_id_on_crtc(intel_crtc, plane_id) {
total_min_blocks += minimum[plane_id];
total_min_blocks += y_minimum[plane_id];
total_min_blocks += uv_minimum[plane_id];
}
if (total_min_blocks > alloc_size) {
@ -4257,14 +4279,14 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *cstate,
*/
total_data_rate = skl_get_total_relative_data_rate(cstate,
plane_data_rate,
plane_y_data_rate);
uv_plane_data_rate);
if (total_data_rate == 0)
return 0;
start = alloc->start;
for_each_plane_id_on_crtc(intel_crtc, plane_id) {
unsigned int data_rate, y_data_rate;
uint16_t plane_blocks, y_plane_blocks = 0;
unsigned int data_rate, uv_data_rate;
uint16_t plane_blocks, uv_plane_blocks;
if (plane_id == PLANE_CURSOR)
continue;
@ -4288,21 +4310,20 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *cstate,
start += plane_blocks;
/*
* allocation for y_plane part of planar format:
*/
y_data_rate = plane_y_data_rate[plane_id];
/* Allocate DDB for UV plane for planar format/NV12 */
uv_data_rate = uv_plane_data_rate[plane_id];
y_plane_blocks = y_minimum[plane_id];
y_plane_blocks += div_u64((uint64_t)alloc_size * y_data_rate,
total_data_rate);
uv_plane_blocks = uv_minimum[plane_id];
uv_plane_blocks += div_u64((uint64_t)alloc_size * uv_data_rate,
total_data_rate);
if (y_data_rate) {
ddb->y_plane[pipe][plane_id].start = start;
ddb->y_plane[pipe][plane_id].end = start + y_plane_blocks;
if (uv_data_rate) {
ddb->uv_plane[pipe][plane_id].start = start;
ddb->uv_plane[pipe][plane_id].end =
start + uv_plane_blocks;
}
start += y_plane_blocks;
start += uv_plane_blocks;
}
return 0;
@ -4398,7 +4419,7 @@ static int
skl_compute_plane_wm_params(const struct drm_i915_private *dev_priv,
struct intel_crtc_state *cstate,
const struct intel_plane_state *intel_pstate,
struct skl_wm_params *wp)
struct skl_wm_params *wp, int plane_id)
{
struct intel_plane *plane = to_intel_plane(intel_pstate->base.plane);
const struct drm_plane_state *pstate = &intel_pstate->base;
@ -4411,6 +4432,12 @@ skl_compute_plane_wm_params(const struct drm_i915_private *dev_priv,
if (!intel_wm_plane_visible(cstate, intel_pstate))
return 0;
/* only NV12 format has two planes */
if (plane_id == 1 && fb->format->format != DRM_FORMAT_NV12) {
DRM_DEBUG_KMS("Non NV12 format have single plane\n");
return -EINVAL;
}
wp->y_tiled = fb->modifier == I915_FORMAT_MOD_Y_TILED ||
fb->modifier == I915_FORMAT_MOD_Yf_TILED ||
fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
@ -4418,6 +4445,7 @@ skl_compute_plane_wm_params(const struct drm_i915_private *dev_priv,
wp->x_tiled = fb->modifier == I915_FORMAT_MOD_X_TILED;
wp->rc_surface = fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS;
wp->is_planar = fb->format->format == DRM_FORMAT_NV12;
if (plane->id == PLANE_CURSOR) {
wp->width = intel_pstate->base.crtc_w;
@ -4430,8 +4458,10 @@ skl_compute_plane_wm_params(const struct drm_i915_private *dev_priv,
wp->width = drm_rect_width(&intel_pstate->base.src) >> 16;
}
wp->cpp = (fb->format->format == DRM_FORMAT_NV12) ? fb->format->cpp[1] :
fb->format->cpp[0];
if (plane_id == 1 && wp->is_planar)
wp->width /= 2;
wp->cpp = fb->format->cpp[plane_id];
wp->plane_pixel_rate = skl_adjusted_plane_pixel_rate(cstate,
intel_pstate);
@ -4499,9 +4529,8 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
uint16_t ddb_allocation,
int level,
const struct skl_wm_params *wp,
uint16_t *out_blocks, /* out */
uint8_t *out_lines, /* out */
bool *enabled /* out */)
const struct skl_wm_level *result_prev,
struct skl_wm_level *result /* out */)
{
const struct drm_plane_state *pstate = &intel_pstate->base;
uint32_t latency = dev_priv->wm.skl_latency[level];
@ -4515,7 +4544,7 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
if (latency == 0 ||
!intel_wm_plane_visible(cstate, intel_pstate)) {
*enabled = false;
result->plane_en = false;
return 0;
}
@ -4568,6 +4597,15 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
} else {
res_blocks++;
}
/*
* Make sure result blocks for higher latency levels are atleast
* as high as level below the current level.
* Assumption in DDB algorithm optimization for special cases.
* Also covers Display WA #1125 for RC.
*/
if (result_prev->plane_res_b > res_blocks)
res_blocks = result_prev->plane_res_b;
}
if (INTEL_GEN(dev_priv) >= 11) {
@ -4596,7 +4634,7 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
if ((level > 0 && res_lines > 31) ||
res_blocks >= ddb_allocation ||
min_disp_buf_needed >= ddb_allocation) {
*enabled = false;
result->plane_en = false;
/*
* If there are no valid level 0 watermarks, then we can't
@ -4615,10 +4653,21 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
}
}
/*
* Display WA #826 (SKL:ALL, BXT:ALL) & #1059 (CNL:A)
* disable wm level 1-7 on NV12 planes
*/
if (wp->is_planar && level >= 1 &&
(IS_SKYLAKE(dev_priv) || IS_BROXTON(dev_priv) ||
IS_CNL_REVID(dev_priv, CNL_REVID_A0, CNL_REVID_A0))) {
result->plane_en = false;
return 0;
}
/* The number of lines are ignored for the level 0 watermark. */
*out_lines = level ? res_lines : 0;
*out_blocks = res_blocks;
*enabled = true;
result->plane_res_b = res_blocks;
result->plane_res_l = res_lines;
result->plane_en = true;
return 0;
}
@ -4629,7 +4678,8 @@ skl_compute_wm_levels(const struct drm_i915_private *dev_priv,
struct intel_crtc_state *cstate,
const struct intel_plane_state *intel_pstate,
const struct skl_wm_params *wm_params,
struct skl_plane_wm *wm)
struct skl_plane_wm *wm,
int plane_id)
{
struct intel_crtc *intel_crtc = to_intel_crtc(cstate->base.crtc);
struct drm_plane *plane = intel_pstate->base.plane;
@ -4637,15 +4687,26 @@ skl_compute_wm_levels(const struct drm_i915_private *dev_priv,
uint16_t ddb_blocks;
enum pipe pipe = intel_crtc->pipe;
int level, max_level = ilk_wm_max_level(dev_priv);
enum plane_id intel_plane_id = intel_plane->id;
int ret;
if (WARN_ON(!intel_pstate->base.fb))
return -EINVAL;
ddb_blocks = skl_ddb_entry_size(&ddb->plane[pipe][intel_plane->id]);
ddb_blocks = plane_id ?
skl_ddb_entry_size(&ddb->uv_plane[pipe][intel_plane_id]) :
skl_ddb_entry_size(&ddb->plane[pipe][intel_plane_id]);
for (level = 0; level <= max_level; level++) {
struct skl_wm_level *result = &wm->wm[level];
struct skl_wm_level *result = plane_id ? &wm->uv_wm[level] :
&wm->wm[level];
struct skl_wm_level *result_prev;
if (level)
result_prev = plane_id ? &wm->uv_wm[level - 1] :
&wm->wm[level - 1];
else
result_prev = plane_id ? &wm->uv_wm[0] : &wm->wm[0];
ret = skl_compute_plane_wm(dev_priv,
cstate,
@ -4653,13 +4714,15 @@ skl_compute_wm_levels(const struct drm_i915_private *dev_priv,
ddb_blocks,
level,
wm_params,
&result->plane_res_b,
&result->plane_res_l,
&result->plane_en);
result_prev,
result);
if (ret)
return ret;
}
if (intel_pstate->base.fb->format->format == DRM_FORMAT_NV12)
wm->is_planar = true;
return 0;
}
@ -4769,20 +4832,39 @@ static int skl_build_pipe_wm(struct intel_crtc_state *cstate,
wm = &pipe_wm->planes[plane_id];
ddb_blocks = skl_ddb_entry_size(&ddb->plane[pipe][plane_id]);
memset(&wm_params, 0, sizeof(struct skl_wm_params));
ret = skl_compute_plane_wm_params(dev_priv, cstate,
intel_pstate, &wm_params);
intel_pstate, &wm_params, 0);
if (ret)
return ret;
ret = skl_compute_wm_levels(dev_priv, ddb, cstate,
intel_pstate, &wm_params, wm);
intel_pstate, &wm_params, wm, 0);
if (ret)
return ret;
skl_compute_transition_wm(cstate, &wm_params, &wm->wm[0],
ddb_blocks, &wm->trans_wm);
/* uv plane watermarks must also be validated for NV12/Planar */
if (wm_params.is_planar) {
memset(&wm_params, 0, sizeof(struct skl_wm_params));
wm->is_planar = true;
ret = skl_compute_plane_wm_params(dev_priv, cstate,
intel_pstate,
&wm_params, 1);
if (ret)
return ret;
ret = skl_compute_wm_levels(dev_priv, ddb, cstate,
intel_pstate, &wm_params,
wm, 1);
if (ret)
return ret;
}
}
pipe_wm->linetime = skl_compute_linetime_wm(cstate);
return 0;
@ -4833,10 +4915,21 @@ static void skl_write_plane_wm(struct intel_crtc *intel_crtc,
skl_ddb_entry_write(dev_priv, PLANE_BUF_CFG(pipe, plane_id),
&ddb->plane[pipe][plane_id]);
if (INTEL_GEN(dev_priv) < 11)
if (INTEL_GEN(dev_priv) >= 11)
return skl_ddb_entry_write(dev_priv,
PLANE_BUF_CFG(pipe, plane_id),
&ddb->plane[pipe][plane_id]);
if (wm->is_planar) {
skl_ddb_entry_write(dev_priv, PLANE_BUF_CFG(pipe, plane_id),
&ddb->uv_plane[pipe][plane_id]);
skl_ddb_entry_write(dev_priv,
PLANE_NV12_BUF_CFG(pipe, plane_id),
&ddb->y_plane[pipe][plane_id]);
&ddb->plane[pipe][plane_id]);
} else {
skl_ddb_entry_write(dev_priv, PLANE_BUF_CFG(pipe, plane_id),
&ddb->plane[pipe][plane_id]);
I915_WRITE(PLANE_NV12_BUF_CFG(pipe, plane_id), 0x0);
}
}
static void skl_write_cursor_wm(struct intel_crtc *intel_crtc,
@ -4944,15 +5037,13 @@ skl_ddb_add_affected_planes(struct intel_crtc_state *cstate)
struct drm_plane *plane;
enum pipe pipe = intel_crtc->pipe;
WARN_ON(!drm_atomic_get_existing_crtc_state(state, crtc));
drm_for_each_plane_mask(plane, dev, cstate->base.plane_mask) {
enum plane_id plane_id = to_intel_plane(plane)->id;
if (skl_ddb_entry_equal(&cur_ddb->plane[pipe][plane_id],
&new_ddb->plane[pipe][plane_id]) &&
skl_ddb_entry_equal(&cur_ddb->y_plane[pipe][plane_id],
&new_ddb->y_plane[pipe][plane_id]))
skl_ddb_entry_equal(&cur_ddb->uv_plane[pipe][plane_id],
&new_ddb->uv_plane[pipe][plane_id]))
continue;
plane_state = drm_atomic_get_plane_state(state, plane);
@ -4966,13 +5057,108 @@ skl_ddb_add_affected_planes(struct intel_crtc_state *cstate)
static int
skl_compute_ddb(struct drm_atomic_state *state)
{
struct drm_device *dev = state->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
const struct drm_i915_private *dev_priv = to_i915(state->dev);
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
struct intel_crtc *intel_crtc;
struct skl_ddb_allocation *ddb = &intel_state->wm_results.ddb;
struct intel_crtc *crtc;
struct intel_crtc_state *cstate;
int ret, i;
memcpy(ddb, &dev_priv->wm.skl_hw.ddb, sizeof(*ddb));
for_each_new_intel_crtc_in_state(intel_state, crtc, cstate, i) {
ret = skl_allocate_pipe_ddb(cstate, ddb);
if (ret)
return ret;
ret = skl_ddb_add_affected_planes(cstate);
if (ret)
return ret;
}
return 0;
}
static void
skl_copy_ddb_for_pipe(struct skl_ddb_values *dst,
struct skl_ddb_values *src,
enum pipe pipe)
{
memcpy(dst->ddb.uv_plane[pipe], src->ddb.uv_plane[pipe],
sizeof(dst->ddb.uv_plane[pipe]));
memcpy(dst->ddb.plane[pipe], src->ddb.plane[pipe],
sizeof(dst->ddb.plane[pipe]));
}
static void
skl_print_wm_changes(const struct drm_atomic_state *state)
{
const struct drm_device *dev = state->dev;
const struct drm_i915_private *dev_priv = to_i915(dev);
const struct intel_atomic_state *intel_state =
to_intel_atomic_state(state);
const struct drm_crtc *crtc;
const struct drm_crtc_state *cstate;
const struct intel_plane *intel_plane;
const struct skl_ddb_allocation *old_ddb = &dev_priv->wm.skl_hw.ddb;
const struct skl_ddb_allocation *new_ddb = &intel_state->wm_results.ddb;
int i;
for_each_new_crtc_in_state(state, crtc, cstate, i) {
const struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
enum pipe pipe = intel_crtc->pipe;
for_each_intel_plane_on_crtc(dev, intel_crtc, intel_plane) {
enum plane_id plane_id = intel_plane->id;
const struct skl_ddb_entry *old, *new;
old = &old_ddb->plane[pipe][plane_id];
new = &new_ddb->plane[pipe][plane_id];
if (skl_ddb_entry_equal(old, new))
continue;
DRM_DEBUG_ATOMIC("[PLANE:%d:%s] ddb (%d - %d) -> (%d - %d)\n",
intel_plane->base.base.id,
intel_plane->base.name,
old->start, old->end,
new->start, new->end);
}
}
}
static int
skl_ddb_add_affected_pipes(struct drm_atomic_state *state, bool *changed)
{
struct drm_device *dev = state->dev;
const struct drm_i915_private *dev_priv = to_i915(dev);
const struct drm_crtc *crtc;
const struct drm_crtc_state *cstate;
struct intel_crtc *intel_crtc;
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
uint32_t realloc_pipes = pipes_modified(state);
int ret;
int ret, i;
/*
* When we distrust bios wm we always need to recompute to set the
* expected DDB allocations for each CRTC.
*/
if (dev_priv->wm.distrust_bios_wm)
(*changed) = true;
/*
* If this transaction isn't actually touching any CRTC's, don't
* bother with watermark calculation. Note that if we pass this
* test, we're guaranteed to hold at least one CRTC state mutex,
* which means we can safely use values like dev_priv->active_crtcs
* since any racing commits that want to update them would need to
* hold _all_ CRTC state mutexes.
*/
for_each_new_crtc_in_state(state, crtc, cstate, i)
(*changed) = true;
if (!*changed)
return 0;
/*
* If this is our first atomic update following hardware readout,
@ -5020,111 +5206,35 @@ skl_compute_ddb(struct drm_atomic_state *state)
* We're not recomputing for the pipes not included in the commit, so
* make sure we start with the current state.
*/
memcpy(ddb, &dev_priv->wm.skl_hw.ddb, sizeof(*ddb));
for_each_intel_crtc_mask(dev, intel_crtc, realloc_pipes) {
struct intel_crtc_state *cstate;
cstate = intel_atomic_get_crtc_state(state, intel_crtc);
if (IS_ERR(cstate))
return PTR_ERR(cstate);
ret = skl_allocate_pipe_ddb(cstate, ddb);
if (ret)
return ret;
ret = skl_ddb_add_affected_planes(cstate);
if (ret)
return ret;
}
return 0;
}
static void
skl_copy_wm_for_pipe(struct skl_wm_values *dst,
struct skl_wm_values *src,
enum pipe pipe)
{
memcpy(dst->ddb.y_plane[pipe], src->ddb.y_plane[pipe],
sizeof(dst->ddb.y_plane[pipe]));
memcpy(dst->ddb.plane[pipe], src->ddb.plane[pipe],
sizeof(dst->ddb.plane[pipe]));
}
static void
skl_print_wm_changes(const struct drm_atomic_state *state)
{
const struct drm_device *dev = state->dev;
const struct drm_i915_private *dev_priv = to_i915(dev);
const struct intel_atomic_state *intel_state =
to_intel_atomic_state(state);
const struct drm_crtc *crtc;
const struct drm_crtc_state *cstate;
const struct intel_plane *intel_plane;
const struct skl_ddb_allocation *old_ddb = &dev_priv->wm.skl_hw.ddb;
const struct skl_ddb_allocation *new_ddb = &intel_state->wm_results.ddb;
int i;
for_each_new_crtc_in_state(state, crtc, cstate, i) {
const struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
enum pipe pipe = intel_crtc->pipe;
for_each_intel_plane_on_crtc(dev, intel_crtc, intel_plane) {
enum plane_id plane_id = intel_plane->id;
const struct skl_ddb_entry *old, *new;
old = &old_ddb->plane[pipe][plane_id];
new = &new_ddb->plane[pipe][plane_id];
if (skl_ddb_entry_equal(old, new))
continue;
DRM_DEBUG_ATOMIC("[PLANE:%d:%s] ddb (%d - %d) -> (%d - %d)\n",
intel_plane->base.base.id,
intel_plane->base.name,
old->start, old->end,
new->start, new->end);
}
}
}
static int
skl_compute_wm(struct drm_atomic_state *state)
{
struct drm_crtc *crtc;
struct drm_crtc_state *cstate;
struct intel_atomic_state *intel_state = to_intel_atomic_state(state);
struct skl_wm_values *results = &intel_state->wm_results;
struct drm_device *dev = state->dev;
struct skl_ddb_values *results = &intel_state->wm_results;
struct skl_pipe_wm *pipe_wm;
bool changed = false;
int ret, i;
/*
* When we distrust bios wm we always need to recompute to set the
* expected DDB allocations for each CRTC.
*/
if (to_i915(dev)->wm.distrust_bios_wm)
changed = true;
/*
* If this transaction isn't actually touching any CRTC's, don't
* bother with watermark calculation. Note that if we pass this
* test, we're guaranteed to hold at least one CRTC state mutex,
* which means we can safely use values like dev_priv->active_crtcs
* since any racing commits that want to update them would need to
* hold _all_ CRTC state mutexes.
*/
for_each_new_crtc_in_state(state, crtc, cstate, i)
changed = true;
if (!changed)
return 0;
/* Clear all dirty flags */
results->dirty_pipes = 0;
ret = skl_ddb_add_affected_pipes(state, &changed);
if (ret || !changed)
return ret;
ret = skl_compute_ddb(state);
if (ret)
return ret;
@ -5197,8 +5307,8 @@ static void skl_initial_wm(struct intel_atomic_state *state,
struct intel_crtc *intel_crtc = to_intel_crtc(cstate->base.crtc);
struct drm_device *dev = intel_crtc->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct skl_wm_values *results = &state->wm_results;
struct skl_wm_values *hw_vals = &dev_priv->wm.skl_hw;
struct skl_ddb_values *results = &state->wm_results;
struct skl_ddb_values *hw_vals = &dev_priv->wm.skl_hw;
enum pipe pipe = intel_crtc->pipe;
if ((results->dirty_pipes & drm_crtc_mask(&intel_crtc->base)) == 0)
@ -5209,7 +5319,7 @@ static void skl_initial_wm(struct intel_atomic_state *state,
if (cstate->base.active_changed)
skl_atomic_update_crtc_wm(state, cstate);
skl_copy_wm_for_pipe(hw_vals, results, pipe);
skl_copy_ddb_for_pipe(hw_vals, results, pipe);
mutex_unlock(&dev_priv->wm.wm_mutex);
}
@ -5341,7 +5451,7 @@ void skl_pipe_wm_get_hw_state(struct drm_crtc *crtc,
void skl_wm_get_hw_state(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct skl_wm_values *hw = &dev_priv->wm.skl_hw;
struct skl_ddb_values *hw = &dev_priv->wm.skl_hw;
struct skl_ddb_allocation *ddb = &dev_priv->wm.skl_hw.ddb;
struct drm_crtc *crtc;
struct intel_crtc *intel_crtc;
@ -6572,7 +6682,7 @@ static void gen6_init_rps_frequencies(struct drm_i915_private *dev_priv)
rps->efficient_freq = rps->rp1_freq;
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv) ||
IS_GEN9_BC(dev_priv) || IS_CANNONLAKE(dev_priv)) {
IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) {
u32 ddcc_status = 0;
if (sandybridge_pcode_read(dev_priv,
@ -6585,7 +6695,7 @@ static void gen6_init_rps_frequencies(struct drm_i915_private *dev_priv)
rps->max_freq);
}
if (IS_GEN9_BC(dev_priv) || IS_CANNONLAKE(dev_priv)) {
if (IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) {
/* Store the frequency values in 16.66 MHZ units, which is
* the natural hardware unit for SKL
*/
@ -6890,15 +7000,18 @@ static void gen6_enable_rps(struct drm_i915_private *dev_priv)
static void gen6_update_ring_freq(struct drm_i915_private *dev_priv)
{
struct intel_rps *rps = &dev_priv->gt_pm.rps;
int min_freq = 15;
const int min_freq = 15;
const int scaling_factor = 180;
unsigned int gpu_freq;
unsigned int max_ia_freq, min_ring_freq;
unsigned int max_gpu_freq, min_gpu_freq;
int scaling_factor = 180;
struct cpufreq_policy *policy;
WARN_ON(!mutex_is_locked(&dev_priv->pcu_lock));
if (rps->max_freq <= rps->min_freq)
return;
policy = cpufreq_cpu_get(0);
if (policy) {
max_ia_freq = policy->cpuinfo.max_freq;
@ -6918,13 +7031,12 @@ static void gen6_update_ring_freq(struct drm_i915_private *dev_priv)
/* convert DDR frequency from units of 266.6MHz to bandwidth */
min_ring_freq = mult_frac(min_ring_freq, 8, 3);
if (IS_GEN9_BC(dev_priv) || IS_CANNONLAKE(dev_priv)) {
min_gpu_freq = rps->min_freq;
max_gpu_freq = rps->max_freq;
if (IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) {
/* Convert GT frequency to 50 HZ units */
min_gpu_freq = rps->min_freq / GEN9_FREQ_SCALER;
max_gpu_freq = rps->max_freq / GEN9_FREQ_SCALER;
} else {
min_gpu_freq = rps->min_freq;
max_gpu_freq = rps->max_freq;
min_gpu_freq /= GEN9_FREQ_SCALER;
max_gpu_freq /= GEN9_FREQ_SCALER;
}
/*
@ -6933,10 +7045,10 @@ static void gen6_update_ring_freq(struct drm_i915_private *dev_priv)
* the PCU should use as a reference to determine the ring frequency.
*/
for (gpu_freq = max_gpu_freq; gpu_freq >= min_gpu_freq; gpu_freq--) {
int diff = max_gpu_freq - gpu_freq;
const int diff = max_gpu_freq - gpu_freq;
unsigned int ia_freq = 0, ring_freq = 0;
if (IS_GEN9_BC(dev_priv) || IS_CANNONLAKE(dev_priv)) {
if (IS_GEN9_BC(dev_priv) || INTEL_GEN(dev_priv) >= 10) {
/*
* ring_freq = 2 * GT. ring_freq is in 100MHz units
* No floor required for ring frequency on SKL.
@ -8026,10 +8138,10 @@ void intel_sanitize_gt_powersave(struct drm_i915_private *dev_priv)
dev_priv->gt_pm.rc6.enabled = true; /* force RC6 disabling */
intel_disable_gt_powersave(dev_priv);
if (INTEL_GEN(dev_priv) < 11)
gen6_reset_rps_interrupts(dev_priv);
if (INTEL_GEN(dev_priv) >= 11)
gen11_reset_rps_interrupts(dev_priv);
else
WARN_ON_ONCE(1);
gen6_reset_rps_interrupts(dev_priv);
}
static inline void intel_disable_llc_pstate(struct drm_i915_private *i915)
@ -8142,8 +8254,6 @@ static void intel_enable_rps(struct drm_i915_private *dev_priv)
cherryview_enable_rps(dev_priv);
} else if (IS_VALLEYVIEW(dev_priv)) {
valleyview_enable_rps(dev_priv);
} else if (WARN_ON_ONCE(INTEL_GEN(dev_priv) >= 11)) {
/* TODO */
} else if (INTEL_GEN(dev_priv) >= 9) {
gen9_enable_rps(dev_priv);
} else if (IS_BROADWELL(dev_priv)) {

327
drivers/gpu/drm/i915/intel_psr.c
View File

@ -93,7 +93,7 @@ static void psr_aux_io_power_put(struct intel_dp *intel_dp)
intel_display_power_put(dev_priv, psr_aux_domain(intel_dp));
}
static bool intel_dp_get_y_cord_status(struct intel_dp *intel_dp)
static bool intel_dp_get_y_coord_required(struct intel_dp *intel_dp)
{
uint8_t psr_caps = 0;
@ -122,6 +122,18 @@ static bool intel_dp_get_alpm_status(struct intel_dp *intel_dp)
return alpm_caps & DP_ALPM_CAP;
}
static u8 intel_dp_get_sink_sync_latency(struct intel_dp *intel_dp)
{
u8 val = 0;
if (drm_dp_dpcd_readb(&intel_dp->aux,
DP_SYNCHRONIZATION_LATENCY_IN_SINK, &val) == 1)
val &= DP_MAX_RESYNC_FRAME_COUNT_MASK;
else
DRM_ERROR("Unable to get sink synchronization latency\n");
return val;
}
void intel_psr_init_dpcd(struct intel_dp *intel_dp)
{
struct drm_i915_private *dev_priv =
@ -130,33 +142,36 @@ void intel_psr_init_dpcd(struct intel_dp *intel_dp)
drm_dp_dpcd_read(&intel_dp->aux, DP_PSR_SUPPORT, intel_dp->psr_dpcd,
sizeof(intel_dp->psr_dpcd));
if (intel_dp->psr_dpcd[0] & DP_PSR_IS_SUPPORTED) {
if (intel_dp->psr_dpcd[0]) {
dev_priv->psr.sink_support = true;
DRM_DEBUG_KMS("Detected EDP PSR Panel.\n");
}
if (INTEL_GEN(dev_priv) >= 9 &&
(intel_dp->psr_dpcd[0] & DP_PSR2_IS_SUPPORTED)) {
uint8_t frame_sync_cap;
(intel_dp->psr_dpcd[0] == DP_PSR2_WITH_Y_COORD_IS_SUPPORTED)) {
/*
* All panels that supports PSR version 03h (PSR2 +
* Y-coordinate) can handle Y-coordinates in VSC but we are
* only sure that it is going to be used when required by the
* panel. This way panel is capable to do selective update
* without a aux frame sync.
*
* To support PSR version 02h and PSR version 03h without
* Y-coordinate requirement panels we would need to enable
* GTC first.
*/
dev_priv->psr.sink_psr2_support =
intel_dp_get_y_coord_required(intel_dp);
DRM_DEBUG_KMS("PSR2 %s on sink", dev_priv->psr.sink_psr2_support
? "supported" : "not supported");
dev_priv->psr.sink_support = true;
if (drm_dp_dpcd_readb(&intel_dp->aux,
DP_SINK_DEVICE_AUX_FRAME_SYNC_CAP,
&frame_sync_cap) != 1)
frame_sync_cap = 0;
dev_priv->psr.aux_frame_sync = frame_sync_cap & DP_AUX_FRAME_SYNC_CAP;
/* PSR2 needs frame sync as well */
dev_priv->psr.psr2_support = dev_priv->psr.aux_frame_sync;
DRM_DEBUG_KMS("PSR2 %s on sink",
dev_priv->psr.psr2_support ? "supported" : "not supported");
if (dev_priv->psr.psr2_support) {
dev_priv->psr.y_cord_support =
intel_dp_get_y_cord_status(intel_dp);
if (dev_priv->psr.sink_psr2_support) {
dev_priv->psr.colorimetry_support =
intel_dp_get_colorimetry_status(intel_dp);
dev_priv->psr.alpm =
intel_dp_get_alpm_status(intel_dp);
dev_priv->psr.sink_sync_latency =
intel_dp_get_sink_sync_latency(intel_dp);
}
}
}
@ -193,21 +208,17 @@ static void hsw_psr_setup_vsc(struct intel_dp *intel_dp,
struct drm_i915_private *dev_priv = to_i915(intel_dig_port->base.base.dev);
struct edp_vsc_psr psr_vsc;
if (dev_priv->psr.psr2_support) {
if (dev_priv->psr.psr2_enabled) {
/* Prepare VSC Header for SU as per EDP 1.4 spec, Table 6.11 */
memset(&psr_vsc, 0, sizeof(psr_vsc));
psr_vsc.sdp_header.HB0 = 0;
psr_vsc.sdp_header.HB1 = 0x7;
if (dev_priv->psr.colorimetry_support &&
dev_priv->psr.y_cord_support) {
if (dev_priv->psr.colorimetry_support) {
psr_vsc.sdp_header.HB2 = 0x5;
psr_vsc.sdp_header.HB3 = 0x13;
} else if (dev_priv->psr.y_cord_support) {
} else {
psr_vsc.sdp_header.HB2 = 0x4;
psr_vsc.sdp_header.HB3 = 0xe;
} else {
psr_vsc.sdp_header.HB2 = 0x3;
psr_vsc.sdp_header.HB3 = 0xc;
}
} else {
/* Prepare VSC packet as per EDP 1.3 spec, Table 3.10 */
@ -228,31 +239,12 @@ static void vlv_psr_enable_sink(struct intel_dp *intel_dp)
DP_PSR_ENABLE | DP_PSR_MAIN_LINK_ACTIVE);
}
static i915_reg_t psr_aux_ctl_reg(struct drm_i915_private *dev_priv,
enum port port)
{
if (INTEL_GEN(dev_priv) >= 9)
return DP_AUX_CH_CTL(port);
else
return EDP_PSR_AUX_CTL;
}
static i915_reg_t psr_aux_data_reg(struct drm_i915_private *dev_priv,
enum port port, int index)
{
if (INTEL_GEN(dev_priv) >= 9)
return DP_AUX_CH_DATA(port, index);
else
return EDP_PSR_AUX_DATA(index);
}
static void hsw_psr_enable_sink(struct intel_dp *intel_dp)
static void hsw_psr_setup_aux(struct intel_dp *intel_dp)
{
struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
struct drm_device *dev = dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
uint32_t aux_clock_divider;
i915_reg_t aux_ctl_reg;
struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
u32 aux_clock_divider, aux_ctl;
int i;
static const uint8_t aux_msg[] = {
[0] = DP_AUX_NATIVE_WRITE << 4,
[1] = DP_SET_POWER >> 8,
@ -260,41 +252,47 @@ static void hsw_psr_enable_sink(struct intel_dp *intel_dp)
[3] = 1 - 1,
[4] = DP_SET_POWER_D0,
};
enum port port = dig_port->base.port;
u32 aux_ctl;
int i;
u32 psr_aux_mask = EDP_PSR_AUX_CTL_TIME_OUT_MASK |
EDP_PSR_AUX_CTL_MESSAGE_SIZE_MASK |
EDP_PSR_AUX_CTL_PRECHARGE_2US_MASK |
EDP_PSR_AUX_CTL_BIT_CLOCK_2X_MASK;
BUILD_BUG_ON(sizeof(aux_msg) > 20);
for (i = 0; i < sizeof(aux_msg); i += 4)
I915_WRITE(EDP_PSR_AUX_DATA(i >> 2),
intel_dp_pack_aux(&aux_msg[i], sizeof(aux_msg) - i));
aux_clock_divider = intel_dp->get_aux_clock_divider(intel_dp, 0);
/* Enable AUX frame sync at sink */
if (dev_priv->psr.aux_frame_sync)
drm_dp_dpcd_writeb(&intel_dp->aux,
DP_SINK_DEVICE_AUX_FRAME_SYNC_CONF,
DP_AUX_FRAME_SYNC_ENABLE);
/* Start with bits set for DDI_AUX_CTL register */
aux_ctl = intel_dp->get_aux_send_ctl(intel_dp, 0, sizeof(aux_msg),
aux_clock_divider);
/* Select only valid bits for SRD_AUX_CTL */
aux_ctl &= psr_aux_mask;
I915_WRITE(EDP_PSR_AUX_CTL, aux_ctl);
}
static void hsw_psr_enable_sink(struct intel_dp *intel_dp)
{
struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
struct drm_device *dev = dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
u8 dpcd_val = DP_PSR_ENABLE;
/* Enable ALPM at sink for psr2 */
if (dev_priv->psr.psr2_support && dev_priv->psr.alpm)
if (dev_priv->psr.psr2_enabled && dev_priv->psr.alpm)
drm_dp_dpcd_writeb(&intel_dp->aux,
DP_RECEIVER_ALPM_CONFIG,
DP_ALPM_ENABLE);
if (dev_priv->psr.psr2_enabled)
dpcd_val |= DP_PSR_ENABLE_PSR2;
if (dev_priv->psr.link_standby)
drm_dp_dpcd_writeb(&intel_dp->aux, DP_PSR_EN_CFG,
DP_PSR_ENABLE | DP_PSR_MAIN_LINK_ACTIVE);
else
drm_dp_dpcd_writeb(&intel_dp->aux, DP_PSR_EN_CFG,
DP_PSR_ENABLE);
dpcd_val |= DP_PSR_MAIN_LINK_ACTIVE;
drm_dp_dpcd_writeb(&intel_dp->aux, DP_PSR_EN_CFG, dpcd_val);
aux_ctl_reg = psr_aux_ctl_reg(dev_priv, port);
/* Setup AUX registers */
for (i = 0; i < sizeof(aux_msg); i += 4)
I915_WRITE(psr_aux_data_reg(dev_priv, port, i >> 2),
intel_dp_pack_aux(&aux_msg[i], sizeof(aux_msg) - i));
aux_ctl = intel_dp->get_aux_send_ctl(intel_dp, 0, sizeof(aux_msg),
aux_clock_divider);
I915_WRITE(aux_ctl_reg, aux_ctl);
drm_dp_dpcd_writeb(&intel_dp->aux, DP_SET_POWER, DP_SET_POWER_D0);
}
static void vlv_psr_enable_source(struct intel_dp *intel_dp,
@ -396,25 +394,17 @@ static void hsw_activate_psr2(struct intel_dp *intel_dp)
* with the 5 or 6 idle patterns.
*/
uint32_t idle_frames = max(6, dev_priv->vbt.psr.idle_frames);
uint32_t val;
uint8_t sink_latency;
val = idle_frames << EDP_PSR_IDLE_FRAME_SHIFT;
u32 val = idle_frames << EDP_PSR2_IDLE_FRAME_SHIFT;
/* FIXME: selective update is probably totally broken because it doesn't
* mesh at all with our frontbuffer tracking. And the hw alone isn't
* good enough. */
val |= EDP_PSR2_ENABLE |
EDP_SU_TRACK_ENABLE;
if (drm_dp_dpcd_readb(&intel_dp->aux,
DP_SYNCHRONIZATION_LATENCY_IN_SINK,
&sink_latency) == 1) {
sink_latency &= DP_MAX_RESYNC_FRAME_COUNT_MASK;
} else {
sink_latency = 0;
val |= EDP_PSR2_ENABLE | EDP_SU_TRACK_ENABLE;
if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv)) {
val |= EDP_Y_COORDINATE_VALID | EDP_Y_COORDINATE_ENABLE;
}
val |= EDP_PSR2_FRAME_BEFORE_SU(sink_latency + 1);
val |= EDP_PSR2_FRAME_BEFORE_SU(dev_priv->psr.sink_sync_latency + 1);
if (dev_priv->vbt.psr.tp2_tp3_wakeup_time > 5)
val |= EDP_PSR2_TP2_TIME_2500;
@ -440,7 +430,7 @@ static void hsw_psr_activate(struct intel_dp *intel_dp)
*/
/* psr1 and psr2 are mutually exclusive.*/
if (dev_priv->psr.psr2_support)
if (dev_priv->psr.psr2_enabled)
hsw_activate_psr2(intel_dp);
else
hsw_activate_psr1(intel_dp);
@ -460,7 +450,7 @@ static bool intel_psr2_config_valid(struct intel_dp *intel_dp,
* dynamically during PSR enable, and extracted from sink
* caps during eDP detection.
*/
if (!dev_priv->psr.psr2_support)
if (!dev_priv->psr.sink_psr2_support)
return false;
if (INTEL_GEN(dev_priv) >= 10 || IS_GEMINILAKE(dev_priv)) {
@ -478,15 +468,6 @@ static bool intel_psr2_config_valid(struct intel_dp *intel_dp,
return false;
}
/*
* FIXME:enable psr2 only for y-cordinate psr2 panels
* After gtc implementation , remove this restriction.
*/
if (!dev_priv->psr.y_cord_support) {
DRM_DEBUG_KMS("PSR2 not enabled, panel does not support Y coordinate\n");
return false;
}
return true;
}
@ -568,7 +549,7 @@ static void intel_psr_activate(struct intel_dp *intel_dp)
struct drm_device *dev = intel_dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
if (dev_priv->psr.psr2_support)
if (dev_priv->psr.psr2_enabled)
WARN_ON(I915_READ(EDP_PSR2_CTL) & EDP_PSR2_ENABLE);
else
WARN_ON(I915_READ(EDP_PSR_CTL) & EDP_PSR_ENABLE);
@ -586,14 +567,24 @@ static void hsw_psr_enable_source(struct intel_dp *intel_dp,
struct drm_device *dev = dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
enum transcoder cpu_transcoder = crtc_state->cpu_transcoder;
u32 chicken;
psr_aux_io_power_get(intel_dp);
if (dev_priv->psr.psr2_support) {
chicken = PSR2_VSC_ENABLE_PROG_HEADER;
if (dev_priv->psr.y_cord_support)
chicken |= PSR2_ADD_VERTICAL_LINE_COUNT;
/* Only HSW and BDW have PSR AUX registers that need to be setup. SKL+
* use hardcoded values PSR AUX transactions
*/
if (IS_HASWELL(dev_priv) || IS_BROADWELL(dev_priv))
hsw_psr_setup_aux(intel_dp);
if (dev_priv->psr.psr2_enabled) {
u32 chicken = I915_READ(CHICKEN_TRANS(cpu_transcoder));
if (INTEL_GEN(dev_priv) == 9 && !IS_GEMINILAKE(dev_priv))
chicken |= (PSR2_VSC_ENABLE_PROG_HEADER
| PSR2_ADD_VERTICAL_LINE_COUNT);
else
chicken &= ~VSC_DATA_SEL_SOFTWARE_CONTROL;
I915_WRITE(CHICKEN_TRANS(cpu_transcoder), chicken);
I915_WRITE(EDP_PSR_DEBUG,
@ -644,7 +635,7 @@ void intel_psr_enable(struct intel_dp *intel_dp,
goto unlock;
}
dev_priv->psr.psr2_support = crtc_state->has_psr2;
dev_priv->psr.psr2_enabled = crtc_state->has_psr2;
dev_priv->psr.busy_frontbuffer_bits = 0;
dev_priv->psr.setup_vsc(intel_dp, crtc_state);
@ -714,12 +705,7 @@ static void hsw_psr_disable(struct intel_dp *intel_dp,
i915_reg_t psr_status;
u32 psr_status_mask;
if (dev_priv->psr.aux_frame_sync)
drm_dp_dpcd_writeb(&intel_dp->aux,
DP_SINK_DEVICE_AUX_FRAME_SYNC_CONF,
0);
if (dev_priv->psr.psr2_support) {
if (dev_priv->psr.psr2_enabled) {
psr_status = EDP_PSR2_STATUS;
psr_status_mask = EDP_PSR2_STATUS_STATE_MASK;
@ -743,7 +729,7 @@ static void hsw_psr_disable(struct intel_dp *intel_dp,
dev_priv->psr.active = false;
} else {
if (dev_priv->psr.psr2_support)
if (dev_priv->psr.psr2_enabled)
WARN_ON(I915_READ(EDP_PSR2_CTL) & EDP_PSR2_ENABLE);
else
WARN_ON(I915_READ(EDP_PSR_CTL) & EDP_PSR_ENABLE);
@ -789,53 +775,59 @@ void intel_psr_disable(struct intel_dp *intel_dp,
cancel_delayed_work_sync(&dev_priv->psr.work);
}
static bool psr_wait_for_idle(struct drm_i915_private *dev_priv)
{
struct intel_dp *intel_dp;
i915_reg_t reg;
u32 mask;
int err;
intel_dp = dev_priv->psr.enabled;
if (!intel_dp)
return false;
if (HAS_DDI(dev_priv)) {
if (dev_priv->psr.psr2_enabled) {
reg = EDP_PSR2_STATUS;
mask = EDP_PSR2_STATUS_STATE_MASK;
} else {
reg = EDP_PSR_STATUS;
mask = EDP_PSR_STATUS_STATE_MASK;
}
} else {
struct drm_crtc *crtc =
dp_to_dig_port(intel_dp)->base.base.crtc;
enum pipe pipe = to_intel_crtc(crtc)->pipe;
reg = VLV_PSRSTAT(pipe);
mask = VLV_EDP_PSR_IN_TRANS;
}
mutex_unlock(&dev_priv->psr.lock);
err = intel_wait_for_register(dev_priv, reg, mask, 0, 50);
if (err)
DRM_ERROR("Timed out waiting for PSR Idle for re-enable\n");
/* After the unlocked wait, verify that PSR is still wanted! */
mutex_lock(&dev_priv->psr.lock);
return err == 0 && dev_priv->psr.enabled;
}
static void intel_psr_work(struct work_struct *work)
{
struct drm_i915_private *dev_priv =
container_of(work, typeof(*dev_priv), psr.work.work);
struct intel_dp *intel_dp = dev_priv->psr.enabled;
struct drm_crtc *crtc = dp_to_dig_port(intel_dp)->base.base.crtc;
enum pipe pipe = to_intel_crtc(crtc)->pipe;
/* We have to make sure PSR is ready for re-enable
mutex_lock(&dev_priv->psr.lock);
/*
* We have to make sure PSR is ready for re-enable
* otherwise it keeps disabled until next full enable/disable cycle.
* PSR might take some time to get fully disabled
* and be ready for re-enable.
*/
if (HAS_DDI(dev_priv)) {
if (dev_priv->psr.psr2_support) {
if (intel_wait_for_register(dev_priv,
EDP_PSR2_STATUS,
EDP_PSR2_STATUS_STATE_MASK,
0,
50)) {
DRM_ERROR("Timed out waiting for PSR2 Idle for re-enable\n");
return;
}
} else {
if (intel_wait_for_register(dev_priv,
EDP_PSR_STATUS,
EDP_PSR_STATUS_STATE_MASK,
0,
50)) {
DRM_ERROR("Timed out waiting for PSR Idle for re-enable\n");
return;
}
}
} else {
if (intel_wait_for_register(dev_priv,
VLV_PSRSTAT(pipe),
VLV_EDP_PSR_IN_TRANS,
0,
1)) {
DRM_ERROR("Timed out waiting for PSR Idle for re-enable\n");
return;
}
}
mutex_lock(&dev_priv->psr.lock);
intel_dp = dev_priv->psr.enabled;
if (!intel_dp)
if (!psr_wait_for_idle(dev_priv))
goto unlock;
/*
@ -846,7 +838,7 @@ static void intel_psr_work(struct work_struct *work)
if (dev_priv->psr.busy_frontbuffer_bits)
goto unlock;
intel_psr_activate(intel_dp);
intel_psr_activate(dev_priv->psr.enabled);
unlock:
mutex_unlock(&dev_priv->psr.lock);
}
@ -862,11 +854,7 @@ static void intel_psr_exit(struct drm_i915_private *dev_priv)
return;
if (HAS_DDI(dev_priv)) {
if (dev_priv->psr.aux_frame_sync)
drm_dp_dpcd_writeb(&intel_dp->aux,
DP_SINK_DEVICE_AUX_FRAME_SYNC_CONF,
0);
if (dev_priv->psr.psr2_support) {
if (dev_priv->psr.psr2_enabled) {
val = I915_READ(EDP_PSR2_CTL);
WARN_ON(!(val & EDP_PSR2_ENABLE));
I915_WRITE(EDP_PSR2_CTL, val & ~EDP_PSR2_ENABLE);
@ -957,6 +945,7 @@ void intel_psr_single_frame_update(struct drm_i915_private *dev_priv,
* intel_psr_invalidate - Invalidade PSR
* @dev_priv: i915 device
* @frontbuffer_bits: frontbuffer plane tracking bits
* @origin: which operation caused the invalidate
*
* Since the hardware frontbuffer tracking has gaps we need to integrate
* with the software frontbuffer tracking. This function gets called every
@ -966,7 +955,7 @@ void intel_psr_single_frame_update(struct drm_i915_private *dev_priv,
* Dirty frontbuffers relevant to PSR are tracked in busy_frontbuffer_bits."
*/
void intel_psr_invalidate(struct drm_i915_private *dev_priv,
unsigned frontbuffer_bits)
unsigned frontbuffer_bits, enum fb_op_origin origin)
{
struct drm_crtc *crtc;
enum pipe pipe;
@ -974,6 +963,9 @@ void intel_psr_invalidate(struct drm_i915_private *dev_priv,
if (!CAN_PSR(dev_priv))
return;
if (dev_priv->psr.has_hw_tracking && origin == ORIGIN_FLIP)
return;
mutex_lock(&dev_priv->psr.lock);
if (!dev_priv->psr.enabled) {
mutex_unlock(&dev_priv->psr.lock);
@ -1014,6 +1006,9 @@ void intel_psr_flush(struct drm_i915_private *dev_priv,
if (!CAN_PSR(dev_priv))
return;
if (dev_priv->psr.has_hw_tracking && origin == ORIGIN_FLIP)
return;
mutex_lock(&dev_priv->psr.lock);
if (!dev_priv->psr.enabled) {
mutex_unlock(&dev_priv->psr.lock);
@ -1027,8 +1022,23 @@ void intel_psr_flush(struct drm_i915_private *dev_priv,
dev_priv->psr.busy_frontbuffer_bits &= ~frontbuffer_bits;
/* By definition flush = invalidate + flush */
if (frontbuffer_bits)
intel_psr_exit(dev_priv);
if (frontbuffer_bits) {
if (dev_priv->psr.psr2_enabled ||
IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
intel_psr_exit(dev_priv);
} else {
/*
* Display WA #0884: all
* This documented WA for bxt can be safely applied
* broadly so we can force HW tracking to exit PSR
* instead of disabling and re-enabling.
* Workaround tells us to write 0 to CUR_SURFLIVE_A,
* but it makes more sense write to the current active
* pipe.
*/
I915_WRITE(CURSURFLIVE(pipe), 0);
}
}
if (!dev_priv->psr.active && !dev_priv->psr.busy_frontbuffer_bits)
if (!work_busy(&dev_priv->psr.work.work))
@ -1090,6 +1100,7 @@ void intel_psr_init(struct drm_i915_private *dev_priv)
dev_priv->psr.activate = vlv_psr_activate;
dev_priv->psr.setup_vsc = vlv_psr_setup_vsc;
} else {
dev_priv->psr.has_hw_tracking = true;
dev_priv->psr.enable_source = hsw_psr_enable_source;
dev_priv->psr.disable_source = hsw_psr_disable;
dev_priv->psr.enable_sink = hsw_psr_enable_sink;

22
drivers/gpu/drm/i915/intel_ringbuffer.c
View File

@ -36,6 +36,7 @@
#include "i915_gem_render_state.h"
#include "i915_trace.h"
#include "intel_drv.h"
#include "intel_workarounds.h"
/* Rough estimate of the typical request size, performing a flush,
* set-context and then emitting the batch.
@ -599,7 +600,7 @@ static int intel_rcs_ctx_init(struct i915_request *rq)
{
int ret;
ret = intel_ring_workarounds_emit(rq);
ret = intel_ctx_workarounds_emit(rq);
if (ret != 0)
return ret;
@ -617,6 +618,10 @@ static int init_render_ring(struct intel_engine_cs *engine)
if (ret)
return ret;
ret = intel_whitelist_workarounds_apply(engine);
if (ret)
return ret;
/* WaTimedSingleVertexDispatch:cl,bw,ctg,elk,ilk,snb */
if (IS_GEN(dev_priv, 4, 6))
I915_WRITE(MI_MODE, _MASKED_BIT_ENABLE(VS_TIMER_DISPATCH));
@ -658,7 +663,7 @@ static int init_render_ring(struct intel_engine_cs *engine)
if (INTEL_GEN(dev_priv) >= 6)
I915_WRITE_IMR(engine, ~engine->irq_keep_mask);
return init_workarounds_ring(engine);
return 0;
}
static u32 *gen6_signal(struct i915_request *rq, u32 *cs)
@ -1593,6 +1598,7 @@ static noinline int wait_for_space(struct intel_ring *ring, unsigned int bytes)
if (intel_ring_update_space(ring) >= bytes)
return 0;
GEM_BUG_ON(list_empty(&ring->request_list));
list_for_each_entry(target, &ring->request_list, ring_link) {
/* Would completion of this request free enough space? */
if (bytes <= __intel_ring_space(target->postfix,
@ -1692,17 +1698,18 @@ u32 *intel_ring_begin(struct i915_request *rq, unsigned int num_dwords)
need_wrap &= ~1;
GEM_BUG_ON(need_wrap > ring->space);
GEM_BUG_ON(ring->emit + need_wrap > ring->size);
GEM_BUG_ON(!IS_ALIGNED(need_wrap, sizeof(u64)));
/* Fill the tail with MI_NOOP */
memset(ring->vaddr + ring->emit, 0, need_wrap);
ring->emit = 0;
memset64(ring->vaddr + ring->emit, 0, need_wrap / sizeof(u64));
ring->space -= need_wrap;
ring->emit = 0;
}
GEM_BUG_ON(ring->emit > ring->size - bytes);
GEM_BUG_ON(ring->space < bytes);
cs = ring->vaddr + ring->emit;
GEM_DEBUG_EXEC(memset(cs, POISON_INUSE, bytes));
GEM_DEBUG_EXEC(memset32(cs, POISON_INUSE, bytes / sizeof(*cs)));
ring->emit += bytes;
ring->space -= bytes;
@ -1943,8 +1950,6 @@ static void intel_ring_init_semaphores(struct drm_i915_private *dev_priv,
static void intel_ring_init_irq(struct drm_i915_private *dev_priv,
struct intel_engine_cs *engine)
{
engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT << engine->irq_shift;
if (INTEL_GEN(dev_priv) >= 6) {
engine->irq_enable = gen6_irq_enable;
engine->irq_disable = gen6_irq_disable;
@ -2029,6 +2034,8 @@ int intel_init_render_ring_buffer(struct intel_engine_cs *engine)
if (HAS_L3_DPF(dev_priv))
engine->irq_keep_mask = GT_RENDER_L3_PARITY_ERROR_INTERRUPT;
engine->irq_enable_mask = GT_RENDER_USER_INTERRUPT;
if (INTEL_GEN(dev_priv) >= 6) {
engine->init_context = intel_rcs_ctx_init;
engine->emit_flush = gen7_render_ring_flush;
@ -2079,7 +2086,6 @@ int intel_init_bsd_ring_buffer(struct intel_engine_cs *engine)
engine->emit_flush = gen6_bsd_ring_flush;
engine->irq_enable_mask = GT_BSD_USER_INTERRUPT;
} else {
engine->mmio_base = BSD_RING_BASE;
engine->emit_flush = bsd_ring_flush;
if (IS_GEN5(dev_priv))
engine->irq_enable_mask = ILK_BSD_USER_INTERRUPT;

43
drivers/gpu/drm/i915/intel_ringbuffer.h
View File

@ -7,9 +7,11 @@
#include "i915_gem_batch_pool.h"
#include "i915_gem_timeline.h"
#include "i915_reg.h"
#include "i915_pmu.h"
#include "i915_request.h"
#include "i915_selftest.h"
#include "intel_gpu_commands.h"
struct drm_printer;
@ -84,7 +86,7 @@ hangcheck_action_to_str(const enum intel_engine_hangcheck_action a)
}
#define I915_MAX_SLICES 3
#define I915_MAX_SUBSLICES 3
#define I915_MAX_SUBSLICES 8
#define instdone_slice_mask(dev_priv__) \
(INTEL_GEN(dev_priv__) == 7 ? \
@ -330,7 +332,6 @@ struct intel_engine_cs {
u8 instance;
u32 context_size;
u32 mmio_base;
unsigned int irq_shift;
struct intel_ring *buffer;
struct intel_timeline *timeline;
@ -561,6 +562,7 @@ struct intel_engine_cs {
#define I915_ENGINE_NEEDS_CMD_PARSER BIT(0)
#define I915_ENGINE_SUPPORTS_STATS BIT(1)
#define I915_ENGINE_HAS_PREEMPTION BIT(2)
unsigned int flags;
/*
@ -620,16 +622,29 @@ struct intel_engine_cs {
} stats;
};
static inline bool intel_engine_needs_cmd_parser(struct intel_engine_cs *engine)
static inline bool
intel_engine_needs_cmd_parser(const struct intel_engine_cs *engine)
{
return engine->flags & I915_ENGINE_NEEDS_CMD_PARSER;
}
static inline bool intel_engine_supports_stats(struct intel_engine_cs *engine)
static inline bool
intel_engine_supports_stats(const struct intel_engine_cs *engine)
{
return engine->flags & I915_ENGINE_SUPPORTS_STATS;
}
static inline bool
intel_engine_has_preemption(const struct intel_engine_cs *engine)
{
return engine->flags & I915_ENGINE_HAS_PREEMPTION;
}
static inline bool __execlists_need_preempt(int prio, int last)
{
return prio > max(0, last);
}
static inline void
execlists_set_active(struct intel_engine_execlists *execlists,
unsigned int bit)
@ -637,6 +652,13 @@ execlists_set_active(struct intel_engine_execlists *execlists,
__set_bit(bit, (unsigned long *)&execlists->active);
}
static inline bool
execlists_set_active_once(struct intel_engine_execlists *execlists,
unsigned int bit)
{
return !__test_and_set_bit(bit, (unsigned long *)&execlists->active);
}
static inline void
execlists_clear_active(struct intel_engine_execlists *execlists,
unsigned int bit)
@ -651,6 +673,10 @@ execlists_is_active(const struct intel_engine_execlists *execlists,
return test_bit(bit, (unsigned long *)&execlists->active);
}
void execlists_user_begin(struct intel_engine_execlists *execlists,
const struct execlist_port *port);
void execlists_user_end(struct intel_engine_execlists *execlists);
void
execlists_cancel_port_requests(struct intel_engine_execlists * const execlists);
@ -663,7 +689,7 @@ execlists_num_ports(const struct intel_engine_execlists * const execlists)
return execlists->port_mask + 1;
}
static inline void
static inline struct execlist_port *
execlists_port_complete(struct intel_engine_execlists * const execlists,
struct execlist_port * const port)
{
@ -674,6 +700,8 @@ execlists_port_complete(struct intel_engine_execlists * const execlists,
memmove(port, port + 1, m * sizeof(struct execlist_port));
memset(port + m, 0, sizeof(struct execlist_port));
return port;
}
static inline unsigned int
@ -857,9 +885,6 @@ static inline u32 intel_engine_last_submit(struct intel_engine_cs *engine)
return READ_ONCE(engine->timeline->seqno);
}
int init_workarounds_ring(struct intel_engine_cs *engine);
int intel_ring_workarounds_emit(struct i915_request *rq);
void intel_engine_get_instdone(struct intel_engine_cs *engine,
struct intel_instdone *instdone);
@ -939,7 +964,7 @@ bool intel_engine_add_wait(struct intel_engine_cs *engine,
struct intel_wait *wait);
void intel_engine_remove_wait(struct intel_engine_cs *engine,
struct intel_wait *wait);
void intel_engine_enable_signaling(struct i915_request *request, bool wakeup);
bool intel_engine_enable_signaling(struct i915_request *request, bool wakeup);
void intel_engine_cancel_signaling(struct i915_request *request);
static inline bool intel_engine_has_waiter(const struct intel_engine_cs *engine)

7
drivers/gpu/drm/i915/intel_sprite.c
View File

@ -48,6 +48,7 @@ bool intel_format_is_yuv(u32 format)
case DRM_FORMAT_UYVY:
case DRM_FORMAT_VYUY:
case DRM_FORMAT_YVYU:
case DRM_FORMAT_NV12:
return true;
default:
return false;
@ -946,6 +947,7 @@ intel_check_sprite_plane(struct intel_plane *plane,
int max_scale, min_scale;
bool can_scale;
int ret;
uint32_t pixel_format = 0;
*src = drm_plane_state_src(&state->base);
*dst = drm_plane_state_dest(&state->base);
@ -969,11 +971,14 @@ intel_check_sprite_plane(struct intel_plane *plane,
/* setup can_scale, min_scale, max_scale */
if (INTEL_GEN(dev_priv) >= 9) {
if (state->base.fb)
pixel_format = state->base.fb->format->format;
/* use scaler when colorkey is not required */
if (!state->ckey.flags) {
can_scale = 1;
min_scale = 1;
max_scale = skl_max_scale(crtc, crtc_state);
max_scale =
skl_max_scale(crtc, crtc_state, pixel_format);
} else {
can_scale = 0;
min_scale = DRM_PLANE_HELPER_NO_SCALING;

138
drivers/gpu/drm/i915/intel_uc.c
View File

@ -69,13 +69,15 @@ static int __get_platform_enable_guc(struct drm_i915_private *dev_priv)
static int __get_default_guc_log_level(struct drm_i915_private *dev_priv)
{
int guc_log_level = 0; /* disabled */
int guc_log_level;
/* Enable if we're running on platform with GuC and debug config */
if (HAS_GUC(dev_priv) && intel_uc_is_using_guc() &&
(IS_ENABLED(CONFIG_DRM_I915_DEBUG) ||
IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM)))
guc_log_level = 1 + GUC_LOG_VERBOSITY_MAX;
if (!HAS_GUC(dev_priv) || !intel_uc_is_using_guc())
guc_log_level = GUC_LOG_LEVEL_DISABLED;
else if (IS_ENABLED(CONFIG_DRM_I915_DEBUG) ||
IS_ENABLED(CONFIG_DRM_I915_DEBUG_GEM))
guc_log_level = GUC_LOG_LEVEL_MAX;
else
guc_log_level = GUC_LOG_LEVEL_NON_VERBOSE;
/* Any platform specific fine-tuning can be done here */
@ -83,7 +85,7 @@ static int __get_default_guc_log_level(struct drm_i915_private *dev_priv)
}
/**
* intel_uc_sanitize_options - sanitize uC related modparam options
* sanitize_options_early - sanitize uC related modparam options
* @dev_priv: device private
*
* In case of "enable_guc" option this function will attempt to modify
@ -99,7 +101,7 @@ static int __get_default_guc_log_level(struct drm_i915_private *dev_priv)
* unless GuC is enabled on given platform and the driver is compiled with
* debug config when this modparam will default to "enable(1..4)".
*/
void intel_uc_sanitize_options(struct drm_i915_private *dev_priv)
static void sanitize_options_early(struct drm_i915_private *dev_priv)
{
struct intel_uc_fw *guc_fw = &dev_priv->guc.fw;
struct intel_uc_fw *huc_fw = &dev_priv->huc.fw;
@ -142,51 +144,53 @@ void intel_uc_sanitize_options(struct drm_i915_private *dev_priv)
i915_modparams.guc_log_level = 0;
}
if (i915_modparams.guc_log_level > 1 + GUC_LOG_VERBOSITY_MAX) {
if (i915_modparams.guc_log_level > GUC_LOG_LEVEL_MAX) {
DRM_WARN("Incompatible option detected: %s=%d, %s!\n",
"guc_log_level", i915_modparams.guc_log_level,
"verbosity too high");
i915_modparams.guc_log_level = 1 + GUC_LOG_VERBOSITY_MAX;
i915_modparams.guc_log_level = GUC_LOG_LEVEL_MAX;
}
DRM_DEBUG_DRIVER("guc_log_level=%d (enabled:%s verbosity:%d)\n",
DRM_DEBUG_DRIVER("guc_log_level=%d (enabled:%s, verbose:%s, verbosity:%d)\n",
i915_modparams.guc_log_level,
yesno(i915_modparams.guc_log_level),
i915_modparams.guc_log_level - 1);
yesno(GUC_LOG_LEVEL_IS_VERBOSE(i915_modparams.guc_log_level)),
GUC_LOG_LEVEL_TO_VERBOSITY(i915_modparams.guc_log_level));
/* Make sure that sanitization was done */
GEM_BUG_ON(i915_modparams.enable_guc < 0);
GEM_BUG_ON(i915_modparams.guc_log_level < 0);
}
void intel_uc_init_early(struct drm_i915_private *dev_priv)
void intel_uc_init_early(struct drm_i915_private *i915)
{
intel_guc_init_early(&dev_priv->guc);
intel_huc_init_early(&dev_priv->huc);
struct intel_guc *guc = &i915->guc;
struct intel_huc *huc = &i915->huc;
intel_guc_init_early(guc);
intel_huc_init_early(huc);
sanitize_options_early(i915);
if (USES_GUC(i915))
intel_uc_fw_fetch(i915, &guc->fw);
if (USES_HUC(i915))
intel_uc_fw_fetch(i915, &huc->fw);
}
void intel_uc_init_fw(struct drm_i915_private *dev_priv)
void intel_uc_cleanup_early(struct drm_i915_private *i915)
{
if (!USES_GUC(dev_priv))
return;
struct intel_guc *guc = &i915->guc;
struct intel_huc *huc = &i915->huc;
if (USES_HUC(dev_priv))
intel_uc_fw_fetch(dev_priv, &dev_priv->huc.fw);
if (USES_HUC(i915))
intel_uc_fw_fini(&huc->fw);
intel_uc_fw_fetch(dev_priv, &dev_priv->guc.fw);
}
if (USES_GUC(i915))
intel_uc_fw_fini(&guc->fw);
void intel_uc_fini_fw(struct drm_i915_private *dev_priv)
{
if (!USES_GUC(dev_priv))
return;
intel_uc_fw_fini(&dev_priv->guc.fw);
if (USES_HUC(dev_priv))
intel_uc_fw_fini(&dev_priv->huc.fw);
guc_free_load_err_log(&dev_priv->guc);
guc_free_load_err_log(guc);
}
/**
@ -223,10 +227,13 @@ static int guc_enable_communication(struct intel_guc *guc)
{
struct drm_i915_private *dev_priv = guc_to_i915(guc);
gen9_enable_guc_interrupts(dev_priv);
if (HAS_GUC_CT(dev_priv))
return intel_guc_enable_ct(guc);
return intel_guc_ct_enable(&guc->ct);
guc->send = intel_guc_send_mmio;
guc->handler = intel_guc_to_host_event_handler_mmio;
return 0;
}
@ -235,9 +242,12 @@ static void guc_disable_communication(struct intel_guc *guc)
struct drm_i915_private *dev_priv = guc_to_i915(guc);
if (HAS_GUC_CT(dev_priv))
intel_guc_disable_ct(guc);
intel_guc_ct_disable(&guc->ct);
gen9_disable_guc_interrupts(dev_priv);
guc->send = intel_guc_send_nop;
guc->handler = intel_guc_to_host_event_handler_nop;
}
int intel_uc_init_misc(struct drm_i915_private *dev_priv)
@ -248,24 +258,13 @@ int intel_uc_init_misc(struct drm_i915_private *dev_priv)
if (!USES_GUC(dev_priv))
return 0;
ret = intel_guc_init_wq(guc);
if (ret) {
DRM_ERROR("Couldn't allocate workqueues for GuC\n");
goto err;
}
intel_guc_init_ggtt_pin_bias(guc);
ret = intel_guc_log_relay_create(guc);
if (ret) {
DRM_ERROR("Couldn't allocate relay for GuC log\n");
goto err_relay;
}
ret = intel_guc_init_wq(guc);
if (ret)
return ret;
return 0;
err_relay:
intel_guc_fini_wq(guc);
err:
return ret;
}
void intel_uc_fini_misc(struct drm_i915_private *dev_priv)
@ -276,8 +275,6 @@ void intel_uc_fini_misc(struct drm_i915_private *dev_priv)
return;
intel_guc_fini_wq(guc);
intel_guc_log_relay_destroy(guc);
}
int intel_uc_init(struct drm_i915_private *dev_priv)
@ -325,6 +322,24 @@ void intel_uc_fini(struct drm_i915_private *dev_priv)
intel_guc_fini(guc);
}
void intel_uc_sanitize(struct drm_i915_private *i915)
{
struct intel_guc *guc = &i915->guc;
struct intel_huc *huc = &i915->huc;
if (!USES_GUC(i915))
return;
GEM_BUG_ON(!HAS_GUC(i915));
guc_disable_communication(guc);
intel_huc_sanitize(huc);
intel_guc_sanitize(guc);
__intel_uc_reset_hw(i915);
}
int intel_uc_init_hw(struct drm_i915_private *dev_priv)
{
struct intel_guc *guc = &dev_priv->guc;
@ -336,14 +351,8 @@ int intel_uc_init_hw(struct drm_i915_private *dev_priv)
GEM_BUG_ON(!HAS_GUC(dev_priv));
guc_disable_communication(guc);
gen9_reset_guc_interrupts(dev_priv);
/* init WOPCM */
I915_WRITE(GUC_WOPCM_SIZE, intel_guc_wopcm_size(dev_priv));
I915_WRITE(DMA_GUC_WOPCM_OFFSET,
GUC_WOPCM_OFFSET_VALUE | HUC_LOADING_AGENT_GUC);
/* WaEnableuKernelHeaderValidFix:skl */
/* WaEnableGuCBootHashCheckNotSet:skl,bxt,kbl */
if (IS_GEN9(dev_priv))
@ -390,12 +399,9 @@ int intel_uc_init_hw(struct drm_i915_private *dev_priv)
}
if (USES_GUC_SUBMISSION(dev_priv)) {
if (i915_modparams.guc_log_level)
gen9_enable_guc_interrupts(dev_priv);
ret = intel_guc_submission_enable(guc);
if (ret)
goto err_interrupts;
goto err_communication;
}
dev_info(dev_priv->drm.dev, "GuC firmware version %u.%u\n",
@ -410,8 +416,6 @@ int intel_uc_init_hw(struct drm_i915_private *dev_priv)
/*
* We've failed to load the firmware :(
*/
err_interrupts:
gen9_disable_guc_interrupts(dev_priv);
err_communication:
guc_disable_communication(guc);
err_log_capture:
@ -441,9 +445,6 @@ void intel_uc_fini_hw(struct drm_i915_private *dev_priv)
intel_guc_submission_disable(guc);
guc_disable_communication(guc);
if (USES_GUC_SUBMISSION(dev_priv))
gen9_disable_guc_interrupts(dev_priv);
}
int intel_uc_suspend(struct drm_i915_private *i915)
@ -479,8 +480,7 @@ int intel_uc_resume(struct drm_i915_private *i915)
if (guc->fw.load_status != INTEL_UC_FIRMWARE_SUCCESS)
return 0;
if (i915_modparams.guc_log_level)
gen9_enable_guc_interrupts(i915);
gen9_enable_guc_interrupts(i915);
err = intel_guc_resume(guc);
if (err) {

5
drivers/gpu/drm/i915/intel_uc.h
View File

@ -28,13 +28,12 @@
#include "intel_huc.h"
#include "i915_params.h"
void intel_uc_sanitize_options(struct drm_i915_private *dev_priv);
void intel_uc_init_early(struct drm_i915_private *dev_priv);
void intel_uc_cleanup_early(struct drm_i915_private *dev_priv);
void intel_uc_init_mmio(struct drm_i915_private *dev_priv);
void intel_uc_init_fw(struct drm_i915_private *dev_priv);
void intel_uc_fini_fw(struct drm_i915_private *dev_priv);
int intel_uc_init_misc(struct drm_i915_private *dev_priv);
void intel_uc_fini_misc(struct drm_i915_private *dev_priv);
void intel_uc_sanitize(struct drm_i915_private *dev_priv);
int intel_uc_init_hw(struct drm_i915_private *dev_priv);
void intel_uc_fini_hw(struct drm_i915_private *dev_priv);
int intel_uc_init(struct drm_i915_private *dev_priv);

13
drivers/gpu/drm/i915/intel_uc_fw.c
View File

@ -95,15 +95,6 @@ void intel_uc_fw_fetch(struct drm_i915_private *dev_priv,
uc_fw->ucode_offset = uc_fw->header_offset + uc_fw->header_size;
uc_fw->ucode_size = (css->size_dw - css->header_size_dw) * sizeof(u32);
/* Header and uCode will be loaded to WOPCM */
size = uc_fw->header_size + uc_fw->ucode_size;
if (size > intel_guc_wopcm_size(dev_priv)) {
DRM_WARN("%s: Firmware is too large to fit in WOPCM\n",
intel_uc_fw_type_repr(uc_fw->type));
err = -E2BIG;
goto fail;
}
/* now RSA */
if (css->key_size_dw != UOS_RSA_SCRATCH_COUNT) {
DRM_WARN("%s: Mismatched firmware RSA key size (%u)\n",
@ -209,6 +200,7 @@ int intel_uc_fw_upload(struct intel_uc_fw *uc_fw,
struct i915_vma *vma))
{
struct i915_vma *vma;
u32 ggtt_pin_bias;
int err;
DRM_DEBUG_DRIVER("%s fw load %s\n",
@ -230,8 +222,9 @@ int intel_uc_fw_upload(struct intel_uc_fw *uc_fw,
goto fail;
}
ggtt_pin_bias = to_i915(uc_fw->obj->base.dev)->guc.ggtt_pin_bias;
vma = i915_gem_object_ggtt_pin(uc_fw->obj, NULL, 0, 0,
PIN_OFFSET_BIAS | GUC_WOPCM_TOP);
PIN_OFFSET_BIAS | ggtt_pin_bias);
if (IS_ERR(vma)) {
err = PTR_ERR(vma);
DRM_DEBUG_DRIVER("%s fw ggtt-pin err=%d\n",

22
drivers/gpu/drm/i915/intel_uc_fw.h
View File

@ -115,6 +115,28 @@ static inline bool intel_uc_fw_is_selected(struct intel_uc_fw *uc_fw)
return uc_fw->path != NULL;
}
static inline void intel_uc_fw_sanitize(struct intel_uc_fw *uc_fw)
{
if (uc_fw->load_status == INTEL_UC_FIRMWARE_SUCCESS)
uc_fw->load_status = INTEL_UC_FIRMWARE_PENDING;
}
/**
* intel_uc_fw_get_upload_size() - Get size of firmware needed to be uploaded.
* @uc_fw: uC firmware.
*
* Get the size of the firmware and header that will be uploaded to WOPCM.
*
* Return: Upload firmware size, or zero on firmware fetch failure.
*/
static inline u32 intel_uc_fw_get_upload_size(struct intel_uc_fw *uc_fw)
{
if (uc_fw->fetch_status != INTEL_UC_FIRMWARE_SUCCESS)
return 0;
return uc_fw->header_size + uc_fw->ucode_size;
}
void intel_uc_fw_fetch(struct drm_i915_private *dev_priv,
struct intel_uc_fw *uc_fw);
int intel_uc_fw_upload(struct intel_uc_fw *uc_fw,

168
drivers/gpu/drm/i915/intel_uncore.c
View File

@ -62,6 +62,11 @@ static inline void
fw_domain_reset(struct drm_i915_private *i915,
const struct intel_uncore_forcewake_domain *d)
{
/*
* We don't really know if the powerwell for the forcewake domain we are
* trying to reset here does exist at this point (engines could be fused
* off in ICL+), so no waiting for acks
*/
__raw_i915_write32(i915, d->reg_set, i915->uncore.fw_reset);
}
@ -1353,6 +1358,23 @@ static void fw_domain_init(struct drm_i915_private *dev_priv,
fw_domain_reset(dev_priv, d);
}
static void fw_domain_fini(struct drm_i915_private *dev_priv,
enum forcewake_domain_id domain_id)
{
struct intel_uncore_forcewake_domain *d;
if (WARN_ON(domain_id >= FW_DOMAIN_ID_COUNT))
return;
d = &dev_priv->uncore.fw_domain[domain_id];
WARN_ON(d->wake_count);
WARN_ON(hrtimer_cancel(&d->timer));
memset(d, 0, sizeof(*d));
dev_priv->uncore.fw_domains &= ~BIT(domain_id);
}
static void intel_uncore_fw_domains_init(struct drm_i915_private *dev_priv)
{
if (INTEL_GEN(dev_priv) <= 5 || intel_vgpu_active(dev_priv))
@ -1565,6 +1587,40 @@ void intel_uncore_init(struct drm_i915_private *dev_priv)
&dev_priv->uncore.pmic_bus_access_nb);
}
/*
* We might have detected that some engines are fused off after we initialized
* the forcewake domains. Prune them, to make sure they only reference existing
* engines.
*/
void intel_uncore_prune(struct drm_i915_private *dev_priv)
{
if (INTEL_GEN(dev_priv) >= 11) {
enum forcewake_domains fw_domains = dev_priv->uncore.fw_domains;
enum forcewake_domain_id domain_id;
int i;
for (i = 0; i < I915_MAX_VCS; i++) {
domain_id = FW_DOMAIN_ID_MEDIA_VDBOX0 + i;
if (HAS_ENGINE(dev_priv, _VCS(i)))
continue;
if (fw_domains & BIT(domain_id))
fw_domain_fini(dev_priv, domain_id);
}
for (i = 0; i < I915_MAX_VECS; i++) {
domain_id = FW_DOMAIN_ID_MEDIA_VEBOX0 + i;
if (HAS_ENGINE(dev_priv, _VECS(i)))
continue;
if (fw_domains & BIT(domain_id))
fw_domain_fini(dev_priv, domain_id);
}
}
}
void intel_uncore_fini(struct drm_i915_private *dev_priv)
{
/* Paranoia: make sure we have disabled everything before we exit. */
@ -1646,11 +1702,10 @@ static void gen3_stop_engine(struct intel_engine_cs *engine)
const i915_reg_t mode = RING_MI_MODE(base);
I915_WRITE_FW(mode, _MASKED_BIT_ENABLE(STOP_RING));
if (intel_wait_for_register_fw(dev_priv,
mode,
MODE_IDLE,
MODE_IDLE,
500))
if (__intel_wait_for_register_fw(dev_priv,
mode, MODE_IDLE, MODE_IDLE,
500, 0,
NULL))
DRM_DEBUG_DRIVER("%s: timed out on STOP_RING\n",
engine->name);
@ -1804,9 +1859,10 @@ static int gen6_hw_domain_reset(struct drm_i915_private *dev_priv,
__raw_i915_write32(dev_priv, GEN6_GDRST, hw_domain_mask);
/* Wait for the device to ack the reset requests */
err = intel_wait_for_register_fw(dev_priv,
GEN6_GDRST, hw_domain_mask, 0,
500);
err = __intel_wait_for_register_fw(dev_priv,
GEN6_GDRST, hw_domain_mask, 0,
500, 0,
NULL);
if (err)
DRM_DEBUG_DRIVER("Wait for 0x%08x engines reset failed\n",
hw_domain_mask);
@ -1853,6 +1909,50 @@ static int gen6_reset_engines(struct drm_i915_private *dev_priv,
return gen6_hw_domain_reset(dev_priv, hw_mask);
}
/**
* gen11_reset_engines - reset individual engines
* @dev_priv: i915 device
* @engine_mask: mask of intel_ring_flag() engines or ALL_ENGINES for full reset
*
* This function will reset the individual engines that are set in engine_mask.
* If you provide ALL_ENGINES as mask, full global domain reset will be issued.
*
* Note: It is responsibility of the caller to handle the difference between
* asking full domain reset versus reset for all available individual engines.
*
* Returns 0 on success, nonzero on error.
*/
static int gen11_reset_engines(struct drm_i915_private *dev_priv,
unsigned engine_mask)
{
struct intel_engine_cs *engine;
const u32 hw_engine_mask[I915_NUM_ENGINES] = {
[RCS] = GEN11_GRDOM_RENDER,
[BCS] = GEN11_GRDOM_BLT,
[VCS] = GEN11_GRDOM_MEDIA,
[VCS2] = GEN11_GRDOM_MEDIA2,
[VCS3] = GEN11_GRDOM_MEDIA3,
[VCS4] = GEN11_GRDOM_MEDIA4,
[VECS] = GEN11_GRDOM_VECS,
[VECS2] = GEN11_GRDOM_VECS2,
};
u32 hw_mask;
BUILD_BUG_ON(VECS2 + 1 != I915_NUM_ENGINES);
if (engine_mask == ALL_ENGINES) {
hw_mask = GEN11_GRDOM_FULL;
} else {
unsigned int tmp;
hw_mask = 0;
for_each_engine_masked(engine, dev_priv, engine_mask, tmp)
hw_mask |= hw_engine_mask[engine->id];
}
return gen6_hw_domain_reset(dev_priv, hw_mask);
}
/**
* __intel_wait_for_register_fw - wait until register matches expected state
* @dev_priv: the i915 device
@ -1940,7 +2040,7 @@ int __intel_wait_for_register(struct drm_i915_private *dev_priv,
u32 reg_value;
int ret;
might_sleep();
might_sleep_if(slow_timeout_ms);
spin_lock_irq(&dev_priv->uncore.lock);
intel_uncore_forcewake_get__locked(dev_priv, fw);
@ -1952,7 +2052,7 @@ int __intel_wait_for_register(struct drm_i915_private *dev_priv,
intel_uncore_forcewake_put__locked(dev_priv, fw);
spin_unlock_irq(&dev_priv->uncore.lock);
if (ret)
if (ret && slow_timeout_ms)
ret = __wait_for(reg_value = I915_READ_NOTRACE(reg),
(reg_value & mask) == value,
slow_timeout_ms * 1000, 10, 1000);
@ -1971,11 +2071,12 @@ static int gen8_reset_engine_start(struct intel_engine_cs *engine)
I915_WRITE_FW(RING_RESET_CTL(engine->mmio_base),
_MASKED_BIT_ENABLE(RESET_CTL_REQUEST_RESET));
ret = intel_wait_for_register_fw(dev_priv,
RING_RESET_CTL(engine->mmio_base),
RESET_CTL_READY_TO_RESET,
RESET_CTL_READY_TO_RESET,
700);
ret = __intel_wait_for_register_fw(dev_priv,
RING_RESET_CTL(engine->mmio_base),
RESET_CTL_READY_TO_RESET,
RESET_CTL_READY_TO_RESET,
700, 0,
NULL);
if (ret)
DRM_ERROR("%s: reset request timeout\n", engine->name);
@ -2000,7 +2101,10 @@ static int gen8_reset_engines(struct drm_i915_private *dev_priv,
if (gen8_reset_engine_start(engine))
goto not_ready;
return gen6_reset_engines(dev_priv, engine_mask);
if (INTEL_GEN(dev_priv) >= 11)
return gen11_reset_engines(dev_priv, engine_mask);
else
return gen6_reset_engines(dev_priv, engine_mask);
not_ready:
for_each_engine_masked(engine, dev_priv, engine_mask, tmp)
@ -2038,15 +2142,31 @@ int intel_gpu_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)
int retry;
int ret;
might_sleep();
/*
* We want to perform per-engine reset from atomic context (e.g.
* softirq), which imposes the constraint that we cannot sleep.
* However, experience suggests that spending a bit of time waiting
* for a reset helps in various cases, so for a full-device reset
* we apply the opposite rule and wait if we want to. As we should
* always follow up a failed per-engine reset with a full device reset,
* being a little faster, stricter and more error prone for the
* atomic case seems an acceptable compromise.
*
* Unfortunately this leads to a bimodal routine, when the goal was
* to have a single reset function that worked for resetting any
* number of engines simultaneously.
*/
might_sleep_if(engine_mask == ALL_ENGINES);
/* If the power well sleeps during the reset, the reset
/*
* If the power well sleeps during the reset, the reset
* request may be dropped and never completes (causing -EIO).
*/
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
for (retry = 0; retry < 3; retry++) {
/* We stop engines, otherwise we might get failed reset and a
/*
* We stop engines, otherwise we might get failed reset and a
* dead gpu (on elk). Also as modern gpu as kbl can suffer
* from system hang if batchbuffer is progressing when
* the reset is issued, regardless of READY_TO_RESET ack.
@ -2060,9 +2180,11 @@ int intel_gpu_reset(struct drm_i915_private *dev_priv, unsigned engine_mask)
i915_stop_engines(dev_priv, engine_mask);
ret = -ENODEV;
if (reset)
if (reset) {
GEM_TRACE("engine_mask=%x\n", engine_mask);
ret = reset(dev_priv, engine_mask);
if (ret != -ETIMEDOUT)
}
if (ret != -ETIMEDOUT || engine_mask != ALL_ENGINES)
break;
cond_resched();
@ -2085,12 +2207,14 @@ bool intel_has_reset_engine(struct drm_i915_private *dev_priv)
int intel_reset_guc(struct drm_i915_private *dev_priv)
{
u32 guc_domain = INTEL_GEN(dev_priv) >= 11 ? GEN11_GRDOM_GUC :
GEN9_GRDOM_GUC;
int ret;
GEM_BUG_ON(!HAS_GUC(dev_priv));
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
ret = gen6_hw_domain_reset(dev_priv, GEN9_GRDOM_GUC);
ret = gen6_hw_domain_reset(dev_priv, guc_domain);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
return ret;

1
drivers/gpu/drm/i915/intel_uncore.h
View File

@ -140,6 +140,7 @@ struct intel_uncore {
void intel_uncore_sanitize(struct drm_i915_private *dev_priv);
void intel_uncore_init(struct drm_i915_private *dev_priv);
void intel_uncore_prune(struct drm_i915_private *dev_priv);
bool intel_uncore_unclaimed_mmio(struct drm_i915_private *dev_priv);
bool intel_uncore_arm_unclaimed_mmio_detection(struct drm_i915_private *dev_priv);
void intel_uncore_fini(struct drm_i915_private *dev_priv);

275
drivers/gpu/drm/i915/intel_wopcm.c 100644
View File

@ -0,0 +1,275 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2017-2018 Intel Corporation
*/
#include "intel_wopcm.h"
#include "i915_drv.h"
/**
* DOC: WOPCM Layout
*
* The layout of the WOPCM will be fixed after writing to GuC WOPCM size and
* offset registers whose values are calculated and determined by HuC/GuC
* firmware size and set of hardware requirements/restrictions as shown below:
*
* ::
*
* +=========> +====================+ <== WOPCM Top
* ^ | HW contexts RSVD |
* | +===> +====================+ <== GuC WOPCM Top
* | ^ | |
* | | | |
* | | | |
* | GuC | |
* | WOPCM | |
* | Size +--------------------+
* WOPCM | | GuC FW RSVD |
* | | +--------------------+
* | | | GuC Stack RSVD |
* | | +------------------- +
* | v | GuC WOPCM RSVD |
* | +===> +====================+ <== GuC WOPCM base
* | | WOPCM RSVD |
* | +------------------- + <== HuC Firmware Top
* v | HuC FW |
* +=========> +====================+ <== WOPCM Base
*
* GuC accessible WOPCM starts at GuC WOPCM base and ends at GuC WOPCM top.
* The top part of the WOPCM is reserved for hardware contexts (e.g. RC6
* context).
*/
/* Default WOPCM size 1MB. */
#define GEN9_WOPCM_SIZE (1024 * 1024)
/* 16KB WOPCM (RSVD WOPCM) is reserved from HuC firmware top. */
#define WOPCM_RESERVED_SIZE (16 * 1024)
/* 16KB reserved at the beginning of GuC WOPCM. */
#define GUC_WOPCM_RESERVED (16 * 1024)
/* 8KB from GUC_WOPCM_RESERVED is reserved for GuC stack. */
#define GUC_WOPCM_STACK_RESERVED (8 * 1024)
/* GuC WOPCM Offset value needs to be aligned to 16KB. */
#define GUC_WOPCM_OFFSET_ALIGNMENT (1UL << GUC_WOPCM_OFFSET_SHIFT)
/* 24KB at the end of WOPCM is reserved for RC6 CTX on BXT. */
#define BXT_WOPCM_RC6_CTX_RESERVED (24 * 1024)
/* 36KB WOPCM reserved at the end of WOPCM on CNL. */
#define CNL_WOPCM_HW_CTX_RESERVED (36 * 1024)
/* 128KB from GUC_WOPCM_RESERVED is reserved for FW on Gen9. */
#define GEN9_GUC_FW_RESERVED (128 * 1024)
#define GEN9_GUC_WOPCM_OFFSET (GUC_WOPCM_RESERVED + GEN9_GUC_FW_RESERVED)
/**
* intel_wopcm_init_early() - Early initialization of the WOPCM.
* @wopcm: pointer to intel_wopcm.
*
* Setup the size of WOPCM which will be used by later on WOPCM partitioning.
*/
void intel_wopcm_init_early(struct intel_wopcm *wopcm)
{
wopcm->size = GEN9_WOPCM_SIZE;
DRM_DEBUG_DRIVER("WOPCM size: %uKiB\n", wopcm->size / 1024);
}
static inline u32 context_reserved_size(struct drm_i915_private *i915)
{
if (IS_GEN9_LP(i915))
return BXT_WOPCM_RC6_CTX_RESERVED;
else if (INTEL_GEN(i915) >= 10)
return CNL_WOPCM_HW_CTX_RESERVED;
else
return 0;
}
static inline int gen9_check_dword_gap(u32 guc_wopcm_base, u32 guc_wopcm_size)
{
u32 offset;
/*
* GuC WOPCM size shall be at least a dword larger than the offset from
* WOPCM base (GuC WOPCM offset from WOPCM base + GEN9_GUC_WOPCM_OFFSET)
* due to hardware limitation on Gen9.
*/
offset = guc_wopcm_base + GEN9_GUC_WOPCM_OFFSET;
if (offset > guc_wopcm_size ||
(guc_wopcm_size - offset) < sizeof(u32)) {
DRM_ERROR("GuC WOPCM size %uKiB is too small. %uKiB needed.\n",
guc_wopcm_size / 1024,
(u32)(offset + sizeof(u32)) / 1024);
return -E2BIG;
}
return 0;
}
static inline int gen9_check_huc_fw_fits(u32 guc_wopcm_size, u32 huc_fw_size)
{
/*
* On Gen9 & CNL A0, hardware requires the total available GuC WOPCM
* size to be larger than or equal to HuC firmware size. Otherwise,
* firmware uploading would fail.
*/
if (huc_fw_size > guc_wopcm_size - GUC_WOPCM_RESERVED) {
DRM_ERROR("HuC FW (%uKiB) won't fit in GuC WOPCM (%uKiB).\n",
huc_fw_size / 1024,
(guc_wopcm_size - GUC_WOPCM_RESERVED) / 1024);
return -E2BIG;
}
return 0;
}
static inline int check_hw_restriction(struct drm_i915_private *i915,
u32 guc_wopcm_base, u32 guc_wopcm_size,
u32 huc_fw_size)
{
int err = 0;
if (IS_GEN9(i915))
err = gen9_check_dword_gap(guc_wopcm_base, guc_wopcm_size);
if (!err &&
(IS_GEN9(i915) || IS_CNL_REVID(i915, CNL_REVID_A0, CNL_REVID_A0)))
err = gen9_check_huc_fw_fits(guc_wopcm_size, huc_fw_size);
return err;
}
/**
* intel_wopcm_init() - Initialize the WOPCM structure.
* @wopcm: pointer to intel_wopcm.
*
* This function will partition WOPCM space based on GuC and HuC firmware sizes
* and will allocate max remaining for use by GuC. This function will also
* enforce platform dependent hardware restrictions on GuC WOPCM offset and
* size. It will fail the WOPCM init if any of these checks were failed, so that
* the following GuC firmware uploading would be aborted.
*
* Return: 0 on success, non-zero error code on failure.
*/
int intel_wopcm_init(struct intel_wopcm *wopcm)
{
struct drm_i915_private *i915 = wopcm_to_i915(wopcm);
u32 guc_fw_size = intel_uc_fw_get_upload_size(&i915->guc.fw);
u32 huc_fw_size = intel_uc_fw_get_upload_size(&i915->huc.fw);
u32 ctx_rsvd = context_reserved_size(i915);
u32 guc_wopcm_base;
u32 guc_wopcm_size;
u32 guc_wopcm_rsvd;
int err;
GEM_BUG_ON(!wopcm->size);
if (guc_fw_size >= wopcm->size) {
DRM_ERROR("GuC FW (%uKiB) is too big to fit in WOPCM.",
guc_fw_size / 1024);
return -E2BIG;
}
if (huc_fw_size >= wopcm->size) {
DRM_ERROR("HuC FW (%uKiB) is too big to fit in WOPCM.",
huc_fw_size / 1024);
return -E2BIG;
}
guc_wopcm_base = ALIGN(huc_fw_size + WOPCM_RESERVED_SIZE,
GUC_WOPCM_OFFSET_ALIGNMENT);
if ((guc_wopcm_base + ctx_rsvd) >= wopcm->size) {
DRM_ERROR("GuC WOPCM base (%uKiB) is too big.\n",
guc_wopcm_base / 1024);
return -E2BIG;
}
guc_wopcm_size = wopcm->size - guc_wopcm_base - ctx_rsvd;
guc_wopcm_size &= GUC_WOPCM_SIZE_MASK;
DRM_DEBUG_DRIVER("Calculated GuC WOPCM Region: [%uKiB, %uKiB)\n",
guc_wopcm_base / 1024, guc_wopcm_size / 1024);
guc_wopcm_rsvd = GUC_WOPCM_RESERVED + GUC_WOPCM_STACK_RESERVED;
if ((guc_fw_size + guc_wopcm_rsvd) > guc_wopcm_size) {
DRM_ERROR("Need %uKiB WOPCM for GuC, %uKiB available.\n",
(guc_fw_size + guc_wopcm_rsvd) / 1024,
guc_wopcm_size / 1024);
return -E2BIG;
}
err = check_hw_restriction(i915, guc_wopcm_base, guc_wopcm_size,
huc_fw_size);
if (err)
return err;
wopcm->guc.base = guc_wopcm_base;
wopcm->guc.size = guc_wopcm_size;
return 0;
}
static inline int write_and_verify(struct drm_i915_private *dev_priv,
i915_reg_t reg, u32 val, u32 mask,
u32 locked_bit)
{
u32 reg_val;
GEM_BUG_ON(val & ~mask);
I915_WRITE(reg, val);
reg_val = I915_READ(reg);
return (reg_val & mask) != (val | locked_bit) ? -EIO : 0;
}
/**
* intel_wopcm_init_hw() - Setup GuC WOPCM registers.
* @wopcm: pointer to intel_wopcm.
*
* Setup the GuC WOPCM size and offset registers with the calculated values. It
* will verify the register values to make sure the registers are locked with
* correct values.
*
* Return: 0 on success. -EIO if registers were locked with incorrect values.
*/
int intel_wopcm_init_hw(struct intel_wopcm *wopcm)
{
struct drm_i915_private *dev_priv = wopcm_to_i915(wopcm);
u32 huc_agent;
u32 mask;
int err;
if (!USES_GUC(dev_priv))
return 0;
GEM_BUG_ON(!HAS_GUC(dev_priv));
GEM_BUG_ON(!wopcm->guc.size);
GEM_BUG_ON(!wopcm->guc.base);
err = write_and_verify(dev_priv, GUC_WOPCM_SIZE, wopcm->guc.size,
GUC_WOPCM_SIZE_MASK | GUC_WOPCM_SIZE_LOCKED,
GUC_WOPCM_SIZE_LOCKED);
if (err)
goto err_out;
huc_agent = USES_HUC(dev_priv) ? HUC_LOADING_AGENT_GUC : 0;
mask = GUC_WOPCM_OFFSET_MASK | GUC_WOPCM_OFFSET_VALID | huc_agent;
err = write_and_verify(dev_priv, DMA_GUC_WOPCM_OFFSET,
wopcm->guc.base | huc_agent, mask,
GUC_WOPCM_OFFSET_VALID);
if (err)
goto err_out;
return 0;
err_out:
DRM_ERROR("Failed to init WOPCM registers:\n");
DRM_ERROR("DMA_GUC_WOPCM_OFFSET=%#x\n",
I915_READ(DMA_GUC_WOPCM_OFFSET));
DRM_ERROR("GUC_WOPCM_SIZE=%#x\n", I915_READ(GUC_WOPCM_SIZE));
return err;
}

31
drivers/gpu/drm/i915/intel_wopcm.h 100644
View File

@ -0,0 +1,31 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2017-2018 Intel Corporation
*/
#ifndef _INTEL_WOPCM_H_
#define _INTEL_WOPCM_H_
#include <linux/types.h>
/**
* struct intel_wopcm - Overall WOPCM info and WOPCM regions.
* @size: Size of overall WOPCM.
* @guc: GuC WOPCM Region info.
* @guc.base: GuC WOPCM base which is offset from WOPCM base.
* @guc.size: Size of the GuC WOPCM region.
*/
struct intel_wopcm {
u32 size;
struct {
u32 base;
u32 size;
} guc;
};
void intel_wopcm_init_early(struct intel_wopcm *wopcm);
int intel_wopcm_init(struct intel_wopcm *wopcm);
int intel_wopcm_init_hw(struct intel_wopcm *wopcm);
#endif

856
drivers/gpu/drm/i915/intel_workarounds.c 100644
View File

@ -0,0 +1,856 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2014-2018 Intel Corporation
*/
#include "i915_drv.h"
#include "intel_workarounds.h"
/**
* DOC: Hardware workarounds
*
* This file is intended as a central place to implement most [1]_ of the
* required workarounds for hardware to work as originally intended. They fall
* in five basic categories depending on how/when they are applied:
*
* - Workarounds that touch registers that are saved/restored to/from the HW
* context image. The list is emitted (via Load Register Immediate commands)
* everytime a new context is created.
* - GT workarounds. The list of these WAs is applied whenever these registers
* revert to default values (on GPU reset, suspend/resume [2]_, etc..).
* - Display workarounds. The list is applied during display clock-gating
* initialization.
* - Workarounds that whitelist a privileged register, so that UMDs can manage
* them directly. This is just a special case of a MMMIO workaround (as we
* write the list of these to/be-whitelisted registers to some special HW
* registers).
* - Workaround batchbuffers, that get executed automatically by the hardware
* on every HW context restore.
*
* .. [1] Please notice that there are other WAs that, due to their nature,
* cannot be applied from a central place. Those are peppered around the rest
* of the code, as needed.
*
* .. [2] Technically, some registers are powercontext saved & restored, so they
* survive a suspend/resume. In practice, writing them again is not too
* costly and simplifies things. We can revisit this in the future.
*
* Layout
* ''''''
*
* Keep things in this file ordered by WA type, as per the above (context, GT,
* display, register whitelist, batchbuffer). Then, inside each type, keep the
* following order:
*
* - Infrastructure functions and macros
* - WAs per platform in standard gen/chrono order
* - Public functions to init or apply the given workaround type.
*/
static int wa_add(struct drm_i915_private *dev_priv,
i915_reg_t addr,
const u32 mask, const u32 val)
{
const unsigned int idx = dev_priv->workarounds.count;
if (WARN_ON(idx >= I915_MAX_WA_REGS))
return -ENOSPC;
dev_priv->workarounds.reg[idx].addr = addr;
dev_priv->workarounds.reg[idx].value = val;
dev_priv->workarounds.reg[idx].mask = mask;
dev_priv->workarounds.count++;
return 0;
}
#define WA_REG(addr, mask, val) do { \
const int r = wa_add(dev_priv, (addr), (mask), (val)); \
if (r) \
return r; \
} while (0)
#define WA_SET_BIT_MASKED(addr, mask) \
WA_REG(addr, (mask), _MASKED_BIT_ENABLE(mask))
#define WA_CLR_BIT_MASKED(addr, mask) \
WA_REG(addr, (mask), _MASKED_BIT_DISABLE(mask))
#define WA_SET_FIELD_MASKED(addr, mask, value) \
WA_REG(addr, (mask), _MASKED_FIELD(mask, value))
static int gen8_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
WA_SET_BIT_MASKED(INSTPM, INSTPM_FORCE_ORDERING);
/* WaDisableAsyncFlipPerfMode:bdw,chv */
WA_SET_BIT_MASKED(MI_MODE, ASYNC_FLIP_PERF_DISABLE);
/* WaDisablePartialInstShootdown:bdw,chv */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
/* Use Force Non-Coherent whenever executing a 3D context. This is a
* workaround for for a possible hang in the unlikely event a TLB
* invalidation occurs during a PSD flush.
*/
/* WaForceEnableNonCoherent:bdw,chv */
/* WaHdcDisableFetchWhenMasked:bdw,chv */
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_DONOT_FETCH_MEM_WHEN_MASKED |
HDC_FORCE_NON_COHERENT);
/* From the Haswell PRM, Command Reference: Registers, CACHE_MODE_0:
* "The Hierarchical Z RAW Stall Optimization allows non-overlapping
* polygons in the same 8x4 pixel/sample area to be processed without
* stalling waiting for the earlier ones to write to Hierarchical Z
* buffer."
*
* This optimization is off by default for BDW and CHV; turn it on.
*/
WA_CLR_BIT_MASKED(CACHE_MODE_0_GEN7, HIZ_RAW_STALL_OPT_DISABLE);
/* Wa4x4STCOptimizationDisable:bdw,chv */
WA_SET_BIT_MASKED(CACHE_MODE_1, GEN8_4x4_STC_OPTIMIZATION_DISABLE);
/*
* BSpec recommends 8x4 when MSAA is used,
* however in practice 16x4 seems fastest.
*
* Note that PS/WM thread counts depend on the WIZ hashing
* disable bit, which we don't touch here, but it's good
* to keep in mind (see 3DSTATE_PS and 3DSTATE_WM).
*/
WA_SET_FIELD_MASKED(GEN7_GT_MODE,
GEN6_WIZ_HASHING_MASK,
GEN6_WIZ_HASHING_16x4);
return 0;
}
static int bdw_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int ret;
ret = gen8_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
/* WaDisableThreadStallDopClockGating:bdw (pre-production) */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
/* WaDisableDopClockGating:bdw
*
* Also see the related UCGTCL1 write in broadwell_init_clock_gating()
* to disable EUTC clock gating.
*/
WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2,
DOP_CLOCK_GATING_DISABLE);
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
GEN8_SAMPLER_POWER_BYPASS_DIS);
WA_SET_BIT_MASKED(HDC_CHICKEN0,
/* WaForceContextSaveRestoreNonCoherent:bdw */
HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
/* WaDisableFenceDestinationToSLM:bdw (pre-prod) */
(IS_BDW_GT3(dev_priv) ? HDC_FENCE_DEST_SLM_DISABLE : 0));
return 0;
}
static int chv_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int ret;
ret = gen8_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
/* WaDisableThreadStallDopClockGating:chv */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, STALL_DOP_GATING_DISABLE);
/* Improve HiZ throughput on CHV. */
WA_SET_BIT_MASKED(HIZ_CHICKEN, CHV_HZ_8X8_MODE_IN_1X);
return 0;
}
static int gen9_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
if (HAS_LLC(dev_priv)) {
/* WaCompressedResourceSamplerPbeMediaNewHashMode:skl,kbl
*
* Must match Display Engine. See
* WaCompressedResourceDisplayNewHashMode.
*/
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN9_PBE_COMPRESSED_HASH_SELECTION);
WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
GEN9_SAMPLER_HASH_COMPRESSED_READ_ADDR);
}
/* WaClearFlowControlGpgpuContextSave:skl,bxt,kbl,glk,cfl */
/* WaDisablePartialInstShootdown:skl,bxt,kbl,glk,cfl */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
FLOW_CONTROL_ENABLE |
PARTIAL_INSTRUCTION_SHOOTDOWN_DISABLE);
/* Syncing dependencies between camera and graphics:skl,bxt,kbl */
if (!IS_COFFEELAKE(dev_priv))
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
GEN9_DISABLE_OCL_OOB_SUPPRESS_LOGIC);
/* WaEnableYV12BugFixInHalfSliceChicken7:skl,bxt,kbl,glk,cfl */
/* WaEnableSamplerGPGPUPreemptionSupport:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN7,
GEN9_ENABLE_YV12_BUGFIX |
GEN9_ENABLE_GPGPU_PREEMPTION);
/* Wa4x4STCOptimizationDisable:skl,bxt,kbl,glk,cfl */
/* WaDisablePartialResolveInVc:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(CACHE_MODE_1,
GEN8_4x4_STC_OPTIMIZATION_DISABLE |
GEN9_PARTIAL_RESOLVE_IN_VC_DISABLE);
/* WaCcsTlbPrefetchDisable:skl,bxt,kbl,glk,cfl */
WA_CLR_BIT_MASKED(GEN9_HALF_SLICE_CHICKEN5,
GEN9_CCS_TLB_PREFETCH_ENABLE);
/* WaForceContextSaveRestoreNonCoherent:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT |
HDC_FORCE_CSR_NON_COHERENT_OVR_DISABLE);
/* WaForceEnableNonCoherent and WaDisableHDCInvalidation are
* both tied to WaForceContextSaveRestoreNonCoherent
* in some hsds for skl. We keep the tie for all gen9. The
* documentation is a bit hazy and so we want to get common behaviour,
* even though there is no clear evidence we would need both on kbl/bxt.
* This area has been source of system hangs so we play it safe
* and mimic the skl regardless of what bspec says.
*
* Use Force Non-Coherent whenever executing a 3D context. This
* is a workaround for a possible hang in the unlikely event
* a TLB invalidation occurs during a PSD flush.
*/
/* WaForceEnableNonCoherent:skl,bxt,kbl,cfl */
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FORCE_NON_COHERENT);
/* WaDisableSamplerPowerBypassForSOPingPong:skl,bxt,kbl,cfl */
if (IS_SKYLAKE(dev_priv) ||
IS_KABYLAKE(dev_priv) ||
IS_COFFEELAKE(dev_priv))
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3,
GEN8_SAMPLER_POWER_BYPASS_DIS);
/* WaDisableSTUnitPowerOptimization:skl,bxt,kbl,glk,cfl */
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN2, GEN8_ST_PO_DISABLE);
/*
* Supporting preemption with fine-granularity requires changes in the
* batch buffer programming. Since we can't break old userspace, we
* need to set our default preemption level to safe value. Userspace is
* still able to use more fine-grained preemption levels, since in
* WaEnablePreemptionGranularityControlByUMD we're whitelisting the
* per-ctx register. As such, WaDisable{3D,GPGPU}MidCmdPreemption are
* not real HW workarounds, but merely a way to start using preemption
* while maintaining old contract with userspace.
*/
/* WaDisable3DMidCmdPreemption:skl,bxt,glk,cfl,[cnl] */
WA_CLR_BIT_MASKED(GEN8_CS_CHICKEN1, GEN9_PREEMPT_3D_OBJECT_LEVEL);
/* WaDisableGPGPUMidCmdPreemption:skl,bxt,blk,cfl,[cnl] */
WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1,
GEN9_PREEMPT_GPGPU_LEVEL_MASK,
GEN9_PREEMPT_GPGPU_COMMAND_LEVEL);
return 0;
}
static int skl_tune_iz_hashing(struct drm_i915_private *dev_priv)
{
u8 vals[3] = { 0, 0, 0 };
unsigned int i;
for (i = 0; i < 3; i++) {
u8 ss;
/*
* Only consider slices where one, and only one, subslice has 7
* EUs
*/
if (!is_power_of_2(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]))
continue;
/*
* subslice_7eu[i] != 0 (because of the check above) and
* ss_max == 4 (maximum number of subslices possible per slice)
*
* -> 0 <= ss <= 3;
*/
ss = ffs(INTEL_INFO(dev_priv)->sseu.subslice_7eu[i]) - 1;
vals[i] = 3 - ss;
}
if (vals[0] == 0 && vals[1] == 0 && vals[2] == 0)
return 0;
/* Tune IZ hashing. See intel_device_info_runtime_init() */
WA_SET_FIELD_MASKED(GEN7_GT_MODE,
GEN9_IZ_HASHING_MASK(2) |
GEN9_IZ_HASHING_MASK(1) |
GEN9_IZ_HASHING_MASK(0),
GEN9_IZ_HASHING(2, vals[2]) |
GEN9_IZ_HASHING(1, vals[1]) |
GEN9_IZ_HASHING(0, vals[0]));
return 0;
}
static int skl_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int ret;
ret = gen9_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
return skl_tune_iz_hashing(dev_priv);
}
static int bxt_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int ret;
ret = gen9_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
/* WaDisableThreadStallDopClockGating:bxt */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN,
STALL_DOP_GATING_DISABLE);
/* WaToEnableHwFixForPushConstHWBug:bxt */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
return 0;
}
static int kbl_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int ret;
ret = gen9_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
/* WaDisableFenceDestinationToSLM:kbl (pre-prod) */
if (IS_KBL_REVID(dev_priv, KBL_REVID_A0, KBL_REVID_A0))
WA_SET_BIT_MASKED(HDC_CHICKEN0,
HDC_FENCE_DEST_SLM_DISABLE);
/* WaToEnableHwFixForPushConstHWBug:kbl */
if (IS_KBL_REVID(dev_priv, KBL_REVID_C0, REVID_FOREVER))
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableSbeCacheDispatchPortSharing:kbl */
WA_SET_BIT_MASKED(GEN7_HALF_SLICE_CHICKEN1,
GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
return 0;
}
static int glk_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int ret;
ret = gen9_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
/* WaToEnableHwFixForPushConstHWBug:glk */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
return 0;
}
static int cfl_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int ret;
ret = gen9_ctx_workarounds_init(dev_priv);
if (ret)
return ret;
/* WaToEnableHwFixForPushConstHWBug:cfl */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableSbeCacheDispatchPortSharing:cfl */
WA_SET_BIT_MASKED(GEN7_HALF_SLICE_CHICKEN1,
GEN7_SBE_SS_CACHE_DISPATCH_PORT_SHARING_DISABLE);
return 0;
}
static int cnl_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
/* WaForceContextSaveRestoreNonCoherent:cnl */
WA_SET_BIT_MASKED(CNL_HDC_CHICKEN0,
HDC_FORCE_CONTEXT_SAVE_RESTORE_NON_COHERENT);
/* WaThrottleEUPerfToAvoidTDBackPressure:cnl(pre-prod) */
if (IS_CNL_REVID(dev_priv, CNL_REVID_B0, CNL_REVID_B0))
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, THROTTLE_12_5);
/* WaDisableReplayBufferBankArbitrationOptimization:cnl */
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_SBE_DISABLE_REPLAY_BUF_OPTIMIZATION);
/* WaDisableEnhancedSBEVertexCaching:cnl (pre-prod) */
if (IS_CNL_REVID(dev_priv, 0, CNL_REVID_B0))
WA_SET_BIT_MASKED(COMMON_SLICE_CHICKEN2,
GEN8_CSC2_SBE_VUE_CACHE_CONSERVATIVE);
/* WaPushConstantDereferenceHoldDisable:cnl */
WA_SET_BIT_MASKED(GEN7_ROW_CHICKEN2, PUSH_CONSTANT_DEREF_DISABLE);
/* FtrEnableFastAnisoL1BankingFix:cnl */
WA_SET_BIT_MASKED(HALF_SLICE_CHICKEN3, CNL_FAST_ANISO_L1_BANKING_FIX);
/* WaDisable3DMidCmdPreemption:cnl */
WA_CLR_BIT_MASKED(GEN8_CS_CHICKEN1, GEN9_PREEMPT_3D_OBJECT_LEVEL);
/* WaDisableGPGPUMidCmdPreemption:cnl */
WA_SET_FIELD_MASKED(GEN8_CS_CHICKEN1,
GEN9_PREEMPT_GPGPU_LEVEL_MASK,
GEN9_PREEMPT_GPGPU_COMMAND_LEVEL);
/* WaDisableEarlyEOT:cnl */
WA_SET_BIT_MASKED(GEN8_ROW_CHICKEN, DISABLE_EARLY_EOT);
return 0;
}
int intel_ctx_workarounds_init(struct drm_i915_private *dev_priv)
{
int err = 0;
dev_priv->workarounds.count = 0;
if (INTEL_GEN(dev_priv) < 8)
err = 0;
else if (IS_BROADWELL(dev_priv))
err = bdw_ctx_workarounds_init(dev_priv);
else if (IS_CHERRYVIEW(dev_priv))
err = chv_ctx_workarounds_init(dev_priv);
else if (IS_SKYLAKE(dev_priv))
err = skl_ctx_workarounds_init(dev_priv);
else if (IS_BROXTON(dev_priv))
err = bxt_ctx_workarounds_init(dev_priv);
else if (IS_KABYLAKE(dev_priv))
err = kbl_ctx_workarounds_init(dev_priv);
else if (IS_GEMINILAKE(dev_priv))
err = glk_ctx_workarounds_init(dev_priv);
else if (IS_COFFEELAKE(dev_priv))
err = cfl_ctx_workarounds_init(dev_priv);
else if (IS_CANNONLAKE(dev_priv))
err = cnl_ctx_workarounds_init(dev_priv);
else
MISSING_CASE(INTEL_GEN(dev_priv));
if (err)
return err;
DRM_DEBUG_DRIVER("Number of context specific w/a: %d\n",
dev_priv->workarounds.count);
return 0;
}
int intel_ctx_workarounds_emit(struct i915_request *rq)
{
struct i915_workarounds *w = &rq->i915->workarounds;
u32 *cs;
int ret, i;
if (w->count == 0)
return 0;
ret = rq->engine->emit_flush(rq, EMIT_BARRIER);
if (ret)
return ret;
cs = intel_ring_begin(rq, (w->count * 2 + 2));
if (IS_ERR(cs))
return PTR_ERR(cs);
*cs++ = MI_LOAD_REGISTER_IMM(w->count);
for (i = 0; i < w->count; i++) {
*cs++ = i915_mmio_reg_offset(w->reg[i].addr);
*cs++ = w->reg[i].value;
}
*cs++ = MI_NOOP;
intel_ring_advance(rq, cs);
ret = rq->engine->emit_flush(rq, EMIT_BARRIER);
if (ret)
return ret;
return 0;
}
static void bdw_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
}
static void chv_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
}
static void gen9_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
/* WaContextSwitchWithConcurrentTLBInvalidate:skl,bxt,kbl,glk,cfl */
I915_WRITE(GEN9_CSFE_CHICKEN1_RCS,
_MASKED_BIT_ENABLE(GEN9_PREEMPT_GPGPU_SYNC_SWITCH_DISABLE));
/* WaEnableLbsSlaRetryTimerDecrement:skl,bxt,kbl,glk,cfl */
I915_WRITE(BDW_SCRATCH1, I915_READ(BDW_SCRATCH1) |
GEN9_LBS_SLA_RETRY_TIMER_DECREMENT_ENABLE);
/* WaDisableKillLogic:bxt,skl,kbl */
if (!IS_COFFEELAKE(dev_priv))
I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
ECOCHK_DIS_TLB);
if (HAS_LLC(dev_priv)) {
/* WaCompressedResourceSamplerPbeMediaNewHashMode:skl,kbl
*
* Must match Display Engine. See
* WaCompressedResourceDisplayNewHashMode.
*/
I915_WRITE(MMCD_MISC_CTRL,
I915_READ(MMCD_MISC_CTRL) |
MMCD_PCLA |
MMCD_HOTSPOT_EN);
}
/* WaDisableHDCInvalidation:skl,bxt,kbl,cfl */
I915_WRITE(GAM_ECOCHK, I915_READ(GAM_ECOCHK) |
BDW_DISABLE_HDC_INVALIDATION);
/* WaProgramL3SqcReg1DefaultForPerf:bxt,glk */
if (IS_GEN9_LP(dev_priv)) {
u32 val = I915_READ(GEN8_L3SQCREG1);
val &= ~L3_PRIO_CREDITS_MASK;
val |= L3_GENERAL_PRIO_CREDITS(62) | L3_HIGH_PRIO_CREDITS(2);
I915_WRITE(GEN8_L3SQCREG1, val);
}
/* WaOCLCoherentLineFlush:skl,bxt,kbl,cfl */
I915_WRITE(GEN8_L3SQCREG4,
I915_READ(GEN8_L3SQCREG4) | GEN8_LQSC_FLUSH_COHERENT_LINES);
/* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,[cnl] */
I915_WRITE(GEN7_FF_SLICE_CS_CHICKEN1,
_MASKED_BIT_ENABLE(GEN9_FFSC_PERCTX_PREEMPT_CTRL));
}
static void skl_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
gen9_gt_workarounds_apply(dev_priv);
/* WaEnableGapsTsvCreditFix:skl */
I915_WRITE(GEN8_GARBCNTL,
I915_READ(GEN8_GARBCNTL) | GEN9_GAPS_TSV_CREDIT_DISABLE);
/* WaDisableGafsUnitClkGating:skl */
I915_WRITE(GEN7_UCGCTL4,
I915_READ(GEN7_UCGCTL4) | GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
/* WaInPlaceDecompressionHang:skl */
if (IS_SKL_REVID(dev_priv, SKL_REVID_H0, REVID_FOREVER))
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
}
static void bxt_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
gen9_gt_workarounds_apply(dev_priv);
/* WaDisablePooledEuLoadBalancingFix:bxt */
I915_WRITE(FF_SLICE_CS_CHICKEN2,
_MASKED_BIT_ENABLE(GEN9_POOLED_EU_LOAD_BALANCING_FIX_DISABLE));
/* WaInPlaceDecompressionHang:bxt */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
}
static void kbl_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
gen9_gt_workarounds_apply(dev_priv);
/* WaEnableGapsTsvCreditFix:kbl */
I915_WRITE(GEN8_GARBCNTL,
I915_READ(GEN8_GARBCNTL) | GEN9_GAPS_TSV_CREDIT_DISABLE);
/* WaDisableDynamicCreditSharing:kbl */
if (IS_KBL_REVID(dev_priv, 0, KBL_REVID_B0))
I915_WRITE(GAMT_CHKN_BIT_REG,
I915_READ(GAMT_CHKN_BIT_REG) |
GAMT_CHKN_DISABLE_DYNAMIC_CREDIT_SHARING);
/* WaDisableGafsUnitClkGating:kbl */
I915_WRITE(GEN7_UCGCTL4,
I915_READ(GEN7_UCGCTL4) | GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
/* WaInPlaceDecompressionHang:kbl */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
}
static void glk_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
gen9_gt_workarounds_apply(dev_priv);
}
static void cfl_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
gen9_gt_workarounds_apply(dev_priv);
/* WaEnableGapsTsvCreditFix:cfl */
I915_WRITE(GEN8_GARBCNTL,
I915_READ(GEN8_GARBCNTL) | GEN9_GAPS_TSV_CREDIT_DISABLE);
/* WaDisableGafsUnitClkGating:cfl */
I915_WRITE(GEN7_UCGCTL4,
I915_READ(GEN7_UCGCTL4) | GEN8_EU_GAUNIT_CLOCK_GATE_DISABLE);
/* WaInPlaceDecompressionHang:cfl */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
}
static void cnl_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
/* WaDisableI2mCycleOnWRPort:cnl (pre-prod) */
if (IS_CNL_REVID(dev_priv, CNL_REVID_B0, CNL_REVID_B0))
I915_WRITE(GAMT_CHKN_BIT_REG,
I915_READ(GAMT_CHKN_BIT_REG) |
GAMT_CHKN_DISABLE_I2M_CYCLE_ON_WR_PORT);
/* WaInPlaceDecompressionHang:cnl */
I915_WRITE(GEN9_GAMT_ECO_REG_RW_IA,
I915_READ(GEN9_GAMT_ECO_REG_RW_IA) |
GAMT_ECO_ENABLE_IN_PLACE_DECOMPRESS);
/* WaEnablePreemptionGranularityControlByUMD:cnl */
I915_WRITE(GEN7_FF_SLICE_CS_CHICKEN1,
_MASKED_BIT_ENABLE(GEN9_FFSC_PERCTX_PREEMPT_CTRL));
}
void intel_gt_workarounds_apply(struct drm_i915_private *dev_priv)
{
if (INTEL_GEN(dev_priv) < 8)
return;
else if (IS_BROADWELL(dev_priv))
bdw_gt_workarounds_apply(dev_priv);
else if (IS_CHERRYVIEW(dev_priv))
chv_gt_workarounds_apply(dev_priv);
else if (IS_SKYLAKE(dev_priv))
skl_gt_workarounds_apply(dev_priv);
else if (IS_BROXTON(dev_priv))
bxt_gt_workarounds_apply(dev_priv);
else if (IS_KABYLAKE(dev_priv))
kbl_gt_workarounds_apply(dev_priv);
else if (IS_GEMINILAKE(dev_priv))
glk_gt_workarounds_apply(dev_priv);
else if (IS_COFFEELAKE(dev_priv))
cfl_gt_workarounds_apply(dev_priv);
else if (IS_CANNONLAKE(dev_priv))
cnl_gt_workarounds_apply(dev_priv);
else
MISSING_CASE(INTEL_GEN(dev_priv));
}
static int wa_ring_whitelist_reg(struct intel_engine_cs *engine,
i915_reg_t reg)
{
struct drm_i915_private *dev_priv = engine->i915;
struct i915_workarounds *wa = &dev_priv->workarounds;
const unsigned int index = wa->hw_whitelist_count[engine->id];
if (WARN_ON(index >= RING_MAX_NONPRIV_SLOTS))
return -EINVAL;
I915_WRITE(RING_FORCE_TO_NONPRIV(engine->mmio_base, index),
i915_mmio_reg_offset(reg));
wa->hw_whitelist_count[engine->id]++;
return 0;
}
static int bdw_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
return 0;
}
static int chv_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
return 0;
}
static int gen9_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
int ret;
/* WaVFEStateAfterPipeControlwithMediaStateClear:skl,bxt,glk,cfl */
ret = wa_ring_whitelist_reg(engine, GEN9_CTX_PREEMPT_REG);
if (ret)
return ret;
/* WaEnablePreemptionGranularityControlByUMD:skl,bxt,kbl,cfl,[cnl] */
ret = wa_ring_whitelist_reg(engine, GEN8_CS_CHICKEN1);
if (ret)
return ret;
/* WaAllowUMDToModifyHDCChicken1:skl,bxt,kbl,glk,cfl */
ret = wa_ring_whitelist_reg(engine, GEN8_HDC_CHICKEN1);
if (ret)
return ret;
return 0;
}
static int skl_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
int ret;
ret = gen9_whitelist_workarounds_apply(engine);
if (ret)
return ret;
/* WaDisableLSQCROPERFforOCL:skl */
ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
if (ret)
return ret;
return 0;
}
static int bxt_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
int ret;
ret = gen9_whitelist_workarounds_apply(engine);
if (ret)
return ret;
return 0;
}
static int kbl_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
int ret;
ret = gen9_whitelist_workarounds_apply(engine);
if (ret)
return ret;
/* WaDisableLSQCROPERFforOCL:kbl */
ret = wa_ring_whitelist_reg(engine, GEN8_L3SQCREG4);
if (ret)
return ret;
return 0;
}
static int glk_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
int ret;
ret = gen9_whitelist_workarounds_apply(engine);
if (ret)
return ret;
/* WA #0862: Userspace has to set "Barrier Mode" to avoid hangs. */
ret = wa_ring_whitelist_reg(engine, GEN9_SLICE_COMMON_ECO_CHICKEN1);
if (ret)
return ret;
return 0;
}
static int cfl_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
int ret;
ret = gen9_whitelist_workarounds_apply(engine);
if (ret)
return ret;
return 0;
}
static int cnl_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
int ret;
/* WaEnablePreemptionGranularityControlByUMD:cnl */
ret = wa_ring_whitelist_reg(engine, GEN8_CS_CHICKEN1);
if (ret)
return ret;
return 0;
}
int intel_whitelist_workarounds_apply(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
int err = 0;
WARN_ON(engine->id != RCS);
dev_priv->workarounds.hw_whitelist_count[engine->id] = 0;
if (INTEL_GEN(dev_priv) < 8)
err = 0;
else if (IS_BROADWELL(dev_priv))
err = bdw_whitelist_workarounds_apply(engine);
else if (IS_CHERRYVIEW(dev_priv))
err = chv_whitelist_workarounds_apply(engine);
else if (IS_SKYLAKE(dev_priv))
err = skl_whitelist_workarounds_apply(engine);
else if (IS_BROXTON(dev_priv))
err = bxt_whitelist_workarounds_apply(engine);
else if (IS_KABYLAKE(dev_priv))
err = kbl_whitelist_workarounds_apply(engine);
else if (IS_GEMINILAKE(dev_priv))
err = glk_whitelist_workarounds_apply(engine);
else if (IS_COFFEELAKE(dev_priv))
err = cfl_whitelist_workarounds_apply(engine);
else if (IS_CANNONLAKE(dev_priv))
err = cnl_whitelist_workarounds_apply(engine);
else
MISSING_CASE(INTEL_GEN(dev_priv));
if (err)
return err;
DRM_DEBUG_DRIVER("%s: Number of whitelist w/a: %d\n", engine->name,
dev_priv->workarounds.hw_whitelist_count[engine->id]);
return 0;
}

17
drivers/gpu/drm/i915/intel_workarounds.h 100644
View File

@ -0,0 +1,17 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2014-2018 Intel Corporation
*/
#ifndef _I915_WORKAROUNDS_H_
#define _I915_WORKAROUNDS_H_
int intel_ctx_workarounds_init(struct drm_i915_private *dev_priv);
int intel_ctx_workarounds_emit(struct i915_request *rq);
void intel_gt_workarounds_apply(struct drm_i915_private *dev_priv);
int intel_whitelist_workarounds_apply(struct intel_engine_cs *engine);
#endif

1
drivers/gpu/drm/i915/selftests/i915_live_selftests.h
View File

@ -20,4 +20,5 @@ selftest(evict, i915_gem_evict_live_selftests)
selftest(hugepages, i915_gem_huge_page_live_selftests)
selftest(contexts, i915_gem_context_live_selftests)
selftest(hangcheck, intel_hangcheck_live_selftests)
selftest(execlists, intel_execlists_live_selftests)
selftest(guc, intel_guc_live_selftest)

1
drivers/gpu/drm/i915/selftests/i915_mock_selftests.h
View File

@ -14,6 +14,7 @@ selftest(fence, i915_sw_fence_mock_selftests)
selftest(scatterlist, scatterlist_mock_selftests)
selftest(syncmap, i915_syncmap_mock_selftests)
selftest(uncore, intel_uncore_mock_selftests)
selftest(engine, intel_engine_cs_mock_selftests)
selftest(breadcrumbs, intel_breadcrumbs_mock_selftests)
selftest(timelines, i915_gem_timeline_mock_selftests)
selftest(requests, i915_request_mock_selftests)

58
drivers/gpu/drm/i915/selftests/intel_engine_cs.c 100644
View File

@ -0,0 +1,58 @@
/*
* SPDX-License-Identifier: GPL-2.0
*
* Copyright © 2018 Intel Corporation
*/
#include "../i915_selftest.h"
static int intel_mmio_bases_check(void *arg)
{
int i, j;
for (i = 0; i < ARRAY_SIZE(intel_engines); i++) {
const struct engine_info *info = &intel_engines[i];
char name[INTEL_ENGINE_CS_MAX_NAME];
u8 prev = U8_MAX;
__sprint_engine_name(name, info);
for (j = 0; j < MAX_MMIO_BASES; j++) {
u8 gen = info->mmio_bases[j].gen;
u32 base = info->mmio_bases[j].base;
if (gen >= prev) {
pr_err("%s: %s: mmio base for gen %x "
"is before the one for gen %x\n",
__func__, name, prev, gen);
return -EINVAL;
}
if (gen == 0)
break;
if (!base) {
pr_err("%s: %s: invalid mmio base (%x) "
"for gen %x at entry %u\n",
__func__, name, base, gen, j);
return -EINVAL;
}
prev = gen;
}
pr_info("%s: min gen supported for %s = %d\n",
__func__, name, prev);
}
return 0;
}
int intel_engine_cs_mock_selftests(void)
{
static const struct i915_subtest tests[] = {
SUBTEST(intel_mmio_bases_check),
};
return i915_subtests(tests, NULL);
}

314
drivers/gpu/drm/i915/selftests/intel_hangcheck.c
View File

@ -25,6 +25,7 @@
#include <linux/kthread.h>
#include "../i915_selftest.h"
#include "i915_random.h"
#include "mock_context.h"
#include "mock_drm.h"
@ -260,8 +261,11 @@ static void wedge_me(struct work_struct *work)
{
struct wedge_me *w = container_of(work, typeof(*w), work.work);
pr_err("%pS timed out, cancelling all further testing.\n",
w->symbol);
pr_err("%pS timed out, cancelling all further testing.\n", w->symbol);
GEM_TRACE("%pS timed out.\n", w->symbol);
GEM_TRACE_DUMP();
i915_gem_set_wedged(w->i915);
}
@ -318,7 +322,7 @@ static void hang_fini(struct hang *h)
flush_test(h->i915, I915_WAIT_LOCKED);
}
static bool wait_for_hang(struct hang *h, struct i915_request *rq)
static bool wait_until_running(struct hang *h, struct i915_request *rq)
{
return !(wait_for_us(i915_seqno_passed(hws_seqno(h, rq),
rq->fence.seqno),
@ -433,7 +437,7 @@ static int igt_global_reset(void *arg)
mutex_lock(&i915->drm.struct_mutex);
reset_count = i915_reset_count(&i915->gpu_error);
i915_reset(i915, I915_RESET_QUIET);
i915_reset(i915, ALL_ENGINES, NULL);
if (i915_reset_count(&i915->gpu_error) == reset_count) {
pr_err("No GPU reset recorded!\n");
@ -483,6 +487,8 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active)
set_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
do {
u32 seqno = intel_engine_get_seqno(engine);
if (active) {
struct i915_request *rq;
@ -498,7 +504,7 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active)
__i915_request_add(rq, true);
mutex_unlock(&i915->drm.struct_mutex);
if (!wait_for_hang(&h, rq)) {
if (!wait_until_running(&h, rq)) {
struct drm_printer p = drm_info_printer(i915->drm.dev);
pr_err("%s: Failed to start request %x, at %x\n",
@ -511,14 +517,12 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active)
break;
}
GEM_BUG_ON(!rq->global_seqno);
seqno = rq->global_seqno - 1;
i915_request_put(rq);
}
engine->hangcheck.stalled = true;
engine->hangcheck.seqno =
intel_engine_get_seqno(engine);
err = i915_reset_engine(engine, I915_RESET_QUIET);
err = i915_reset_engine(engine, NULL);
if (err) {
pr_err("i915_reset_engine failed\n");
break;
@ -538,8 +542,6 @@ static int __igt_reset_engine(struct drm_i915_private *i915, bool active)
err = -EINVAL;
break;
}
engine->hangcheck.stalled = false;
} while (time_before(jiffies, end_time));
clear_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
@ -573,11 +575,25 @@ static int igt_reset_active_engine(void *arg)
return __igt_reset_engine(arg, true);
}
struct active_engine {
struct task_struct *task;
struct intel_engine_cs *engine;
unsigned long resets;
unsigned int flags;
};
#define TEST_ACTIVE BIT(0)
#define TEST_OTHERS BIT(1)
#define TEST_SELF BIT(2)
#define TEST_PRIORITY BIT(3)
static int active_engine(void *data)
{
struct intel_engine_cs *engine = data;
struct i915_request *rq[2] = {};
struct i915_gem_context *ctx[2];
I915_RND_STATE(prng);
struct active_engine *arg = data;
struct intel_engine_cs *engine = arg->engine;
struct i915_request *rq[8] = {};
struct i915_gem_context *ctx[ARRAY_SIZE(rq)];
struct drm_file *file;
unsigned long count = 0;
int err = 0;
@ -586,25 +602,20 @@ static int active_engine(void *data)
if (IS_ERR(file))
return PTR_ERR(file);
mutex_lock(&engine->i915->drm.struct_mutex);
ctx[0] = live_context(engine->i915, file);
mutex_unlock(&engine->i915->drm.struct_mutex);
if (IS_ERR(ctx[0])) {
err = PTR_ERR(ctx[0]);
goto err_file;
}
mutex_lock(&engine->i915->drm.struct_mutex);
ctx[1] = live_context(engine->i915, file);
mutex_unlock(&engine->i915->drm.struct_mutex);
if (IS_ERR(ctx[1])) {
err = PTR_ERR(ctx[1]);
i915_gem_context_put(ctx[0]);
goto err_file;
for (count = 0; count < ARRAY_SIZE(ctx); count++) {
mutex_lock(&engine->i915->drm.struct_mutex);
ctx[count] = live_context(engine->i915, file);
mutex_unlock(&engine->i915->drm.struct_mutex);
if (IS_ERR(ctx[count])) {
err = PTR_ERR(ctx[count]);
while (--count)
i915_gem_context_put(ctx[count]);
goto err_file;
}
}
while (!kthread_should_stop()) {
unsigned int idx = count++ & 1;
unsigned int idx = count++ & (ARRAY_SIZE(rq) - 1);
struct i915_request *old = rq[idx];
struct i915_request *new;
@ -616,14 +627,28 @@ static int active_engine(void *data)
break;
}
if (arg->flags & TEST_PRIORITY)
ctx[idx]->priority =
i915_prandom_u32_max_state(512, &prng);
rq[idx] = i915_request_get(new);
i915_request_add(new);
mutex_unlock(&engine->i915->drm.struct_mutex);
if (old) {
i915_request_wait(old, 0, MAX_SCHEDULE_TIMEOUT);
if (i915_request_wait(old, 0, HZ) < 0) {
GEM_TRACE("%s timed out.\n", engine->name);
GEM_TRACE_DUMP();
i915_gem_set_wedged(engine->i915);
i915_request_put(old);
err = -EIO;
break;
}
i915_request_put(old);
}
cond_resched();
}
for (count = 0; count < ARRAY_SIZE(rq); count++)
@ -634,8 +659,9 @@ err_file:
return err;
}
static int __igt_reset_engine_others(struct drm_i915_private *i915,
bool active)
static int __igt_reset_engines(struct drm_i915_private *i915,
const char *test_name,
unsigned int flags)
{
struct intel_engine_cs *engine, *other;
enum intel_engine_id id, tmp;
@ -649,50 +675,61 @@ static int __igt_reset_engine_others(struct drm_i915_private *i915,
if (!intel_has_reset_engine(i915))
return 0;
if (active) {
if (flags & TEST_ACTIVE) {
mutex_lock(&i915->drm.struct_mutex);
err = hang_init(&h, i915);
mutex_unlock(&i915->drm.struct_mutex);
if (err)
return err;
if (flags & TEST_PRIORITY)
h.ctx->priority = 1024;
}
for_each_engine(engine, i915, id) {
struct task_struct *threads[I915_NUM_ENGINES] = {};
unsigned long resets[I915_NUM_ENGINES];
struct active_engine threads[I915_NUM_ENGINES] = {};
unsigned long global = i915_reset_count(&i915->gpu_error);
unsigned long count = 0;
unsigned long count = 0, reported;
IGT_TIMEOUT(end_time);
if (active && !intel_engine_can_store_dword(engine))
if (flags & TEST_ACTIVE &&
!intel_engine_can_store_dword(engine))
continue;
memset(threads, 0, sizeof(threads));
for_each_engine(other, i915, tmp) {
struct task_struct *tsk;
resets[tmp] = i915_reset_engine_count(&i915->gpu_error,
other);
threads[tmp].resets =
i915_reset_engine_count(&i915->gpu_error,
other);
if (other == engine)
if (!(flags & TEST_OTHERS))
continue;
tsk = kthread_run(active_engine, other,
if (other == engine && !(flags & TEST_SELF))
continue;
threads[tmp].engine = other;
threads[tmp].flags = flags;
tsk = kthread_run(active_engine, &threads[tmp],
"igt/%s", other->name);
if (IS_ERR(tsk)) {
err = PTR_ERR(tsk);
goto unwind;
}
threads[tmp] = tsk;
threads[tmp].task = tsk;
get_task_struct(tsk);
}
set_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
do {
if (active) {
struct i915_request *rq;
u32 seqno = intel_engine_get_seqno(engine);
struct i915_request *rq = NULL;
if (flags & TEST_ACTIVE) {
mutex_lock(&i915->drm.struct_mutex);
rq = hang_create_request(&h, engine);
if (IS_ERR(rq)) {
@ -705,7 +742,7 @@ static int __igt_reset_engine_others(struct drm_i915_private *i915,
__i915_request_add(rq, true);
mutex_unlock(&i915->drm.struct_mutex);
if (!wait_for_hang(&h, rq)) {
if (!wait_until_running(&h, rq)) {
struct drm_printer p = drm_info_printer(i915->drm.dev);
pr_err("%s: Failed to start request %x, at %x\n",
@ -718,33 +755,34 @@ static int __igt_reset_engine_others(struct drm_i915_private *i915,
break;
}
i915_request_put(rq);
GEM_BUG_ON(!rq->global_seqno);
seqno = rq->global_seqno - 1;
}
engine->hangcheck.stalled = true;
engine->hangcheck.seqno =
intel_engine_get_seqno(engine);
err = i915_reset_engine(engine, I915_RESET_QUIET);
err = i915_reset_engine(engine, NULL);
if (err) {
pr_err("i915_reset_engine(%s:%s) failed, err=%d\n",
engine->name, active ? "active" : "idle", err);
pr_err("i915_reset_engine(%s:%s): failed, err=%d\n",
engine->name, test_name, err);
break;
}
engine->hangcheck.stalled = false;
count++;
if (rq) {
i915_request_wait(rq, 0, MAX_SCHEDULE_TIMEOUT);
i915_request_put(rq);
}
} while (time_before(jiffies, end_time));
clear_bit(I915_RESET_ENGINE + id, &i915->gpu_error.flags);
pr_info("i915_reset_engine(%s:%s): %lu resets\n",
engine->name, active ? "active" : "idle", count);
engine->name, test_name, count);
if (i915_reset_engine_count(&i915->gpu_error, engine) -
resets[engine->id] != (active ? count : 0)) {
pr_err("i915_reset_engine(%s:%s): reset %lu times, but reported %lu\n",
engine->name, active ? "active" : "idle", count,
i915_reset_engine_count(&i915->gpu_error,
engine) - resets[engine->id]);
reported = i915_reset_engine_count(&i915->gpu_error, engine);
reported -= threads[engine->id].resets;
if (reported != (flags & TEST_ACTIVE ? count : 0)) {
pr_err("i915_reset_engine(%s:%s): reset %lu times, but reported %lu, expected %lu reported\n",
engine->name, test_name, count, reported,
(flags & TEST_ACTIVE ? count : 0));
if (!err)
err = -EINVAL;
}
@ -753,24 +791,26 @@ unwind:
for_each_engine(other, i915, tmp) {
int ret;
if (!threads[tmp])
if (!threads[tmp].task)
continue;
ret = kthread_stop(threads[tmp]);
ret = kthread_stop(threads[tmp].task);
if (ret) {
pr_err("kthread for other engine %s failed, err=%d\n",
other->name, ret);
if (!err)
err = ret;
}
put_task_struct(threads[tmp]);
put_task_struct(threads[tmp].task);
if (resets[tmp] != i915_reset_engine_count(&i915->gpu_error,
other)) {
if (other != engine &&
threads[tmp].resets !=
i915_reset_engine_count(&i915->gpu_error, other)) {
pr_err("Innocent engine %s was reset (count=%ld)\n",
other->name,
i915_reset_engine_count(&i915->gpu_error,
other) - resets[tmp]);
other) -
threads[tmp].resets);
if (!err)
err = -EINVAL;
}
@ -794,7 +834,7 @@ unwind:
if (i915_terminally_wedged(&i915->gpu_error))
err = -EIO;
if (active) {
if (flags & TEST_ACTIVE) {
mutex_lock(&i915->drm.struct_mutex);
hang_fini(&h);
mutex_unlock(&i915->drm.struct_mutex);
@ -803,27 +843,56 @@ unwind:
return err;
}
static int igt_reset_idle_engine_others(void *arg)
static int igt_reset_engines(void *arg)
{
return __igt_reset_engine_others(arg, false);
static const struct {
const char *name;
unsigned int flags;
} phases[] = {
{ "idle", 0 },
{ "active", TEST_ACTIVE },
{ "others-idle", TEST_OTHERS },
{ "others-active", TEST_OTHERS | TEST_ACTIVE },
{
"others-priority",
TEST_OTHERS | TEST_ACTIVE | TEST_PRIORITY
},
{
"self-priority",
TEST_OTHERS | TEST_ACTIVE | TEST_PRIORITY | TEST_SELF,
},
{ }
};
struct drm_i915_private *i915 = arg;
typeof(*phases) *p;
int err;
for (p = phases; p->name; p++) {
if (p->flags & TEST_PRIORITY) {
if (!(i915->caps.scheduler & I915_SCHEDULER_CAP_PRIORITY))
continue;
}
err = __igt_reset_engines(arg, p->name, p->flags);
if (err)
return err;
}
return 0;
}
static int igt_reset_active_engine_others(void *arg)
static u32 fake_hangcheck(struct i915_request *rq, u32 mask)
{
return __igt_reset_engine_others(arg, true);
}
struct i915_gpu_error *error = &rq->i915->gpu_error;
u32 reset_count = i915_reset_count(error);
static u32 fake_hangcheck(struct i915_request *rq)
{
u32 reset_count;
error->stalled_mask = mask;
rq->engine->hangcheck.stalled = true;
rq->engine->hangcheck.seqno = intel_engine_get_seqno(rq->engine);
/* set_bit() must be after we have setup the backchannel (mask) */
smp_mb__before_atomic();
set_bit(I915_RESET_HANDOFF, &error->flags);
reset_count = i915_reset_count(&rq->i915->gpu_error);
set_bit(I915_RESET_HANDOFF, &rq->i915->gpu_error.flags);
wake_up_all(&rq->i915->gpu_error.wait_queue);
wake_up_all(&error->wait_queue);
return reset_count;
}
@ -858,21 +927,20 @@ static int igt_wait_reset(void *arg)
i915_request_get(rq);
__i915_request_add(rq, true);
if (!wait_for_hang(&h, rq)) {
if (!wait_until_running(&h, rq)) {
struct drm_printer p = drm_info_printer(i915->drm.dev);
pr_err("%s: Failed to start request %x, at %x\n",
__func__, rq->fence.seqno, hws_seqno(&h, rq));
intel_engine_dump(rq->engine, &p, "%s\n", rq->engine->name);
i915_reset(i915, 0);
i915_gem_set_wedged(i915);
err = -EIO;
goto out_rq;
}
reset_count = fake_hangcheck(rq);
reset_count = fake_hangcheck(rq, ALL_ENGINES);
timeout = i915_request_wait(rq, I915_WAIT_LOCKED, 10);
if (timeout < 0) {
@ -903,6 +971,23 @@ unlock:
return err;
}
static int wait_for_others(struct drm_i915_private *i915,
struct intel_engine_cs *exclude)
{
struct intel_engine_cs *engine;
enum intel_engine_id id;
for_each_engine(engine, i915, id) {
if (engine == exclude)
continue;
if (wait_for(intel_engine_is_idle(engine), 10))
return -EIO;
}
return 0;
}
static int igt_reset_queue(void *arg)
{
struct drm_i915_private *i915 = arg;
@ -951,27 +1036,49 @@ static int igt_reset_queue(void *arg)
i915_request_get(rq);
__i915_request_add(rq, true);
if (!wait_for_hang(&h, prev)) {
/*
* XXX We don't handle resetting the kernel context
* very well. If we trigger a device reset twice in
* quick succession while the kernel context is
* executing, we may end up skipping the breadcrumb.
* This is really only a problem for the selftest as
* normally there is a large interlude between resets
* (hangcheck), or we focus on resetting just one
* engine and so avoid repeatedly resetting innocents.
*/
err = wait_for_others(i915, engine);
if (err) {
pr_err("%s(%s): Failed to idle other inactive engines after device reset\n",
__func__, engine->name);
i915_request_put(rq);
i915_request_put(prev);
GEM_TRACE_DUMP();
i915_gem_set_wedged(i915);
goto fini;
}
if (!wait_until_running(&h, prev)) {
struct drm_printer p = drm_info_printer(i915->drm.dev);
pr_err("%s: Failed to start request %x, at %x\n",
__func__, prev->fence.seqno, hws_seqno(&h, prev));
intel_engine_dump(prev->engine, &p,
"%s\n", prev->engine->name);
pr_err("%s(%s): Failed to start request %x, at %x\n",
__func__, engine->name,
prev->fence.seqno, hws_seqno(&h, prev));
intel_engine_dump(engine, &p,
"%s\n", engine->name);
i915_request_put(rq);
i915_request_put(prev);
i915_reset(i915, 0);
i915_gem_set_wedged(i915);
err = -EIO;
goto fini;
}
reset_count = fake_hangcheck(prev);
reset_count = fake_hangcheck(prev, ENGINE_MASK(id));
i915_reset(i915, I915_RESET_QUIET);
i915_reset(i915, ENGINE_MASK(id), NULL);
GEM_BUG_ON(test_bit(I915_RESET_HANDOFF,
&i915->gpu_error.flags));
@ -1044,7 +1151,7 @@ static int igt_handle_error(void *arg)
if (!intel_has_reset_engine(i915))
return 0;
if (!intel_engine_can_store_dword(i915->engine[RCS]))
if (!engine || !intel_engine_can_store_dword(engine))
return 0;
mutex_lock(&i915->drm.struct_mutex);
@ -1062,14 +1169,13 @@ static int igt_handle_error(void *arg)
i915_request_get(rq);
__i915_request_add(rq, true);
if (!wait_for_hang(&h, rq)) {
if (!wait_until_running(&h, rq)) {
struct drm_printer p = drm_info_printer(i915->drm.dev);
pr_err("%s: Failed to start request %x, at %x\n",
__func__, rq->fence.seqno, hws_seqno(&h, rq));
intel_engine_dump(rq->engine, &p, "%s\n", rq->engine->name);
i915_reset(i915, 0);
i915_gem_set_wedged(i915);
err = -EIO;
@ -1081,10 +1187,7 @@ static int igt_handle_error(void *arg)
/* Temporarily disable error capture */
error = xchg(&i915->gpu_error.first_error, (void *)-1);
engine->hangcheck.stalled = true;
engine->hangcheck.seqno = intel_engine_get_seqno(engine);
i915_handle_error(i915, intel_engine_flag(engine), "%s", __func__);
i915_handle_error(i915, ENGINE_MASK(engine->id), 0, NULL);
xchg(&i915->gpu_error.first_error, error);
@ -1112,8 +1215,7 @@ int intel_hangcheck_live_selftests(struct drm_i915_private *i915)
SUBTEST(igt_hang_sanitycheck),
SUBTEST(igt_reset_idle_engine),
SUBTEST(igt_reset_active_engine),
SUBTEST(igt_reset_idle_engine_others),
SUBTEST(igt_reset_active_engine_others),
SUBTEST(igt_reset_engines),
SUBTEST(igt_wait_reset),
SUBTEST(igt_reset_queue),
SUBTEST(igt_handle_error),
@ -1129,6 +1231,10 @@ int intel_hangcheck_live_selftests(struct drm_i915_private *i915)
err = i915_subtests(tests, i915);
mutex_lock(&i915->drm.struct_mutex);
flush_test(i915, I915_WAIT_LOCKED);
mutex_unlock(&i915->drm.struct_mutex);
i915_modparams.enable_hangcheck = saved_hangcheck;
intel_runtime_pm_put(i915);

507
drivers/gpu/drm/i915/selftests/intel_lrc.c 100644
View File

@ -0,0 +1,507 @@
/*
* SPDX-License-Identifier: MIT
*
* Copyright © 2018 Intel Corporation
*/
#include "../i915_selftest.h"
#include "mock_context.h"
struct spinner {
struct drm_i915_private *i915;
struct drm_i915_gem_object *hws;
struct drm_i915_gem_object *obj;
u32 *batch;
void *seqno;
};
static int spinner_init(struct spinner *spin, struct drm_i915_private *i915)
{
unsigned int mode;
void *vaddr;
int err;
GEM_BUG_ON(INTEL_GEN(i915) < 8);
memset(spin, 0, sizeof(*spin));
spin->i915 = i915;
spin->hws = i915_gem_object_create_internal(i915, PAGE_SIZE);
if (IS_ERR(spin->hws)) {
err = PTR_ERR(spin->hws);
goto err;
}
spin->obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
if (IS_ERR(spin->obj)) {
err = PTR_ERR(spin->obj);
goto err_hws;
}
i915_gem_object_set_cache_level(spin->hws, I915_CACHE_LLC);
vaddr = i915_gem_object_pin_map(spin->hws, I915_MAP_WB);
if (IS_ERR(vaddr)) {
err = PTR_ERR(vaddr);
goto err_obj;
}
spin->seqno = memset(vaddr, 0xff, PAGE_SIZE);
mode = HAS_LLC(i915) ? I915_MAP_WB : I915_MAP_WC;
vaddr = i915_gem_object_pin_map(spin->obj, mode);
if (IS_ERR(vaddr)) {
err = PTR_ERR(vaddr);
goto err_unpin_hws;
}
spin->batch = vaddr;
return 0;
err_unpin_hws:
i915_gem_object_unpin_map(spin->hws);
err_obj:
i915_gem_object_put(spin->obj);
err_hws:
i915_gem_object_put(spin->hws);
err:
return err;
}
static unsigned int seqno_offset(u64 fence)
{
return offset_in_page(sizeof(u32) * fence);
}
static u64 hws_address(const struct i915_vma *hws,
const struct i915_request *rq)
{
return hws->node.start + seqno_offset(rq->fence.context);
}
static int emit_recurse_batch(struct spinner *spin,
struct i915_request *rq,
u32 arbitration_command)
{
struct i915_address_space *vm = &rq->ctx->ppgtt->base;
struct i915_vma *hws, *vma;
u32 *batch;
int err;
vma = i915_vma_instance(spin->obj, vm, NULL);
if (IS_ERR(vma))
return PTR_ERR(vma);
hws = i915_vma_instance(spin->hws, vm, NULL);
if (IS_ERR(hws))
return PTR_ERR(hws);
err = i915_vma_pin(vma, 0, 0, PIN_USER);
if (err)
return err;
err = i915_vma_pin(hws, 0, 0, PIN_USER);
if (err)
goto unpin_vma;
i915_vma_move_to_active(vma, rq, 0);
if (!i915_gem_object_has_active_reference(vma->obj)) {
i915_gem_object_get(vma->obj);
i915_gem_object_set_active_reference(vma->obj);
}
i915_vma_move_to_active(hws, rq, 0);
if (!i915_gem_object_has_active_reference(hws->obj)) {
i915_gem_object_get(hws->obj);
i915_gem_object_set_active_reference(hws->obj);
}
batch = spin->batch;
*batch++ = MI_STORE_DWORD_IMM_GEN4;
*batch++ = lower_32_bits(hws_address(hws, rq));
*batch++ = upper_32_bits(hws_address(hws, rq));
*batch++ = rq->fence.seqno;
*batch++ = arbitration_command;
*batch++ = MI_BATCH_BUFFER_START | 1 << 8 | 1;
*batch++ = lower_32_bits(vma->node.start);
*batch++ = upper_32_bits(vma->node.start);
*batch++ = MI_BATCH_BUFFER_END; /* not reached */
i915_gem_chipset_flush(spin->i915);
err = rq->engine->emit_bb_start(rq, vma->node.start, PAGE_SIZE, 0);
i915_vma_unpin(hws);
unpin_vma:
i915_vma_unpin(vma);
return err;
}
static struct i915_request *
spinner_create_request(struct spinner *spin,
struct i915_gem_context *ctx,
struct intel_engine_cs *engine,
u32 arbitration_command)
{
struct i915_request *rq;
int err;
rq = i915_request_alloc(engine, ctx);
if (IS_ERR(rq))
return rq;
err = emit_recurse_batch(spin, rq, arbitration_command);
if (err) {
__i915_request_add(rq, false);
return ERR_PTR(err);
}
return rq;
}
static u32 hws_seqno(const struct spinner *spin, const struct i915_request *rq)
{
u32 *seqno = spin->seqno + seqno_offset(rq->fence.context);
return READ_ONCE(*seqno);
}
struct wedge_me {
struct delayed_work work;
struct drm_i915_private *i915;
const void *symbol;
};
static void wedge_me(struct work_struct *work)
{
struct wedge_me *w = container_of(work, typeof(*w), work.work);
pr_err("%pS timed out, cancelling all further testing.\n", w->symbol);
GEM_TRACE("%pS timed out.\n", w->symbol);
GEM_TRACE_DUMP();
i915_gem_set_wedged(w->i915);
}
static void __init_wedge(struct wedge_me *w,
struct drm_i915_private *i915,
long timeout,
const void *symbol)
{
w->i915 = i915;
w->symbol = symbol;
INIT_DELAYED_WORK_ONSTACK(&w->work, wedge_me);
schedule_delayed_work(&w->work, timeout);
}
static void __fini_wedge(struct wedge_me *w)
{
cancel_delayed_work_sync(&w->work);
destroy_delayed_work_on_stack(&w->work);
w->i915 = NULL;
}
#define wedge_on_timeout(W, DEV, TIMEOUT) \
for (__init_wedge((W), (DEV), (TIMEOUT), __builtin_return_address(0)); \
(W)->i915; \
__fini_wedge((W)))
static noinline int
flush_test(struct drm_i915_private *i915, unsigned int flags)
{
struct wedge_me w;
cond_resched();
wedge_on_timeout(&w, i915, HZ)
i915_gem_wait_for_idle(i915, flags);
return i915_terminally_wedged(&i915->gpu_error) ? -EIO : 0;
}
static void spinner_end(struct spinner *spin)
{
*spin->batch = MI_BATCH_BUFFER_END;
i915_gem_chipset_flush(spin->i915);
}
static void spinner_fini(struct spinner *spin)
{
spinner_end(spin);
i915_gem_object_unpin_map(spin->obj);
i915_gem_object_put(spin->obj);
i915_gem_object_unpin_map(spin->hws);
i915_gem_object_put(spin->hws);
}
static bool wait_for_spinner(struct spinner *spin, struct i915_request *rq)
{
if (!wait_event_timeout(rq->execute,
READ_ONCE(rq->global_seqno),
msecs_to_jiffies(10)))
return false;
return !(wait_for_us(i915_seqno_passed(hws_seqno(spin, rq),
rq->fence.seqno),
10) &&
wait_for(i915_seqno_passed(hws_seqno(spin, rq),
rq->fence.seqno),
1000));
}
static int live_sanitycheck(void *arg)
{
struct drm_i915_private *i915 = arg;
struct intel_engine_cs *engine;
struct i915_gem_context *ctx;
enum intel_engine_id id;
struct spinner spin;
int err = -ENOMEM;
if (!HAS_LOGICAL_RING_CONTEXTS(i915))
return 0;
mutex_lock(&i915->drm.struct_mutex);
if (spinner_init(&spin, i915))
goto err_unlock;
ctx = kernel_context(i915);
if (!ctx)
goto err_spin;
for_each_engine(engine, i915, id) {
struct i915_request *rq;
rq = spinner_create_request(&spin, ctx, engine, MI_NOOP);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
goto err_ctx;
}
i915_request_add(rq);
if (!wait_for_spinner(&spin, rq)) {
GEM_TRACE("spinner failed to start\n");
GEM_TRACE_DUMP();
i915_gem_set_wedged(i915);
err = -EIO;
goto err_ctx;
}
spinner_end(&spin);
if (flush_test(i915, I915_WAIT_LOCKED)) {
err = -EIO;
goto err_ctx;
}
}
err = 0;
err_ctx:
kernel_context_close(ctx);
err_spin:
spinner_fini(&spin);
err_unlock:
flush_test(i915, I915_WAIT_LOCKED);
mutex_unlock(&i915->drm.struct_mutex);
return err;
}
static int live_preempt(void *arg)
{
struct drm_i915_private *i915 = arg;
struct i915_gem_context *ctx_hi, *ctx_lo;
struct spinner spin_hi, spin_lo;
struct intel_engine_cs *engine;
enum intel_engine_id id;
int err = -ENOMEM;
if (!HAS_LOGICAL_RING_PREEMPTION(i915))
return 0;
mutex_lock(&i915->drm.struct_mutex);
if (spinner_init(&spin_hi, i915))
goto err_unlock;
if (spinner_init(&spin_lo, i915))
goto err_spin_hi;
ctx_hi = kernel_context(i915);
if (!ctx_hi)
goto err_spin_lo;
ctx_hi->priority = I915_CONTEXT_MAX_USER_PRIORITY;
ctx_lo = kernel_context(i915);
if (!ctx_lo)
goto err_ctx_hi;
ctx_lo->priority = I915_CONTEXT_MIN_USER_PRIORITY;
for_each_engine(engine, i915, id) {
struct i915_request *rq;
rq = spinner_create_request(&spin_lo, ctx_lo, engine,
MI_ARB_CHECK);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
goto err_ctx_lo;
}
i915_request_add(rq);
if (!wait_for_spinner(&spin_lo, rq)) {
GEM_TRACE("lo spinner failed to start\n");
GEM_TRACE_DUMP();
i915_gem_set_wedged(i915);
err = -EIO;
goto err_ctx_lo;
}
rq = spinner_create_request(&spin_hi, ctx_hi, engine,
MI_ARB_CHECK);
if (IS_ERR(rq)) {
spinner_end(&spin_lo);
err = PTR_ERR(rq);
goto err_ctx_lo;
}
i915_request_add(rq);
if (!wait_for_spinner(&spin_hi, rq)) {
GEM_TRACE("hi spinner failed to start\n");
GEM_TRACE_DUMP();
i915_gem_set_wedged(i915);
err = -EIO;
goto err_ctx_lo;
}
spinner_end(&spin_hi);
spinner_end(&spin_lo);
if (flush_test(i915, I915_WAIT_LOCKED)) {
err = -EIO;
goto err_ctx_lo;
}
}
err = 0;
err_ctx_lo:
kernel_context_close(ctx_lo);
err_ctx_hi:
kernel_context_close(ctx_hi);
err_spin_lo:
spinner_fini(&spin_lo);
err_spin_hi:
spinner_fini(&spin_hi);
err_unlock:
flush_test(i915, I915_WAIT_LOCKED);
mutex_unlock(&i915->drm.struct_mutex);
return err;
}
static int live_late_preempt(void *arg)
{
struct drm_i915_private *i915 = arg;
struct i915_gem_context *ctx_hi, *ctx_lo;
struct spinner spin_hi, spin_lo;
struct intel_engine_cs *engine;
enum intel_engine_id id;
int err = -ENOMEM;
if (!HAS_LOGICAL_RING_PREEMPTION(i915))
return 0;
mutex_lock(&i915->drm.struct_mutex);
if (spinner_init(&spin_hi, i915))
goto err_unlock;
if (spinner_init(&spin_lo, i915))
goto err_spin_hi;
ctx_hi = kernel_context(i915);
if (!ctx_hi)
goto err_spin_lo;
ctx_lo = kernel_context(i915);
if (!ctx_lo)
goto err_ctx_hi;
for_each_engine(engine, i915, id) {
struct i915_request *rq;
rq = spinner_create_request(&spin_lo, ctx_lo, engine,
MI_ARB_CHECK);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
goto err_ctx_lo;
}
i915_request_add(rq);
if (!wait_for_spinner(&spin_lo, rq)) {
pr_err("First context failed to start\n");
goto err_wedged;
}
rq = spinner_create_request(&spin_hi, ctx_hi, engine, MI_NOOP);
if (IS_ERR(rq)) {
spinner_end(&spin_lo);
err = PTR_ERR(rq);
goto err_ctx_lo;
}
i915_request_add(rq);
if (wait_for_spinner(&spin_hi, rq)) {
pr_err("Second context overtook first?\n");
goto err_wedged;
}
engine->schedule(rq, I915_PRIORITY_MAX);
if (!wait_for_spinner(&spin_hi, rq)) {
pr_err("High priority context failed to preempt the low priority context\n");
GEM_TRACE_DUMP();
goto err_wedged;
}
spinner_end(&spin_hi);
spinner_end(&spin_lo);
if (flush_test(i915, I915_WAIT_LOCKED)) {
err = -EIO;
goto err_ctx_lo;
}
}
err = 0;
err_ctx_lo:
kernel_context_close(ctx_lo);
err_ctx_hi:
kernel_context_close(ctx_hi);
err_spin_lo:
spinner_fini(&spin_lo);
err_spin_hi:
spinner_fini(&spin_hi);
err_unlock:
flush_test(i915, I915_WAIT_LOCKED);
mutex_unlock(&i915->drm.struct_mutex);
return err;
err_wedged:
spinner_end(&spin_hi);
spinner_end(&spin_lo);
i915_gem_set_wedged(i915);
err = -EIO;
goto err_ctx_lo;
}
int intel_execlists_live_selftests(struct drm_i915_private *i915)
{
static const struct i915_subtest tests[] = {
SUBTEST(live_sanitycheck),
SUBTEST(live_preempt),
SUBTEST(live_late_preempt),
};
return i915_subtests(tests, i915);
}

10
include/drm/drm_dp_helper.h
View File

@ -478,6 +478,7 @@
# define DP_PSR_FRAME_CAPTURE (1 << 3)
# define DP_PSR_SELECTIVE_UPDATE (1 << 4)
# define DP_PSR_IRQ_HPD_WITH_CRC_ERRORS (1 << 5)
# define DP_PSR_ENABLE_PSR2 (1 << 6) /* eDP 1.4a */
#define DP_ADAPTER_CTRL 0x1a0
# define DP_ADAPTER_CTRL_FORCE_LOAD_SENSE (1 << 0)
@ -794,6 +795,15 @@
# define DP_LAST_ACTUAL_SYNCHRONIZATION_LATENCY_MASK (0xf << 4)
# define DP_LAST_ACTUAL_SYNCHRONIZATION_LATENCY_SHIFT 4
#define DP_LAST_RECEIVED_PSR_SDP 0x200a /* eDP 1.2 */
# define DP_PSR_STATE_BIT (1 << 0) /* eDP 1.2 */
# define DP_UPDATE_RFB_BIT (1 << 1) /* eDP 1.2 */
# define DP_CRC_VALID_BIT (1 << 2) /* eDP 1.2 */
# define DP_SU_VALID (1 << 3) /* eDP 1.4 */
# define DP_FIRST_SCAN_LINE_SU_REGION (1 << 4) /* eDP 1.4 */
# define DP_LAST_SCAN_LINE_SU_REGION (1 << 5) /* eDP 1.4 */
# define DP_Y_COORDINATE_VALID (1 << 6) /* eDP 1.4a */
#define DP_RECEIVER_ALPM_STATUS 0x200b /* eDP 1.4 */
# define DP_ALPM_LOCK_TIMEOUT_ERROR (1 << 0)