redonkable
/
remarkable-linux
1
0
Fork 0
Code Releases Activity

Merge tag 'drm-intel-next-2017-08-18' of git://anongit.freedesktop.org/git/drm-intel into drm-next 

Final pile of features for 4.14

- New ioctl to change NOA configurations, plus prep (Lionel)
- CCS (color compression) scanout support, based on the fancy new
  modifier additions (Ville&Ben)
- Document i915 register macro style (Jani)
- Many more gen10/cnl patches (Rodrigo, Pualo, ...)
- More gpu reset vs. modeset duct-tape to restore the old way.
- prep work for cnl: hpd_pin reorg (Rodrigo), support for more power
  wells (Imre), i2c pin reorg (Anusha)
- drm_syncobj support (Jason Ekstrand)
- forcewake vs gpu reset fix (Chris)
- execbuf speedup for the no-relocs fastpath, anv/vk low-overhead ftw (Chris)
- switch to idr/radixtree instead of the resizing ht for execbuf id->vma
  lookups (Chris)

gvt:
- MMIO save/restore optimization (Changbin)
- Split workload scan vs. dispatch for more parallel exec (Ping)
- vGPU full 48bit ppgtt support (Joonas, Tina)
- vGPU hw id expose for perf (Zhenyu)

Bunch of work all over to make the igt CI runs more complete/stable.
Watch https://intel-gfx-ci.01.org/tree/drm-tip/shards-all.html for
progress in getting this ready. Next week we're going into production
mode (i.e. will send results to intel-gfx) on hsw, more platforms to
come.

Also, a new maintainer tram, I'm stepping out. Huge thanks to Jani for
being an awesome co-maintainer the past few years, and all the best
for Jani, Joonas&Rodrigo as the new maintainers!

* tag 'drm-intel-next-2017-08-18' of git://anongit.freedesktop.org/git/drm-intel: (179 commits)
  drm/i915: Update DRIVER_DATE to 20170818
  drm/i915/bxt: use NULL for GPIO connection ID
  drm/i915: Mark the GT as busy before idling the previous request
  drm/i915: Trivial grammar fix s/opt of/opt out of/ in comment
  drm/i915: Replace execbuf vma ht with an idr
  drm/i915: Simplify eb_lookup_vmas()
  drm/i915: Convert execbuf to use struct-of-array packing for critical fields
  drm/i915: Check context status before looking up our obj/vma
  drm/i915: Don't use MI_STORE_DWORD_IMM on Sandybridge/vcs
  drm/i915: Stop touching forcewake following a gen6+ engine reset
  MAINTAINERS: drm/i915 has a new maintainer team
  drm/i915: Split pin mapping into per platform functions
  drm/i915/opregion: let user specify override VBT via firmware load
  drm/i915/cnl: Reuse skl_wm_get_hw_state on Cannonlake.
  drm/i915/gen10: implement gen 10 watermarks calculations
  drm/i915/cnl: Fix LSPCON support.
  drm/i915/vbt: ignore extraneous child devices for a port
  drm/i915/cnl: Setup PAT Index.
  drm/i915/edp: Allow alternate fixed mode for eDP if available.
  drm/i915: Add support for drm syncobjs
  ...
zero-colors
Dave Airlie 2017-08-22 10:03:07 +10:00
parent 3aadb888b1 a42894ebb5
commit 735f463af7
121 changed files with 4761 additions and 32752 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
Documentation/gpu/i915.rst
View File

@ -417,6 +417,10 @@ integrate with drm/i915 and to handle the `DRM_I915_PERF_OPEN` ioctl.
:functions: i915_perf_open_ioctl
.. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
:functions: i915_perf_release
.. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
:functions: i915_perf_add_config_ioctl
.. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
:functions: i915_perf_remove_config_ioctl
i915 Perf Stream
----------------
@ -477,4 +481,16 @@ specific details than found in the more high-level sections.
.. kernel-doc:: drivers/gpu/drm/i915/i915_perf.c
:internal:
.. WARNING: DOCPROC directive not supported: !Cdrivers/gpu/drm/i915/i915_irq.c
Style
=====
The drm/i915 driver codebase has some style rules in addition to (and, in some
cases, deviating from) the kernel coding style.
Register macro definition style
-------------------------------
The style guide for ``i915_reg.h``.
.. kernel-doc:: drivers/gpu/drm/i915/i915_reg.h
:doc: The i915 register macro definition style guide

3
MAINTAINERS
View File

@ -6764,8 +6764,9 @@ S: Supported
F: drivers/scsi/isci/
INTEL DRM DRIVERS (excluding Poulsbo, Moorestown and derivative chipsets)
M: Daniel Vetter <daniel.vetter@intel.com>
M: Jani Nikula <jani.nikula@linux.intel.com>
M: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
M: Rodrigo Vivi <rodrigo.vivi@intel.com>
L: intel-gfx@lists.freedesktop.org
W: https://01.org/linuxgraphics/
B: https://01.org/linuxgraphics/documentation/how-report-bugs

18
drivers/gpu/drm/drm_atomic_helper.c
View File

@ -2716,6 +2716,7 @@ int drm_atomic_helper_disable_all(struct drm_device *dev,
struct drm_plane *plane;
struct drm_crtc_state *crtc_state;
struct drm_crtc *crtc;
unsigned plane_mask = 0;
int ret, i;
state = drm_atomic_state_alloc(dev);
@ -2758,10 +2759,14 @@ int drm_atomic_helper_disable_all(struct drm_device *dev,
goto free;
drm_atomic_set_fb_for_plane(plane_state, NULL);
plane_mask |= BIT(drm_plane_index(plane));
plane->old_fb = plane->fb;
}
ret = drm_atomic_commit(state);
free:
if (plane_mask)
drm_atomic_clean_old_fb(dev, plane_mask, ret);
drm_atomic_state_put(state);
return ret;
}
@ -2892,11 +2897,16 @@ int drm_atomic_helper_commit_duplicated_state(struct drm_atomic_state *state,
struct drm_connector_state *new_conn_state;
struct drm_crtc *crtc;
struct drm_crtc_state *new_crtc_state;
unsigned plane_mask = 0;
struct drm_device *dev = state->dev;
int ret;
state->acquire_ctx = ctx;
for_each_new_plane_in_state(state, plane, new_plane_state, i)
for_each_new_plane_in_state(state, plane, new_plane_state, i) {
plane_mask |= BIT(drm_plane_index(plane));
state->planes[i].old_state = plane->state;
}
for_each_new_crtc_in_state(state, crtc, new_crtc_state, i)
state->crtcs[i].old_state = crtc->state;
@ -2904,7 +2914,11 @@ int drm_atomic_helper_commit_duplicated_state(struct drm_atomic_state *state,
for_each_new_connector_in_state(state, connector, new_conn_state, i)
state->connectors[i].old_state = connector->state;
return drm_atomic_commit(state);
ret = drm_atomic_commit(state);
if (plane_mask)
drm_atomic_clean_old_fb(dev, plane_mask, ret);
return ret;
}
EXPORT_SYMBOL(drm_atomic_helper_commit_duplicated_state);

1
drivers/gpu/drm/i915/Kconfig.debug
View File

@ -25,6 +25,7 @@ config DRM_I915_DEBUG
select DRM_VGEM # used by igt/prime_vgem (dmabuf interop checks)
select DRM_DEBUG_MM if DRM=y
select DRM_DEBUG_MM_SELFTEST
select SW_SYNC # signaling validation framework (igt/syncobj*)
select DRM_I915_SW_FENCE_DEBUG_OBJECTS
select DRM_I915_SELFTEST
default n

1
drivers/gpu/drm/i915/Makefile
View File

@ -39,6 +39,7 @@ i915-y += i915_cmd_parser.o \
i915_gem_gtt.o \
i915_gem_internal.o \
i915_gem.o \
i915_gem_object.o \
i915_gem_render_state.o \
i915_gem_request.o \
i915_gem_shrinker.o \

4
drivers/gpu/drm/i915/gvt/aperture_gm.c
View File

@ -285,8 +285,8 @@ static int alloc_resource(struct intel_vgpu *vgpu,
return 0;
no_enough_resource:
gvt_vgpu_err("fail to allocate resource %s\n", item);
gvt_vgpu_err("request %luMB avail %luMB max %luMB taken %luMB\n",
gvt_err("fail to allocate resource %s\n", item);
gvt_err("request %luMB avail %luMB max %luMB taken %luMB\n",
BYTES_TO_MB(request), BYTES_TO_MB(avail),
BYTES_TO_MB(max), BYTES_TO_MB(taken));
return -ENOSPC;

8
drivers/gpu/drm/i915/gvt/cmd_parser.c
View File

@ -1382,13 +1382,13 @@ static inline int cmd_address_audit(struct parser_exec_state *s,
ret = -EINVAL;
goto err;
}
} else if ((!vgpu_gmadr_is_valid(s->vgpu, guest_gma)) ||
(!vgpu_gmadr_is_valid(s->vgpu,
guest_gma + op_size - 1))) {
} else if (!intel_gvt_ggtt_validate_range(vgpu, guest_gma, op_size)) {
ret = -EINVAL;
goto err;
}
return 0;
err:
gvt_vgpu_err("cmd_parser: Malicious %s detected, addr=0x%lx, len=%d!\n",
s->info->name, guest_gma, op_size);
@ -2647,7 +2647,7 @@ static int shadow_workload_ring_buffer(struct intel_vgpu_workload *workload)
return 0;
}
int intel_gvt_scan_and_shadow_workload(struct intel_vgpu_workload *workload)
int intel_gvt_scan_and_shadow_ringbuffer(struct intel_vgpu_workload *workload)
{
int ret;
struct intel_vgpu *vgpu = workload->vgpu;

2
drivers/gpu/drm/i915/gvt/cmd_parser.h
View File

@ -42,7 +42,7 @@ void intel_gvt_clean_cmd_parser(struct intel_gvt *gvt);
int intel_gvt_init_cmd_parser(struct intel_gvt *gvt);
int intel_gvt_scan_and_shadow_workload(struct intel_vgpu_workload *workload);
int intel_gvt_scan_and_shadow_ringbuffer(struct intel_vgpu_workload *workload);
int intel_gvt_scan_and_shadow_wa_ctx(struct intel_shadow_wa_ctx *wa_ctx);

6
drivers/gpu/drm/i915/gvt/display.c
View File

@ -178,9 +178,9 @@ static void emulate_monitor_status_change(struct intel_vgpu *vgpu)
SDE_PORTE_HOTPLUG_SPT);
vgpu_vreg(vgpu, SKL_FUSE_STATUS) |=
SKL_FUSE_DOWNLOAD_STATUS |
SKL_FUSE_PG0_DIST_STATUS |
SKL_FUSE_PG1_DIST_STATUS |
SKL_FUSE_PG2_DIST_STATUS;
SKL_FUSE_PG_DIST_STATUS(SKL_PG0) |
SKL_FUSE_PG_DIST_STATUS(SKL_PG1) |
SKL_FUSE_PG_DIST_STATUS(SKL_PG2);
vgpu_vreg(vgpu, LCPLL1_CTL) |=
LCPLL_PLL_ENABLE |
LCPLL_PLL_LOCK;

15
drivers/gpu/drm/i915/gvt/execlist.c
View File

@ -622,6 +622,7 @@ static int submit_context(struct intel_vgpu *vgpu, int ring_id,
struct list_head *q = workload_q_head(vgpu, ring_id);
struct intel_vgpu_workload *last_workload = get_last_workload(q);
struct intel_vgpu_workload *workload = NULL;
struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
u64 ring_context_gpa;
u32 head, tail, start, ctl, ctx_ctl, per_ctx, indirect_ctx;
int ret;
@ -685,6 +686,7 @@ static int submit_context(struct intel_vgpu *vgpu, int ring_id,
workload->complete = complete_execlist_workload;
workload->status = -EINPROGRESS;
workload->emulate_schedule_in = emulate_schedule_in;
workload->shadowed = false;
if (ring_id == RCS) {
intel_gvt_hypervisor_read_gpa(vgpu, ring_context_gpa +
@ -718,6 +720,17 @@ static int submit_context(struct intel_vgpu *vgpu, int ring_id,
return ret;
}
/* Only scan and shadow the first workload in the queue
* as there is only one pre-allocated buf-obj for shadow.
*/
if (list_empty(workload_q_head(vgpu, ring_id))) {
intel_runtime_pm_get(dev_priv);
mutex_lock(&dev_priv->drm.struct_mutex);
intel_gvt_scan_and_shadow_workload(workload);
mutex_unlock(&dev_priv->drm.struct_mutex);
intel_runtime_pm_put(dev_priv);
}
queue_workload(workload);
return 0;
}
@ -800,6 +813,8 @@ static void clean_workloads(struct intel_vgpu *vgpu, unsigned long engine_mask)
list_del_init(&pos->list);
free_workload(pos);
}
clear_bit(engine->id, vgpu->shadow_ctx_desc_updated);
}
}

128
drivers/gpu/drm/i915/gvt/gtt.c
View File

@ -259,7 +259,7 @@ static void write_pte64(struct drm_i915_private *dev_priv,
writeq(pte, addr);
}
static inline struct intel_gvt_gtt_entry *gtt_get_entry64(void *pt,
static inline int gtt_get_entry64(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index, bool hypervisor_access, unsigned long gpa,
struct intel_vgpu *vgpu)
@ -268,22 +268,23 @@ static inline struct intel_gvt_gtt_entry *gtt_get_entry64(void *pt,
int ret;
if (WARN_ON(info->gtt_entry_size != 8))
return e;
return -EINVAL;
if (hypervisor_access) {
ret = intel_gvt_hypervisor_read_gpa(vgpu, gpa +
(index << info->gtt_entry_size_shift),
&e->val64, 8);
WARN_ON(ret);
if (WARN_ON(ret))
return ret;
} else if (!pt) {
e->val64 = read_pte64(vgpu->gvt->dev_priv, index);
} else {
e->val64 = *((u64 *)pt + index);
}
return e;
return 0;
}
static inline struct intel_gvt_gtt_entry *gtt_set_entry64(void *pt,
static inline int gtt_set_entry64(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index, bool hypervisor_access, unsigned long gpa,
struct intel_vgpu *vgpu)
@ -292,19 +293,20 @@ static inline struct intel_gvt_gtt_entry *gtt_set_entry64(void *pt,
int ret;
if (WARN_ON(info->gtt_entry_size != 8))
return e;
return -EINVAL;
if (hypervisor_access) {
ret = intel_gvt_hypervisor_write_gpa(vgpu, gpa +
(index << info->gtt_entry_size_shift),
&e->val64, 8);
WARN_ON(ret);
if (WARN_ON(ret))
return ret;
} else if (!pt) {
write_pte64(vgpu->gvt->dev_priv, index, e->val64);
} else {
*((u64 *)pt + index) = e->val64;
}
return e;
return 0;
}
#define GTT_HAW 46
@ -445,21 +447,25 @@ static int gtt_entry_p2m(struct intel_vgpu *vgpu, struct intel_gvt_gtt_entry *p,
/*
* MM helpers.
*/
struct intel_gvt_gtt_entry *intel_vgpu_mm_get_entry(struct intel_vgpu_mm *mm,
int intel_vgpu_mm_get_entry(struct intel_vgpu_mm *mm,
void *page_table, struct intel_gvt_gtt_entry *e,
unsigned long index)
{
struct intel_gvt *gvt = mm->vgpu->gvt;
struct intel_gvt_gtt_pte_ops *ops = gvt->gtt.pte_ops;
int ret;
e->type = mm->page_table_entry_type;
ops->get_entry(page_table, e, index, false, 0, mm->vgpu);
ret = ops->get_entry(page_table, e, index, false, 0, mm->vgpu);
if (ret)
return ret;
ops->test_pse(e);
return e;
return 0;
}
struct intel_gvt_gtt_entry *intel_vgpu_mm_set_entry(struct intel_vgpu_mm *mm,
int intel_vgpu_mm_set_entry(struct intel_vgpu_mm *mm,
void *page_table, struct intel_gvt_gtt_entry *e,
unsigned long index)
{
@ -472,7 +478,7 @@ struct intel_gvt_gtt_entry *intel_vgpu_mm_set_entry(struct intel_vgpu_mm *mm,
/*
* PPGTT shadow page table helpers.
*/
static inline struct intel_gvt_gtt_entry *ppgtt_spt_get_entry(
static inline int ppgtt_spt_get_entry(
struct intel_vgpu_ppgtt_spt *spt,
void *page_table, int type,
struct intel_gvt_gtt_entry *e, unsigned long index,
@ -480,20 +486,24 @@ static inline struct intel_gvt_gtt_entry *ppgtt_spt_get_entry(
{
struct intel_gvt *gvt = spt->vgpu->gvt;
struct intel_gvt_gtt_pte_ops *ops = gvt->gtt.pte_ops;
int ret;
e->type = get_entry_type(type);
if (WARN(!gtt_type_is_entry(e->type), "invalid entry type\n"))
return e;
return -EINVAL;
ops->get_entry(page_table, e, index, guest,
ret = ops->get_entry(page_table, e, index, guest,
spt->guest_page.gfn << GTT_PAGE_SHIFT,
spt->vgpu);
if (ret)
return ret;
ops->test_pse(e);
return e;
return 0;
}
static inline struct intel_gvt_gtt_entry *ppgtt_spt_set_entry(
static inline int ppgtt_spt_set_entry(
struct intel_vgpu_ppgtt_spt *spt,
void *page_table, int type,
struct intel_gvt_gtt_entry *e, unsigned long index,
@ -503,7 +513,7 @@ static inline struct intel_gvt_gtt_entry *ppgtt_spt_set_entry(
struct intel_gvt_gtt_pte_ops *ops = gvt->gtt.pte_ops;
if (WARN(!gtt_type_is_entry(e->type), "invalid entry type\n"))
return e;
return -EINVAL;
return ops->set_entry(page_table, e, index, guest,
spt->guest_page.gfn << GTT_PAGE_SHIFT,
@ -792,13 +802,13 @@ static struct intel_vgpu_ppgtt_spt *ppgtt_find_shadow_page(
#define for_each_present_guest_entry(spt, e, i) \
for (i = 0; i < pt_entries(spt); i++) \
if (spt->vgpu->gvt->gtt.pte_ops->test_present( \
ppgtt_get_guest_entry(spt, e, i)))
if (!ppgtt_get_guest_entry(spt, e, i) && \
spt->vgpu->gvt->gtt.pte_ops->test_present(e))
#define for_each_present_shadow_entry(spt, e, i) \
for (i = 0; i < pt_entries(spt); i++) \
if (spt->vgpu->gvt->gtt.pte_ops->test_present( \
ppgtt_get_shadow_entry(spt, e, i)))
if (!ppgtt_get_shadow_entry(spt, e, i) && \
spt->vgpu->gvt->gtt.pte_ops->test_present(e))
static void ppgtt_get_shadow_page(struct intel_vgpu_ppgtt_spt *spt)
{
@ -979,29 +989,26 @@ fail:
}
static int ppgtt_handle_guest_entry_removal(struct intel_vgpu_guest_page *gpt,
unsigned long index)
struct intel_gvt_gtt_entry *se, unsigned long index)
{
struct intel_vgpu_ppgtt_spt *spt = guest_page_to_ppgtt_spt(gpt);
struct intel_vgpu_shadow_page *sp = &spt->shadow_page;
struct intel_vgpu *vgpu = spt->vgpu;
struct intel_gvt_gtt_pte_ops *ops = vgpu->gvt->gtt.pte_ops;
struct intel_gvt_gtt_entry e;
int ret;
ppgtt_get_shadow_entry(spt, &e, index);
trace_gpt_change(spt->vgpu->id, "remove", spt, sp->type, e.val64,
trace_gpt_change(spt->vgpu->id, "remove", spt, sp->type, se->val64,
index);
if (!ops->test_present(&e))
if (!ops->test_present(se))
return 0;
if (ops->get_pfn(&e) == vgpu->gtt.scratch_pt[sp->type].page_mfn)
if (ops->get_pfn(se) == vgpu->gtt.scratch_pt[sp->type].page_mfn)
return 0;
if (gtt_type_is_pt(get_next_pt_type(e.type))) {
if (gtt_type_is_pt(get_next_pt_type(se->type))) {
struct intel_vgpu_ppgtt_spt *s =
ppgtt_find_shadow_page(vgpu, ops->get_pfn(&e));
ppgtt_find_shadow_page(vgpu, ops->get_pfn(se));
if (!s) {
gvt_vgpu_err("fail to find guest page\n");
ret = -ENXIO;
@ -1011,12 +1018,10 @@ static int ppgtt_handle_guest_entry_removal(struct intel_vgpu_guest_page *gpt,
if (ret)
goto fail;
}
ops->set_pfn(&e, vgpu->gtt.scratch_pt[sp->type].page_mfn);
ppgtt_set_shadow_entry(spt, &e, index);
return 0;
fail:
gvt_vgpu_err("fail: shadow page %p guest entry 0x%llx type %d\n",
spt, e.val64, e.type);
spt, se->val64, se->type);
return ret;
}
@ -1236,22 +1241,37 @@ static int ppgtt_handle_guest_write_page_table(
{
struct intel_vgpu_ppgtt_spt *spt = guest_page_to_ppgtt_spt(gpt);
struct intel_vgpu *vgpu = spt->vgpu;
int type = spt->shadow_page.type;
struct intel_gvt_gtt_pte_ops *ops = vgpu->gvt->gtt.pte_ops;
struct intel_gvt_gtt_entry se;
int ret;
int new_present;
new_present = ops->test_present(we);
ret = ppgtt_handle_guest_entry_removal(gpt, index);
if (ret)
goto fail;
/*
* Adding the new entry first and then removing the old one, that can
* guarantee the ppgtt table is validated during the window between
* adding and removal.
*/
ppgtt_get_shadow_entry(spt, &se, index);
if (new_present) {
ret = ppgtt_handle_guest_entry_add(gpt, we, index);
if (ret)
goto fail;
}
ret = ppgtt_handle_guest_entry_removal(gpt, &se, index);
if (ret)
goto fail;
if (!new_present) {
ops->set_pfn(&se, vgpu->gtt.scratch_pt[type].page_mfn);
ppgtt_set_shadow_entry(spt, &se, index);
}
return 0;
fail:
gvt_vgpu_err("fail: shadow page %p guest entry 0x%llx type %d.\n",
@ -1323,7 +1343,7 @@ static int ppgtt_handle_guest_write_page_table_bytes(void *gp,
struct intel_vgpu *vgpu = spt->vgpu;
struct intel_gvt_gtt_pte_ops *ops = vgpu->gvt->gtt.pte_ops;
const struct intel_gvt_device_info *info = &vgpu->gvt->device_info;
struct intel_gvt_gtt_entry we;
struct intel_gvt_gtt_entry we, se;
unsigned long index;
int ret;
@ -1339,7 +1359,8 @@ static int ppgtt_handle_guest_write_page_table_bytes(void *gp,
return ret;
} else {
if (!test_bit(index, spt->post_shadow_bitmap)) {
ret = ppgtt_handle_guest_entry_removal(gpt, index);
ppgtt_get_shadow_entry(spt, &se, index);
ret = ppgtt_handle_guest_entry_removal(gpt, &se, index);
if (ret)
return ret;
}
@ -1713,8 +1734,10 @@ unsigned long intel_vgpu_gma_to_gpa(struct intel_vgpu_mm *mm, unsigned long gma)
if (!vgpu_gmadr_is_valid(vgpu, gma))
goto err;
ggtt_get_guest_entry(mm, &e,
gma_ops->gma_to_ggtt_pte_index(gma));
ret = ggtt_get_guest_entry(mm, &e,
gma_ops->gma_to_ggtt_pte_index(gma));
if (ret)
goto err;
gpa = (pte_ops->get_pfn(&e) << GTT_PAGE_SHIFT)
+ (gma & ~GTT_PAGE_MASK);
@ -1724,7 +1747,9 @@ unsigned long intel_vgpu_gma_to_gpa(struct intel_vgpu_mm *mm, unsigned long gma)
switch (mm->page_table_level) {
case 4:
ppgtt_get_shadow_root_entry(mm, &e, 0);
ret = ppgtt_get_shadow_root_entry(mm, &e, 0);
if (ret)
goto err;
gma_index[0] = gma_ops->gma_to_pml4_index(gma);
gma_index[1] = gma_ops->gma_to_l4_pdp_index(gma);
gma_index[2] = gma_ops->gma_to_pde_index(gma);
@ -1732,15 +1757,19 @@ unsigned long intel_vgpu_gma_to_gpa(struct intel_vgpu_mm *mm, unsigned long gma)
index = 4;
break;
case 3:
ppgtt_get_shadow_root_entry(mm, &e,
ret = ppgtt_get_shadow_root_entry(mm, &e,
gma_ops->gma_to_l3_pdp_index(gma));
if (ret)
goto err;
gma_index[0] = gma_ops->gma_to_pde_index(gma);
gma_index[1] = gma_ops->gma_to_pte_index(gma);
index = 2;
break;
case 2:
ppgtt_get_shadow_root_entry(mm, &e,
ret = ppgtt_get_shadow_root_entry(mm, &e,
gma_ops->gma_to_pde_index(gma));
if (ret)
goto err;
gma_index[0] = gma_ops->gma_to_pte_index(gma);
index = 1;
break;
@ -1755,6 +1784,11 @@ unsigned long intel_vgpu_gma_to_gpa(struct intel_vgpu_mm *mm, unsigned long gma)
(i == index - 1));
if (ret)
goto err;
if (!pte_ops->test_present(&e)) {
gvt_dbg_core("GMA 0x%lx is not present\n", gma);
goto err;
}
}
gpa = (pte_ops->get_pfn(&e) << GTT_PAGE_SHIFT)
@ -2329,13 +2363,12 @@ void intel_vgpu_reset_ggtt(struct intel_vgpu *vgpu)
/**
* intel_vgpu_reset_gtt - reset the all GTT related status
* @vgpu: a vGPU
* @dmlr: true for vGPU Device Model Level Reset, false for GT Reset
*
* This function is called from vfio core to reset reset all
* GTT related status, including GGTT, PPGTT, scratch page.
*
*/
void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr)
void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu)
{
int i;
@ -2347,9 +2380,6 @@ void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr)
*/
intel_vgpu_free_mm(vgpu, INTEL_GVT_MM_PPGTT);
if (!dmlr)
return;
intel_vgpu_reset_ggtt(vgpu);
/* clear scratch page for security */

26
drivers/gpu/drm/i915/gvt/gtt.h
View File

@ -49,14 +49,18 @@ struct intel_gvt_gtt_entry {
};
struct intel_gvt_gtt_pte_ops {
struct intel_gvt_gtt_entry *(*get_entry)(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index, bool hypervisor_access, unsigned long gpa,
struct intel_vgpu *vgpu);
struct intel_gvt_gtt_entry *(*set_entry)(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index, bool hypervisor_access, unsigned long gpa,
struct intel_vgpu *vgpu);
int (*get_entry)(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index,
bool hypervisor_access,
unsigned long gpa,
struct intel_vgpu *vgpu);
int (*set_entry)(void *pt,
struct intel_gvt_gtt_entry *e,
unsigned long index,
bool hypervisor_access,
unsigned long gpa,
struct intel_vgpu *vgpu);
bool (*test_present)(struct intel_gvt_gtt_entry *e);
void (*clear_present)(struct intel_gvt_gtt_entry *e);
bool (*test_pse)(struct intel_gvt_gtt_entry *e);
@ -143,12 +147,12 @@ struct intel_vgpu_mm {
struct intel_vgpu *vgpu;
};
extern struct intel_gvt_gtt_entry *intel_vgpu_mm_get_entry(
extern int intel_vgpu_mm_get_entry(
struct intel_vgpu_mm *mm,
void *page_table, struct intel_gvt_gtt_entry *e,
unsigned long index);
extern struct intel_gvt_gtt_entry *intel_vgpu_mm_set_entry(
extern int intel_vgpu_mm_set_entry(
struct intel_vgpu_mm *mm,
void *page_table, struct intel_gvt_gtt_entry *e,
unsigned long index);
@ -208,7 +212,7 @@ extern void intel_vgpu_clean_gtt(struct intel_vgpu *vgpu);
void intel_vgpu_reset_ggtt(struct intel_vgpu *vgpu);
extern int intel_gvt_init_gtt(struct intel_gvt *gvt);
extern void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu, bool dmlr);
void intel_vgpu_reset_gtt(struct intel_vgpu *vgpu);
extern void intel_gvt_clean_gtt(struct intel_gvt *gvt);
extern struct intel_vgpu_mm *intel_gvt_find_ppgtt_mm(struct intel_vgpu *vgpu,

3
drivers/gpu/drm/i915/gvt/gvt.h
View File

@ -167,6 +167,7 @@ struct intel_vgpu {
atomic_t running_workload_num;
DECLARE_BITMAP(tlb_handle_pending, I915_NUM_ENGINES);
struct i915_gem_context *shadow_ctx;
DECLARE_BITMAP(shadow_ctx_desc_updated, I915_NUM_ENGINES);
#if IS_ENABLED(CONFIG_DRM_I915_GVT_KVMGT)
struct {
@ -482,6 +483,8 @@ int intel_vgpu_init_opregion(struct intel_vgpu *vgpu, u32 gpa);
int intel_vgpu_emulate_opregion_request(struct intel_vgpu *vgpu, u32 swsci);
void populate_pvinfo_page(struct intel_vgpu *vgpu);
int intel_gvt_scan_and_shadow_workload(struct intel_vgpu_workload *workload);
struct intel_gvt_ops {
int (*emulate_cfg_read)(struct intel_vgpu *, unsigned int, void *,
unsigned int);

50
drivers/gpu/drm/i915/gvt/handlers.c
View File

@ -113,9 +113,17 @@ static int new_mmio_info(struct intel_gvt *gvt,
info->offset = i;
p = find_mmio_info(gvt, info->offset);
if (p)
gvt_err("dup mmio definition offset %x\n",
if (p) {
WARN(1, "dup mmio definition offset %x\n",
info->offset);
kfree(info);
/* We return -EEXIST here to make GVT-g load fail.
* So duplicated MMIO can be found as soon as
* possible.
*/
return -EEXIST;
}
info->ro_mask = ro_mask;
info->device = device;
@ -1222,10 +1230,12 @@ static int power_well_ctl_mmio_write(struct intel_vgpu *vgpu,
{
write_vreg(vgpu, offset, p_data, bytes);
if (vgpu_vreg(vgpu, offset) & HSW_PWR_WELL_ENABLE_REQUEST)
vgpu_vreg(vgpu, offset) |= HSW_PWR_WELL_STATE_ENABLED;
if (vgpu_vreg(vgpu, offset) & HSW_PWR_WELL_CTL_REQ(HSW_DISP_PW_GLOBAL))
vgpu_vreg(vgpu, offset) |=
HSW_PWR_WELL_CTL_STATE(HSW_DISP_PW_GLOBAL);
else
vgpu_vreg(vgpu, offset) &= ~HSW_PWR_WELL_STATE_ENABLED;
vgpu_vreg(vgpu, offset) &=
~HSW_PWR_WELL_CTL_STATE(HSW_DISP_PW_GLOBAL);
return 0;
}
@ -2242,10 +2252,17 @@ static int init_generic_mmio_info(struct intel_gvt *gvt)
MMIO_D(GEN6_RC6p_THRESHOLD, D_ALL);
MMIO_D(GEN6_RC6pp_THRESHOLD, D_ALL);
MMIO_D(GEN6_PMINTRMSK, D_ALL);
MMIO_DH(HSW_PWR_WELL_BIOS, D_BDW, NULL, power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_DRIVER, D_BDW, NULL, power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_KVMR, D_BDW, NULL, power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_DEBUG, D_BDW, NULL, power_well_ctl_mmio_write);
/*
* Use an arbitrary power well controlled by the PWR_WELL_CTL
* register.
*/
MMIO_DH(HSW_PWR_WELL_CTL_BIOS(HSW_DISP_PW_GLOBAL), D_BDW, NULL,
power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_CTL_DRIVER(HSW_DISP_PW_GLOBAL), D_BDW, NULL,
power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_CTL_KVMR, D_BDW, NULL, power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_CTL_DEBUG(HSW_DISP_PW_GLOBAL), D_BDW, NULL,
power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_CTL5, D_BDW, NULL, power_well_ctl_mmio_write);
MMIO_DH(HSW_PWR_WELL_CTL6, D_BDW, NULL, power_well_ctl_mmio_write);
@ -2581,7 +2598,6 @@ static int init_broadwell_mmio_info(struct intel_gvt *gvt)
MMIO_F(0x24d0, 48, F_CMD_ACCESS, 0, 0, D_BDW_PLUS,
NULL, force_nonpriv_write);
MMIO_D(0x22040, D_BDW_PLUS);
MMIO_D(0x44484, D_BDW_PLUS);
MMIO_D(0x4448c, D_BDW_PLUS);
@ -2636,9 +2652,13 @@ static int init_skl_mmio_info(struct intel_gvt *gvt)
MMIO_F(_DPD_AUX_CH_CTL, 6 * 4, 0, 0, 0, D_SKL_PLUS, NULL,
dp_aux_ch_ctl_mmio_write);
MMIO_D(HSW_PWR_WELL_BIOS, D_SKL_PLUS);
MMIO_DH(HSW_PWR_WELL_DRIVER, D_SKL_PLUS, NULL,
skl_power_well_ctl_write);
/*
* Use an arbitrary power well controlled by the PWR_WELL_CTL
* register.
*/
MMIO_D(HSW_PWR_WELL_CTL_BIOS(SKL_DISP_PW_MISC_IO), D_SKL_PLUS);
MMIO_DH(HSW_PWR_WELL_CTL_DRIVER(SKL_DISP_PW_MISC_IO), D_SKL_PLUS, NULL,
skl_power_well_ctl_write);
MMIO_DH(GEN6_PCODE_MAILBOX, D_SKL_PLUS, NULL, mailbox_write);
MMIO_D(0xa210, D_SKL_PLUS);
@ -2831,7 +2851,6 @@ static int init_skl_mmio_info(struct intel_gvt *gvt)
MMIO_D(0x320f0, D_SKL | D_KBL);
MMIO_DFH(_REG_VCS2_EXCC, D_SKL_PLUS, F_CMD_ACCESS, NULL, NULL);
MMIO_DFH(_REG_VECS_EXCC, D_SKL_PLUS, F_CMD_ACCESS, NULL, NULL);
MMIO_D(0x70034, D_SKL_PLUS);
MMIO_D(0x71034, D_SKL_PLUS);
MMIO_D(0x72034, D_SKL_PLUS);
@ -2849,10 +2868,7 @@ static int init_skl_mmio_info(struct intel_gvt *gvt)
NULL, NULL);
MMIO_D(0x4ab8, D_KBL);
MMIO_D(0x940c, D_SKL_PLUS);
MMIO_D(0x2248, D_SKL_PLUS | D_KBL);
MMIO_D(0x4ab0, D_SKL | D_KBL);
MMIO_D(0x20d4, D_SKL | D_KBL);
return 0;
}

17
drivers/gpu/drm/i915/gvt/kvmgt.c
View File

@ -1170,10 +1170,27 @@ vgpu_id_show(struct device *dev, struct device_attribute *attr,
return sprintf(buf, "\n");
}
static ssize_t
hw_id_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
struct mdev_device *mdev = mdev_from_dev(dev);
if (mdev) {
struct intel_vgpu *vgpu = (struct intel_vgpu *)
mdev_get_drvdata(mdev);
return sprintf(buf, "%u\n",
vgpu->shadow_ctx->hw_id);
}
return sprintf(buf, "\n");
}
static DEVICE_ATTR_RO(vgpu_id);
static DEVICE_ATTR_RO(hw_id);
static struct attribute *intel_vgpu_attrs[] = {
&dev_attr_vgpu_id.attr,
&dev_attr_hw_id.attr,
NULL
};

54
drivers/gpu/drm/i915/gvt/render.c
View File

@ -207,18 +207,16 @@ static void load_mocs(struct intel_vgpu *vgpu, int ring_id)
offset.reg = regs[ring_id];
for (i = 0; i < 64; i++) {
gen9_render_mocs[ring_id][i] = I915_READ(offset);
gen9_render_mocs[ring_id][i] = I915_READ_FW(offset);
I915_WRITE(offset, vgpu_vreg(vgpu, offset));
POSTING_READ(offset);
offset.reg += 4;
}
if (ring_id == RCS) {
l3_offset.reg = 0xb020;
for (i = 0; i < 32; i++) {
gen9_render_mocs_L3[i] = I915_READ(l3_offset);
I915_WRITE(l3_offset, vgpu_vreg(vgpu, l3_offset));
POSTING_READ(l3_offset);
gen9_render_mocs_L3[i] = I915_READ_FW(l3_offset);
I915_WRITE_FW(l3_offset, vgpu_vreg(vgpu, l3_offset));
l3_offset.reg += 4;
}
}
@ -242,18 +240,16 @@ static void restore_mocs(struct intel_vgpu *vgpu, int ring_id)
offset.reg = regs[ring_id];
for (i = 0; i < 64; i++) {
vgpu_vreg(vgpu, offset) = I915_READ(offset);
I915_WRITE(offset, gen9_render_mocs[ring_id][i]);
POSTING_READ(offset);
vgpu_vreg(vgpu, offset) = I915_READ_FW(offset);
I915_WRITE_FW(offset, gen9_render_mocs[ring_id][i]);
offset.reg += 4;
}
if (ring_id == RCS) {
l3_offset.reg = 0xb020;
for (i = 0; i < 32; i++) {
vgpu_vreg(vgpu, l3_offset) = I915_READ(l3_offset);
I915_WRITE(l3_offset, gen9_render_mocs_L3[i]);
POSTING_READ(l3_offset);
vgpu_vreg(vgpu, l3_offset) = I915_READ_FW(l3_offset);
I915_WRITE_FW(l3_offset, gen9_render_mocs_L3[i]);
l3_offset.reg += 4;
}
}
@ -272,6 +268,7 @@ static void switch_mmio_to_vgpu(struct intel_vgpu *vgpu, int ring_id)
u32 ctx_ctrl = reg_state[CTX_CONTEXT_CONTROL_VAL];
u32 inhibit_mask =
_MASKED_BIT_ENABLE(CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT);
i915_reg_t last_reg = _MMIO(0);
if (IS_SKYLAKE(vgpu->gvt->dev_priv)
|| IS_KABYLAKE(vgpu->gvt->dev_priv)) {
@ -287,7 +284,7 @@ static void switch_mmio_to_vgpu(struct intel_vgpu *vgpu, int ring_id)
if (mmio->ring_id != ring_id)
continue;
mmio->value = I915_READ(mmio->reg);
mmio->value = I915_READ_FW(mmio->reg);
/*
* if it is an inhibit context, load in_context mmio
@ -304,13 +301,18 @@ static void switch_mmio_to_vgpu(struct intel_vgpu *vgpu, int ring_id)
else
v = vgpu_vreg(vgpu, mmio->reg);
I915_WRITE(mmio->reg, v);
POSTING_READ(mmio->reg);
I915_WRITE_FW(mmio->reg, v);
last_reg = mmio->reg;
trace_render_mmio(vgpu->id, "load",
i915_mmio_reg_offset(mmio->reg),
mmio->value, v);
}
/* Make sure the swiched MMIOs has taken effect. */
if (likely(INTEL_GVT_MMIO_OFFSET(last_reg)))
I915_READ_FW(last_reg);
handle_tlb_pending_event(vgpu, ring_id);
}
@ -319,6 +321,7 @@ static void switch_mmio_to_host(struct intel_vgpu *vgpu, int ring_id)
{
struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
struct render_mmio *mmio;
i915_reg_t last_reg = _MMIO(0);
u32 v;
int i, array_size;
@ -335,7 +338,7 @@ static void switch_mmio_to_host(struct intel_vgpu *vgpu, int ring_id)
if (mmio->ring_id != ring_id)
continue;
vgpu_vreg(vgpu, mmio->reg) = I915_READ(mmio->reg);
vgpu_vreg(vgpu, mmio->reg) = I915_READ_FW(mmio->reg);
if (mmio->mask) {
vgpu_vreg(vgpu, mmio->reg) &= ~(mmio->mask << 16);
@ -346,13 +349,17 @@ static void switch_mmio_to_host(struct intel_vgpu *vgpu, int ring_id)
if (mmio->in_context)
continue;
I915_WRITE(mmio->reg, v);
POSTING_READ(mmio->reg);
I915_WRITE_FW(mmio->reg, v);
last_reg = mmio->reg;
trace_render_mmio(vgpu->id, "restore",
i915_mmio_reg_offset(mmio->reg),
mmio->value, v);
}
/* Make sure the swiched MMIOs has taken effect. */
if (likely(INTEL_GVT_MMIO_OFFSET(last_reg)))
I915_READ_FW(last_reg);
}
/**
@ -367,12 +374,23 @@ static void switch_mmio_to_host(struct intel_vgpu *vgpu, int ring_id)
void intel_gvt_switch_mmio(struct intel_vgpu *pre,
struct intel_vgpu *next, int ring_id)
{
struct drm_i915_private *dev_priv;
if (WARN_ON(!pre && !next))
return;
gvt_dbg_render("switch ring %d from %s to %s\n", ring_id,
pre ? "vGPU" : "host", next ? "vGPU" : "HOST");
dev_priv = pre ? pre->gvt->dev_priv : next->gvt->dev_priv;
/**
* We are using raw mmio access wrapper to improve the
* performace for batch mmio read/write, so we need
* handle forcewake mannually.
*/
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
/**
* TODO: Optimize for vGPU to vGPU switch by merging
* switch_mmio_to_host() and switch_mmio_to_vgpu().
@ -382,4 +400,6 @@ void intel_gvt_switch_mmio(struct intel_vgpu *pre,
if (next)
switch_mmio_to_vgpu(next, ring_id);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}

111
drivers/gpu/drm/i915/gvt/scheduler.c
View File

@ -184,41 +184,52 @@ static int shadow_context_status_change(struct notifier_block *nb,
return NOTIFY_OK;
}
static int dispatch_workload(struct intel_vgpu_workload *workload)
static void shadow_context_descriptor_update(struct i915_gem_context *ctx,
struct intel_engine_cs *engine)
{
struct intel_context *ce = &ctx->engine[engine->id];
u64 desc = 0;
desc = ce->lrc_desc;
/* Update bits 0-11 of the context descriptor which includes flags
* like GEN8_CTX_* cached in desc_template
*/
desc &= U64_MAX << 12;
desc |= ctx->desc_template & ((1ULL << 12) - 1);
ce->lrc_desc = desc;
}
/**
* intel_gvt_scan_and_shadow_workload - audit the workload by scanning and
* shadow it as well, include ringbuffer,wa_ctx and ctx.
* @workload: an abstract entity for each execlist submission.
*
* This function is called before the workload submitting to i915, to make
* sure the content of the workload is valid.
*/
int intel_gvt_scan_and_shadow_workload(struct intel_vgpu_workload *workload)
{
int ring_id = workload->ring_id;
struct i915_gem_context *shadow_ctx = workload->vgpu->shadow_ctx;
struct drm_i915_private *dev_priv = workload->vgpu->gvt->dev_priv;
struct intel_engine_cs *engine = dev_priv->engine[ring_id];
struct drm_i915_gem_request *rq;
struct intel_vgpu *vgpu = workload->vgpu;
struct intel_ring *ring;
int ret;
gvt_dbg_sched("ring id %d prepare to dispatch workload %p\n",
ring_id, workload);
lockdep_assert_held(&dev_priv->drm.struct_mutex);
if (workload->shadowed)
return 0;
shadow_ctx->desc_template &= ~(0x3 << GEN8_CTX_ADDRESSING_MODE_SHIFT);
shadow_ctx->desc_template |= workload->ctx_desc.addressing_mode <<
GEN8_CTX_ADDRESSING_MODE_SHIFT;
mutex_lock(&dev_priv->drm.struct_mutex);
/* pin shadow context by gvt even the shadow context will be pinned
* when i915 alloc request. That is because gvt will update the guest
* context from shadow context when workload is completed, and at that
* moment, i915 may already unpined the shadow context to make the
* shadow_ctx pages invalid. So gvt need to pin itself. After update
* the guest context, gvt can unpin the shadow_ctx safely.
*/
ring = engine->context_pin(engine, shadow_ctx);
if (IS_ERR(ring)) {
ret = PTR_ERR(ring);
gvt_vgpu_err("fail to pin shadow context\n");
workload->status = ret;
mutex_unlock(&dev_priv->drm.struct_mutex);
return ret;
}
if (!test_and_set_bit(ring_id, vgpu->shadow_ctx_desc_updated))
shadow_context_descriptor_update(shadow_ctx,
dev_priv->engine[ring_id]);
rq = i915_gem_request_alloc(dev_priv->engine[ring_id], shadow_ctx);
if (IS_ERR(rq)) {
@ -231,7 +242,7 @@ static int dispatch_workload(struct intel_vgpu_workload *workload)
workload->req = i915_gem_request_get(rq);
ret = intel_gvt_scan_and_shadow_workload(workload);
ret = intel_gvt_scan_and_shadow_ringbuffer(workload);
if (ret)
goto out;
@ -246,25 +257,61 @@ static int dispatch_workload(struct intel_vgpu_workload *workload)
if (ret)
goto out;
workload->shadowed = true;
out:
return ret;
}
static int dispatch_workload(struct intel_vgpu_workload *workload)
{
int ring_id = workload->ring_id;
struct i915_gem_context *shadow_ctx = workload->vgpu->shadow_ctx;
struct drm_i915_private *dev_priv = workload->vgpu->gvt->dev_priv;
struct intel_engine_cs *engine = dev_priv->engine[ring_id];
struct intel_vgpu *vgpu = workload->vgpu;
struct intel_ring *ring;
int ret = 0;
gvt_dbg_sched("ring id %d prepare to dispatch workload %p\n",
ring_id, workload);
mutex_lock(&dev_priv->drm.struct_mutex);
ret = intel_gvt_scan_and_shadow_workload(workload);
if (ret)
goto out;
if (workload->prepare) {
ret = workload->prepare(workload);
if (ret)
goto out;
}
gvt_dbg_sched("ring id %d submit workload to i915 %p\n",
ring_id, workload->req);
/* pin shadow context by gvt even the shadow context will be pinned
* when i915 alloc request. That is because gvt will update the guest
* context from shadow context when workload is completed, and at that
* moment, i915 may already unpined the shadow context to make the
* shadow_ctx pages invalid. So gvt need to pin itself. After update
* the guest context, gvt can unpin the shadow_ctx safely.
*/
ring = engine->context_pin(engine, shadow_ctx);
if (IS_ERR(ring)) {
ret = PTR_ERR(ring);
gvt_vgpu_err("fail to pin shadow context\n");
goto out;
}
ret = 0;
workload->dispatched = true;
out:
if (ret)
workload->status = ret;
if (!IS_ERR_OR_NULL(rq))
i915_add_request(rq);
else
engine->context_unpin(engine, shadow_ctx);
if (!IS_ERR_OR_NULL(workload->req)) {
gvt_dbg_sched("ring id %d submit workload to i915 %p\n",
ring_id, workload->req);
i915_add_request(workload->req);
workload->dispatched = true;
}
mutex_unlock(&dev_priv->drm.struct_mutex);
return ret;
@ -631,5 +678,7 @@ int intel_vgpu_init_gvt_context(struct intel_vgpu *vgpu)
vgpu->shadow_ctx->engine[RCS].initialised = true;
bitmap_zero(vgpu->shadow_ctx_desc_updated, I915_NUM_ENGINES);
return 0;
}

1
drivers/gpu/drm/i915/gvt/scheduler.h
View File

@ -82,6 +82,7 @@ struct intel_vgpu_workload {
struct drm_i915_gem_request *req;
/* if this workload has been dispatched to i915? */
bool dispatched;
bool shadowed;
int status;
struct intel_vgpu_mm *shadow_mm;

7
drivers/gpu/drm/i915/gvt/vgpu.c
View File

@ -43,6 +43,7 @@ void populate_pvinfo_page(struct intel_vgpu *vgpu)
vgpu_vreg(vgpu, vgtif_reg(version_minor)) = 0;
vgpu_vreg(vgpu, vgtif_reg(display_ready)) = 0;
vgpu_vreg(vgpu, vgtif_reg(vgt_id)) = vgpu->id;
vgpu_vreg(vgpu, vgtif_reg(vgt_caps)) = VGT_CAPS_FULL_48BIT_PPGTT;
vgpu_vreg(vgpu, vgtif_reg(avail_rs.mappable_gmadr.base)) =
vgpu_aperture_gmadr_base(vgpu);
vgpu_vreg(vgpu, vgtif_reg(avail_rs.mappable_gmadr.size)) =
@ -504,11 +505,11 @@ void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr,
/* full GPU reset or device model level reset */
if (engine_mask == ALL_ENGINES || dmlr) {
intel_vgpu_reset_gtt(vgpu, dmlr);
/*fence will not be reset during virtual reset */
if (dmlr)
if (dmlr) {
intel_vgpu_reset_gtt(vgpu);
intel_vgpu_reset_resource(vgpu);
}
intel_vgpu_reset_mmio(vgpu, dmlr);
populate_pvinfo_page(vgpu);

99
drivers/gpu/drm/i915/i915_debugfs.c
View File

@ -543,75 +543,6 @@ static int i915_gem_gtt_info(struct seq_file *m, void *data)
return 0;
}
static int i915_gem_pageflip_info(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
struct drm_device *dev = &dev_priv->drm;
struct intel_crtc *crtc;
int ret;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
return ret;
for_each_intel_crtc(dev, crtc) {
const char pipe = pipe_name(crtc->pipe);
const char plane = plane_name(crtc->plane);
struct intel_flip_work *work;
spin_lock_irq(&dev->event_lock);
work = crtc->flip_work;
if (work == NULL) {
seq_printf(m, "No flip due on pipe %c (plane %c)\n",
pipe, plane);
} else {
u32 pending;
u32 addr;
pending = atomic_read(&work->pending);
if (pending) {
seq_printf(m, "Flip ioctl preparing on pipe %c (plane %c)\n",
pipe, plane);
} else {
seq_printf(m, "Flip pending (waiting for vsync) on pipe %c (plane %c)\n",
pipe, plane);
}
if (work->flip_queued_req) {
struct intel_engine_cs *engine = work->flip_queued_req->engine;
seq_printf(m, "Flip queued on %s at seqno %x, last submitted seqno %x [current breadcrumb %x], completed? %d\n",
engine->name,
work->flip_queued_req->global_seqno,
intel_engine_last_submit(engine),
intel_engine_get_seqno(engine),
i915_gem_request_completed(work->flip_queued_req));
} else
seq_printf(m, "Flip not associated with any ring\n");
seq_printf(m, "Flip queued on frame %d, (was ready on frame %d), now %d\n",
work->flip_queued_vblank,
work->flip_ready_vblank,
intel_crtc_get_vblank_counter(crtc));
seq_printf(m, "%d prepares\n", atomic_read(&work->pending));
if (INTEL_GEN(dev_priv) >= 4)
addr = I915_HI_DISPBASE(I915_READ(DSPSURF(crtc->plane)));
else
addr = I915_READ(DSPADDR(crtc->plane));
seq_printf(m, "Current scanout address 0x%08x\n", addr);
if (work->pending_flip_obj) {
seq_printf(m, "New framebuffer address 0x%08lx\n", (long)work->gtt_offset);
seq_printf(m, "MMIO update completed? %d\n", addr == work->gtt_offset);
}
}
spin_unlock_irq(&dev->event_lock);
}
mutex_unlock(&dev->struct_mutex);
return 0;
}
static int i915_gem_batch_pool_info(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
@ -2023,12 +1954,6 @@ static int i915_context_status(struct seq_file *m, void *unused)
seq_putc(m, '\n');
}
seq_printf(m,
"\tvma hashtable size=%u (actual %lu), count=%u\n",
ctx->vma_lut.ht_size,
BIT(ctx->vma_lut.ht_bits),
ctx->vma_lut.ht_count);
seq_putc(m, '\n');
}
@ -2852,7 +2777,7 @@ out:
static int i915_energy_uJ(struct seq_file *m, void *data)
{
struct drm_i915_private *dev_priv = node_to_i915(m->private);
u64 power;
unsigned long long power;
u32 units;
if (INTEL_GEN(dev_priv) < 6)
@ -2860,15 +2785,18 @@ static int i915_energy_uJ(struct seq_file *m, void *data)
intel_runtime_pm_get(dev_priv);
rdmsrl(MSR_RAPL_POWER_UNIT, power);
power = (power & 0x1f00) >> 8;
units = 1000000 / (1 << power); /* convert to uJ */
if (rdmsrl_safe(MSR_RAPL_POWER_UNIT, &power)) {
intel_runtime_pm_put(dev_priv);
return -ENODEV;
}
units = (power & 0x1f00) >> 8;
power = I915_READ(MCH_SECP_NRG_STTS);
power *= units;
power = (1000000 * power) >> units; /* convert to uJ */
intel_runtime_pm_put(dev_priv);
seq_printf(m, "%llu", (long long unsigned)power);
seq_printf(m, "%llu", power);
return 0;
}
@ -3394,8 +3322,10 @@ static int i915_engine_info(struct seq_file *m, void *unused)
ptr = I915_READ(RING_CONTEXT_STATUS_PTR(engine));
read = GEN8_CSB_READ_PTR(ptr);
write = GEN8_CSB_WRITE_PTR(ptr);
seq_printf(m, "\tExeclist CSB read %d, write %d\n",
read, write);
seq_printf(m, "\tExeclist CSB read %d, write %d, interrupt posted? %s\n",
read, write,
yesno(test_bit(ENGINE_IRQ_EXECLIST,
&engine->irq_posted)));
if (read >= GEN8_CSB_ENTRIES)
read = 0;
if (write >= GEN8_CSB_ENTRIES)
@ -4624,7 +4554,7 @@ static void gen9_sseu_device_status(struct drm_i915_private *dev_priv,
sseu->slice_mask |= BIT(s);
if (IS_GEN9_BC(dev_priv))
if (IS_GEN9_BC(dev_priv) || IS_CANNONLAKE(dev_priv))
sseu->subslice_mask =
INTEL_INFO(dev_priv)->sseu.subslice_mask;
@ -4854,7 +4784,6 @@ static const struct drm_info_list i915_debugfs_list[] = {
{"i915_gem_gtt", i915_gem_gtt_info, 0},
{"i915_gem_pin_display", i915_gem_gtt_info, 0, (void *)1},
{"i915_gem_stolen", i915_gem_stolen_list_info },
{"i915_gem_pageflip", i915_gem_pageflip_info, 0},
{"i915_gem_request", i915_gem_request_info, 0},
{"i915_gem_seqno", i915_gem_seqno_info, 0},
{"i915_gem_fence_regs", i915_gem_fence_regs_info, 0},

92
drivers/gpu/drm/i915/i915_drv.c
View File

@ -237,17 +237,17 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv)
!IS_KABYLAKE(dev_priv));
} else if (id == INTEL_PCH_KBP_DEVICE_ID_TYPE) {
dev_priv->pch_type = PCH_KBP;
DRM_DEBUG_KMS("Found KabyPoint PCH\n");
DRM_DEBUG_KMS("Found Kaby Lake PCH (KBP)\n");
WARN_ON(!IS_SKYLAKE(dev_priv) &&
!IS_KABYLAKE(dev_priv));
} else if (id == INTEL_PCH_CNP_DEVICE_ID_TYPE) {
dev_priv->pch_type = PCH_CNP;
DRM_DEBUG_KMS("Found CannonPoint PCH\n");
DRM_DEBUG_KMS("Found Cannon Lake PCH (CNP)\n");
WARN_ON(!IS_CANNONLAKE(dev_priv) &&
!IS_COFFEELAKE(dev_priv));
} else if (id == INTEL_PCH_CNP_LP_DEVICE_ID_TYPE) {
dev_priv->pch_type = PCH_CNP;
DRM_DEBUG_KMS("Found CannonPoint LP PCH\n");
DRM_DEBUG_KMS("Found Cannon Lake LP PCH (CNP-LP)\n");
WARN_ON(!IS_CANNONLAKE(dev_priv) &&
!IS_COFFEELAKE(dev_priv));
} else if (id == INTEL_PCH_P2X_DEVICE_ID_TYPE ||
@ -388,6 +388,7 @@ static int i915_getparam(struct drm_device *dev, void *data,
case I915_PARAM_HAS_EXEC_FENCE:
case I915_PARAM_HAS_EXEC_CAPTURE:
case I915_PARAM_HAS_EXEC_BATCH_FIRST:
case I915_PARAM_HAS_EXEC_FENCE_ARRAY:
/* For the time being all of these are always true;
* if some supported hardware does not have one of these
* features this value needs to be provided from
@ -596,7 +597,8 @@ static const struct vga_switcheroo_client_ops i915_switcheroo_ops = {
static void i915_gem_fini(struct drm_i915_private *dev_priv)
{
flush_workqueue(dev_priv->wq);
/* Flush any outstanding unpin_work. */
i915_gem_drain_workqueue(dev_priv);
mutex_lock(&dev_priv->drm.struct_mutex);
intel_uc_fini_hw(dev_priv);
@ -875,7 +877,6 @@ static int i915_driver_init_early(struct drm_i915_private *dev_priv,
spin_lock_init(&dev_priv->uncore.lock);
spin_lock_init(&dev_priv->mm.object_stat_lock);
spin_lock_init(&dev_priv->mmio_flip_lock);
mutex_init(&dev_priv->sb_lock);
mutex_init(&dev_priv->modeset_restore_lock);
mutex_init(&dev_priv->av_mutex);
@ -1240,6 +1241,7 @@ static void i915_driver_register(struct drm_i915_private *dev_priv)
*/
static void i915_driver_unregister(struct drm_i915_private *dev_priv)
{
intel_fbdev_unregister(dev_priv);
intel_audio_deinit(dev_priv);
intel_gpu_ips_teardown();
@ -1371,7 +1373,7 @@ void i915_driver_unload(struct drm_device *dev)
struct drm_i915_private *dev_priv = to_i915(dev);
struct pci_dev *pdev = dev_priv->drm.pdev;
intel_fbdev_fini(dev);
i915_driver_unregister(dev_priv);
if (i915_gem_suspend(dev_priv))
DRM_ERROR("failed to idle hardware; continuing to unload!\n");
@ -1382,8 +1384,6 @@ void i915_driver_unload(struct drm_device *dev)
intel_gvt_cleanup(dev_priv);
i915_driver_unregister(dev_priv);
intel_modeset_cleanup(dev);
/*
@ -1409,9 +1409,6 @@ void i915_driver_unload(struct drm_device *dev)
cancel_delayed_work_sync(&dev_priv->gpu_error.hangcheck_work);
i915_reset_error_state(dev_priv);
/* Flush any outstanding unpin_work. */
drain_workqueue(dev_priv->wq);
i915_gem_fini(dev_priv);
intel_uc_fini_fw(dev_priv);
intel_fbc_cleanup_cfb(dev_priv);
@ -1835,7 +1832,8 @@ static int i915_resume_switcheroo(struct drm_device *dev)
/**
* i915_reset - reset chip after a hang
* @dev_priv: device private to reset
* @i915: #drm_i915_private to reset
* @flags: Instructions
*
* Reset the chip. Useful if a hang is detected. Marks the device as wedged
* on failure.
@ -1850,33 +1848,34 @@ static int i915_resume_switcheroo(struct drm_device *dev)
* - re-init interrupt state
* - re-init display
*/
void i915_reset(struct drm_i915_private *dev_priv)
void i915_reset(struct drm_i915_private *i915, unsigned int flags)
{
struct i915_gpu_error *error = &dev_priv->gpu_error;
struct i915_gpu_error *error = &i915->gpu_error;
int ret;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
lockdep_assert_held(&i915->drm.struct_mutex);
GEM_BUG_ON(!test_bit(I915_RESET_BACKOFF, &error->flags));
if (!test_bit(I915_RESET_HANDOFF, &error->flags))
return;
/* Clear any previous failed attempts at recovery. Time to try again. */
if (!i915_gem_unset_wedged(dev_priv))
if (!i915_gem_unset_wedged(i915))
goto wakeup;
if (!(flags & I915_RESET_QUIET))
dev_notice(i915->drm.dev, "Resetting chip after gpu hang\n");
error->reset_count++;
pr_notice("drm/i915: Resetting chip after gpu hang\n");
disable_irq(dev_priv->drm.irq);
ret = i915_gem_reset_prepare(dev_priv);
disable_irq(i915->drm.irq);
ret = i915_gem_reset_prepare(i915);
if (ret) {
DRM_ERROR("GPU recovery failed\n");
intel_gpu_reset(dev_priv, ALL_ENGINES);
intel_gpu_reset(i915, ALL_ENGINES);
goto error;
}
ret = intel_gpu_reset(dev_priv, ALL_ENGINES);
ret = intel_gpu_reset(i915, ALL_ENGINES);
if (ret) {
if (ret != -ENODEV)
DRM_ERROR("Failed to reset chip: %i\n", ret);
@ -1885,8 +1884,8 @@ void i915_reset(struct drm_i915_private *dev_priv)
goto error;
}
i915_gem_reset(dev_priv);
intel_overlay_reset(dev_priv);
i915_gem_reset(i915);
intel_overlay_reset(i915);
/* Ok, now get things going again... */
@ -1902,17 +1901,17 @@ void i915_reset(struct drm_i915_private *dev_priv)
* was running at the time of the reset (i.e. we weren't VT
* switched away).
*/
ret = i915_gem_init_hw(dev_priv);
ret = i915_gem_init_hw(i915);
if (ret) {
DRM_ERROR("Failed hw init on reset %d\n", ret);
goto error;
}
i915_queue_hangcheck(dev_priv);
i915_queue_hangcheck(i915);
finish:
i915_gem_reset_finish(dev_priv);
enable_irq(dev_priv->drm.irq);
i915_gem_reset_finish(i915);
enable_irq(i915->drm.irq);
wakeup:
clear_bit(I915_RESET_HANDOFF, &error->flags);
@ -1920,14 +1919,15 @@ wakeup:
return;
error:
i915_gem_set_wedged(dev_priv);
i915_gem_retire_requests(dev_priv);
i915_gem_set_wedged(i915);
i915_gem_retire_requests(i915);
goto finish;
}
/**
* i915_reset_engine - reset GPU engine to recover from a hang
* @engine: engine to reset
* @flags: options
*
* Reset a specific GPU engine. Useful if a hang is detected.
* Returns zero on successful reset or otherwise an error code.
@ -1937,7 +1937,7 @@ error:
* - reset engine (which will force the engine to idle)
* - re-init/configure engine
*/
int i915_reset_engine(struct intel_engine_cs *engine)
int i915_reset_engine(struct intel_engine_cs *engine, unsigned int flags)
{
struct i915_gpu_error *error = &engine->i915->gpu_error;
struct drm_i915_gem_request *active_request;
@ -1945,7 +1945,11 @@ int i915_reset_engine(struct intel_engine_cs *engine)
GEM_BUG_ON(!test_bit(I915_RESET_ENGINE + engine->id, &error->flags));
DRM_DEBUG_DRIVER("resetting %s\n", engine->name);
if (!(flags & I915_RESET_QUIET)) {
dev_notice(engine->i915->drm.dev,
"Resetting %s after gpu hang\n", engine->name);
}
error->reset_engine_count[engine->id]++;
active_request = i915_gem_reset_prepare_engine(engine);
if (IS_ERR(active_request)) {
@ -1954,6 +1958,14 @@ int i915_reset_engine(struct intel_engine_cs *engine)
goto out;
}
ret = intel_gpu_reset(engine->i915, intel_engine_flag(engine));
if (ret) {
/* If we fail here, we expect to fallback to a global reset */
DRM_DEBUG_DRIVER("Failed to reset %s, ret=%d\n",
engine->name, ret);
goto out;
}
/*
* The request that caused the hang is stuck on elsp, we know the
* active request and can drop it, adjust head to skip the offending
@ -1961,18 +1973,6 @@ int i915_reset_engine(struct intel_engine_cs *engine)
*/
i915_gem_reset_engine(engine, active_request);
/* Finally, reset just this engine. */
ret = intel_gpu_reset(engine->i915, intel_engine_flag(engine));
i915_gem_reset_finish_engine(engine);
if (ret) {
/* If we fail here, we expect to fallback to a global reset */
DRM_DEBUG_DRIVER("Failed to reset %s, ret=%d\n",
engine->name, ret);
goto out;
}
/*
* The engine and its registers (and workarounds in case of render)
* have been reset to their default values. Follow the init_ring
@ -1982,8 +1982,8 @@ int i915_reset_engine(struct intel_engine_cs *engine)
if (ret)
goto out;
error->reset_engine_count[engine->id]++;
out:
i915_gem_reset_finish_engine(engine);
return ret;
}
@ -2730,6 +2730,8 @@ static const struct drm_ioctl_desc i915_ioctls[] = {
DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_GETPARAM, i915_gem_context_getparam_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_GEM_CONTEXT_SETPARAM, i915_gem_context_setparam_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_PERF_OPEN, i915_perf_open_ioctl, DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_PERF_ADD_CONFIG, i915_perf_add_config_ioctl, DRM_UNLOCKED|DRM_RENDER_ALLOW),
DRM_IOCTL_DEF_DRV(I915_PERF_REMOVE_CONFIG, i915_perf_remove_config_ioctl, DRM_UNLOCKED|DRM_RENDER_ALLOW),
};
static struct drm_driver driver = {
@ -2738,7 +2740,7 @@ static struct drm_driver driver = {
*/
.driver_features =
DRIVER_HAVE_IRQ | DRIVER_IRQ_SHARED | DRIVER_GEM | DRIVER_PRIME |
DRIVER_RENDER | DRIVER_MODESET | DRIVER_ATOMIC,
DRIVER_RENDER | DRIVER_MODESET | DRIVER_ATOMIC | DRIVER_SYNCOBJ,
.release = i915_driver_release,
.open = i915_driver_open,
.lastclose = i915_driver_lastclose,

186
drivers/gpu/drm/i915/i915_drv.h
View File

@ -80,8 +80,8 @@
#define DRIVER_NAME "i915"
#define DRIVER_DESC "Intel Graphics"
#define DRIVER_DATE "20170717"
#define DRIVER_TIMESTAMP 1500275179
#define DRIVER_DATE "20170818"
#define DRIVER_TIMESTAMP 1503088845
/* Use I915_STATE_WARN(x) and I915_STATE_WARN_ON() (rather than WARN() and
* WARN_ON()) for hw state sanity checks to check for unexpected conditions
@ -602,7 +602,7 @@ struct drm_i915_file_private {
* to limit the badly behaving clients access to gpu.
*/
#define I915_MAX_CLIENT_CONTEXT_BANS 3
int context_bans;
atomic_t context_bans;
};
/* Used by dp and fdi links */
@ -646,6 +646,7 @@ struct intel_opregion {
u32 swsci_sbcb_sub_functions;
struct opregion_asle *asle;
void *rvda;
void *vbt_firmware;
const void *vbt;
u32 vbt_size;
u32 *lid_state;
@ -715,11 +716,6 @@ struct drm_i915_display_funcs {
void (*fdi_link_train)(struct intel_crtc *crtc,
const struct intel_crtc_state *crtc_state);
void (*init_clock_gating)(struct drm_i915_private *dev_priv);
int (*queue_flip)(struct drm_device *dev, struct drm_crtc *crtc,
struct drm_framebuffer *fb,
struct drm_i915_gem_object *obj,
struct drm_i915_gem_request *req,
uint32_t flags);
void (*hpd_irq_setup)(struct drm_i915_private *dev_priv);
/* clock updates for mode set */
/* cursor updates */
@ -1063,6 +1059,11 @@ struct intel_fbc {
bool underrun_detected;
struct work_struct underrun_work;
/*
* Due to the atomic rules we can't access some structures without the
* appropriate locking, so we cache information here in order to avoid
* these problems.
*/
struct intel_fbc_state_cache {
struct i915_vma *vma;
@ -1084,6 +1085,13 @@ struct intel_fbc {
} fb;
} state_cache;
/*
* This structure contains everything that's relevant to program the
* hardware registers. When we want to figure out if we need to disable
* and re-enable FBC for a new configuration we just check if there's
* something different in the struct. The genx_fbc_activate functions
* are supposed to read from it in order to program the registers.
*/
struct intel_fbc_reg_params {
struct i915_vma *vma;
@ -1159,8 +1167,8 @@ enum intel_pch {
PCH_CPT, /* Cougarpoint/Pantherpoint PCH */
PCH_LPT, /* Lynxpoint/Wildcatpoint PCH */
PCH_SPT, /* Sunrisepoint PCH */
PCH_KBP, /* Kabypoint PCH */
PCH_CNP, /* Cannonpoint PCH */
PCH_KBP, /* Kaby Lake PCH */
PCH_CNP, /* Cannon Lake PCH */
PCH_NOP,
};
@ -1388,12 +1396,23 @@ struct i915_power_well {
bool hw_enabled;
u64 domains;
/* unique identifier for this power well */
unsigned long id;
enum i915_power_well_id id;
/*
* Arbitraty data associated with this power well. Platform and power
* well specific.
*/
unsigned long data;
union {
struct {
enum dpio_phy phy;
} bxt;
struct {
/* Mask of pipes whose IRQ logic is backed by the pw */
u8 irq_pipe_mask;
/* The pw is backing the VGA functionality */
bool has_vga:1;
bool has_fuses:1;
} hsw;
};
const struct i915_power_well_ops *ops;
};
@ -1510,6 +1529,8 @@ struct i915_gpu_error {
/* Protected by the above dev->gpu_error.lock. */
struct i915_gpu_state *first_error;
atomic_t pending_fb_pin;
unsigned long missed_irq_rings;
/**
@ -1569,6 +1590,7 @@ struct i915_gpu_error {
unsigned long flags;
#define I915_RESET_BACKOFF 0
#define I915_RESET_HANDOFF 1
#define I915_RESET_MODESET 2
#define I915_WEDGED (BITS_PER_LONG - 1)
#define I915_RESET_ENGINE (I915_WEDGED - I915_NUM_ENGINES)
@ -1884,6 +1906,7 @@ struct i915_workarounds {
struct i915_virtual_gpu {
bool active;
u32 caps;
};
/* used in computing the new watermarks state */
@ -1903,6 +1926,24 @@ struct i915_oa_reg {
u32 value;
};
struct i915_oa_config {
char uuid[UUID_STRING_LEN + 1];
int id;
const struct i915_oa_reg *mux_regs;
u32 mux_regs_len;
const struct i915_oa_reg *b_counter_regs;
u32 b_counter_regs_len;
const struct i915_oa_reg *flex_regs;
u32 flex_regs_len;
struct attribute_group sysfs_metric;
struct attribute *attrs[2];
struct device_attribute sysfs_metric_id;
atomic_t ref_count;
};
struct i915_perf_stream;
/**
@ -2015,12 +2056,36 @@ struct i915_perf_stream {
* type of configured stream.
*/
const struct i915_perf_stream_ops *ops;
/**
* @oa_config: The OA configuration used by the stream.
*/
struct i915_oa_config *oa_config;
};
/**
* struct i915_oa_ops - Gen specific implementation of an OA unit stream
*/
struct i915_oa_ops {
/**
* @is_valid_b_counter_reg: Validates register's address for
* programming boolean counters for a particular platform.
*/
bool (*is_valid_b_counter_reg)(struct drm_i915_private *dev_priv,
u32 addr);
/**
* @is_valid_mux_reg: Validates register's address for programming mux
* for a particular platform.
*/
bool (*is_valid_mux_reg)(struct drm_i915_private *dev_priv, u32 addr);
/**
* @is_valid_flex_reg: Validates register's address for programming
* flex EU filtering for a particular platform.
*/
bool (*is_valid_flex_reg)(struct drm_i915_private *dev_priv, u32 addr);
/**
* @init_oa_buffer: Resets the head and tail pointers of the
* circular buffer for periodic OA reports.
@ -2038,21 +2103,14 @@ struct i915_oa_ops {
*/
void (*init_oa_buffer)(struct drm_i915_private *dev_priv);
/**
* @select_metric_set: The auto generated code that checks whether a
* requested OA config is applicable to the system and if so sets up
* the mux, oa and flex eu register config pointers according to the
* current dev_priv->perf.oa.metrics_set.
*/
int (*select_metric_set)(struct drm_i915_private *dev_priv);
/**
* @enable_metric_set: Selects and applies any MUX configuration to set
* up the Boolean and Custom (B/C) counters that are part of the
* counter reports being sampled. May apply system constraints such as
* disabling EU clock gating as required.
*/
int (*enable_metric_set)(struct drm_i915_private *dev_priv);
int (*enable_metric_set)(struct drm_i915_private *dev_priv,
const struct i915_oa_config *oa_config);
/**
* @disable_metric_set: Remove system constraints associated with using
@ -2098,6 +2156,7 @@ struct drm_i915_private {
struct kmem_cache *objects;
struct kmem_cache *vmas;
struct kmem_cache *luts;
struct kmem_cache *requests;
struct kmem_cache *dependencies;
struct kmem_cache *priorities;
@ -2148,9 +2207,6 @@ struct drm_i915_private {
/* protects the irq masks */
spinlock_t irq_lock;
/* protects the mmio flip data */
spinlock_t mmio_flip_lock;
bool display_irqs_enabled;
/* To control wakeup latency, e.g. for irq-driven dp aux transfers. */
@ -2255,7 +2311,6 @@ struct drm_i915_private {
struct intel_crtc *plane_to_crtc_mapping[I915_MAX_PIPES];
struct intel_crtc *pipe_to_crtc_mapping[I915_MAX_PIPES];
wait_queue_head_t pending_flip_queue;
#ifdef CONFIG_DEBUG_FS
struct intel_pipe_crc pipe_crc[I915_MAX_PIPES];
@ -2416,10 +2471,32 @@ struct drm_i915_private {
struct kobject *metrics_kobj;
struct ctl_table_header *sysctl_header;
/*
* Lock associated with adding/modifying/removing OA configs
* in dev_priv->perf.metrics_idr.
*/
struct mutex metrics_lock;
/*
* List of dynamic configurations, you need to hold
* dev_priv->perf.metrics_lock to access it.
*/
struct idr metrics_idr;
/*
* Lock associated with anything below within this structure
* except exclusive_stream.
*/
struct mutex lock;
struct list_head streams;
struct {
/*
* The stream currently using the OA unit. If accessed
* outside a syscall associated to its file
* descriptor, you need to hold
* dev_priv->drm.struct_mutex.
*/
struct i915_perf_stream *exclusive_stream;
u32 specific_ctx_id;
@ -2438,16 +2515,7 @@ struct drm_i915_private {
int period_exponent;
int timestamp_frequency;
int metrics_set;
const struct i915_oa_reg *mux_regs[6];
int mux_regs_lens[6];
int n_mux_configs;
const struct i915_oa_reg *b_counter_regs;
int b_counter_regs_len;
const struct i915_oa_reg *flex_regs;
int flex_regs_len;
struct i915_oa_config test_config;
struct {
struct i915_vma *vma;
@ -2534,7 +2602,6 @@ struct drm_i915_private {
struct i915_oa_ops ops;
const struct i915_oa_format *oa_formats;
int n_builtin_sets;
} oa;
} perf;
@ -3108,8 +3175,12 @@ extern int i915_driver_load(struct pci_dev *pdev,
extern void i915_driver_unload(struct drm_device *dev);
extern int intel_gpu_reset(struct drm_i915_private *dev_priv, u32 engine_mask);
extern bool intel_has_gpu_reset(struct drm_i915_private *dev_priv);
extern void i915_reset(struct drm_i915_private *dev_priv);
extern int i915_reset_engine(struct intel_engine_cs *engine);
#define I915_RESET_QUIET BIT(0)
extern void i915_reset(struct drm_i915_private *i915, unsigned int flags);
extern int i915_reset_engine(struct intel_engine_cs *engine,
unsigned int flags);
extern bool intel_has_reset_engine(struct drm_i915_private *dev_priv);
extern int intel_guc_reset(struct drm_i915_private *dev_priv);
extern void intel_engine_init_hangcheck(struct intel_engine_cs *engine);
@ -3129,7 +3200,8 @@ void intel_hpd_irq_handler(struct drm_i915_private *dev_priv,
void intel_hpd_init(struct drm_i915_private *dev_priv);
void intel_hpd_init_work(struct drm_i915_private *dev_priv);
void intel_hpd_cancel_work(struct drm_i915_private *dev_priv);
bool intel_hpd_pin_to_port(enum hpd_pin pin, enum port *port);
enum port intel_hpd_pin_to_port(enum hpd_pin pin);
enum hpd_pin intel_hpd_pin(enum port port);
bool intel_hpd_disable(struct drm_i915_private *dev_priv, enum hpd_pin pin);
void intel_hpd_enable(struct drm_i915_private *dev_priv, enum hpd_pin pin);
@ -3298,6 +3370,26 @@ static inline void i915_gem_drain_freed_objects(struct drm_i915_private *i915)
} while (flush_work(&i915->mm.free_work));
}
static inline void i915_gem_drain_workqueue(struct drm_i915_private *i915)
{
/*
* Similar to objects above (see i915_gem_drain_freed-objects), in
* general we have workers that are armed by RCU and then rearm
* themselves in their callbacks. To be paranoid, we need to
* drain the workqueue a second time after waiting for the RCU
* grace period so that we catch work queued via RCU from the first
* pass. As neither drain_workqueue() nor flush_workqueue() report
* a result, we make an assumption that we only don't require more
* than 2 passes to catch all recursive RCU delayed work.
*
*/
int pass = 2;
do {
rcu_barrier();
drain_workqueue(i915->wq);
} while (--pass);
}
struct i915_vma * __must_check
i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj,
const struct i915_ggtt_view *view,
@ -3595,6 +3687,10 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx,
int i915_perf_open_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
int i915_perf_add_config_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
int i915_perf_remove_config_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
void i915_oa_init_reg_state(struct intel_engine_cs *engine,
struct i915_gem_context *ctx,
uint32_t *reg_state);
@ -4082,6 +4178,11 @@ static inline unsigned long msecs_to_jiffies_timeout(const unsigned int m)
static inline unsigned long nsecs_to_jiffies_timeout(const u64 n)
{
/* nsecs_to_jiffies64() does not guard against overflow */
if (NSEC_PER_SEC % HZ &&
div_u64(n, NSEC_PER_SEC) >= MAX_JIFFY_OFFSET / HZ)
return MAX_JIFFY_OFFSET;
return min_t(u64, MAX_JIFFY_OFFSET, nsecs_to_jiffies64(n) + 1);
}
@ -4228,10 +4329,11 @@ int remap_io_mapping(struct vm_area_struct *vma,
unsigned long addr, unsigned long pfn, unsigned long size,
struct io_mapping *iomap);
static inline bool i915_gem_object_is_coherent(struct drm_i915_gem_object *obj)
static inline bool
intel_engine_can_store_dword(struct intel_engine_cs *engine)
{
return (obj->cache_level != I915_CACHE_NONE ||
HAS_LLC(to_i915(obj->base.dev)));
return __intel_engine_can_store_dword(INTEL_GEN(engine->i915),
engine->class);
}
#endif

283
drivers/gpu/drm/i915/i915_gem.c
View File

@ -52,7 +52,7 @@ static bool cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
if (obj->cache_dirty)
return false;
if (!obj->cache_coherent)
if (!(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE))
return true;
return obj->pin_display;
@ -253,7 +253,7 @@ __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
if (needs_clflush &&
(obj->base.read_domains & I915_GEM_DOMAIN_CPU) == 0 &&
!obj->cache_coherent)
!(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_READ))
drm_clflush_sg(pages);
__start_cpu_write(obj);
@ -561,46 +561,6 @@ static struct intel_rps_client *to_rps_client(struct drm_file *file)
return &fpriv->rps;
}
int
i915_gem_object_attach_phys(struct drm_i915_gem_object *obj,
int align)
{
int ret;
if (align > obj->base.size)
return -EINVAL;
if (obj->ops == &i915_gem_phys_ops)
return 0;
if (obj->mm.madv != I915_MADV_WILLNEED)
return -EFAULT;
if (obj->base.filp == NULL)
return -EINVAL;
ret = i915_gem_object_unbind(obj);
if (ret)
return ret;
__i915_gem_object_put_pages(obj, I915_MM_NORMAL);
if (obj->mm.pages)
return -EBUSY;
GEM_BUG_ON(obj->ops != &i915_gem_object_ops);
obj->ops = &i915_gem_phys_ops;
ret = i915_gem_object_pin_pages(obj);
if (ret)
goto err_xfer;
return 0;
err_xfer:
obj->ops = &i915_gem_object_ops;
return ret;
}
static int
i915_gem_phys_pwrite(struct drm_i915_gem_object *obj,
struct drm_i915_gem_pwrite *args,
@ -840,7 +800,8 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,
if (ret)
return ret;
if (obj->cache_coherent || !static_cpu_has(X86_FEATURE_CLFLUSH)) {
if (obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_READ ||
!static_cpu_has(X86_FEATURE_CLFLUSH)) {
ret = i915_gem_object_set_to_cpu_domain(obj, false);
if (ret)
goto err_unpin;
@ -892,7 +853,8 @@ int i915_gem_obj_prepare_shmem_write(struct drm_i915_gem_object *obj,
if (ret)
return ret;
if (obj->cache_coherent || !static_cpu_has(X86_FEATURE_CLFLUSH)) {
if (obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE ||
!static_cpu_has(X86_FEATURE_CLFLUSH)) {
ret = i915_gem_object_set_to_cpu_domain(obj, true);
if (ret)
goto err_unpin;
@ -2740,34 +2702,38 @@ i915_gem_object_pwrite_gtt(struct drm_i915_gem_object *obj,
return 0;
}
static bool ban_context(const struct i915_gem_context *ctx)
static bool ban_context(const struct i915_gem_context *ctx,
unsigned int score)
{
return (i915_gem_context_is_bannable(ctx) &&
ctx->ban_score >= CONTEXT_SCORE_BAN_THRESHOLD);
score >= CONTEXT_SCORE_BAN_THRESHOLD);
}
static void i915_gem_context_mark_guilty(struct i915_gem_context *ctx)
{
ctx->guilty_count++;
ctx->ban_score += CONTEXT_SCORE_GUILTY;
if (ban_context(ctx))
i915_gem_context_set_banned(ctx);
unsigned int score;
bool banned;
atomic_inc(&ctx->guilty_count);
score = atomic_add_return(CONTEXT_SCORE_GUILTY, &ctx->ban_score);
banned = ban_context(ctx, score);
DRM_DEBUG_DRIVER("context %s marked guilty (score %d) banned? %s\n",
ctx->name, ctx->ban_score,
yesno(i915_gem_context_is_banned(ctx)));
if (!i915_gem_context_is_banned(ctx) || IS_ERR_OR_NULL(ctx->file_priv))
ctx->name, score, yesno(banned));
if (!banned)
return;
ctx->file_priv->context_bans++;
DRM_DEBUG_DRIVER("client %s has had %d context banned\n",
ctx->name, ctx->file_priv->context_bans);
i915_gem_context_set_banned(ctx);
if (!IS_ERR_OR_NULL(ctx->file_priv)) {
atomic_inc(&ctx->file_priv->context_bans);
DRM_DEBUG_DRIVER("client %s has had %d context banned\n",
ctx->name, atomic_read(&ctx->file_priv->context_bans));
}
}
static void i915_gem_context_mark_innocent(struct i915_gem_context *ctx)
{
ctx->active_count++;
atomic_inc(&ctx->active_count);
}
struct drm_i915_gem_request *
@ -2850,11 +2816,9 @@ i915_gem_reset_prepare_engine(struct intel_engine_cs *engine)
if (engine->irq_seqno_barrier)
engine->irq_seqno_barrier(engine);
if (engine_stalled(engine)) {
request = i915_gem_find_active_request(engine);
if (request && request->fence.error == -EIO)
request = ERR_PTR(-EIO); /* Previous reset failed! */
}
request = i915_gem_find_active_request(engine);
if (request && request->fence.error == -EIO)
request = ERR_PTR(-EIO); /* Previous reset failed! */
return request;
}
@ -2923,12 +2887,11 @@ static void engine_skip_context(struct drm_i915_gem_request *request)
spin_unlock_irqrestore(&engine->timeline->lock, flags);
}
/* Returns true if the request was guilty of hang */
static bool i915_gem_reset_request(struct drm_i915_gem_request *request)
/* Returns the request if it was guilty of the hang */
static struct drm_i915_gem_request *
i915_gem_reset_request(struct intel_engine_cs *engine,
struct drm_i915_gem_request *request)
{
/* Read once and return the resolution */
const bool guilty = !i915_gem_request_completed(request);
/* The guilty request will get skipped on a hung engine.
*
* Users of client default contexts do not rely on logical
@ -2950,27 +2913,47 @@ static bool i915_gem_reset_request(struct drm_i915_gem_request *request)
* subsequent hangs.
*/
if (guilty) {
if (engine_stalled(engine)) {
i915_gem_context_mark_guilty(request->ctx);
skip_request(request);
/* If this context is now banned, skip all pending requests. */
if (i915_gem_context_is_banned(request->ctx))
engine_skip_context(request);
} else {
i915_gem_context_mark_innocent(request->ctx);
dma_fence_set_error(&request->fence, -EAGAIN);
/*
* Since this is not the hung engine, it may have advanced
* since the hang declaration. Double check by refinding
* the active request at the time of the reset.
*/
request = i915_gem_find_active_request(engine);
if (request) {
i915_gem_context_mark_innocent(request->ctx);
dma_fence_set_error(&request->fence, -EAGAIN);
/* Rewind the engine to replay the incomplete rq */
spin_lock_irq(&engine->timeline->lock);
request = list_prev_entry(request, link);
if (&request->link == &engine->timeline->requests)
request = NULL;
spin_unlock_irq(&engine->timeline->lock);
}
}
return guilty;
return request;
}
void i915_gem_reset_engine(struct intel_engine_cs *engine,
struct drm_i915_gem_request *request)
{
if (request && i915_gem_reset_request(request)) {
engine->irq_posted = 0;
if (request)
request = i915_gem_reset_request(engine, request);
if (request) {
DRM_DEBUG_DRIVER("resetting %s to restart from tail of request 0x%x\n",
engine->name, request->global_seqno);
/* If this context is now banned, skip all pending requests. */
if (i915_gem_context_is_banned(request->ctx))
engine_skip_context(request);
}
/* Setup the CS to resume from the breadcrumb of the hung request */
@ -3026,6 +3009,7 @@ void i915_gem_reset_finish(struct drm_i915_private *dev_priv)
static void nop_submit_request(struct drm_i915_gem_request *request)
{
GEM_BUG_ON(!i915_terminally_wedged(&request->i915->gpu_error));
dma_fence_set_error(&request->fence, -EIO);
i915_gem_request_submit(request);
intel_engine_init_global_seqno(request->engine, request->global_seqno);
@ -3051,13 +3035,6 @@ static void engine_set_wedged(struct intel_engine_cs *engine)
dma_fence_set_error(&request->fence, -EIO);
spin_unlock_irqrestore(&engine->timeline->lock, flags);
/* Mark all pending requests as complete so that any concurrent
* (lockless) lookup doesn't try and wait upon the request as we
* reset it.
*/
intel_engine_init_global_seqno(engine,
intel_engine_last_submit(engine));
/*
* Clear the execlists queue up before freeing the requests, as those
* are the ones that keep the context and ringbuffer backing objects
@ -3086,6 +3063,13 @@ static void engine_set_wedged(struct intel_engine_cs *engine)
*/
clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
}
/* Mark all pending requests as complete so that any concurrent
* (lockless) lookup doesn't try and wait upon the request as we
* reset it.
*/
intel_engine_init_global_seqno(engine,
intel_engine_last_submit(engine));
}
static int __i915_gem_set_wedged_BKL(void *data)
@ -3094,10 +3078,12 @@ static int __i915_gem_set_wedged_BKL(void *data)
struct intel_engine_cs *engine;
enum intel_engine_id id;
set_bit(I915_WEDGED, &i915->gpu_error.flags);
for_each_engine(engine, i915, id)
engine_set_wedged(engine);
set_bit(I915_WEDGED, &i915->gpu_error.flags);
wake_up_all(&i915->gpu_error.reset_queue);
return 0;
}
@ -3256,25 +3242,33 @@ out_rearm:
void i915_gem_close_object(struct drm_gem_object *gem, struct drm_file *file)
{
struct drm_i915_private *i915 = to_i915(gem->dev);
struct drm_i915_gem_object *obj = to_intel_bo(gem);
struct drm_i915_file_private *fpriv = file->driver_priv;
struct i915_vma *vma, *vn;
struct i915_lut_handle *lut, *ln;
mutex_lock(&obj->base.dev->struct_mutex);
list_for_each_entry_safe(vma, vn, &obj->vma_list, obj_link)
if (vma->vm->file == fpriv)
mutex_lock(&i915->drm.struct_mutex);
list_for_each_entry_safe(lut, ln, &obj->lut_list, obj_link) {
struct i915_gem_context *ctx = lut->ctx;
struct i915_vma *vma;
if (ctx->file_priv != fpriv)
continue;
vma = radix_tree_delete(&ctx->handles_vma, lut->handle);
if (!i915_vma_is_ggtt(vma))
i915_vma_close(vma);
vma = obj->vma_hashed;
if (vma && vma->ctx->file_priv == fpriv)
i915_vma_unlink_ctx(vma);
list_del(&lut->obj_link);
list_del(&lut->ctx_link);
if (i915_gem_object_is_active(obj) &&
!i915_gem_object_has_active_reference(obj)) {
i915_gem_object_set_active_reference(obj);
i915_gem_object_get(obj);
kmem_cache_free(i915->luts, lut);
__i915_gem_object_release_unless_active(obj);
}
mutex_unlock(&obj->base.dev->struct_mutex);
mutex_unlock(&i915->drm.struct_mutex);
}
static unsigned long to_wait_timeout(s64 timeout_ns)
@ -3300,7 +3294,7 @@ static unsigned long to_wait_timeout(s64 timeout_ns)
* -ERESTARTSYS: signal interrupted the wait
* -ENONENT: object doesn't exist
* Also possible, but rare:
* -EAGAIN: GPU wedged
* -EAGAIN: incomplete, restart syscall
* -ENOMEM: damn
* -ENODEV: Internal IRQ fail
* -E?: The add request failed
@ -3348,6 +3342,10 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
*/
if (ret == -ETIME && !nsecs_to_jiffies(args->timeout_ns))
args->timeout_ns = 0;
/* Asked to wait beyond the jiffie/scheduler precision? */
if (ret == -ETIME && args->timeout_ns)
ret = -EAGAIN;
}
i915_gem_object_put(obj);
@ -3689,8 +3687,7 @@ restart:
list_for_each_entry(vma, &obj->vma_list, obj_link)
vma->node.color = cache_level;
obj->cache_level = cache_level;
obj->cache_coherent = i915_gem_object_is_coherent(obj);
i915_gem_object_set_cache_coherency(obj, cache_level);
obj->cache_dirty = true; /* Always invalidate stale cachelines */
return 0;
@ -4263,6 +4260,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
INIT_LIST_HEAD(&obj->global_link);
INIT_LIST_HEAD(&obj->userfault_link);
INIT_LIST_HEAD(&obj->vma_list);
INIT_LIST_HEAD(&obj->lut_list);
INIT_LIST_HEAD(&obj->batch_pool_link);
obj->ops = ops;
@ -4295,6 +4293,7 @@ i915_gem_object_create(struct drm_i915_private *dev_priv, u64 size)
{
struct drm_i915_gem_object *obj;
struct address_space *mapping;
unsigned int cache_level;
gfp_t mask;
int ret;
@ -4333,7 +4332,7 @@ i915_gem_object_create(struct drm_i915_private *dev_priv, u64 size)
obj->base.write_domain = I915_GEM_DOMAIN_CPU;
obj->base.read_domains = I915_GEM_DOMAIN_CPU;
if (HAS_LLC(dev_priv)) {
if (HAS_LLC(dev_priv))
/* On some devices, we can have the GPU use the LLC (the CPU
* cache) for about a 10% performance improvement
* compared to uncached. Graphics requests other than
@ -4346,12 +4345,11 @@ i915_gem_object_create(struct drm_i915_private *dev_priv, u64 size)
* However, we maintain the display planes as UC, and so
* need to rebind when first used as such.
*/
obj->cache_level = I915_CACHE_LLC;
} else
obj->cache_level = I915_CACHE_NONE;
cache_level = I915_CACHE_LLC;
else
cache_level = I915_CACHE_NONE;
obj->cache_coherent = i915_gem_object_is_coherent(obj);
obj->cache_dirty = !obj->cache_coherent;
i915_gem_object_set_cache_coherency(obj, cache_level);
trace_i915_gem_object_create(obj);
@ -4506,8 +4504,8 @@ void __i915_gem_object_release_unless_active(struct drm_i915_gem_object *obj)
{
lockdep_assert_held(&obj->base.dev->struct_mutex);
GEM_BUG_ON(i915_gem_object_has_active_reference(obj));
if (i915_gem_object_is_active(obj))
if (!i915_gem_object_has_active_reference(obj) &&
i915_gem_object_is_active(obj))
i915_gem_object_set_active_reference(obj);
else
i915_gem_object_put(obj);
@ -4899,12 +4897,16 @@ i915_gem_load_init(struct drm_i915_private *dev_priv)
if (!dev_priv->vmas)
goto err_objects;
dev_priv->luts = KMEM_CACHE(i915_lut_handle, 0);
if (!dev_priv->luts)
goto err_vmas;
dev_priv->requests = KMEM_CACHE(drm_i915_gem_request,
SLAB_HWCACHE_ALIGN |
SLAB_RECLAIM_ACCOUNT |
SLAB_TYPESAFE_BY_RCU);
if (!dev_priv->requests)
goto err_vmas;
goto err_luts;
dev_priv->dependencies = KMEM_CACHE(i915_dependency,
SLAB_HWCACHE_ALIGN |
@ -4936,8 +4938,6 @@ i915_gem_load_init(struct drm_i915_private *dev_priv)
init_waitqueue_head(&dev_priv->gpu_error.wait_queue);
init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
init_waitqueue_head(&dev_priv->pending_flip_queue);
atomic_set(&dev_priv->mm.bsd_engine_dispatch_index, 0);
spin_lock_init(&dev_priv->fb_tracking.lock);
@ -4950,6 +4950,8 @@ err_dependencies:
kmem_cache_destroy(dev_priv->dependencies);
err_requests:
kmem_cache_destroy(dev_priv->requests);
err_luts:
kmem_cache_destroy(dev_priv->luts);
err_vmas:
kmem_cache_destroy(dev_priv->vmas);
err_objects:
@ -4972,6 +4974,7 @@ void i915_gem_load_cleanup(struct drm_i915_private *dev_priv)
kmem_cache_destroy(dev_priv->priorities);
kmem_cache_destroy(dev_priv->dependencies);
kmem_cache_destroy(dev_priv->requests);
kmem_cache_destroy(dev_priv->luts);
kmem_cache_destroy(dev_priv->vmas);
kmem_cache_destroy(dev_priv->objects);
@ -5304,6 +5307,64 @@ i915_gem_object_get_dma_address(struct drm_i915_gem_object *obj,
return sg_dma_address(sg) + (offset << PAGE_SHIFT);
}
int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
{
struct sg_table *pages;
int err;
if (align > obj->base.size)
return -EINVAL;
if (obj->ops == &i915_gem_phys_ops)
return 0;
if (obj->ops != &i915_gem_object_ops)
return -EINVAL;
err = i915_gem_object_unbind(obj);
if (err)
return err;
mutex_lock(&obj->mm.lock);
if (obj->mm.madv != I915_MADV_WILLNEED) {
err = -EFAULT;
goto err_unlock;
}
if (obj->mm.quirked) {
err = -EFAULT;
goto err_unlock;
}
if (obj->mm.mapping) {
err = -EBUSY;
goto err_unlock;
}
pages = obj->mm.pages;
obj->ops = &i915_gem_phys_ops;
err = ____i915_gem_object_get_pages(obj);
if (err)
goto err_xfer;
/* Perma-pin (until release) the physical set of pages */
__i915_gem_object_pin_pages(obj);
if (!IS_ERR_OR_NULL(pages))
i915_gem_object_ops.put_pages(obj, pages);
mutex_unlock(&obj->mm.lock);
return 0;
err_xfer:
obj->ops = &i915_gem_object_ops;
obj->mm.pages = pages;
err_unlock:
mutex_unlock(&obj->mm.lock);
return err;
}
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftests/scatterlist.c"
#include "selftests/mock_gem_device.c"

3
drivers/gpu/drm/i915/i915_gem_clflush.c
View File

@ -139,7 +139,8 @@ bool i915_gem_clflush_object(struct drm_i915_gem_object *obj,
* snooping behaviour occurs naturally as the result of our domain
* tracking.
*/
if (!(flags & I915_CLFLUSH_FORCE) && obj->cache_coherent)
if (!(flags & I915_CLFLUSH_FORCE) &&
obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_READ)
return false;
trace_i915_gem_object_clflush(obj);

108
drivers/gpu/drm/i915/i915_gem_context.c
View File

@ -93,69 +93,28 @@
#define ALL_L3_SLICES(dev) (1 << NUM_L3_SLICES(dev)) - 1
/* Initial size (as log2) to preallocate the handle->object hashtable */
#define VMA_HT_BITS 2u /* 4 x 2 pointers, 64 bytes minimum */
static void resize_vma_ht(struct work_struct *work)
static void lut_close(struct i915_gem_context *ctx)
{
struct i915_gem_context_vma_lut *lut =
container_of(work, typeof(*lut), resize);
unsigned int bits, new_bits, size, i;
struct hlist_head *new_ht;
struct i915_lut_handle *lut, *ln;
struct radix_tree_iter iter;
void __rcu **slot;
GEM_BUG_ON(!(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS));
bits = 1 + ilog2(4*lut->ht_count/3 + 1);
new_bits = min_t(unsigned int,
max(bits, VMA_HT_BITS),
sizeof(unsigned int) * BITS_PER_BYTE - 1);
if (new_bits == lut->ht_bits)
goto out;
new_ht = kzalloc(sizeof(*new_ht)<<new_bits, GFP_KERNEL | __GFP_NOWARN);
if (!new_ht)
new_ht = vzalloc(sizeof(*new_ht)<<new_bits);
if (!new_ht)
/* Pretend resize succeeded and stop calling us for a bit! */
goto out;
size = BIT(lut->ht_bits);
for (i = 0; i < size; i++) {
struct i915_vma *vma;
struct hlist_node *tmp;
hlist_for_each_entry_safe(vma, tmp, &lut->ht[i], ctx_node)
hlist_add_head(&vma->ctx_node,
&new_ht[hash_32(vma->ctx_handle,
new_bits)]);
list_for_each_entry_safe(lut, ln, &ctx->handles_list, ctx_link) {
list_del(&lut->obj_link);
kmem_cache_free(ctx->i915->luts, lut);
}
kvfree(lut->ht);
lut->ht = new_ht;
lut->ht_bits = new_bits;
out:
smp_store_release(&lut->ht_size, BIT(bits));
GEM_BUG_ON(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS);
}
static void vma_lut_free(struct i915_gem_context *ctx)
{
struct i915_gem_context_vma_lut *lut = &ctx->vma_lut;
unsigned int i, size;
radix_tree_for_each_slot(slot, &ctx->handles_vma, &iter, 0) {
struct i915_vma *vma = rcu_dereference_raw(*slot);
struct drm_i915_gem_object *obj = vma->obj;
if (lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS)
cancel_work_sync(&lut->resize);
radix_tree_iter_delete(&ctx->handles_vma, &iter, slot);
size = BIT(lut->ht_bits);
for (i = 0; i < size; i++) {
struct i915_vma *vma;
if (!i915_vma_is_ggtt(vma))
i915_vma_close(vma);
hlist_for_each_entry(vma, &lut->ht[i], ctx_node) {
vma->obj->vma_hashed = NULL;
vma->ctx = NULL;
i915_vma_put(vma);
}
__i915_gem_object_release_unless_active(obj);
}
kvfree(lut->ht);
}
static void i915_gem_context_free(struct i915_gem_context *ctx)
@ -165,7 +124,6 @@ static void i915_gem_context_free(struct i915_gem_context *ctx)
lockdep_assert_held(&ctx->i915->drm.struct_mutex);
GEM_BUG_ON(!i915_gem_context_is_closed(ctx));
vma_lut_free(ctx);
i915_ppgtt_put(ctx->ppgtt);
for (i = 0; i < I915_NUM_ENGINES; i++) {
@ -239,8 +197,11 @@ void i915_gem_context_release(struct kref *ref)
static void context_close(struct i915_gem_context *ctx)
{
i915_gem_context_set_closed(ctx);
lut_close(ctx);
if (ctx->ppgtt)
i915_ppgtt_close(&ctx->ppgtt->base);
ctx->file_priv = ERR_PTR(-EBADF);
i915_gem_context_put(ctx);
}
@ -313,16 +274,8 @@ __create_hw_context(struct drm_i915_private *dev_priv,
ctx->i915 = dev_priv;
ctx->priority = I915_PRIORITY_NORMAL;
ctx->vma_lut.ht_bits = VMA_HT_BITS;
ctx->vma_lut.ht_size = BIT(VMA_HT_BITS);
BUILD_BUG_ON(BIT(VMA_HT_BITS) == I915_CTX_RESIZE_IN_PROGRESS);
ctx->vma_lut.ht = kcalloc(ctx->vma_lut.ht_size,
sizeof(*ctx->vma_lut.ht),
GFP_KERNEL);
if (!ctx->vma_lut.ht)
goto err_out;
INIT_WORK(&ctx->vma_lut.resize, resize_vma_ht);
INIT_RADIX_TREE(&ctx->handles_vma, GFP_KERNEL);
INIT_LIST_HEAD(&ctx->handles_list);
/* Default context will never have a file_priv */
ret = DEFAULT_CONTEXT_HANDLE;
@ -372,8 +325,6 @@ err_pid:
put_pid(ctx->pid);
idr_remove(&file_priv->context_idr, ctx->user_handle);
err_lut:
kvfree(ctx->vma_lut.ht);
err_out:
context_close(ctx);
return ERR_PTR(ret);
}
@ -741,19 +692,19 @@ static inline bool skip_rcs_switch(struct i915_hw_ppgtt *ppgtt,
}
static bool
needs_pd_load_pre(struct i915_hw_ppgtt *ppgtt,
struct intel_engine_cs *engine,
struct i915_gem_context *to)
needs_pd_load_pre(struct i915_hw_ppgtt *ppgtt, struct intel_engine_cs *engine)
{
struct i915_gem_context *from = engine->legacy_active_context;
if (!ppgtt)
return false;
/* Always load the ppgtt on first use */
if (!engine->legacy_active_context)
if (!from)
return true;
/* Same context without new entries, skip */
if (engine->legacy_active_context == to &&
if ((!from->ppgtt || from->ppgtt == ppgtt) &&
!(intel_engine_flag(engine) & ppgtt->pd_dirty_rings))
return false;
@ -797,7 +748,7 @@ static int do_rcs_switch(struct drm_i915_gem_request *req)
if (skip_rcs_switch(ppgtt, engine, to))
return 0;
if (needs_pd_load_pre(ppgtt, engine, to)) {
if (needs_pd_load_pre(ppgtt, engine)) {
/* Older GENs and non render rings still want the load first,
* "PP_DCLV followed by PP_DIR_BASE register through Load
* Register Immediate commands in Ring Buffer before submitting
@ -894,7 +845,7 @@ int i915_switch_context(struct drm_i915_gem_request *req)
struct i915_hw_ppgtt *ppgtt =
to->ppgtt ?: req->i915->mm.aliasing_ppgtt;
if (needs_pd_load_pre(ppgtt, engine, to)) {
if (needs_pd_load_pre(ppgtt, engine)) {
int ret;
trace_switch_mm(engine, to);
@ -905,6 +856,7 @@ int i915_switch_context(struct drm_i915_gem_request *req)
ppgtt->pd_dirty_rings &= ~intel_engine_flag(engine);
}
engine->legacy_active_context = to;
return 0;
}
@ -977,7 +929,7 @@ int i915_gem_switch_to_kernel_context(struct drm_i915_private *dev_priv)
static bool client_is_banned(struct drm_i915_file_private *file_priv)
{
return file_priv->context_bans > I915_MAX_CLIENT_CONTEXT_BANS;
return atomic_read(&file_priv->context_bans) > I915_MAX_CLIENT_CONTEXT_BANS;
}
int i915_gem_context_create_ioctl(struct drm_device *dev, void *data,
@ -1179,8 +1131,8 @@ int i915_gem_context_reset_stats_ioctl(struct drm_device *dev,
else
args->reset_count = 0;
args->batch_active = READ_ONCE(ctx->guilty_count);
args->batch_pending = READ_ONCE(ctx->active_count);
args->batch_active = atomic_read(&ctx->guilty_count);
args->batch_pending = atomic_read(&ctx->active_count);
ret = 0;
out:

45
drivers/gpu/drm/i915/i915_gem_context.h
View File

@ -27,6 +27,7 @@
#include <linux/bitops.h>
#include <linux/list.h>
#include <linux/radix-tree.h>
struct pid;
@ -149,32 +150,6 @@ struct i915_gem_context {
/** ggtt_offset_bias: placement restriction for context objects */
u32 ggtt_offset_bias;
struct i915_gem_context_vma_lut {
/** ht_size: last request size to allocate the hashtable for. */
unsigned int ht_size;
#define I915_CTX_RESIZE_IN_PROGRESS BIT(0)
/** ht_bits: real log2(size) of hashtable. */
unsigned int ht_bits;
/** ht_count: current number of entries inside the hashtable */
unsigned int ht_count;
/** ht: the array of buckets comprising the simple hashtable */
struct hlist_head *ht;
/**
* resize: After an execbuf completes, we check the load factor
* of the hashtable. If the hashtable is too full, or too empty,
* we schedule a task to resize the hashtable. During the
* resize, the entries are moved between different buckets and
* so we cannot simultaneously read the hashtable as it is
* being resized (unlike rhashtable). Therefore we treat the
* active work as a strong barrier, pausing a subsequent
* execbuf to wait for the resize worker to complete, if
* required.
*/
struct work_struct resize;
} vma_lut;
/** engine: per-engine logical HW state */
struct intel_context {
struct i915_vma *state;
@ -191,20 +166,32 @@ struct i915_gem_context {
u32 desc_template;
/** guilty_count: How many times this context has caused a GPU hang. */
unsigned int guilty_count;
atomic_t guilty_count;
/**
* @active_count: How many times this context was active during a GPU
* hang, but did not cause it.
*/
unsigned int active_count;
atomic_t active_count;
#define CONTEXT_SCORE_GUILTY 10
#define CONTEXT_SCORE_BAN_THRESHOLD 40
/** ban_score: Accumulated score of all hangs caused by this context. */
int ban_score;
atomic_t ban_score;
/** remap_slice: Bitmask of cache lines that need remapping */
u8 remap_slice;
/** handles_vma: rbtree to look up our context specific obj/vma for
* the user handle. (user handles are per fd, but the binding is
* per vm, which may be one per context or shared with the global GTT)
*/
struct radix_tree_root handles_vma;
/** handles_list: reverse list of all the rbtree entries in use for
* this context, which allows us to free all the allocations on
* context close.
*/
struct list_head handles_list;
};
static inline bool i915_gem_context_is_closed(const struct i915_gem_context *ctx)

4
drivers/gpu/drm/i915/i915_gem_evict.c
View File

@ -318,8 +318,8 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
/* Overlap of objects in the same batch? */
if (i915_vma_is_pinned(vma)) {
ret = -ENOSPC;
if (vma->exec_entry &&
vma->exec_entry->flags & EXEC_OBJECT_PINNED)
if (vma->exec_flags &&
*vma->exec_flags & EXEC_OBJECT_PINNED)
ret = -EINVAL;
break;
}

625
drivers/gpu/drm/i915/i915_gem_execbuffer.c
View File

@ -32,6 +32,7 @@
#include <linux/uaccess.h>
#include <drm/drmP.h>
#include <drm/drm_syncobj.h>
#include <drm/i915_drm.h>
#include "i915_drv.h"
@ -191,6 +192,8 @@ struct i915_execbuffer {
struct drm_file *file; /** per-file lookup tables and limits */
struct drm_i915_gem_execbuffer2 *args; /** ioctl parameters */
struct drm_i915_gem_exec_object2 *exec; /** ioctl execobj[] */
struct i915_vma **vma;
unsigned int *flags;
struct intel_engine_cs *engine; /** engine to queue the request to */
struct i915_gem_context *ctx; /** context for building the request */
@ -244,13 +247,7 @@ struct i915_execbuffer {
struct hlist_head *buckets; /** ht for relocation handles */
};
/*
* As an alternative to creating a hashtable of handle-to-vma for a batch,
* we used the last available reserved field in the execobject[] and stash
* a link from the execobj to its vma.
*/
#define __exec_to_vma(ee) (ee)->rsvd2
#define exec_to_vma(ee) u64_to_ptr(struct i915_vma, __exec_to_vma(ee))
#define exec_entry(EB, VMA) (&(EB)->exec[(VMA)->exec_flags - (EB)->flags])
/*
* Used to convert any address to canonical form.
@ -319,85 +316,82 @@ static int eb_create(struct i915_execbuffer *eb)
static bool
eb_vma_misplaced(const struct drm_i915_gem_exec_object2 *entry,
const struct i915_vma *vma)
const struct i915_vma *vma,
unsigned int flags)
{
if (!(entry->flags & __EXEC_OBJECT_HAS_PIN))
return true;
if (vma->node.size < entry->pad_to_size)
return true;
if (entry->alignment && !IS_ALIGNED(vma->node.start, entry->alignment))
return true;
if (entry->flags & EXEC_OBJECT_PINNED &&
if (flags & EXEC_OBJECT_PINNED &&
vma->node.start != entry->offset)
return true;
if (entry->flags & __EXEC_OBJECT_NEEDS_BIAS &&
if (flags & __EXEC_OBJECT_NEEDS_BIAS &&
vma->node.start < BATCH_OFFSET_BIAS)
return true;
if (!(entry->flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS) &&
if (!(flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS) &&
(vma->node.start + vma->node.size - 1) >> 32)
return true;
return false;
}
static inline void
static inline bool
eb_pin_vma(struct i915_execbuffer *eb,
struct drm_i915_gem_exec_object2 *entry,
const struct drm_i915_gem_exec_object2 *entry,
struct i915_vma *vma)
{
u64 flags;
unsigned int exec_flags = *vma->exec_flags;
u64 pin_flags;
if (vma->node.size)
flags = vma->node.start;
pin_flags = vma->node.start;
else
flags = entry->offset & PIN_OFFSET_MASK;
pin_flags = entry->offset & PIN_OFFSET_MASK;
flags |= PIN_USER | PIN_NOEVICT | PIN_OFFSET_FIXED;
if (unlikely(entry->flags & EXEC_OBJECT_NEEDS_GTT))
flags |= PIN_GLOBAL;
pin_flags |= PIN_USER | PIN_NOEVICT | PIN_OFFSET_FIXED;
if (unlikely(exec_flags & EXEC_OBJECT_NEEDS_GTT))
pin_flags |= PIN_GLOBAL;
if (unlikely(i915_vma_pin(vma, 0, 0, flags)))
return;
if (unlikely(i915_vma_pin(vma, 0, 0, pin_flags)))
return false;
if (unlikely(entry->flags & EXEC_OBJECT_NEEDS_FENCE)) {
if (unlikely(exec_flags & EXEC_OBJECT_NEEDS_FENCE)) {
if (unlikely(i915_vma_get_fence(vma))) {
i915_vma_unpin(vma);
return;
return false;
}
if (i915_vma_pin_fence(vma))
entry->flags |= __EXEC_OBJECT_HAS_FENCE;
exec_flags |= __EXEC_OBJECT_HAS_FENCE;
}
entry->flags |= __EXEC_OBJECT_HAS_PIN;
*vma->exec_flags = exec_flags | __EXEC_OBJECT_HAS_PIN;
return !eb_vma_misplaced(entry, vma, exec_flags);
}
static inline void
__eb_unreserve_vma(struct i915_vma *vma,
const struct drm_i915_gem_exec_object2 *entry)
static inline void __eb_unreserve_vma(struct i915_vma *vma, unsigned int flags)
{
GEM_BUG_ON(!(entry->flags & __EXEC_OBJECT_HAS_PIN));
GEM_BUG_ON(!(flags & __EXEC_OBJECT_HAS_PIN));
if (unlikely(entry->flags & __EXEC_OBJECT_HAS_FENCE))
if (unlikely(flags & __EXEC_OBJECT_HAS_FENCE))
i915_vma_unpin_fence(vma);
__i915_vma_unpin(vma);
}
static inline void
eb_unreserve_vma(struct i915_vma *vma,
struct drm_i915_gem_exec_object2 *entry)
eb_unreserve_vma(struct i915_vma *vma, unsigned int *flags)
{
if (!(entry->flags & __EXEC_OBJECT_HAS_PIN))
if (!(*flags & __EXEC_OBJECT_HAS_PIN))
return;
__eb_unreserve_vma(vma, entry);
entry->flags &= ~__EXEC_OBJECT_RESERVED;
__eb_unreserve_vma(vma, *flags);
*flags &= ~__EXEC_OBJECT_RESERVED;
}
static int
@ -427,7 +421,7 @@ eb_validate_vma(struct i915_execbuffer *eb,
entry->pad_to_size = 0;
}
if (unlikely(vma->exec_entry)) {
if (unlikely(vma->exec_flags)) {
DRM_DEBUG("Object [handle %d, index %d] appears more than once in object list\n",
entry->handle, (int)(entry - eb->exec));
return -EINVAL;
@ -440,14 +434,25 @@ eb_validate_vma(struct i915_execbuffer *eb,
*/
entry->offset = gen8_noncanonical_addr(entry->offset);
if (!eb->reloc_cache.has_fence) {
entry->flags &= ~EXEC_OBJECT_NEEDS_FENCE;
} else {
if ((entry->flags & EXEC_OBJECT_NEEDS_FENCE ||
eb->reloc_cache.needs_unfenced) &&
i915_gem_object_is_tiled(vma->obj))
entry->flags |= EXEC_OBJECT_NEEDS_GTT | __EXEC_OBJECT_NEEDS_MAP;
}
if (!(entry->flags & EXEC_OBJECT_PINNED))
entry->flags |= eb->context_flags;
return 0;
}
static int
eb_add_vma(struct i915_execbuffer *eb,
struct drm_i915_gem_exec_object2 *entry,
struct i915_vma *vma)
eb_add_vma(struct i915_execbuffer *eb, unsigned int i, struct i915_vma *vma)
{
struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
int err;
GEM_BUG_ON(i915_vma_is_closed(vma));
@ -468,40 +473,28 @@ eb_add_vma(struct i915_execbuffer *eb,
if (entry->relocation_count)
list_add_tail(&vma->reloc_link, &eb->relocs);
if (!eb->reloc_cache.has_fence) {
entry->flags &= ~EXEC_OBJECT_NEEDS_FENCE;
} else {
if ((entry->flags & EXEC_OBJECT_NEEDS_FENCE ||
eb->reloc_cache.needs_unfenced) &&
i915_gem_object_is_tiled(vma->obj))
entry->flags |= EXEC_OBJECT_NEEDS_GTT | __EXEC_OBJECT_NEEDS_MAP;
}
if (!(entry->flags & EXEC_OBJECT_PINNED))
entry->flags |= eb->context_flags;
/*
* Stash a pointer from the vma to execobj, so we can query its flags,
* size, alignment etc as provided by the user. Also we stash a pointer
* to the vma inside the execobj so that we can use a direct lookup
* to find the right target VMA when doing relocations.
*/
vma->exec_entry = entry;
__exec_to_vma(entry) = (uintptr_t)vma;
eb->vma[i] = vma;
eb->flags[i] = entry->flags;
vma->exec_flags = &eb->flags[i];
err = 0;
eb_pin_vma(eb, entry, vma);
if (eb_vma_misplaced(entry, vma)) {
eb_unreserve_vma(vma, entry);
list_add_tail(&vma->exec_link, &eb->unbound);
if (drm_mm_node_allocated(&vma->node))
err = i915_vma_unbind(vma);
} else {
if (eb_pin_vma(eb, entry, vma)) {
if (entry->offset != vma->node.start) {
entry->offset = vma->node.start | UPDATE;
eb->args->flags |= __EXEC_HAS_RELOC;
}
} else {
eb_unreserve_vma(vma, vma->exec_flags);
list_add_tail(&vma->exec_link, &eb->unbound);
if (drm_mm_node_allocated(&vma->node))
err = i915_vma_unbind(vma);
}
return err;
}
@ -526,32 +519,35 @@ static inline int use_cpu_reloc(const struct reloc_cache *cache,
static int eb_reserve_vma(const struct i915_execbuffer *eb,
struct i915_vma *vma)
{
struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
u64 flags;
struct drm_i915_gem_exec_object2 *entry = exec_entry(eb, vma);
unsigned int exec_flags = *vma->exec_flags;
u64 pin_flags;
int err;
flags = PIN_USER | PIN_NONBLOCK;
if (entry->flags & EXEC_OBJECT_NEEDS_GTT)
flags |= PIN_GLOBAL;
pin_flags = PIN_USER | PIN_NONBLOCK;
if (exec_flags & EXEC_OBJECT_NEEDS_GTT)
pin_flags |= PIN_GLOBAL;
/*
* Wa32bitGeneralStateOffset & Wa32bitInstructionBaseOffset,
* limit address to the first 4GBs for unflagged objects.
*/
if (!(entry->flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS))
flags |= PIN_ZONE_4G;
if (!(exec_flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS))
pin_flags |= PIN_ZONE_4G;
if (entry->flags & __EXEC_OBJECT_NEEDS_MAP)
flags |= PIN_MAPPABLE;
if (exec_flags & __EXEC_OBJECT_NEEDS_MAP)
pin_flags |= PIN_MAPPABLE;
if (entry->flags & EXEC_OBJECT_PINNED) {
flags |= entry->offset | PIN_OFFSET_FIXED;
flags &= ~PIN_NONBLOCK; /* force overlapping PINNED checks */
} else if (entry->flags & __EXEC_OBJECT_NEEDS_BIAS) {
flags |= BATCH_OFFSET_BIAS | PIN_OFFSET_BIAS;
if (exec_flags & EXEC_OBJECT_PINNED) {
pin_flags |= entry->offset | PIN_OFFSET_FIXED;
pin_flags &= ~PIN_NONBLOCK; /* force overlapping checks */
} else if (exec_flags & __EXEC_OBJECT_NEEDS_BIAS) {
pin_flags |= BATCH_OFFSET_BIAS | PIN_OFFSET_BIAS;
}
err = i915_vma_pin(vma, entry->pad_to_size, entry->alignment, flags);
err = i915_vma_pin(vma,
entry->pad_to_size, entry->alignment,
pin_flags);
if (err)
return err;
@ -560,7 +556,7 @@ static int eb_reserve_vma(const struct i915_execbuffer *eb,
eb->args->flags |= __EXEC_HAS_RELOC;
}
if (unlikely(entry->flags & EXEC_OBJECT_NEEDS_FENCE)) {
if (unlikely(exec_flags & EXEC_OBJECT_NEEDS_FENCE)) {
err = i915_vma_get_fence(vma);
if (unlikely(err)) {
i915_vma_unpin(vma);
@ -568,11 +564,11 @@ static int eb_reserve_vma(const struct i915_execbuffer *eb,
}
if (i915_vma_pin_fence(vma))
entry->flags |= __EXEC_OBJECT_HAS_FENCE;
exec_flags |= __EXEC_OBJECT_HAS_FENCE;
}
entry->flags |= __EXEC_OBJECT_HAS_PIN;
GEM_BUG_ON(eb_vma_misplaced(entry, vma));
*vma->exec_flags = exec_flags | __EXEC_OBJECT_HAS_PIN;
GEM_BUG_ON(eb_vma_misplaced(entry, vma, exec_flags));
return 0;
}
@ -614,18 +610,18 @@ static int eb_reserve(struct i915_execbuffer *eb)
INIT_LIST_HEAD(&eb->unbound);
INIT_LIST_HEAD(&last);
for (i = 0; i < count; i++) {
struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
unsigned int flags = eb->flags[i];
struct i915_vma *vma = eb->vma[i];
if (entry->flags & EXEC_OBJECT_PINNED &&
entry->flags & __EXEC_OBJECT_HAS_PIN)
if (flags & EXEC_OBJECT_PINNED &&
flags & __EXEC_OBJECT_HAS_PIN)
continue;
vma = exec_to_vma(entry);
eb_unreserve_vma(vma, entry);
eb_unreserve_vma(vma, &eb->flags[i]);
if (entry->flags & EXEC_OBJECT_PINNED)
if (flags & EXEC_OBJECT_PINNED)
list_add(&vma->exec_link, &eb->unbound);
else if (entry->flags & __EXEC_OBJECT_NEEDS_MAP)
else if (flags & __EXEC_OBJECT_NEEDS_MAP)
list_add_tail(&vma->exec_link, &eb->unbound);
else
list_add_tail(&vma->exec_link, &last);
@ -649,19 +645,6 @@ static int eb_reserve(struct i915_execbuffer *eb)
} while (1);
}
static inline struct hlist_head *
ht_head(const struct i915_gem_context_vma_lut *lut, u32 handle)
{
return &lut->ht[hash_32(handle, lut->ht_bits)];
}
static inline bool
ht_needs_resize(const struct i915_gem_context_vma_lut *lut)
{
return (4*lut->ht_count > 3*lut->ht_size ||
4*lut->ht_count + 1 < lut->ht_size);
}
static unsigned int eb_batch_index(const struct i915_execbuffer *eb)
{
if (eb->args->flags & I915_EXEC_BATCH_FIRST)
@ -678,13 +661,6 @@ static int eb_select_context(struct i915_execbuffer *eb)
if (unlikely(!ctx))
return -ENOENT;
if (unlikely(i915_gem_context_is_banned(ctx))) {
DRM_DEBUG("Context %u tried to submit while banned\n",
ctx->user_handle);
i915_gem_context_put(ctx);
return -EIO;
}
eb->ctx = ctx;
eb->vm = ctx->ppgtt ? &ctx->ppgtt->base : &eb->i915->ggtt.base;
@ -697,132 +673,74 @@ static int eb_select_context(struct i915_execbuffer *eb)
static int eb_lookup_vmas(struct i915_execbuffer *eb)
{
#define INTERMEDIATE BIT(0)
const unsigned int count = eb->buffer_count;
struct i915_gem_context_vma_lut *lut = &eb->ctx->vma_lut;
struct i915_vma *vma;
struct idr *idr;
struct radix_tree_root *handles_vma = &eb->ctx->handles_vma;
struct drm_i915_gem_object *uninitialized_var(obj);
unsigned int i;
int slow_pass = -1;
int err;
if (unlikely(i915_gem_context_is_closed(eb->ctx)))
return -ENOENT;
if (unlikely(i915_gem_context_is_banned(eb->ctx)))
return -EIO;
INIT_LIST_HEAD(&eb->relocs);
INIT_LIST_HEAD(&eb->unbound);
if (unlikely(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS))
flush_work(&lut->resize);
GEM_BUG_ON(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS);
for (i = 0; i < eb->buffer_count; i++) {
u32 handle = eb->exec[i].handle;
struct i915_lut_handle *lut;
struct i915_vma *vma;
for (i = 0; i < count; i++) {
__exec_to_vma(&eb->exec[i]) = 0;
vma = radix_tree_lookup(handles_vma, handle);
if (likely(vma))
goto add_vma;
hlist_for_each_entry(vma,
ht_head(lut, eb->exec[i].handle),
ctx_node) {
if (vma->ctx_handle != eb->exec[i].handle)
continue;
err = eb_add_vma(eb, &eb->exec[i], vma);
if (unlikely(err))
return err;
goto next_vma;
}
if (slow_pass < 0)
slow_pass = i;
next_vma: ;
}
if (slow_pass < 0)
goto out;
spin_lock(&eb->file->table_lock);
/*
* Grab a reference to the object and release the lock so we can lookup
* or create the VMA without using GFP_ATOMIC
*/
idr = &eb->file->object_idr;
for (i = slow_pass; i < count; i++) {
struct drm_i915_gem_object *obj;
if (__exec_to_vma(&eb->exec[i]))
continue;
obj = to_intel_bo(idr_find(idr, eb->exec[i].handle));
obj = i915_gem_object_lookup(eb->file, handle);
if (unlikely(!obj)) {
spin_unlock(&eb->file->table_lock);
DRM_DEBUG("Invalid object handle %d at index %d\n",
eb->exec[i].handle, i);
err = -ENOENT;
goto err;
goto err_vma;
}
__exec_to_vma(&eb->exec[i]) = INTERMEDIATE | (uintptr_t)obj;
}
spin_unlock(&eb->file->table_lock);
for (i = slow_pass; i < count; i++) {
struct drm_i915_gem_object *obj;
if (!(__exec_to_vma(&eb->exec[i]) & INTERMEDIATE))
continue;
/*
* NOTE: We can leak any vmas created here when something fails
* later on. But that's no issue since vma_unbind can deal with
* vmas which are not actually bound. And since only
* lookup_or_create exists as an interface to get at the vma
* from the (obj, vm) we don't run the risk of creating
* duplicated vmas for the same vm.
*/
obj = u64_to_ptr(typeof(*obj),
__exec_to_vma(&eb->exec[i]) & ~INTERMEDIATE);
vma = i915_vma_instance(obj, eb->vm, NULL);
if (unlikely(IS_ERR(vma))) {
DRM_DEBUG("Failed to lookup VMA\n");
err = PTR_ERR(vma);
goto err;
goto err_obj;
}
/* First come, first served */
if (!vma->ctx) {
vma->ctx = eb->ctx;
vma->ctx_handle = eb->exec[i].handle;
hlist_add_head(&vma->ctx_node,
ht_head(lut, eb->exec[i].handle));
lut->ht_count++;
lut->ht_size |= I915_CTX_RESIZE_IN_PROGRESS;
if (i915_vma_is_ggtt(vma)) {
GEM_BUG_ON(obj->vma_hashed);
obj->vma_hashed = vma;
}
i915_vma_get(vma);
lut = kmem_cache_alloc(eb->i915->luts, GFP_KERNEL);
if (unlikely(!lut)) {
err = -ENOMEM;
goto err_obj;
}
err = eb_add_vma(eb, &eb->exec[i], vma);
err = radix_tree_insert(handles_vma, handle, vma);
if (unlikely(err)) {
kfree(lut);
goto err_obj;
}
list_add(&lut->obj_link, &obj->lut_list);
list_add(&lut->ctx_link, &eb->ctx->handles_list);
lut->ctx = eb->ctx;
lut->handle = handle;
/* transfer ref to ctx */
obj = NULL;
add_vma:
err = eb_add_vma(eb, i, vma);
if (unlikely(err))
goto err;
goto err_obj;
/* Only after we validated the user didn't use our bits */
if (vma->ctx != eb->ctx) {
i915_vma_get(vma);
eb->exec[i].flags |= __EXEC_OBJECT_HAS_REF;
}
GEM_BUG_ON(vma != eb->vma[i]);
GEM_BUG_ON(vma->exec_flags != &eb->flags[i]);
}
if (lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS) {
if (ht_needs_resize(lut))
queue_work(system_highpri_wq, &lut->resize);
else
lut->ht_size &= ~I915_CTX_RESIZE_IN_PROGRESS;
}
out:
/* take note of the batch buffer before we might reorder the lists */
i = eb_batch_index(eb);
eb->batch = exec_to_vma(&eb->exec[i]);
eb->batch = eb->vma[i];
GEM_BUG_ON(eb->batch->exec_flags != &eb->flags[i]);
/*
* SNA is doing fancy tricks with compressing batch buffers, which leads
@ -833,22 +751,20 @@ out:
* Note that actual hangs have only been observed on gen7, but for
* paranoia do it everywhere.
*/
if (!(eb->exec[i].flags & EXEC_OBJECT_PINNED))
eb->exec[i].flags |= __EXEC_OBJECT_NEEDS_BIAS;
if (!(eb->flags[i] & EXEC_OBJECT_PINNED))
eb->flags[i] |= __EXEC_OBJECT_NEEDS_BIAS;
if (eb->reloc_cache.has_fence)
eb->exec[i].flags |= EXEC_OBJECT_NEEDS_FENCE;
eb->flags[i] |= EXEC_OBJECT_NEEDS_FENCE;
eb->args->flags |= __EXEC_VALIDATED;
return eb_reserve(eb);
err:
for (i = slow_pass; i < count; i++) {
if (__exec_to_vma(&eb->exec[i]) & INTERMEDIATE)
__exec_to_vma(&eb->exec[i]) = 0;
}
lut->ht_size &= ~I915_CTX_RESIZE_IN_PROGRESS;
err_obj:
if (obj)
i915_gem_object_put(obj);
err_vma:
eb->vma[i] = NULL;
return err;
#undef INTERMEDIATE
}
static struct i915_vma *
@ -857,7 +773,7 @@ eb_get_vma(const struct i915_execbuffer *eb, unsigned long handle)
if (eb->lut_size < 0) {
if (handle >= -eb->lut_size)
return NULL;
return exec_to_vma(&eb->exec[handle]);
return eb->vma[handle];
} else {
struct hlist_head *head;
struct i915_vma *vma;
@ -877,24 +793,21 @@ static void eb_release_vmas(const struct i915_execbuffer *eb)
unsigned int i;
for (i = 0; i < count; i++) {
struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
struct i915_vma *vma = exec_to_vma(entry);
struct i915_vma *vma = eb->vma[i];
unsigned int flags = eb->flags[i];
if (!vma)
continue;
break;
GEM_BUG_ON(vma->exec_entry != entry);
vma->exec_entry = NULL;
__exec_to_vma(entry) = 0;
GEM_BUG_ON(vma->exec_flags != &eb->flags[i]);
vma->exec_flags = NULL;
eb->vma[i] = NULL;
if (entry->flags & __EXEC_OBJECT_HAS_PIN)
__eb_unreserve_vma(vma, entry);
if (flags & __EXEC_OBJECT_HAS_PIN)
__eb_unreserve_vma(vma, flags);
if (entry->flags & __EXEC_OBJECT_HAS_REF)
if (flags & __EXEC_OBJECT_HAS_REF)
i915_vma_put(vma);
entry->flags &=
~(__EXEC_OBJECT_RESERVED | __EXEC_OBJECT_HAS_REF);
}
}
@ -1267,7 +1180,9 @@ relocate_entry(struct i915_vma *vma,
if (!eb->reloc_cache.vaddr &&
(DBG_FORCE_RELOC == FORCE_GPU_RELOC ||
!reservation_object_test_signaled_rcu(vma->resv, true))) {
!reservation_object_test_signaled_rcu(vma->resv, true)) &&
__intel_engine_can_store_dword(eb->reloc_cache.gen,
eb->engine->class)) {
const unsigned int gen = eb->reloc_cache.gen;
unsigned int len;
u32 *batch;
@ -1277,10 +1192,8 @@ relocate_entry(struct i915_vma *vma,
len = offset & 7 ? 8 : 5;
else if (gen >= 4)
len = 4;
else if (gen >= 3)
else
len = 3;
else /* On gen2 MI_STORE_DWORD_IMM uses a physical address */
goto repeat;
batch = reloc_gpu(eb, vma, len);
if (IS_ERR(batch))
@ -1383,7 +1296,7 @@ eb_relocate_entry(struct i915_execbuffer *eb,
}
if (reloc->write_domain) {
target->exec_entry->flags |= EXEC_OBJECT_WRITE;
*target->exec_flags |= EXEC_OBJECT_WRITE;
/*
* Sandybridge PPGTT errata: We need a global gtt mapping
@ -1433,9 +1346,9 @@ eb_relocate_entry(struct i915_execbuffer *eb,
* patching using the GPU (though that should be serialised by the
* timeline). To be completely sure, and since we are required to
* do relocations we are already stalling, disable the user's opt
* of our synchronisation.
* out of our synchronisation.
*/
vma->exec_entry->flags &= ~EXEC_OBJECT_ASYNC;
*vma->exec_flags &= ~EXEC_OBJECT_ASYNC;
/* and update the user's relocation entry */
return relocate_entry(vma, reloc, eb, target);
@ -1446,7 +1359,7 @@ static int eb_relocate_vma(struct i915_execbuffer *eb, struct i915_vma *vma)
#define N_RELOC(x) ((x) / sizeof(struct drm_i915_gem_relocation_entry))
struct drm_i915_gem_relocation_entry stack[N_RELOC(512)];
struct drm_i915_gem_relocation_entry __user *urelocs;
const struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
const struct drm_i915_gem_exec_object2 *entry = exec_entry(eb, vma);
unsigned int remain;
urelocs = u64_to_user_ptr(entry->relocs_ptr);
@ -1529,7 +1442,7 @@ out:
static int
eb_relocate_vma_slow(struct i915_execbuffer *eb, struct i915_vma *vma)
{
const struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
const struct drm_i915_gem_exec_object2 *entry = exec_entry(eb, vma);
struct drm_i915_gem_relocation_entry *relocs =
u64_to_ptr(typeof(*relocs), entry->relocs_ptr);
unsigned int i;
@ -1733,6 +1646,8 @@ repeat:
if (err)
goto err;
GEM_BUG_ON(!eb->batch);
list_for_each_entry(vma, &eb->relocs, reloc_link) {
if (!have_copy) {
pagefault_disable();
@ -1826,11 +1741,11 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
int err;
for (i = 0; i < count; i++) {
struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
struct i915_vma *vma = exec_to_vma(entry);
unsigned int flags = eb->flags[i];
struct i915_vma *vma = eb->vma[i];
struct drm_i915_gem_object *obj = vma->obj;
if (entry->flags & EXEC_OBJECT_CAPTURE) {
if (flags & EXEC_OBJECT_CAPTURE) {
struct i915_gem_capture_list *capture;
capture = kmalloc(sizeof(*capture), GFP_KERNEL);
@ -1838,35 +1753,47 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
return -ENOMEM;
capture->next = eb->request->capture_list;
capture->vma = vma;
capture->vma = eb->vma[i];
eb->request->capture_list = capture;
}
if (unlikely(obj->cache_dirty && !obj->cache_coherent)) {
/*
* If the GPU is not _reading_ through the CPU cache, we need
* to make sure that any writes (both previous GPU writes from
* before a change in snooping levels and normal CPU writes)
* caught in that cache are flushed to main memory.
*
* We want to say
* obj->cache_dirty &&
* !(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_READ)
* but gcc's optimiser doesn't handle that as well and emits
* two jumps instead of one. Maybe one day...
*/
if (unlikely(obj->cache_dirty & ~obj->cache_coherent)) {
if (i915_gem_clflush_object(obj, 0))
entry->flags &= ~EXEC_OBJECT_ASYNC;
flags &= ~EXEC_OBJECT_ASYNC;
}
if (entry->flags & EXEC_OBJECT_ASYNC)
goto skip_flushes;
if (flags & EXEC_OBJECT_ASYNC)
continue;
err = i915_gem_request_await_object
(eb->request, obj, entry->flags & EXEC_OBJECT_WRITE);
(eb->request, obj, flags & EXEC_OBJECT_WRITE);
if (err)
return err;
skip_flushes:
i915_vma_move_to_active(vma, eb->request, entry->flags);
__eb_unreserve_vma(vma, entry);
vma->exec_entry = NULL;
}
for (i = 0; i < count; i++) {
const struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
struct i915_vma *vma = exec_to_vma(entry);
unsigned int flags = eb->flags[i];
struct i915_vma *vma = eb->vma[i];
eb_export_fence(vma, eb->request, entry->flags);
if (unlikely(entry->flags & __EXEC_OBJECT_HAS_REF))
i915_vma_move_to_active(vma, eb->request, flags);
eb_export_fence(vma, eb->request, flags);
__eb_unreserve_vma(vma, flags);
vma->exec_flags = NULL;
if (unlikely(flags & __EXEC_OBJECT_HAS_REF))
i915_vma_put(vma);
}
eb->exec = NULL;
@ -1884,8 +1811,10 @@ static bool i915_gem_check_execbuffer(struct drm_i915_gem_execbuffer2 *exec)
return false;
/* Kernel clipping was a DRI1 misfeature */
if (exec->num_cliprects || exec->cliprects_ptr)
return false;
if (!(exec->flags & I915_EXEC_FENCE_ARRAY)) {
if (exec->num_cliprects || exec->cliprects_ptr)
return false;
}
if (exec->DR4 == 0xffffffff) {
DRM_DEBUG("UXA submitting garbage DR4, fixing up\n");
@ -1993,11 +1922,11 @@ static struct i915_vma *eb_parse(struct i915_execbuffer *eb, bool is_master)
if (IS_ERR(vma))
goto out;
vma->exec_entry =
memset(&eb->exec[eb->buffer_count++],
0, sizeof(*vma->exec_entry));
vma->exec_entry->flags = __EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_REF;
__exec_to_vma(vma->exec_entry) = (uintptr_t)i915_vma_get(vma);
eb->vma[eb->buffer_count] = i915_vma_get(vma);
eb->flags[eb->buffer_count] =
__EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_REF;
vma->exec_flags = &eb->flags[eb->buffer_count];
eb->buffer_count++;
out:
i915_gem_object_unpin_pages(shadow_batch_obj);
@ -2116,11 +2045,131 @@ eb_select_engine(struct drm_i915_private *dev_priv,
return engine;
}
static void
__free_fence_array(struct drm_syncobj **fences, unsigned int n)
{
while (n--)
drm_syncobj_put(ptr_mask_bits(fences[n], 2));
kvfree(fences);
}
static struct drm_syncobj **
get_fence_array(struct drm_i915_gem_execbuffer2 *args,
struct drm_file *file)
{
const unsigned int nfences = args->num_cliprects;
struct drm_i915_gem_exec_fence __user *user;
struct drm_syncobj **fences;
unsigned int n;
int err;
if (!(args->flags & I915_EXEC_FENCE_ARRAY))
return NULL;
if (nfences > SIZE_MAX / sizeof(*fences))
return ERR_PTR(-EINVAL);
user = u64_to_user_ptr(args->cliprects_ptr);
if (!access_ok(VERIFY_READ, user, nfences * 2 * sizeof(u32)))
return ERR_PTR(-EFAULT);
fences = kvmalloc_array(args->num_cliprects, sizeof(*fences),
__GFP_NOWARN | GFP_TEMPORARY);
if (!fences)
return ERR_PTR(-ENOMEM);
for (n = 0; n < nfences; n++) {
struct drm_i915_gem_exec_fence fence;
struct drm_syncobj *syncobj;
if (__copy_from_user(&fence, user++, sizeof(fence))) {
err = -EFAULT;
goto err;
}
syncobj = drm_syncobj_find(file, fence.handle);
if (!syncobj) {
DRM_DEBUG("Invalid syncobj handle provided\n");
err = -ENOENT;
goto err;
}
fences[n] = ptr_pack_bits(syncobj, fence.flags, 2);
}
return fences;
err:
__free_fence_array(fences, n);
return ERR_PTR(err);
}
static void
put_fence_array(struct drm_i915_gem_execbuffer2 *args,
struct drm_syncobj **fences)
{
if (fences)
__free_fence_array(fences, args->num_cliprects);
}
static int
await_fence_array(struct i915_execbuffer *eb,
struct drm_syncobj **fences)
{
const unsigned int nfences = eb->args->num_cliprects;
unsigned int n;
int err;
for (n = 0; n < nfences; n++) {
struct drm_syncobj *syncobj;
struct dma_fence *fence;
unsigned int flags;
syncobj = ptr_unpack_bits(fences[n], &flags, 2);
if (!(flags & I915_EXEC_FENCE_WAIT))
continue;
rcu_read_lock();
fence = dma_fence_get_rcu_safe(&syncobj->fence);
rcu_read_unlock();
if (!fence)
return -EINVAL;
err = i915_gem_request_await_dma_fence(eb->request, fence);
dma_fence_put(fence);
if (err < 0)
return err;
}
return 0;
}
static void
signal_fence_array(struct i915_execbuffer *eb,
struct drm_syncobj **fences)
{
const unsigned int nfences = eb->args->num_cliprects;
struct dma_fence * const fence = &eb->request->fence;
unsigned int n;
for (n = 0; n < nfences; n++) {
struct drm_syncobj *syncobj;
unsigned int flags;
syncobj = ptr_unpack_bits(fences[n], &flags, 2);
if (!(flags & I915_EXEC_FENCE_SIGNAL))
continue;
drm_syncobj_replace_fence(syncobj, fence);
}
}
static int
i915_gem_do_execbuffer(struct drm_device *dev,
struct drm_file *file,
struct drm_i915_gem_execbuffer2 *args,
struct drm_i915_gem_exec_object2 *exec)
struct drm_i915_gem_exec_object2 *exec,
struct drm_syncobj **fences)
{
struct i915_execbuffer eb;
struct dma_fence *in_fence = NULL;
@ -2136,7 +2185,12 @@ i915_gem_do_execbuffer(struct drm_device *dev,
eb.args = args;
if (DBG_FORCE_RELOC || !(args->flags & I915_EXEC_NO_RELOC))
args->flags |= __EXEC_HAS_RELOC;
eb.exec = exec;
eb.vma = (struct i915_vma **)(exec + args->buffer_count + 1);
eb.vma[0] = NULL;
eb.flags = (unsigned int *)(eb.vma + args->buffer_count + 1);
eb.invalid_flags = __EXEC_OBJECT_UNKNOWN_FLAGS;
if (USES_FULL_PPGTT(eb.i915))
eb.invalid_flags |= EXEC_OBJECT_NEEDS_GTT;
@ -2224,7 +2278,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
goto err_vma;
}
if (unlikely(eb.batch->exec_entry->flags & EXEC_OBJECT_WRITE)) {
if (unlikely(*eb.batch->exec_flags & EXEC_OBJECT_WRITE)) {
DRM_DEBUG("Attempting to use self-modifying batch buffer\n");
err = -EINVAL;
goto err_vma;
@ -2306,6 +2360,12 @@ i915_gem_do_execbuffer(struct drm_device *dev,
goto err_request;
}
if (fences) {
err = await_fence_array(&eb, fences);
if (err)
goto err_request;
}
if (out_fence_fd != -1) {
out_fence = sync_file_create(&eb.request->fence);
if (!out_fence) {
@ -2329,6 +2389,9 @@ err_request:
__i915_add_request(eb.request, err == 0);
add_to_client(eb.request, file);
if (fences)
signal_fence_array(&eb, fences);
if (out_fence) {
if (err == 0) {
fd_install(out_fence_fd, out_fence->file);
@ -2368,7 +2431,9 @@ int
i915_gem_execbuffer(struct drm_device *dev, void *data,
struct drm_file *file)
{
const size_t sz = sizeof(struct drm_i915_gem_exec_object2);
const size_t sz = (sizeof(struct drm_i915_gem_exec_object2) +
sizeof(struct i915_vma *) +
sizeof(unsigned int));
struct drm_i915_gem_execbuffer *args = data;
struct drm_i915_gem_execbuffer2 exec2;
struct drm_i915_gem_exec_object *exec_list = NULL;
@ -2430,7 +2495,7 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
exec2_list[i].flags = 0;
}
err = i915_gem_do_execbuffer(dev, file, &exec2, exec2_list);
err = i915_gem_do_execbuffer(dev, file, &exec2, exec2_list, NULL);
if (exec2.flags & __EXEC_HAS_RELOC) {
struct drm_i915_gem_exec_object __user *user_exec_list =
u64_to_user_ptr(args->buffers_ptr);
@ -2459,9 +2524,12 @@ int
i915_gem_execbuffer2(struct drm_device *dev, void *data,
struct drm_file *file)
{
const size_t sz = sizeof(struct drm_i915_gem_exec_object2);
const size_t sz = (sizeof(struct drm_i915_gem_exec_object2) +
sizeof(struct i915_vma *) +
sizeof(unsigned int));
struct drm_i915_gem_execbuffer2 *args = data;
struct drm_i915_gem_exec_object2 *exec2_list;
struct drm_syncobj **fences = NULL;
int err;
if (args->buffer_count < 1 || args->buffer_count > SIZE_MAX / sz - 1) {
@ -2488,7 +2556,15 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
return -EFAULT;
}
err = i915_gem_do_execbuffer(dev, file, args, exec2_list);
if (args->flags & I915_EXEC_FENCE_ARRAY) {
fences = get_fence_array(args, file);
if (IS_ERR(fences)) {
kvfree(exec2_list);
return PTR_ERR(fences);
}
}
err = i915_gem_do_execbuffer(dev, file, args, exec2_list, fences);
/*
* Now that we have begun execution of the batchbuffer, we ignore
@ -2518,6 +2594,7 @@ end_user:
}
args->flags &= ~__I915_EXEC_UNKNOWN_FLAGS;
put_fence_array(args, fences);
kvfree(exec2_list);
return err;
}

43
drivers/gpu/drm/i915/i915_gem_gtt.c
View File

@ -144,9 +144,9 @@ int intel_sanitize_enable_ppgtt(struct drm_i915_private *dev_priv,
has_full_48bit_ppgtt = dev_priv->info.has_full_48bit_ppgtt;
if (intel_vgpu_active(dev_priv)) {
/* emulation is too hard */
/* GVT-g has no support for 32bit ppgtt */
has_full_ppgtt = false;
has_full_48bit_ppgtt = false;
has_full_48bit_ppgtt = intel_vgpu_has_full_48bit_ppgtt(dev_priv);
}
if (!has_aliasing_ppgtt)
@ -180,10 +180,15 @@ int intel_sanitize_enable_ppgtt(struct drm_i915_private *dev_priv,
return 0;
}
if (INTEL_GEN(dev_priv) >= 8 && i915.enable_execlists && has_full_ppgtt)
return has_full_48bit_ppgtt ? 3 : 2;
else
return has_aliasing_ppgtt ? 1 : 0;
if (INTEL_GEN(dev_priv) >= 8 && i915.enable_execlists) {
if (has_full_48bit_ppgtt)
return 3;
if (has_full_ppgtt)
return 2;
}
return has_aliasing_ppgtt ? 1 : 0;
}
static int ppgtt_bind_vma(struct i915_vma *vma,
@ -2737,6 +2742,24 @@ static int ggtt_probe_common(struct i915_ggtt *ggtt, u64 size)
return 0;
}
static void cnl_setup_private_ppat(struct drm_i915_private *dev_priv)
{
/* XXX: spec is unclear if this is still needed for CNL+ */
if (!USES_PPGTT(dev_priv)) {
I915_WRITE(GEN10_PAT_INDEX(0), GEN8_PPAT_UC);
return;
}
I915_WRITE(GEN10_PAT_INDEX(0), GEN8_PPAT_WB | GEN8_PPAT_LLC);
I915_WRITE(GEN10_PAT_INDEX(1), GEN8_PPAT_WC | GEN8_PPAT_LLCELLC);
I915_WRITE(GEN10_PAT_INDEX(2), GEN8_PPAT_WT | GEN8_PPAT_LLCELLC);
I915_WRITE(GEN10_PAT_INDEX(3), GEN8_PPAT_UC);
I915_WRITE(GEN10_PAT_INDEX(4), GEN8_PPAT_LLCELLC | GEN8_PPAT_AGE(0));
I915_WRITE(GEN10_PAT_INDEX(5), GEN8_PPAT_LLCELLC | GEN8_PPAT_AGE(1));
I915_WRITE(GEN10_PAT_INDEX(6), GEN8_PPAT_LLCELLC | GEN8_PPAT_AGE(2));
I915_WRITE(GEN10_PAT_INDEX(7), GEN8_PPAT_LLCELLC | GEN8_PPAT_AGE(3));
}
/* The GGTT and PPGTT need a private PPAT setup in order to handle cacheability
* bits. When using advanced contexts each context stores its own PAT, but
* writing this data shouldn't be harmful even in those cases. */
@ -2851,7 +2874,9 @@ static int gen8_gmch_probe(struct i915_ggtt *ggtt)
ggtt->base.total = (size / sizeof(gen8_pte_t)) << PAGE_SHIFT;
if (IS_CHERRYVIEW(dev_priv) || IS_GEN9_LP(dev_priv))
if (INTEL_GEN(dev_priv) >= 10)
cnl_setup_private_ppat(dev_priv);
else if (IS_CHERRYVIEW(dev_priv) || IS_GEN9_LP(dev_priv))
chv_setup_private_ppat(dev_priv);
else
bdw_setup_private_ppat(dev_priv);
@ -3133,7 +3158,9 @@ void i915_gem_restore_gtt_mappings(struct drm_i915_private *dev_priv)
ggtt->base.closed = false;
if (INTEL_GEN(dev_priv) >= 8) {
if (IS_CHERRYVIEW(dev_priv) || IS_GEN9_LP(dev_priv))
if (INTEL_GEN(dev_priv) >= 10)
cnl_setup_private_ppat(dev_priv);
else if (IS_CHERRYVIEW(dev_priv) || IS_GEN9_LP(dev_priv))
chv_setup_private_ppat(dev_priv);
else
bdw_setup_private_ppat(dev_priv);

7
drivers/gpu/drm/i915/i915_gem_internal.c
View File

@ -174,6 +174,7 @@ i915_gem_object_create_internal(struct drm_i915_private *i915,
phys_addr_t size)
{
struct drm_i915_gem_object *obj;
unsigned int cache_level;
GEM_BUG_ON(!size);
GEM_BUG_ON(!IS_ALIGNED(size, PAGE_SIZE));
@ -190,9 +191,9 @@ i915_gem_object_create_internal(struct drm_i915_private *i915,
obj->base.read_domains = I915_GEM_DOMAIN_CPU;
obj->base.write_domain = I915_GEM_DOMAIN_CPU;
obj->cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
obj->cache_coherent = i915_gem_object_is_coherent(obj);
obj->cache_dirty = !obj->cache_coherent;
cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
i915_gem_object_set_cache_coherency(obj, cache_level);
return obj;
}

48
drivers/gpu/drm/i915/i915_gem_object.c 100644
View File

@ -0,0 +1,48 @@
/*
* Copyright © 2017 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*
*/
#include "i915_drv.h"
#include "i915_gem_object.h"
/**
* Mark up the object's coherency levels for a given cache_level
* @obj: #drm_i915_gem_object
* @cache_level: cache level
*/
void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj,
unsigned int cache_level)
{
obj->cache_level = cache_level;
if (cache_level != I915_CACHE_NONE)
obj->cache_coherent = (I915_BO_CACHE_COHERENT_FOR_READ |
I915_BO_CACHE_COHERENT_FOR_WRITE);
else if (HAS_LLC(to_i915(obj->base.dev)))
obj->cache_coherent = I915_BO_CACHE_COHERENT_FOR_READ;
else
obj->cache_coherent = 0;
obj->cache_dirty =
!(obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE);
}

32
drivers/gpu/drm/i915/i915_gem_object.h
View File

@ -33,8 +33,24 @@
#include <drm/i915_drm.h>
#include "i915_gem_request.h"
#include "i915_selftest.h"
struct drm_i915_gem_object;
/*
* struct i915_lut_handle tracks the fast lookups from handle to vma used
* for execbuf. Although we use a radixtree for that mapping, in order to
* remove them as the object or context is closed, we need a secondary list
* and a translation entry (i915_lut_handle).
*/
struct i915_lut_handle {
struct list_head obj_link;
struct list_head ctx_link;
struct i915_gem_context *ctx;
u32 handle;
};
struct drm_i915_gem_object_ops {
unsigned int flags;
#define I915_GEM_OBJECT_HAS_STRUCT_PAGE BIT(0)
@ -86,7 +102,15 @@ struct drm_i915_gem_object {
* They are also added to @vma_list for easy iteration.
*/
struct rb_root vma_tree;
struct i915_vma *vma_hashed;
/**
* @lut_list: List of vma lookup entries in use for this object.
*
* If this object is closed, we need to remove all of its VMA from
* the fast lookup index in associated contexts; @lut_list provides
* this translation from object to context->handles_vma.
*/
struct list_head lut_list;
/** Stolen memory for this object, instead of being backed by shmem. */
struct drm_mm_node *stolen;
@ -118,8 +142,10 @@ struct drm_i915_gem_object {
*/
unsigned long gt_ro:1;
unsigned int cache_level:3;
unsigned int cache_coherent:2;
#define I915_BO_CACHE_COHERENT_FOR_READ BIT(0)
#define I915_BO_CACHE_COHERENT_FOR_WRITE BIT(1)
unsigned int cache_dirty:1;
unsigned int cache_coherent:1;
atomic_t frontbuffer_bits;
unsigned int frontbuffer_ggtt_origin; /* write once */
@ -391,6 +417,8 @@ i915_gem_object_last_write_engine(struct drm_i915_gem_object *obj)
return engine;
}
void i915_gem_object_set_cache_coherency(struct drm_i915_gem_object *obj,
unsigned int cache_level);
void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj);
#endif

4
drivers/gpu/drm/i915/i915_gem_render_state.c
View File

@ -242,6 +242,10 @@ int i915_gem_render_state_emit(struct drm_i915_gem_request *req)
goto err_unpin;
}
ret = req->engine->emit_flush(req, EMIT_INVALIDATE);
if (ret)
goto err_unpin;
ret = req->engine->emit_bb_start(req,
so->batch_offset, so->batch_size,
I915_DISPATCH_SECURE);

97
drivers/gpu/drm/i915/i915_gem_request.c
View File

@ -213,6 +213,10 @@ static int reset_all_global_seqno(struct drm_i915_private *i915, u32 seqno)
cond_resched();
}
/* Check we are idle before we fiddle with hw state! */
GEM_BUG_ON(!intel_engine_is_idle(engine));
GEM_BUG_ON(i915_gem_active_isset(&engine->timeline->last_request));
/* Finally reset hw state */
intel_engine_init_global_seqno(engine, seqno);
tl->seqno = seqno;
@ -240,27 +244,60 @@ int i915_gem_set_global_seqno(struct drm_device *dev, u32 seqno)
return reset_all_global_seqno(dev_priv, seqno - 1);
}
static int reserve_seqno(struct intel_engine_cs *engine)
static void mark_busy(struct drm_i915_private *i915)
{
if (i915->gt.awake)
return;
GEM_BUG_ON(!i915->gt.active_requests);
intel_runtime_pm_get_noresume(i915);
i915->gt.awake = true;
intel_enable_gt_powersave(i915);
i915_update_gfx_val(i915);
if (INTEL_GEN(i915) >= 6)
gen6_rps_busy(i915);
queue_delayed_work(i915->wq,
&i915->gt.retire_work,
round_jiffies_up_relative(HZ));
}
static int reserve_engine(struct intel_engine_cs *engine)
{
struct drm_i915_private *i915 = engine->i915;
u32 active = ++engine->timeline->inflight_seqnos;
u32 seqno = engine->timeline->seqno;
int ret;
/* Reservation is fine until we need to wrap around */
if (likely(!add_overflows(seqno, active)))
return 0;
ret = reset_all_global_seqno(engine->i915, 0);
if (ret) {
engine->timeline->inflight_seqnos--;
return ret;
if (unlikely(add_overflows(seqno, active))) {
ret = reset_all_global_seqno(i915, 0);
if (ret) {
engine->timeline->inflight_seqnos--;
return ret;
}
}
if (!i915->gt.active_requests++)
mark_busy(i915);
return 0;
}
static void unreserve_seqno(struct intel_engine_cs *engine)
static void unreserve_engine(struct intel_engine_cs *engine)
{
struct drm_i915_private *i915 = engine->i915;
if (!--i915->gt.active_requests) {
/* Cancel the mark_busy() from our reserve_engine() */
GEM_BUG_ON(!i915->gt.awake);
mod_delayed_work(i915->wq,
&i915->gt.idle_work,
msecs_to_jiffies(100));
}
GEM_BUG_ON(!engine->timeline->inflight_seqnos);
engine->timeline->inflight_seqnos--;
}
@ -329,13 +366,7 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
list_del_init(&request->link);
spin_unlock_irq(&engine->timeline->lock);
if (!--request->i915->gt.active_requests) {
GEM_BUG_ON(!request->i915->gt.awake);
mod_delayed_work(request->i915->wq,
&request->i915->gt.idle_work,
msecs_to_jiffies(100));
}
unreserve_seqno(request->engine);
unreserve_engine(request->engine);
advance_ring(request);
free_capture_list(request);
@ -370,8 +401,7 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
i915_gem_request_remove_from_client(request);
/* Retirement decays the ban score as it is a sign of ctx progress */
if (request->ctx->ban_score > 0)
request->ctx->ban_score--;
atomic_dec_if_positive(&request->ctx->ban_score);
/* The backing object for the context is done after switching to the
* *next* context. Therefore we cannot retire the previous context until
@ -572,7 +602,7 @@ i915_gem_request_alloc(struct intel_engine_cs *engine,
return ERR_CAST(ring);
GEM_BUG_ON(!ring);
ret = reserve_seqno(engine);
ret = reserve_engine(engine);
if (ret)
goto err_unpin;
@ -678,7 +708,7 @@ err_ctx:
kmem_cache_free(dev_priv->requests, req);
err_unreserve:
unreserve_seqno(engine);
unreserve_engine(engine);
err_unpin:
engine->context_unpin(engine, ctx);
return ERR_PTR(ret);
@ -860,28 +890,6 @@ i915_gem_request_await_object(struct drm_i915_gem_request *to,
return ret;
}
static void i915_gem_mark_busy(const struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
if (dev_priv->gt.awake)
return;
GEM_BUG_ON(!dev_priv->gt.active_requests);
intel_runtime_pm_get_noresume(dev_priv);
dev_priv->gt.awake = true;
intel_enable_gt_powersave(dev_priv);
i915_update_gfx_val(dev_priv);
if (INTEL_GEN(dev_priv) >= 6)
gen6_rps_busy(dev_priv);
queue_delayed_work(dev_priv->wq,
&dev_priv->gt.retire_work,
round_jiffies_up_relative(HZ));
}
/*
* NB: This function is not allowed to fail. Doing so would mean the the
* request is not being tracked for completion but the work itself is
@ -963,9 +971,6 @@ void __i915_add_request(struct drm_i915_gem_request *request, bool flush_caches)
list_add_tail(&request->ring_link, &ring->request_list);
request->emitted_jiffies = jiffies;
if (!request->i915->gt.active_requests++)
i915_gem_mark_busy(engine);
/* Let the backend know a new request has arrived that may need
* to adjust the existing execution schedule due to a high priority
* request - i.e. we may want to preempt the current request in order
@ -1068,7 +1073,7 @@ static bool __i915_wait_request_check_and_reset(struct drm_i915_gem_request *req
return false;
__set_current_state(TASK_RUNNING);
i915_reset(request->i915);
i915_reset(request->i915, 0);
return true;
}

8
drivers/gpu/drm/i915/i915_gem_stolen.c
View File

@ -254,9 +254,10 @@ static dma_addr_t i915_stolen_to_dma(struct drm_i915_private *dev_priv)
* This is a BIOS w/a: Some BIOS wrap stolen in the root
* PCI bus, but have an off-by-one error. Hence retry the
* reservation starting from 1 instead of 0.
* There's also BIOS with off-by-one on the other end.
*/
r = devm_request_mem_region(dev_priv->drm.dev, base + 1,
ggtt->stolen_size - 1,
ggtt->stolen_size - 2,
"Graphics Stolen Memory");
/*
* GEN3 firmware likes to smash pci bridges into the stolen
@ -579,6 +580,7 @@ _i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
struct drm_mm_node *stolen)
{
struct drm_i915_gem_object *obj;
unsigned int cache_level;
obj = i915_gem_object_alloc(dev_priv);
if (obj == NULL)
@ -589,8 +591,8 @@ _i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
obj->stolen = stolen;
obj->base.read_domains = I915_GEM_DOMAIN_CPU | I915_GEM_DOMAIN_GTT;
obj->cache_level = HAS_LLC(dev_priv) ? I915_CACHE_LLC : I915_CACHE_NONE;
obj->cache_coherent = true; /* assumptions! more like cache_oblivious */
cache_level = HAS_LLC(dev_priv) ? I915_CACHE_LLC : I915_CACHE_NONE;
i915_gem_object_set_cache_coherency(obj, cache_level);
if (i915_gem_object_pin_pages(obj))
goto cleanup;

4
drivers/gpu/drm/i915/i915_gem_userptr.c
View File

@ -804,9 +804,7 @@ i915_gem_userptr_ioctl(struct drm_device *dev, void *data, struct drm_file *file
i915_gem_object_init(obj, &i915_gem_userptr_ops);
obj->base.read_domains = I915_GEM_DOMAIN_CPU;
obj->base.write_domain = I915_GEM_DOMAIN_CPU;
obj->cache_level = I915_CACHE_LLC;
obj->cache_coherent = i915_gem_object_is_coherent(obj);
obj->cache_dirty = !obj->cache_coherent;
i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);
obj->userptr.ptr = args->user_ptr;
obj->userptr.read_only = !!(args->flags & I915_USERPTR_READ_ONLY);

8
drivers/gpu/drm/i915/i915_gpu_error.c
View File

@ -1266,7 +1266,7 @@ static void record_request(struct drm_i915_gem_request *request,
struct drm_i915_error_request *erq)
{
erq->context = request->ctx->hw_id;
erq->ban_score = request->ctx->ban_score;
erq->ban_score = atomic_read(&request->ctx->ban_score);
erq->seqno = request->global_seqno;
erq->jiffies = request->emitted_jiffies;
erq->head = request->head;
@ -1357,9 +1357,9 @@ static void record_context(struct drm_i915_error_context *e,
e->handle = ctx->user_handle;
e->hw_id = ctx->hw_id;
e->ban_score = ctx->ban_score;
e->guilty = ctx->guilty_count;
e->active = ctx->active_count;
e->ban_score = atomic_read(&ctx->ban_score);
e->guilty = atomic_read(&ctx->guilty_count);
e->active = atomic_read(&ctx->active_count);
}
static void request_record_user_bo(struct drm_i915_gem_request *request,

222
drivers/gpu/drm/i915/i915_irq.c
View File

@ -275,17 +275,17 @@ void gen5_disable_gt_irq(struct drm_i915_private *dev_priv, uint32_t mask)
static i915_reg_t gen6_pm_iir(struct drm_i915_private *dev_priv)
{
return INTEL_INFO(dev_priv)->gen >= 8 ? GEN8_GT_IIR(2) : GEN6_PMIIR;
return INTEL_GEN(dev_priv) >= 8 ? GEN8_GT_IIR(2) : GEN6_PMIIR;
}
static i915_reg_t gen6_pm_imr(struct drm_i915_private *dev_priv)
{
return INTEL_INFO(dev_priv)->gen >= 8 ? GEN8_GT_IMR(2) : GEN6_PMIMR;
return INTEL_GEN(dev_priv) >= 8 ? GEN8_GT_IMR(2) : GEN6_PMIMR;
}
static i915_reg_t gen6_pm_ier(struct drm_i915_private *dev_priv)
{
return INTEL_INFO(dev_priv)->gen >= 8 ? GEN8_GT_IER(2) : GEN6_PMIER;
return INTEL_GEN(dev_priv) >= 8 ? GEN8_GT_IER(2) : GEN6_PMIER;
}
/**
@ -1501,7 +1501,8 @@ static void intel_get_hpd_pins(u32 *pin_mask, u32 *long_mask,
*pin_mask |= BIT(i);
if (!intel_hpd_pin_to_port(i, &port))
port = intel_hpd_pin_to_port(i);
if (port == PORT_NONE)
continue;
if (long_pulse_detect(port, dig_hotplug_reg))
@ -1661,7 +1662,7 @@ static void gen6_rps_irq_handler(struct drm_i915_private *dev_priv, u32 pm_iir)
spin_unlock(&dev_priv->irq_lock);
}
if (INTEL_INFO(dev_priv)->gen >= 8)
if (INTEL_GEN(dev_priv) >= 8)
return;
if (HAS_VEBOX(dev_priv)) {
@ -1708,18 +1709,6 @@ static void gen9_guc_irq_handler(struct drm_i915_private *dev_priv, u32 gt_iir)
}
}
static bool intel_pipe_handle_vblank(struct drm_i915_private *dev_priv,
enum pipe pipe)
{
bool ret;
ret = drm_handle_vblank(&dev_priv->drm, pipe);
if (ret)
intel_finish_page_flip_mmio(dev_priv, pipe);
return ret;
}
static void valleyview_pipestat_irq_ack(struct drm_i915_private *dev_priv,
u32 iir, u32 pipe_stats[I915_MAX_PIPES])
{
@ -1784,12 +1773,8 @@ static void valleyview_pipestat_irq_handler(struct drm_i915_private *dev_priv,
enum pipe pipe;
for_each_pipe(dev_priv, pipe) {
if (pipe_stats[pipe] & PIPE_START_VBLANK_INTERRUPT_STATUS &&
intel_pipe_handle_vblank(dev_priv, pipe))
intel_check_page_flip(dev_priv, pipe);
if (pipe_stats[pipe] & PLANE_FLIP_DONE_INT_STATUS_VLV)
intel_finish_page_flip_cs(dev_priv, pipe);
if (pipe_stats[pipe] & PIPE_START_VBLANK_INTERRUPT_STATUS)
drm_handle_vblank(&dev_priv->drm, pipe);
if (pipe_stats[pipe] & PIPE_CRC_DONE_INTERRUPT_STATUS)
i9xx_pipe_crc_irq_handler(dev_priv, pipe);
@ -2086,10 +2071,10 @@ static void ibx_irq_handler(struct drm_i915_private *dev_priv, u32 pch_iir)
DRM_DEBUG_DRIVER("PCH transcoder CRC error interrupt\n");
if (pch_iir & SDE_TRANSA_FIFO_UNDER)
intel_pch_fifo_underrun_irq_handler(dev_priv, TRANSCODER_A);
intel_pch_fifo_underrun_irq_handler(dev_priv, PIPE_A);
if (pch_iir & SDE_TRANSB_FIFO_UNDER)
intel_pch_fifo_underrun_irq_handler(dev_priv, TRANSCODER_B);
intel_pch_fifo_underrun_irq_handler(dev_priv, PIPE_B);
}
static void ivb_err_int_handler(struct drm_i915_private *dev_priv)
@ -2123,13 +2108,13 @@ static void cpt_serr_int_handler(struct drm_i915_private *dev_priv)
DRM_ERROR("PCH poison interrupt\n");
if (serr_int & SERR_INT_TRANS_A_FIFO_UNDERRUN)
intel_pch_fifo_underrun_irq_handler(dev_priv, TRANSCODER_A);
intel_pch_fifo_underrun_irq_handler(dev_priv, PIPE_A);
if (serr_int & SERR_INT_TRANS_B_FIFO_UNDERRUN)
intel_pch_fifo_underrun_irq_handler(dev_priv, TRANSCODER_B);
intel_pch_fifo_underrun_irq_handler(dev_priv, PIPE_B);
if (serr_int & SERR_INT_TRANS_C_FIFO_UNDERRUN)
intel_pch_fifo_underrun_irq_handler(dev_priv, TRANSCODER_C);
intel_pch_fifo_underrun_irq_handler(dev_priv, PIPE_C);
I915_WRITE(SERR_INT, serr_int);
}
@ -2241,19 +2226,14 @@ static void ilk_display_irq_handler(struct drm_i915_private *dev_priv,
DRM_ERROR("Poison interrupt\n");
for_each_pipe(dev_priv, pipe) {
if (de_iir & DE_PIPE_VBLANK(pipe) &&
intel_pipe_handle_vblank(dev_priv, pipe))
intel_check_page_flip(dev_priv, pipe);
if (de_iir & DE_PIPE_VBLANK(pipe))
drm_handle_vblank(&dev_priv->drm, pipe);
if (de_iir & DE_PIPE_FIFO_UNDERRUN(pipe))
intel_cpu_fifo_underrun_irq_handler(dev_priv, pipe);
if (de_iir & DE_PIPE_CRC_DONE(pipe))
i9xx_pipe_crc_irq_handler(dev_priv, pipe);
/* plane/pipes map 1:1 on ilk+ */
if (de_iir & DE_PLANE_FLIP_DONE(pipe))
intel_finish_page_flip_cs(dev_priv, pipe);
}
/* check event from PCH */
@ -2292,13 +2272,8 @@ static void ivb_display_irq_handler(struct drm_i915_private *dev_priv,
intel_opregion_asle_intr(dev_priv);
for_each_pipe(dev_priv, pipe) {
if (de_iir & (DE_PIPE_VBLANK_IVB(pipe)) &&
intel_pipe_handle_vblank(dev_priv, pipe))
intel_check_page_flip(dev_priv, pipe);
/* plane/pipes map 1:1 on ilk+ */
if (de_iir & DE_PLANE_FLIP_DONE_IVB(pipe))
intel_finish_page_flip_cs(dev_priv, pipe);
if (de_iir & (DE_PIPE_VBLANK_IVB(pipe)))
drm_handle_vblank(&dev_priv->drm, pipe);
}
/* check event from PCH */
@ -2440,7 +2415,7 @@ gen8_de_irq_handler(struct drm_i915_private *dev_priv, u32 master_ctl)
ret = IRQ_HANDLED;
tmp_mask = GEN8_AUX_CHANNEL_A;
if (INTEL_INFO(dev_priv)->gen >= 9)
if (INTEL_GEN(dev_priv) >= 9)
tmp_mask |= GEN9_AUX_CHANNEL_B |
GEN9_AUX_CHANNEL_C |
GEN9_AUX_CHANNEL_D;
@ -2479,7 +2454,7 @@ gen8_de_irq_handler(struct drm_i915_private *dev_priv, u32 master_ctl)
}
for_each_pipe(dev_priv, pipe) {
u32 flip_done, fault_errors;
u32 fault_errors;
if (!(master_ctl & GEN8_DE_PIPE_IRQ(pipe)))
continue;
@ -2493,18 +2468,8 @@ gen8_de_irq_handler(struct drm_i915_private *dev_priv, u32 master_ctl)
ret = IRQ_HANDLED;
I915_WRITE(GEN8_DE_PIPE_IIR(pipe), iir);
if (iir & GEN8_PIPE_VBLANK &&
intel_pipe_handle_vblank(dev_priv, pipe))
intel_check_page_flip(dev_priv, pipe);
flip_done = iir;
if (INTEL_INFO(dev_priv)->gen >= 9)
flip_done &= GEN9_PIPE_PLANE1_FLIP_DONE;
else
flip_done &= GEN8_PIPE_PRIMARY_FLIP_DONE;
if (flip_done)
intel_finish_page_flip_cs(dev_priv, pipe);
if (iir & GEN8_PIPE_VBLANK)
drm_handle_vblank(&dev_priv->drm, pipe);
if (iir & GEN8_PIPE_CDCLK_CRC_DONE)
hsw_pipe_crc_irq_handler(dev_priv, pipe);
@ -2513,7 +2478,7 @@ gen8_de_irq_handler(struct drm_i915_private *dev_priv, u32 master_ctl)
intel_cpu_fifo_underrun_irq_handler(dev_priv, pipe);
fault_errors = iir;
if (INTEL_INFO(dev_priv)->gen >= 9)
if (INTEL_GEN(dev_priv) >= 9)
fault_errors &= GEN9_DE_PIPE_IRQ_FAULT_ERRORS;
else
fault_errors &= GEN8_DE_PIPE_IRQ_FAULT_ERRORS;
@ -2660,7 +2625,7 @@ static void i915_reset_device(struct drm_i915_private *dev_priv)
*/
do {
if (mutex_trylock(&dev_priv->drm.struct_mutex)) {
i915_reset(dev_priv);
i915_reset(dev_priv, 0);
mutex_unlock(&dev_priv->drm.struct_mutex);
}
} while (wait_on_bit_timeout(&dev_priv->gpu_error.flags,
@ -2676,32 +2641,6 @@ static void i915_reset_device(struct drm_i915_private *dev_priv)
KOBJ_CHANGE, reset_done_event);
}
static inline void
i915_err_print_instdone(struct drm_i915_private *dev_priv,
struct intel_instdone *instdone)
{
int slice;
int subslice;
pr_err(" INSTDONE: 0x%08x\n", instdone->instdone);
if (INTEL_GEN(dev_priv) <= 3)
return;
pr_err(" SC_INSTDONE: 0x%08x\n", instdone->slice_common);
if (INTEL_GEN(dev_priv) <= 6)
return;
for_each_instdone_slice_subslice(dev_priv, slice, subslice)
pr_err(" SAMPLER_INSTDONE[%d][%d]: 0x%08x\n",
slice, subslice, instdone->sampler[slice][subslice]);
for_each_instdone_slice_subslice(dev_priv, slice, subslice)
pr_err(" ROW_INSTDONE[%d][%d]: 0x%08x\n",
slice, subslice, instdone->row[slice][subslice]);
}
static void i915_clear_error_registers(struct drm_i915_private *dev_priv)
{
u32 eir;
@ -2770,12 +2709,12 @@ void i915_handle_error(struct drm_i915_private *dev_priv,
*/
if (intel_has_reset_engine(dev_priv)) {
for_each_engine_masked(engine, dev_priv, engine_mask, tmp) {
BUILD_BUG_ON(I915_RESET_HANDOFF >= I915_RESET_ENGINE);
BUILD_BUG_ON(I915_RESET_MODESET >= I915_RESET_ENGINE);
if (test_and_set_bit(I915_RESET_ENGINE + engine->id,
&dev_priv->gpu_error.flags))
continue;
if (i915_reset_engine(engine) == 0)
if (i915_reset_engine(engine, 0) == 0)
engine_mask &= ~intel_engine_flag(engine);
clear_bit(I915_RESET_ENGINE + engine->id,
@ -3074,7 +3013,7 @@ static void gen8_irq_reset(struct drm_device *dev)
}
void gen8_irq_power_well_post_enable(struct drm_i915_private *dev_priv,
unsigned int pipe_mask)
u8 pipe_mask)
{
uint32_t extra_ier = GEN8_PIPE_VBLANK | GEN8_PIPE_FIFO_UNDERRUN;
enum pipe pipe;
@ -3088,7 +3027,7 @@ void gen8_irq_power_well_post_enable(struct drm_i915_private *dev_priv,
}
void gen8_irq_power_well_pre_disable(struct drm_i915_private *dev_priv,
unsigned int pipe_mask)
u8 pipe_mask)
{
enum pipe pipe;
@ -3492,7 +3431,7 @@ static void gen8_de_irq_postinstall(struct drm_i915_private *dev_priv)
u32 de_misc_masked = GEN8_DE_MISC_GSE;
enum pipe pipe;
if (INTEL_INFO(dev_priv)->gen >= 9) {
if (INTEL_GEN(dev_priv) >= 9) {
de_pipe_masked |= GEN9_PIPE_PLANE1_FLIP_DONE |
GEN9_DE_PIPE_IRQ_FAULT_ERRORS;
de_port_masked |= GEN9_AUX_CHANNEL_B | GEN9_AUX_CHANNEL_C |
@ -3675,34 +3614,6 @@ static int i8xx_irq_postinstall(struct drm_device *dev)
/*
* Returns true when a page flip has completed.
*/
static bool i8xx_handle_vblank(struct drm_i915_private *dev_priv,
int plane, int pipe, u32 iir)
{
u16 flip_pending = DISPLAY_PLANE_FLIP_PENDING(plane);
if (!intel_pipe_handle_vblank(dev_priv, pipe))
return false;
if ((iir & flip_pending) == 0)
goto check_page_flip;
/* We detect FlipDone by looking for the change in PendingFlip from '1'
* to '0' on the following vblank, i.e. IIR has the Pendingflip
* asserted following the MI_DISPLAY_FLIP, but ISR is deasserted, hence
* the flip is completed (no longer pending). Since this doesn't raise
* an interrupt per se, we watch for the change at vblank.
*/
if (I915_READ16(ISR) & flip_pending)
goto check_page_flip;
intel_finish_page_flip_cs(dev_priv, pipe);
return true;
check_page_flip:
intel_check_page_flip(dev_priv, pipe);
return false;
}
static irqreturn_t i8xx_irq_handler(int irq, void *arg)
{
struct drm_device *dev = arg;
@ -3710,9 +3621,6 @@ static irqreturn_t i8xx_irq_handler(int irq, void *arg)
u16 iir, new_iir;
u32 pipe_stats[2];
int pipe;
u16 flip_mask =
I915_DISPLAY_PLANE_A_FLIP_PENDING_INTERRUPT |
I915_DISPLAY_PLANE_B_FLIP_PENDING_INTERRUPT;
irqreturn_t ret;
if (!intel_irqs_enabled(dev_priv))
@ -3726,7 +3634,7 @@ static irqreturn_t i8xx_irq_handler(int irq, void *arg)
if (iir == 0)
goto out;
while (iir & ~flip_mask) {
while (iir) {
/* Can't rely on pipestat interrupt bit in iir as it might
* have been cleared after the pipestat interrupt was received.
* It doesn't set the bit in iir again, but it still produces
@ -3748,7 +3656,7 @@ static irqreturn_t i8xx_irq_handler(int irq, void *arg)
}
spin_unlock(&dev_priv->irq_lock);
I915_WRITE16(IIR, iir & ~flip_mask);
I915_WRITE16(IIR, iir);
new_iir = I915_READ16(IIR); /* Flush posted writes */
if (iir & I915_USER_INTERRUPT)
@ -3759,9 +3667,8 @@ static irqreturn_t i8xx_irq_handler(int irq, void *arg)
if (HAS_FBC(dev_priv))
plane = !plane;
if (pipe_stats[pipe] & PIPE_VBLANK_INTERRUPT_STATUS &&
i8xx_handle_vblank(dev_priv, plane, pipe, iir))
flip_mask &= ~DISPLAY_PLANE_FLIP_PENDING(plane);
if (pipe_stats[pipe] & PIPE_VBLANK_INTERRUPT_STATUS)
drm_handle_vblank(&dev_priv->drm, pipe);
if (pipe_stats[pipe] & PIPE_CRC_DONE_INTERRUPT_STATUS)
i9xx_pipe_crc_irq_handler(dev_priv, pipe);
@ -3861,45 +3768,11 @@ static int i915_irq_postinstall(struct drm_device *dev)
return 0;
}
/*
* Returns true when a page flip has completed.
*/
static bool i915_handle_vblank(struct drm_i915_private *dev_priv,
int plane, int pipe, u32 iir)
{
u32 flip_pending = DISPLAY_PLANE_FLIP_PENDING(plane);
if (!intel_pipe_handle_vblank(dev_priv, pipe))
return false;
if ((iir & flip_pending) == 0)
goto check_page_flip;
/* We detect FlipDone by looking for the change in PendingFlip from '1'
* to '0' on the following vblank, i.e. IIR has the Pendingflip
* asserted following the MI_DISPLAY_FLIP, but ISR is deasserted, hence
* the flip is completed (no longer pending). Since this doesn't raise
* an interrupt per se, we watch for the change at vblank.
*/
if (I915_READ(ISR) & flip_pending)
goto check_page_flip;
intel_finish_page_flip_cs(dev_priv, pipe);
return true;
check_page_flip:
intel_check_page_flip(dev_priv, pipe);
return false;
}
static irqreturn_t i915_irq_handler(int irq, void *arg)
{
struct drm_device *dev = arg;
struct drm_i915_private *dev_priv = to_i915(dev);
u32 iir, new_iir, pipe_stats[I915_MAX_PIPES];
u32 flip_mask =
I915_DISPLAY_PLANE_A_FLIP_PENDING_INTERRUPT |
I915_DISPLAY_PLANE_B_FLIP_PENDING_INTERRUPT;
int pipe, ret = IRQ_NONE;
if (!intel_irqs_enabled(dev_priv))
@ -3910,7 +3783,7 @@ static irqreturn_t i915_irq_handler(int irq, void *arg)
iir = I915_READ(IIR);
do {
bool irq_received = (iir & ~flip_mask) != 0;
bool irq_received = (iir) != 0;
bool blc_event = false;
/* Can't rely on pipestat interrupt bit in iir as it might
@ -3945,7 +3818,7 @@ static irqreturn_t i915_irq_handler(int irq, void *arg)
i9xx_hpd_irq_handler(dev_priv, hotplug_status);
}
I915_WRITE(IIR, iir & ~flip_mask);
I915_WRITE(IIR, iir);
new_iir = I915_READ(IIR); /* Flush posted writes */
if (iir & I915_USER_INTERRUPT)
@ -3956,9 +3829,8 @@ static irqreturn_t i915_irq_handler(int irq, void *arg)
if (HAS_FBC(dev_priv))
plane = !plane;
if (pipe_stats[pipe] & PIPE_VBLANK_INTERRUPT_STATUS &&
i915_handle_vblank(dev_priv, plane, pipe, iir))
flip_mask &= ~DISPLAY_PLANE_FLIP_PENDING(plane);
if (pipe_stats[pipe] & PIPE_VBLANK_INTERRUPT_STATUS)
drm_handle_vblank(&dev_priv->drm, pipe);
if (pipe_stats[pipe] & PIPE_LEGACY_BLC_EVENT_STATUS)
blc_event = true;
@ -3991,7 +3863,7 @@ static irqreturn_t i915_irq_handler(int irq, void *arg)
*/
ret = IRQ_HANDLED;
iir = new_iir;
} while (iir & ~flip_mask);
} while (iir);
enable_rpm_wakeref_asserts(dev_priv);
@ -4126,9 +3998,6 @@ static irqreturn_t i965_irq_handler(int irq, void *arg)
u32 iir, new_iir;
u32 pipe_stats[I915_MAX_PIPES];
int ret = IRQ_NONE, pipe;
u32 flip_mask =
I915_DISPLAY_PLANE_A_FLIP_PENDING_INTERRUPT |
I915_DISPLAY_PLANE_B_FLIP_PENDING_INTERRUPT;
if (!intel_irqs_enabled(dev_priv))
return IRQ_NONE;
@ -4139,7 +4008,7 @@ static irqreturn_t i965_irq_handler(int irq, void *arg)
iir = I915_READ(IIR);
for (;;) {
bool irq_received = (iir & ~flip_mask) != 0;
bool irq_received = (iir) != 0;
bool blc_event = false;
/* Can't rely on pipestat interrupt bit in iir as it might
@ -4177,7 +4046,7 @@ static irqreturn_t i965_irq_handler(int irq, void *arg)
i9xx_hpd_irq_handler(dev_priv, hotplug_status);
}
I915_WRITE(IIR, iir & ~flip_mask);
I915_WRITE(IIR, iir);
new_iir = I915_READ(IIR); /* Flush posted writes */
if (iir & I915_USER_INTERRUPT)
@ -4186,9 +4055,8 @@ static irqreturn_t i965_irq_handler(int irq, void *arg)
notify_ring(dev_priv->engine[VCS]);
for_each_pipe(dev_priv, pipe) {
if (pipe_stats[pipe] & PIPE_START_VBLANK_INTERRUPT_STATUS &&
i915_handle_vblank(dev_priv, pipe, pipe, iir))
flip_mask &= ~DISPLAY_PLANE_FLIP_PENDING(pipe);
if (pipe_stats[pipe] & PIPE_START_VBLANK_INTERRUPT_STATUS)
drm_handle_vblank(&dev_priv->drm, pipe);
if (pipe_stats[pipe] & PIPE_LEGACY_BLC_EVENT_STATUS)
blc_event = true;
@ -4290,16 +4158,16 @@ void intel_irq_init(struct drm_i915_private *dev_priv)
*
* TODO: verify if this can be reproduced on VLV,CHV.
*/
if (INTEL_INFO(dev_priv)->gen <= 7)
if (INTEL_GEN(dev_priv) <= 7)
dev_priv->rps.pm_intrmsk_mbz |= GEN6_PM_RP_UP_EI_EXPIRED;
if (INTEL_INFO(dev_priv)->gen >= 8)
if (INTEL_GEN(dev_priv) >= 8)
dev_priv->rps.pm_intrmsk_mbz |= GEN8_PMINTR_DISABLE_REDIRECT_TO_GUC;
if (IS_GEN2(dev_priv)) {
/* Gen2 doesn't have a hardware frame counter */
dev->max_vblank_count = 0;
} else if (IS_G4X(dev_priv) || INTEL_INFO(dev_priv)->gen >= 5) {
} else if (IS_G4X(dev_priv) || INTEL_GEN(dev_priv) >= 5) {
dev->max_vblank_count = 0xffffffff; /* full 32 bit counter */
dev->driver->get_vblank_counter = g4x_get_vblank_counter;
} else {
@ -4346,7 +4214,7 @@ void intel_irq_init(struct drm_i915_private *dev_priv)
dev->driver->enable_vblank = i965_enable_vblank;
dev->driver->disable_vblank = i965_disable_vblank;
dev_priv->display.hpd_irq_setup = i915_hpd_irq_setup;
} else if (INTEL_INFO(dev_priv)->gen >= 8) {
} else if (INTEL_GEN(dev_priv) >= 8) {
dev->driver->irq_handler = gen8_irq_handler;
dev->driver->irq_preinstall = gen8_irq_reset;
dev->driver->irq_postinstall = gen8_irq_postinstall;

5316
drivers/gpu/drm/i915/i915_oa_bdw.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_bdw.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_BDW_H__
#define __I915_OA_BDW_H__
extern int i915_oa_n_builtin_metric_sets_bdw;
extern int i915_oa_select_metric_set_bdw(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_bdw(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_bdw(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_bdw(struct drm_i915_private *dev_priv);
#endif

2630
drivers/gpu/drm/i915/i915_oa_bxt.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_bxt.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_BXT_H__
#define __I915_OA_BXT_H__
extern int i915_oa_n_builtin_metric_sets_bxt;
extern int i915_oa_select_metric_set_bxt(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_bxt(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_bxt(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_bxt(struct drm_i915_private *dev_priv);
#endif

2814
drivers/gpu/drm/i915/i915_oa_chv.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_chv.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_CHV_H__
#define __I915_OA_CHV_H__
extern int i915_oa_n_builtin_metric_sets_chv;
extern int i915_oa_select_metric_set_chv(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_chv(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_chv(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_chv(struct drm_i915_private *dev_priv);
#endif

2542
drivers/gpu/drm/i915/i915_oa_glk.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_glk.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_GLK_H__
#define __I915_OA_GLK_H__
extern int i915_oa_n_builtin_metric_sets_glk;
extern int i915_oa_select_metric_set_glk(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_glk(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_glk(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_glk(struct drm_i915_private *dev_priv);
#endif

773
drivers/gpu/drm/i915/i915_oa_hsw.c
View File

@ -31,17 +31,6 @@
#include "i915_drv.h"
#include "i915_oa_hsw.h"
enum metric_set_id {
METRIC_SET_ID_RENDER_BASIC = 1,
METRIC_SET_ID_COMPUTE_BASIC,
METRIC_SET_ID_COMPUTE_EXTENDED,
METRIC_SET_ID_MEMORY_READS,
METRIC_SET_ID_MEMORY_WRITES,
METRIC_SET_ID_SAMPLER_BALANCE,
};
int i915_oa_n_builtin_metric_sets_hsw = 6;
static const struct i915_oa_reg b_counter_config_render_basic[] = {
{ _MMIO(0x2724), 0x00800000 },
{ _MMIO(0x2720), 0x00000000 },
@ -53,6 +42,7 @@ static const struct i915_oa_reg flex_eu_config_render_basic[] = {
};
static const struct i915_oa_reg mux_config_render_basic[] = {
{ _MMIO(0x9840), 0x00000080 },
{ _MMIO(0x253a4), 0x01600000 },
{ _MMIO(0x25440), 0x00100000 },
{ _MMIO(0x25128), 0x00000000 },
@ -114,750 +104,35 @@ static const struct i915_oa_reg mux_config_render_basic[] = {
{ _MMIO(0x25428), 0x00042049 },
};
static int
get_render_basic_mux_config(struct drm_i915_private *dev_priv,
const struct i915_oa_reg **regs,
int *lens)
{
int n = 0;
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
regs[n] = mux_config_render_basic;
lens[n] = ARRAY_SIZE(mux_config_render_basic);
n++;
return n;
}
static const struct i915_oa_reg b_counter_config_compute_basic[] = {
{ _MMIO(0x2710), 0x00000000 },
{ _MMIO(0x2714), 0x00800000 },
{ _MMIO(0x2718), 0xaaaaaaaa },
{ _MMIO(0x271c), 0xaaaaaaaa },
{ _MMIO(0x2720), 0x00000000 },
{ _MMIO(0x2724), 0x00800000 },
{ _MMIO(0x2728), 0xaaaaaaaa },
{ _MMIO(0x272c), 0xaaaaaaaa },
{ _MMIO(0x2740), 0x00000000 },
{ _MMIO(0x2744), 0x00000000 },
{ _MMIO(0x2748), 0x00000000 },
{ _MMIO(0x274c), 0x00000000 },
{ _MMIO(0x2750), 0x00000000 },
{ _MMIO(0x2754), 0x00000000 },
{ _MMIO(0x2758), 0x00000000 },
{ _MMIO(0x275c), 0x00000000 },
{ _MMIO(0x236c), 0x00000000 },
};
static const struct i915_oa_reg flex_eu_config_compute_basic[] = {
};
static const struct i915_oa_reg mux_config_compute_basic[] = {
{ _MMIO(0x253a4), 0x00000000 },
{ _MMIO(0x2681c), 0x01f00800 },
{ _MMIO(0x26820), 0x00001000 },
{ _MMIO(0x2781c), 0x01f00800 },
{ _MMIO(0x26520), 0x00000007 },
{ _MMIO(0x265a0), 0x00000007 },
{ _MMIO(0x25380), 0x00000010 },
{ _MMIO(0x2538c), 0x00300000 },
{ _MMIO(0x25384), 0xaa8aaaaa },
{ _MMIO(0x25404), 0xffffffff },
{ _MMIO(0x26800), 0x00004202 },
{ _MMIO(0x26808), 0x00605817 },
{ _MMIO(0x2680c), 0x10001005 },
{ _MMIO(0x26804), 0x00000000 },
{ _MMIO(0x27800), 0x00000102 },
{ _MMIO(0x27808), 0x0c0701e0 },
{ _MMIO(0x2780c), 0x000200a0 },
{ _MMIO(0x27804), 0x00000000 },
{ _MMIO(0x26484), 0x44000000 },
{ _MMIO(0x26704), 0x44000000 },
{ _MMIO(0x26500), 0x00000006 },
{ _MMIO(0x26510), 0x00000001 },
{ _MMIO(0x26504), 0x88000000 },
{ _MMIO(0x26580), 0x00000006 },
{ _MMIO(0x26590), 0x00000020 },
{ _MMIO(0x26584), 0x00000000 },
{ _MMIO(0x26104), 0x55822222 },
{ _MMIO(0x26184), 0xaa866666 },
{ _MMIO(0x25420), 0x08320c83 },
{ _MMIO(0x25424), 0x06820c83 },
{ _MMIO(0x2541c), 0x00000000 },
{ _MMIO(0x25428), 0x00000c03 },
};
static int
get_compute_basic_mux_config(struct drm_i915_private *dev_priv,
const struct i915_oa_reg **regs,
int *lens)
{
int n = 0;
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
regs[n] = mux_config_compute_basic;
lens[n] = ARRAY_SIZE(mux_config_compute_basic);
n++;
return n;
}
static const struct i915_oa_reg b_counter_config_compute_extended[] = {
{ _MMIO(0x2724), 0xf0800000 },
{ _MMIO(0x2720), 0x00000000 },
{ _MMIO(0x2714), 0xf0800000 },
{ _MMIO(0x2710), 0x00000000 },
{ _MMIO(0x2770), 0x0007fe2a },
{ _MMIO(0x2774), 0x0000ff00 },
{ _MMIO(0x2778), 0x0007fe6a },
{ _MMIO(0x277c), 0x0000ff00 },
{ _MMIO(0x2780), 0x0007fe92 },
{ _MMIO(0x2784), 0x0000ff00 },
{ _MMIO(0x2788), 0x0007fea2 },
{ _MMIO(0x278c), 0x0000ff00 },
{ _MMIO(0x2790), 0x0007fe32 },
{ _MMIO(0x2794), 0x0000ff00 },
{ _MMIO(0x2798), 0x0007fe9a },
{ _MMIO(0x279c), 0x0000ff00 },
{ _MMIO(0x27a0), 0x0007ff23 },
{ _MMIO(0x27a4), 0x0000ff00 },
{ _MMIO(0x27a8), 0x0007fff3 },
{ _MMIO(0x27ac), 0x0000fffe },
};
static const struct i915_oa_reg flex_eu_config_compute_extended[] = {
};
static const struct i915_oa_reg mux_config_compute_extended[] = {
{ _MMIO(0x2681c), 0x3eb00800 },
{ _MMIO(0x26820), 0x00900000 },
{ _MMIO(0x25384), 0x02aaaaaa },
{ _MMIO(0x25404), 0x03ffffff },
{ _MMIO(0x26800), 0x00142284 },
{ _MMIO(0x26808), 0x0e629062 },
{ _MMIO(0x2680c), 0x3f6f55cb },
{ _MMIO(0x26810), 0x00000014 },
{ _MMIO(0x26804), 0x00000000 },
{ _MMIO(0x26104), 0x02aaaaaa },
{ _MMIO(0x26184), 0x02aaaaaa },
{ _MMIO(0x25420), 0x00000000 },
{ _MMIO(0x25424), 0x00000000 },
{ _MMIO(0x2541c), 0x00000000 },
{ _MMIO(0x25428), 0x00000000 },
};
static int
get_compute_extended_mux_config(struct drm_i915_private *dev_priv,
const struct i915_oa_reg **regs,
int *lens)
{
int n = 0;
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
regs[n] = mux_config_compute_extended;
lens[n] = ARRAY_SIZE(mux_config_compute_extended);
n++;
return n;
}
static const struct i915_oa_reg b_counter_config_memory_reads[] = {
{ _MMIO(0x2724), 0xf0800000 },
{ _MMIO(0x2720), 0x00000000 },
{ _MMIO(0x2714), 0xf0800000 },
{ _MMIO(0x2710), 0x00000000 },
{ _MMIO(0x274c), 0x76543298 },
{ _MMIO(0x2748), 0x98989898 },
{ _MMIO(0x2744), 0x000000e4 },
{ _MMIO(0x2740), 0x00000000 },
{ _MMIO(0x275c), 0x98a98a98 },
{ _MMIO(0x2758), 0x88888888 },
{ _MMIO(0x2754), 0x000c5500 },
{ _MMIO(0x2750), 0x00000000 },
{ _MMIO(0x2770), 0x0007f81a },
{ _MMIO(0x2774), 0x0000fc00 },
{ _MMIO(0x2778), 0x0007f82a },
{ _MMIO(0x277c), 0x0000fc00 },
{ _MMIO(0x2780), 0x0007f872 },
{ _MMIO(0x2784), 0x0000fc00 },
{ _MMIO(0x2788), 0x0007f8ba },
{ _MMIO(0x278c), 0x0000fc00 },
{ _MMIO(0x2790), 0x0007f87a },
{ _MMIO(0x2794), 0x0000fc00 },
{ _MMIO(0x2798), 0x0007f8ea },
{ _MMIO(0x279c), 0x0000fc00 },
{ _MMIO(0x27a0), 0x0007f8e2 },
{ _MMIO(0x27a4), 0x0000fc00 },
{ _MMIO(0x27a8), 0x0007f8f2 },
{ _MMIO(0x27ac), 0x0000fc00 },
};
static const struct i915_oa_reg flex_eu_config_memory_reads[] = {
};
static const struct i915_oa_reg mux_config_memory_reads[] = {
{ _MMIO(0x253a4), 0x34300000 },
{ _MMIO(0x25440), 0x2d800000 },
{ _MMIO(0x25444), 0x00000008 },
{ _MMIO(0x25128), 0x0e600000 },
{ _MMIO(0x25380), 0x00000450 },
{ _MMIO(0x25390), 0x00052c43 },
{ _MMIO(0x25384), 0x00000000 },
{ _MMIO(0x25400), 0x00006144 },
{ _MMIO(0x25408), 0x0a418820 },
{ _MMIO(0x2540c), 0x000820e6 },
{ _MMIO(0x25404), 0xff500000 },
{ _MMIO(0x25100), 0x000005d6 },
{ _MMIO(0x2510c), 0x0ef00000 },
{ _MMIO(0x25104), 0x00000000 },
{ _MMIO(0x25420), 0x02108421 },
{ _MMIO(0x25424), 0x00008421 },
{ _MMIO(0x2541c), 0x00000000 },
{ _MMIO(0x25428), 0x00000000 },
};
static int
get_memory_reads_mux_config(struct drm_i915_private *dev_priv,
const struct i915_oa_reg **regs,
int *lens)
{
int n = 0;
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
regs[n] = mux_config_memory_reads;
lens[n] = ARRAY_SIZE(mux_config_memory_reads);
n++;
return n;
}
static const struct i915_oa_reg b_counter_config_memory_writes[] = {
{ _MMIO(0x2724), 0xf0800000 },
{ _MMIO(0x2720), 0x00000000 },
{ _MMIO(0x2714), 0xf0800000 },
{ _MMIO(0x2710), 0x00000000 },
{ _MMIO(0x274c), 0x76543298 },
{ _MMIO(0x2748), 0x98989898 },
{ _MMIO(0x2744), 0x000000e4 },
{ _MMIO(0x2740), 0x00000000 },
{ _MMIO(0x275c), 0xbabababa },
{ _MMIO(0x2758), 0x88888888 },
{ _MMIO(0x2754), 0x000c5500 },
{ _MMIO(0x2750), 0x00000000 },
{ _MMIO(0x2770), 0x0007f81a },
{ _MMIO(0x2774), 0x0000fc00 },
{ _MMIO(0x2778), 0x0007f82a },
{ _MMIO(0x277c), 0x0000fc00 },
{ _MMIO(0x2780), 0x0007f822 },
{ _MMIO(0x2784), 0x0000fc00 },
{ _MMIO(0x2788), 0x0007f8ba },
{ _MMIO(0x278c), 0x0000fc00 },
{ _MMIO(0x2790), 0x0007f87a },
{ _MMIO(0x2794), 0x0000fc00 },
{ _MMIO(0x2798), 0x0007f8ea },
{ _MMIO(0x279c), 0x0000fc00 },
{ _MMIO(0x27a0), 0x0007f8e2 },
{ _MMIO(0x27a4), 0x0000fc00 },
{ _MMIO(0x27a8), 0x0007f8f2 },
{ _MMIO(0x27ac), 0x0000fc00 },
};
static const struct i915_oa_reg flex_eu_config_memory_writes[] = {
};
static const struct i915_oa_reg mux_config_memory_writes[] = {
{ _MMIO(0x253a4), 0x34300000 },
{ _MMIO(0x25440), 0x01500000 },
{ _MMIO(0x25444), 0x00000120 },
{ _MMIO(0x25128), 0x0c200000 },
{ _MMIO(0x25380), 0x00000450 },
{ _MMIO(0x25390), 0x00052c43 },
{ _MMIO(0x25384), 0x00000000 },
{ _MMIO(0x25400), 0x00007184 },
{ _MMIO(0x25408), 0x0a418820 },
{ _MMIO(0x2540c), 0x000820e6 },
{ _MMIO(0x25404), 0xff500000 },
{ _MMIO(0x25100), 0x000005d6 },
{ _MMIO(0x2510c), 0x1e700000 },
{ _MMIO(0x25104), 0x00000000 },
{ _MMIO(0x25420), 0x02108421 },
{ _MMIO(0x25424), 0x00008421 },
{ _MMIO(0x2541c), 0x00000000 },
{ _MMIO(0x25428), 0x00000000 },
};
static int
get_memory_writes_mux_config(struct drm_i915_private *dev_priv,
const struct i915_oa_reg **regs,
int *lens)
{
int n = 0;
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
regs[n] = mux_config_memory_writes;
lens[n] = ARRAY_SIZE(mux_config_memory_writes);
n++;
return n;
}
static const struct i915_oa_reg b_counter_config_sampler_balance[] = {
{ _MMIO(0x2740), 0x00000000 },
{ _MMIO(0x2744), 0x00800000 },
{ _MMIO(0x2710), 0x00000000 },
{ _MMIO(0x2714), 0x00800000 },
{ _MMIO(0x2720), 0x00000000 },
{ _MMIO(0x2724), 0x00800000 },
};
static const struct i915_oa_reg flex_eu_config_sampler_balance[] = {
};
static const struct i915_oa_reg mux_config_sampler_balance[] = {
{ _MMIO(0x2eb9c), 0x01906400 },
{ _MMIO(0x2fb9c), 0x01906400 },
{ _MMIO(0x253a4), 0x00000000 },
{ _MMIO(0x26b9c), 0x01906400 },
{ _MMIO(0x27b9c), 0x01906400 },
{ _MMIO(0x27104), 0x00a00000 },
{ _MMIO(0x27184), 0x00a50000 },
{ _MMIO(0x2e804), 0x00500000 },
{ _MMIO(0x2e984), 0x00500000 },
{ _MMIO(0x2eb04), 0x00500000 },
{ _MMIO(0x2eb80), 0x00000084 },
{ _MMIO(0x2eb8c), 0x14200000 },
{ _MMIO(0x2eb84), 0x00000000 },
{ _MMIO(0x2f804), 0x00050000 },
{ _MMIO(0x2f984), 0x00050000 },
{ _MMIO(0x2fb04), 0x00050000 },
{ _MMIO(0x2fb80), 0x00000084 },
{ _MMIO(0x2fb8c), 0x00050800 },
{ _MMIO(0x2fb84), 0x00000000 },
{ _MMIO(0x25380), 0x00000010 },
{ _MMIO(0x2538c), 0x000000c0 },
{ _MMIO(0x25384), 0xaa550000 },
{ _MMIO(0x25404), 0xffffc000 },
{ _MMIO(0x26804), 0x50000000 },
{ _MMIO(0x26984), 0x50000000 },
{ _MMIO(0x26b04), 0x50000000 },
{ _MMIO(0x26b80), 0x00000084 },
{ _MMIO(0x26b90), 0x00050800 },
{ _MMIO(0x26b84), 0x00000000 },
{ _MMIO(0x27804), 0x05000000 },
{ _MMIO(0x27984), 0x05000000 },
{ _MMIO(0x27b04), 0x05000000 },
{ _MMIO(0x27b80), 0x00000084 },
{ _MMIO(0x27b90), 0x00000142 },
{ _MMIO(0x27b84), 0x00000000 },
{ _MMIO(0x26104), 0xa0000000 },
{ _MMIO(0x26184), 0xa5000000 },
{ _MMIO(0x25424), 0x00008620 },
{ _MMIO(0x2541c), 0x00000000 },
{ _MMIO(0x25428), 0x0004a54a },
};
static int
get_sampler_balance_mux_config(struct drm_i915_private *dev_priv,
const struct i915_oa_reg **regs,
int *lens)
{
int n = 0;
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
regs[n] = mux_config_sampler_balance;
lens[n] = ARRAY_SIZE(mux_config_sampler_balance);
n++;
return n;
}
int i915_oa_select_metric_set_hsw(struct drm_i915_private *dev_priv)
{
dev_priv->perf.oa.n_mux_configs = 0;
dev_priv->perf.oa.b_counter_regs = NULL;
dev_priv->perf.oa.b_counter_regs_len = 0;
switch (dev_priv->perf.oa.metrics_set) {
case METRIC_SET_ID_RENDER_BASIC:
dev_priv->perf.oa.n_mux_configs =
get_render_basic_mux_config(dev_priv,
dev_priv->perf.oa.mux_regs,
dev_priv->perf.oa.mux_regs_lens);
if (dev_priv->perf.oa.n_mux_configs == 0) {
DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_BASIC\" metric set\n");
/* EINVAL because *_register_sysfs already checked this
* and so it wouldn't have been advertised to userspace and
* so shouldn't have been requested
*/
return -EINVAL;
}
dev_priv->perf.oa.b_counter_regs =
b_counter_config_render_basic;
dev_priv->perf.oa.b_counter_regs_len =
ARRAY_SIZE(b_counter_config_render_basic);
dev_priv->perf.oa.flex_regs =
flex_eu_config_render_basic;
dev_priv->perf.oa.flex_regs_len =
ARRAY_SIZE(flex_eu_config_render_basic);
return 0;
case METRIC_SET_ID_COMPUTE_BASIC:
dev_priv->perf.oa.n_mux_configs =
get_compute_basic_mux_config(dev_priv,
dev_priv->perf.oa.mux_regs,
dev_priv->perf.oa.mux_regs_lens);
if (dev_priv->perf.oa.n_mux_configs == 0) {
DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_BASIC\" metric set\n");
/* EINVAL because *_register_sysfs already checked this
* and so it wouldn't have been advertised to userspace and
* so shouldn't have been requested
*/
return -EINVAL;
}
dev_priv->perf.oa.b_counter_regs =
b_counter_config_compute_basic;
dev_priv->perf.oa.b_counter_regs_len =
ARRAY_SIZE(b_counter_config_compute_basic);
dev_priv->perf.oa.flex_regs =
flex_eu_config_compute_basic;
dev_priv->perf.oa.flex_regs_len =
ARRAY_SIZE(flex_eu_config_compute_basic);
return 0;
case METRIC_SET_ID_COMPUTE_EXTENDED:
dev_priv->perf.oa.n_mux_configs =
get_compute_extended_mux_config(dev_priv,
dev_priv->perf.oa.mux_regs,
dev_priv->perf.oa.mux_regs_lens);
if (dev_priv->perf.oa.n_mux_configs == 0) {
DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_EXTENDED\" metric set\n");
/* EINVAL because *_register_sysfs already checked this
* and so it wouldn't have been advertised to userspace and
* so shouldn't have been requested
*/
return -EINVAL;
}
dev_priv->perf.oa.b_counter_regs =
b_counter_config_compute_extended;
dev_priv->perf.oa.b_counter_regs_len =
ARRAY_SIZE(b_counter_config_compute_extended);
dev_priv->perf.oa.flex_regs =
flex_eu_config_compute_extended;
dev_priv->perf.oa.flex_regs_len =
ARRAY_SIZE(flex_eu_config_compute_extended);
return 0;
case METRIC_SET_ID_MEMORY_READS:
dev_priv->perf.oa.n_mux_configs =
get_memory_reads_mux_config(dev_priv,
dev_priv->perf.oa.mux_regs,
dev_priv->perf.oa.mux_regs_lens);
if (dev_priv->perf.oa.n_mux_configs == 0) {
DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_READS\" metric set\n");
/* EINVAL because *_register_sysfs already checked this
* and so it wouldn't have been advertised to userspace and
* so shouldn't have been requested
*/
return -EINVAL;
}
dev_priv->perf.oa.b_counter_regs =
b_counter_config_memory_reads;
dev_priv->perf.oa.b_counter_regs_len =
ARRAY_SIZE(b_counter_config_memory_reads);
dev_priv->perf.oa.flex_regs =
flex_eu_config_memory_reads;
dev_priv->perf.oa.flex_regs_len =
ARRAY_SIZE(flex_eu_config_memory_reads);
return 0;
case METRIC_SET_ID_MEMORY_WRITES:
dev_priv->perf.oa.n_mux_configs =
get_memory_writes_mux_config(dev_priv,
dev_priv->perf.oa.mux_regs,
dev_priv->perf.oa.mux_regs_lens);
if (dev_priv->perf.oa.n_mux_configs == 0) {
DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_WRITES\" metric set\n");
/* EINVAL because *_register_sysfs already checked this
* and so it wouldn't have been advertised to userspace and
* so shouldn't have been requested
*/
return -EINVAL;
}
dev_priv->perf.oa.b_counter_regs =
b_counter_config_memory_writes;
dev_priv->perf.oa.b_counter_regs_len =
ARRAY_SIZE(b_counter_config_memory_writes);
dev_priv->perf.oa.flex_regs =
flex_eu_config_memory_writes;
dev_priv->perf.oa.flex_regs_len =
ARRAY_SIZE(flex_eu_config_memory_writes);
return 0;
case METRIC_SET_ID_SAMPLER_BALANCE:
dev_priv->perf.oa.n_mux_configs =
get_sampler_balance_mux_config(dev_priv,
dev_priv->perf.oa.mux_regs,
dev_priv->perf.oa.mux_regs_lens);
if (dev_priv->perf.oa.n_mux_configs == 0) {
DRM_DEBUG_DRIVER("No suitable MUX config for \"SAMPLER_BALANCE\" metric set\n");
/* EINVAL because *_register_sysfs already checked this
* and so it wouldn't have been advertised to userspace and
* so shouldn't have been requested
*/
return -EINVAL;
}
dev_priv->perf.oa.b_counter_regs =
b_counter_config_sampler_balance;
dev_priv->perf.oa.b_counter_regs_len =
ARRAY_SIZE(b_counter_config_sampler_balance);
dev_priv->perf.oa.flex_regs =
flex_eu_config_sampler_balance;
dev_priv->perf.oa.flex_regs_len =
ARRAY_SIZE(flex_eu_config_sampler_balance);
return 0;
default:
return -ENODEV;
}
}
static ssize_t
show_render_basic_id(struct device *kdev, struct device_attribute *attr, char *buf)
{
return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_BASIC);
}
static struct device_attribute dev_attr_render_basic_id = {
.attr = { .name = "id", .mode = 0444 },
.show = show_render_basic_id,
.store = NULL,
};
static struct attribute *attrs_render_basic[] = {
&dev_attr_render_basic_id.attr,
NULL,
};
static struct attribute_group group_render_basic = {
.name = "403d8832-1a27-4aa6-a64e-f5389ce7b212",
.attrs = attrs_render_basic,
};
static ssize_t
show_compute_basic_id(struct device *kdev, struct device_attribute *attr, char *buf)
{
return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_BASIC);
}
static struct device_attribute dev_attr_compute_basic_id = {
.attr = { .name = "id", .mode = 0444 },
.show = show_compute_basic_id,
.store = NULL,
};
static struct attribute *attrs_compute_basic[] = {
&dev_attr_compute_basic_id.attr,
NULL,
};
static struct attribute_group group_compute_basic = {
.name = "39ad14bc-2380-45c4-91eb-fbcb3aa7ae7b",
.attrs = attrs_compute_basic,
};
static ssize_t
show_compute_extended_id(struct device *kdev, struct device_attribute *attr, char *buf)
{
return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_EXTENDED);
}
static struct device_attribute dev_attr_compute_extended_id = {
.attr = { .name = "id", .mode = 0444 },
.show = show_compute_extended_id,
.store = NULL,
};
static struct attribute *attrs_compute_extended[] = {
&dev_attr_compute_extended_id.attr,
NULL,
};
static struct attribute_group group_compute_extended = {
.name = "3865be28-6982-49fe-9494-e4d1b4795413",
.attrs = attrs_compute_extended,
};
static ssize_t
show_memory_reads_id(struct device *kdev, struct device_attribute *attr, char *buf)
{
return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_READS);
}
static struct device_attribute dev_attr_memory_reads_id = {
.attr = { .name = "id", .mode = 0444 },
.show = show_memory_reads_id,
.store = NULL,
};
static struct attribute *attrs_memory_reads[] = {
&dev_attr_memory_reads_id.attr,
NULL,
};
static struct attribute_group group_memory_reads = {
.name = "bb5ed49b-2497-4095-94f6-26ba294db88a",
.attrs = attrs_memory_reads,
};
static ssize_t
show_memory_writes_id(struct device *kdev, struct device_attribute *attr, char *buf)
{
return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_WRITES);
}
static struct device_attribute dev_attr_memory_writes_id = {
.attr = { .name = "id", .mode = 0444 },
.show = show_memory_writes_id,
.store = NULL,
};
static struct attribute *attrs_memory_writes[] = {
&dev_attr_memory_writes_id.attr,
NULL,
};
static struct attribute_group group_memory_writes = {
.name = "3358d639-9b5f-45ab-976d-9b08cbfc6240",
.attrs = attrs_memory_writes,
};
static ssize_t
show_sampler_balance_id(struct device *kdev, struct device_attribute *attr, char *buf)
{
return sprintf(buf, "%d\n", METRIC_SET_ID_SAMPLER_BALANCE);
}
static struct device_attribute dev_attr_sampler_balance_id = {
.attr = { .name = "id", .mode = 0444 },
.show = show_sampler_balance_id,
.store = NULL,
};
static struct attribute *attrs_sampler_balance[] = {
&dev_attr_sampler_balance_id.attr,
NULL,
};
static struct attribute_group group_sampler_balance = {
.name = "bc274488-b4b6-40c7-90da-b77d7ad16189",
.attrs = attrs_sampler_balance,
};
int
i915_perf_register_sysfs_hsw(struct drm_i915_private *dev_priv)
{
const struct i915_oa_reg *mux_regs[ARRAY_SIZE(dev_priv->perf.oa.mux_regs)];
int mux_lens[ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens)];
int ret = 0;
if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_render_basic);
if (ret)
goto error_render_basic;
}
if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
if (ret)
goto error_compute_basic;
}
if (get_compute_extended_mux_config(dev_priv, mux_regs, mux_lens)) {
ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_extended);
if (ret)
goto error_compute_extended;
}
if (get_memory_reads_mux_config(dev_priv, mux_regs, mux_lens)) {
ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_memory_reads);
if (ret)
goto error_memory_reads;
}
if (get_memory_writes_mux_config(dev_priv, mux_regs, mux_lens)) {
ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_memory_writes);
if (ret)
goto error_memory_writes;
}
if (get_sampler_balance_mux_config(dev_priv, mux_regs, mux_lens)) {
ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_sampler_balance);
if (ret)
goto error_sampler_balance;
}
return 0;
error_sampler_balance:
if (get_memory_writes_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_writes);
error_memory_writes:
if (get_memory_reads_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_reads);
error_memory_reads:
if (get_compute_extended_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_extended);
error_compute_extended:
if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
error_compute_basic:
if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_basic);
error_render_basic:
return ret;
return sprintf(buf, "1\n");
}
void
i915_perf_unregister_sysfs_hsw(struct drm_i915_private *dev_priv)
i915_perf_load_test_config_hsw(struct drm_i915_private *dev_priv)
{
const struct i915_oa_reg *mux_regs[ARRAY_SIZE(dev_priv->perf.oa.mux_regs)];
int mux_lens[ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens)];
strncpy(dev_priv->perf.oa.test_config.uuid,
"403d8832-1a27-4aa6-a64e-f5389ce7b212",
UUID_STRING_LEN);
dev_priv->perf.oa.test_config.id = 1;
if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_basic);
if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
if (get_compute_extended_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_extended);
if (get_memory_reads_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_reads);
if (get_memory_writes_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_writes);
if (get_sampler_balance_mux_config(dev_priv, mux_regs, mux_lens))
sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_sampler_balance);
dev_priv->perf.oa.test_config.mux_regs = mux_config_render_basic;
dev_priv->perf.oa.test_config.mux_regs_len = ARRAY_SIZE(mux_config_render_basic);
dev_priv->perf.oa.test_config.b_counter_regs = b_counter_config_render_basic;
dev_priv->perf.oa.test_config.b_counter_regs_len = ARRAY_SIZE(b_counter_config_render_basic);
dev_priv->perf.oa.test_config.flex_regs = flex_eu_config_render_basic;
dev_priv->perf.oa.test_config.flex_regs_len = ARRAY_SIZE(flex_eu_config_render_basic);
dev_priv->perf.oa.test_config.sysfs_metric.name = "403d8832-1a27-4aa6-a64e-f5389ce7b212";
dev_priv->perf.oa.test_config.sysfs_metric.attrs = dev_priv->perf.oa.test_config.attrs;
dev_priv->perf.oa.test_config.attrs[0] = &dev_priv->perf.oa.test_config.sysfs_metric_id.attr;
dev_priv->perf.oa.test_config.sysfs_metric_id.attr.name = "id";
dev_priv->perf.oa.test_config.sysfs_metric_id.attr.mode = 0444;
dev_priv->perf.oa.test_config.sysfs_metric_id.show = show_render_basic_id;
}

8
drivers/gpu/drm/i915/i915_oa_hsw.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_HSW_H__
#define __I915_OA_HSW_H__
extern int i915_oa_n_builtin_metric_sets_hsw;
extern int i915_oa_select_metric_set_hsw(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_hsw(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_hsw(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_hsw(struct drm_i915_private *dev_priv);
#endif

2930
drivers/gpu/drm/i915/i915_oa_kblgt2.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_kblgt2.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_KBLGT2_H__
#define __I915_OA_KBLGT2_H__
extern int i915_oa_n_builtin_metric_sets_kblgt2;
extern int i915_oa_select_metric_set_kblgt2(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_kblgt2(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_kblgt2(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_kblgt2(struct drm_i915_private *dev_priv);
#endif

2979
drivers/gpu/drm/i915/i915_oa_kblgt3.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_kblgt3.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_KBLGT3_H__
#define __I915_OA_KBLGT3_H__
extern int i915_oa_n_builtin_metric_sets_kblgt3;
extern int i915_oa_select_metric_set_kblgt3(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_kblgt3(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_kblgt3(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_kblgt3(struct drm_i915_private *dev_priv);
#endif

3419
drivers/gpu/drm/i915/i915_oa_sklgt2.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_sklgt2.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_SKLGT2_H__
#define __I915_OA_SKLGT2_H__
extern int i915_oa_n_builtin_metric_sets_sklgt2;
extern int i915_oa_select_metric_set_sklgt2(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_sklgt2(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_sklgt2(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_sklgt2(struct drm_i915_private *dev_priv);
#endif

2978
drivers/gpu/drm/i915/i915_oa_sklgt3.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_sklgt3.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_SKLGT3_H__
#define __I915_OA_SKLGT3_H__
extern int i915_oa_n_builtin_metric_sets_sklgt3;
extern int i915_oa_select_metric_set_sklgt3(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_sklgt3(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_sklgt3(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_sklgt3(struct drm_i915_private *dev_priv);
#endif

3032
drivers/gpu/drm/i915/i915_oa_sklgt4.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_oa_sklgt4.h
View File

@ -29,12 +29,6 @@
#ifndef __I915_OA_SKLGT4_H__
#define __I915_OA_SKLGT4_H__
extern int i915_oa_n_builtin_metric_sets_sklgt4;
extern int i915_oa_select_metric_set_sklgt4(struct drm_i915_private *dev_priv);
extern int i915_perf_register_sysfs_sklgt4(struct drm_i915_private *dev_priv);
extern void i915_perf_unregister_sysfs_sklgt4(struct drm_i915_private *dev_priv);
extern void i915_perf_load_test_config_sklgt4(struct drm_i915_private *dev_priv);
#endif

16
drivers/gpu/drm/i915/i915_params.c
View File

@ -63,9 +63,8 @@ struct i915_params i915 __read_mostly = {
.huc_firmware_path = NULL,
.enable_dp_mst = true,
.inject_load_failure = 0,
.enable_dpcd_backlight = -1,
.enable_dpcd_backlight = false,
.enable_gvt = false,
.enable_dbc = true,
};
module_param_named(modeset, i915.modeset, int, 0400);
@ -119,6 +118,10 @@ MODULE_PARM_DESC(vbt_sdvo_panel_type,
module_param_named_unsafe(reset, i915.reset, int, 0600);
MODULE_PARM_DESC(reset, "Attempt GPU resets (0=disabled, 1=full gpu reset, 2=engine reset [default])");
module_param_named_unsafe(vbt_firmware, i915.vbt_firmware, charp, 0400);
MODULE_PARM_DESC(vbt_firmware,
"Load VBT from specified file under /lib/firmware");
#if IS_ENABLED(CONFIG_DRM_I915_CAPTURE_ERROR)
module_param_named(error_capture, i915.error_capture, bool, 0600);
MODULE_PARM_DESC(error_capture,
@ -247,15 +250,10 @@ MODULE_PARM_DESC(enable_dp_mst,
module_param_named_unsafe(inject_load_failure, i915.inject_load_failure, uint, 0400);
MODULE_PARM_DESC(inject_load_failure,
"Force an error after a number of failure check points (0:disabled (default), N:force failure at the Nth failure check point)");
module_param_named_unsafe(enable_dpcd_backlight, i915.enable_dpcd_backlight, int, 0600);
module_param_named(enable_dpcd_backlight, i915.enable_dpcd_backlight, bool, 0600);
MODULE_PARM_DESC(enable_dpcd_backlight,
"Enable support for DPCD backlight control "
"(-1:auto (default), 0:force disable, 1:force enabled if supported");
"Enable support for DPCD backlight control (default:false)");
module_param_named(enable_gvt, i915.enable_gvt, bool, 0400);
MODULE_PARM_DESC(enable_gvt,
"Enable support for Intel GVT-g graphics virtualization host support(default:false)");
module_param_named_unsafe(enable_dbc, i915.enable_dbc, bool, 0600);
MODULE_PARM_DESC(enable_dbc,
"Enable support for dynamic backlight control (default:true)");

6
drivers/gpu/drm/i915/i915_params.h
View File

@ -28,6 +28,7 @@
#include <linux/cache.h> /* for __read_mostly */
#define I915_PARAMS_FOR_EACH(func) \
func(char *, vbt_firmware); \
func(int, modeset); \
func(int, panel_ignore_lid); \
func(int, semaphores); \
@ -53,7 +54,6 @@
func(int, edp_vswing); \
func(int, reset); \
func(unsigned int, inject_load_failure); \
func(int, enable_dpcd_backlight); \
/* leave bools at the end to not create holes */ \
func(bool, alpha_support); \
func(bool, enable_cmd_parser); \
@ -67,8 +67,8 @@
func(bool, verbose_state_checks); \
func(bool, nuclear_pageflip); \
func(bool, enable_dp_mst); \
func(bool, enable_gvt); \
func(bool, enable_dbc)
func(bool, enable_dpcd_backlight); \
func(bool, enable_gvt)
#define MEMBER(T, member) T member
struct i915_params {

1
drivers/gpu/drm/i915/i915_pci.c
View File

@ -398,6 +398,7 @@ static const struct intel_device_info intel_broxton_info = {
GEN9_LP_FEATURES,
.platform = INTEL_BROXTON,
.ddb_size = 512,
.has_reset_engine = false,
};
static const struct intel_device_info intel_geminilake_info = {

803
drivers/gpu/drm/i915/i915_perf.c
View File

File diff suppressed because it is too large Load Diff

8
drivers/gpu/drm/i915/i915_pvinfo.h
View File

@ -49,12 +49,18 @@ enum vgt_g2v_type {
VGT_G2V_MAX,
};
/*
* VGT capabilities type
*/
#define VGT_CAPS_FULL_48BIT_PPGTT BIT(2)
struct vgt_if {
u64 magic; /* VGT_MAGIC */
u16 version_major;
u16 version_minor;
u32 vgt_id; /* ID of vGT instance */
u32 rsv1[12]; /* pad to offset 0x40 */
u32 vgt_caps; /* VGT capabilities */
u32 rsv1[11]; /* pad to offset 0x40 */
/*
* Data structure to describe the balooning info of resources.
* Each VM can only have one portion of continuous area for now.

511
drivers/gpu/drm/i915/i915_reg.h
View File

@ -25,6 +25,97 @@
#ifndef _I915_REG_H_
#define _I915_REG_H_
/**
* DOC: The i915 register macro definition style guide
*
* Follow the style described here for new macros, and while changing existing
* macros. Do **not** mass change existing definitions just to update the style.
*
* Layout
* ''''''
*
* Keep helper macros near the top. For example, _PIPE() and friends.
*
* Prefix macros that generally should not be used outside of this file with
* underscore '_'. For example, _PIPE() and friends, single instances of
* registers that are defined solely for the use by function-like macros.
*
* Avoid using the underscore prefixed macros outside of this file. There are
* exceptions, but keep them to a minimum.
*
* There are two basic types of register definitions: Single registers and
* register groups. Register groups are registers which have two or more
* instances, for example one per pipe, port, transcoder, etc. Register groups
* should be defined using function-like macros.
*
* For single registers, define the register offset first, followed by register
* contents.
*
* For register groups, define the register instance offsets first, prefixed
* with underscore, followed by a function-like macro choosing the right
* instance based on the parameter, followed by register contents.
*
* Define the register contents (i.e. bit and bit field macros) from most
* significant to least significant bit. Indent the register content macros
* using two extra spaces between ``#define`` and the macro name.
*
* For bit fields, define a ``_MASK`` and a ``_SHIFT`` macro. Define bit field
* contents so that they are already shifted in place, and can be directly
* OR'd. For convenience, function-like macros may be used to define bit fields,
* but do note that the macros may be needed to read as well as write the
* register contents.
*
* Define bits using ``(1 << N)`` instead of ``BIT(N)``. We may change this in
* the future, but this is the prevailing style. Do **not** add ``_BIT`` suffix
* to the name.
*
* Group the register and its contents together without blank lines, separate
* from other registers and their contents with one blank line.
*
* Indent macro values from macro names using TABs. Align values vertically. Use
* braces in macro values as needed to avoid unintended precedence after macro
* substitution. Use spaces in macro values according to kernel coding
* style. Use lower case in hexadecimal values.
*
* Naming
* ''''''
*
* Try to name registers according to the specs. If the register name changes in
* the specs from platform to another, stick to the original name.
*
* Try to re-use existing register macro definitions. Only add new macros for
* new register offsets, or when the register contents have changed enough to
* warrant a full redefinition.
*
* When a register macro changes for a new platform, prefix the new macro using
* the platform acronym or generation. For example, ``SKL_`` or ``GEN8_``. The
* prefix signifies the start platform/generation using the register.
*
* When a bit (field) macro changes or gets added for a new platform, while
* retaining the existing register macro, add a platform acronym or generation
* suffix to the name. For example, ``_SKL`` or ``_GEN8``.
*
* Examples
* ''''''''
*
* (Note that the values in the example are indented using spaces instead of
* TABs to avoid misalignment in generated documentation. Use TABs in the
* definitions.)::
*
* #define _FOO_A 0xf000
* #define _FOO_B 0xf001
* #define FOO(pipe) _MMIO_PIPE(pipe, _FOO_A, _FOO_B)
* #define FOO_ENABLE (1 << 31)
* #define FOO_MODE_MASK (0xf << 16)
* #define FOO_MODE_SHIFT 16
* #define FOO_MODE_BAR (0 << 16)
* #define FOO_MODE_BAZ (1 << 16)
* #define FOO_MODE_QUX_SNB (2 << 16)
*
* #define BAR _MMIO(0xb000)
* #define GEN8_BAR _MMIO(0xb888)
*/
typedef struct {
uint32_t reg;
} i915_reg_t;
@ -229,6 +320,28 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define GEN8_RPCS_EU_MIN_SHIFT 0
#define GEN8_RPCS_EU_MIN_MASK (0xf << GEN8_RPCS_EU_MIN_SHIFT)
#define WAIT_FOR_RC6_EXIT _MMIO(0x20CC)
/* HSW only */
#define HSW_SELECTIVE_READ_ADDRESSING_SHIFT 2
#define HSW_SELECTIVE_READ_ADDRESSING_MASK (0x3 << HSW_SLECTIVE_READ_ADDRESSING_SHIFT)
#define HSW_SELECTIVE_WRITE_ADDRESS_SHIFT 4
#define HSW_SELECTIVE_WRITE_ADDRESS_MASK (0x7 << HSW_SELECTIVE_WRITE_ADDRESS_SHIFT)
/* HSW+ */
#define HSW_WAIT_FOR_RC6_EXIT_ENABLE (1 << 0)
#define HSW_RCS_CONTEXT_ENABLE (1 << 7)
#define HSW_RCS_INHIBIT (1 << 8)
/* Gen8 */
#define GEN8_SELECTIVE_WRITE_ADDRESS_SHIFT 4
#define GEN8_SELECTIVE_WRITE_ADDRESS_MASK (0x3 << GEN8_SELECTIVE_WRITE_ADDRESS_SHIFT)
#define GEN8_SELECTIVE_WRITE_ADDRESS_SHIFT 4
#define GEN8_SELECTIVE_WRITE_ADDRESS_MASK (0x3 << GEN8_SELECTIVE_WRITE_ADDRESS_SHIFT)
#define GEN8_SELECTIVE_WRITE_ADDRESSING_ENABLE (1 << 6)
#define GEN8_SELECTIVE_READ_SUBSLICE_SELECT_SHIFT 9
#define GEN8_SELECTIVE_READ_SUBSLICE_SELECT_MASK (0x3 << GEN8_SELECTIVE_READ_SUBSLICE_SELECT_SHIFT)
#define GEN8_SELECTIVE_READ_SLICE_SELECT_SHIFT 11
#define GEN8_SELECTIVE_READ_SLICE_SELECT_MASK (0x3 << GEN8_SELECTIVE_READ_SLICE_SELECT_SHIFT)
#define GEN8_SELECTIVE_READ_ADDRESSING_ENABLE (1 << 13)
#define GAM_ECOCHK _MMIO(0x4090)
#define BDW_DISABLE_HDC_INVALIDATION (1<<25)
#define ECOCHK_SNB_BIT (1<<10)
@ -729,119 +842,10 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define EU_PERF_CNTL5 _MMIO(0xe55c)
#define EU_PERF_CNTL6 _MMIO(0xe65c)
#define GDT_CHICKEN_BITS _MMIO(0x9840)
#define GT_NOA_ENABLE 0x00000080
/*
* OA Boolean state
*/
#define OAREPORTTRIG1 _MMIO(0x2740)
#define OAREPORTTRIG1_THRESHOLD_MASK 0xffff
#define OAREPORTTRIG1_EDGE_LEVEL_TRIGER_SELECT_MASK 0xffff0000 /* 0=level */
#define OAREPORTTRIG2 _MMIO(0x2744)
#define OAREPORTTRIG2_INVERT_A_0 (1<<0)
#define OAREPORTTRIG2_INVERT_A_1 (1<<1)
#define OAREPORTTRIG2_INVERT_A_2 (1<<2)
#define OAREPORTTRIG2_INVERT_A_3 (1<<3)
#define OAREPORTTRIG2_INVERT_A_4 (1<<4)
#define OAREPORTTRIG2_INVERT_A_5 (1<<5)
#define OAREPORTTRIG2_INVERT_A_6 (1<<6)
#define OAREPORTTRIG2_INVERT_A_7 (1<<7)
#define OAREPORTTRIG2_INVERT_A_8 (1<<8)
#define OAREPORTTRIG2_INVERT_A_9 (1<<9)
#define OAREPORTTRIG2_INVERT_A_10 (1<<10)
#define OAREPORTTRIG2_INVERT_A_11 (1<<11)
#define OAREPORTTRIG2_INVERT_A_12 (1<<12)
#define OAREPORTTRIG2_INVERT_A_13 (1<<13)
#define OAREPORTTRIG2_INVERT_A_14 (1<<14)
#define OAREPORTTRIG2_INVERT_A_15 (1<<15)
#define OAREPORTTRIG2_INVERT_B_0 (1<<16)
#define OAREPORTTRIG2_INVERT_B_1 (1<<17)
#define OAREPORTTRIG2_INVERT_B_2 (1<<18)
#define OAREPORTTRIG2_INVERT_B_3 (1<<19)
#define OAREPORTTRIG2_INVERT_C_0 (1<<20)
#define OAREPORTTRIG2_INVERT_C_1 (1<<21)
#define OAREPORTTRIG2_INVERT_D_0 (1<<22)
#define OAREPORTTRIG2_THRESHOLD_ENABLE (1<<23)
#define OAREPORTTRIG2_REPORT_TRIGGER_ENABLE (1<<31)
#define OAREPORTTRIG3 _MMIO(0x2748)
#define OAREPORTTRIG3_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG3_NOA_SELECT_8_SHIFT 0
#define OAREPORTTRIG3_NOA_SELECT_9_SHIFT 4
#define OAREPORTTRIG3_NOA_SELECT_10_SHIFT 8
#define OAREPORTTRIG3_NOA_SELECT_11_SHIFT 12
#define OAREPORTTRIG3_NOA_SELECT_12_SHIFT 16
#define OAREPORTTRIG3_NOA_SELECT_13_SHIFT 20
#define OAREPORTTRIG3_NOA_SELECT_14_SHIFT 24
#define OAREPORTTRIG3_NOA_SELECT_15_SHIFT 28
#define OAREPORTTRIG4 _MMIO(0x274c)
#define OAREPORTTRIG4_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG4_NOA_SELECT_0_SHIFT 0
#define OAREPORTTRIG4_NOA_SELECT_1_SHIFT 4
#define OAREPORTTRIG4_NOA_SELECT_2_SHIFT 8
#define OAREPORTTRIG4_NOA_SELECT_3_SHIFT 12
#define OAREPORTTRIG4_NOA_SELECT_4_SHIFT 16
#define OAREPORTTRIG4_NOA_SELECT_5_SHIFT 20
#define OAREPORTTRIG4_NOA_SELECT_6_SHIFT 24
#define OAREPORTTRIG4_NOA_SELECT_7_SHIFT 28
#define OAREPORTTRIG5 _MMIO(0x2750)
#define OAREPORTTRIG5_THRESHOLD_MASK 0xffff
#define OAREPORTTRIG5_EDGE_LEVEL_TRIGER_SELECT_MASK 0xffff0000 /* 0=level */
#define OAREPORTTRIG6 _MMIO(0x2754)
#define OAREPORTTRIG6_INVERT_A_0 (1<<0)
#define OAREPORTTRIG6_INVERT_A_1 (1<<1)
#define OAREPORTTRIG6_INVERT_A_2 (1<<2)
#define OAREPORTTRIG6_INVERT_A_3 (1<<3)
#define OAREPORTTRIG6_INVERT_A_4 (1<<4)
#define OAREPORTTRIG6_INVERT_A_5 (1<<5)
#define OAREPORTTRIG6_INVERT_A_6 (1<<6)
#define OAREPORTTRIG6_INVERT_A_7 (1<<7)
#define OAREPORTTRIG6_INVERT_A_8 (1<<8)
#define OAREPORTTRIG6_INVERT_A_9 (1<<9)
#define OAREPORTTRIG6_INVERT_A_10 (1<<10)
#define OAREPORTTRIG6_INVERT_A_11 (1<<11)
#define OAREPORTTRIG6_INVERT_A_12 (1<<12)
#define OAREPORTTRIG6_INVERT_A_13 (1<<13)
#define OAREPORTTRIG6_INVERT_A_14 (1<<14)
#define OAREPORTTRIG6_INVERT_A_15 (1<<15)
#define OAREPORTTRIG6_INVERT_B_0 (1<<16)
#define OAREPORTTRIG6_INVERT_B_1 (1<<17)
#define OAREPORTTRIG6_INVERT_B_2 (1<<18)
#define OAREPORTTRIG6_INVERT_B_3 (1<<19)
#define OAREPORTTRIG6_INVERT_C_0 (1<<20)
#define OAREPORTTRIG6_INVERT_C_1 (1<<21)
#define OAREPORTTRIG6_INVERT_D_0 (1<<22)
#define OAREPORTTRIG6_THRESHOLD_ENABLE (1<<23)
#define OAREPORTTRIG6_REPORT_TRIGGER_ENABLE (1<<31)
#define OAREPORTTRIG7 _MMIO(0x2758)
#define OAREPORTTRIG7_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG7_NOA_SELECT_8_SHIFT 0
#define OAREPORTTRIG7_NOA_SELECT_9_SHIFT 4
#define OAREPORTTRIG7_NOA_SELECT_10_SHIFT 8
#define OAREPORTTRIG7_NOA_SELECT_11_SHIFT 12
#define OAREPORTTRIG7_NOA_SELECT_12_SHIFT 16
#define OAREPORTTRIG7_NOA_SELECT_13_SHIFT 20
#define OAREPORTTRIG7_NOA_SELECT_14_SHIFT 24
#define OAREPORTTRIG7_NOA_SELECT_15_SHIFT 28
#define OAREPORTTRIG8 _MMIO(0x275c)
#define OAREPORTTRIG8_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG8_NOA_SELECT_0_SHIFT 0
#define OAREPORTTRIG8_NOA_SELECT_1_SHIFT 4
#define OAREPORTTRIG8_NOA_SELECT_2_SHIFT 8
#define OAREPORTTRIG8_NOA_SELECT_3_SHIFT 12
#define OAREPORTTRIG8_NOA_SELECT_4_SHIFT 16
#define OAREPORTTRIG8_NOA_SELECT_5_SHIFT 20
#define OAREPORTTRIG8_NOA_SELECT_6_SHIFT 24
#define OAREPORTTRIG8_NOA_SELECT_7_SHIFT 28
#define OASTARTTRIG1 _MMIO(0x2710)
#define OASTARTTRIG1_THRESHOLD_COUNT_MASK_MBZ 0xffff0000
#define OASTARTTRIG1_THRESHOLD_MASK 0xffff
@ -956,6 +960,112 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define OASTARTTRIG8_NOA_SELECT_6_SHIFT 24
#define OASTARTTRIG8_NOA_SELECT_7_SHIFT 28
#define OAREPORTTRIG1 _MMIO(0x2740)
#define OAREPORTTRIG1_THRESHOLD_MASK 0xffff
#define OAREPORTTRIG1_EDGE_LEVEL_TRIGER_SELECT_MASK 0xffff0000 /* 0=level */
#define OAREPORTTRIG2 _MMIO(0x2744)
#define OAREPORTTRIG2_INVERT_A_0 (1<<0)
#define OAREPORTTRIG2_INVERT_A_1 (1<<1)
#define OAREPORTTRIG2_INVERT_A_2 (1<<2)
#define OAREPORTTRIG2_INVERT_A_3 (1<<3)
#define OAREPORTTRIG2_INVERT_A_4 (1<<4)
#define OAREPORTTRIG2_INVERT_A_5 (1<<5)
#define OAREPORTTRIG2_INVERT_A_6 (1<<6)
#define OAREPORTTRIG2_INVERT_A_7 (1<<7)
#define OAREPORTTRIG2_INVERT_A_8 (1<<8)
#define OAREPORTTRIG2_INVERT_A_9 (1<<9)
#define OAREPORTTRIG2_INVERT_A_10 (1<<10)
#define OAREPORTTRIG2_INVERT_A_11 (1<<11)
#define OAREPORTTRIG2_INVERT_A_12 (1<<12)
#define OAREPORTTRIG2_INVERT_A_13 (1<<13)
#define OAREPORTTRIG2_INVERT_A_14 (1<<14)
#define OAREPORTTRIG2_INVERT_A_15 (1<<15)
#define OAREPORTTRIG2_INVERT_B_0 (1<<16)
#define OAREPORTTRIG2_INVERT_B_1 (1<<17)
#define OAREPORTTRIG2_INVERT_B_2 (1<<18)
#define OAREPORTTRIG2_INVERT_B_3 (1<<19)
#define OAREPORTTRIG2_INVERT_C_0 (1<<20)
#define OAREPORTTRIG2_INVERT_C_1 (1<<21)
#define OAREPORTTRIG2_INVERT_D_0 (1<<22)
#define OAREPORTTRIG2_THRESHOLD_ENABLE (1<<23)
#define OAREPORTTRIG2_REPORT_TRIGGER_ENABLE (1<<31)
#define OAREPORTTRIG3 _MMIO(0x2748)
#define OAREPORTTRIG3_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG3_NOA_SELECT_8_SHIFT 0
#define OAREPORTTRIG3_NOA_SELECT_9_SHIFT 4
#define OAREPORTTRIG3_NOA_SELECT_10_SHIFT 8
#define OAREPORTTRIG3_NOA_SELECT_11_SHIFT 12
#define OAREPORTTRIG3_NOA_SELECT_12_SHIFT 16
#define OAREPORTTRIG3_NOA_SELECT_13_SHIFT 20
#define OAREPORTTRIG3_NOA_SELECT_14_SHIFT 24
#define OAREPORTTRIG3_NOA_SELECT_15_SHIFT 28
#define OAREPORTTRIG4 _MMIO(0x274c)
#define OAREPORTTRIG4_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG4_NOA_SELECT_0_SHIFT 0
#define OAREPORTTRIG4_NOA_SELECT_1_SHIFT 4
#define OAREPORTTRIG4_NOA_SELECT_2_SHIFT 8
#define OAREPORTTRIG4_NOA_SELECT_3_SHIFT 12
#define OAREPORTTRIG4_NOA_SELECT_4_SHIFT 16
#define OAREPORTTRIG4_NOA_SELECT_5_SHIFT 20
#define OAREPORTTRIG4_NOA_SELECT_6_SHIFT 24
#define OAREPORTTRIG4_NOA_SELECT_7_SHIFT 28
#define OAREPORTTRIG5 _MMIO(0x2750)
#define OAREPORTTRIG5_THRESHOLD_MASK 0xffff
#define OAREPORTTRIG5_EDGE_LEVEL_TRIGER_SELECT_MASK 0xffff0000 /* 0=level */
#define OAREPORTTRIG6 _MMIO(0x2754)
#define OAREPORTTRIG6_INVERT_A_0 (1<<0)
#define OAREPORTTRIG6_INVERT_A_1 (1<<1)
#define OAREPORTTRIG6_INVERT_A_2 (1<<2)
#define OAREPORTTRIG6_INVERT_A_3 (1<<3)
#define OAREPORTTRIG6_INVERT_A_4 (1<<4)
#define OAREPORTTRIG6_INVERT_A_5 (1<<5)
#define OAREPORTTRIG6_INVERT_A_6 (1<<6)
#define OAREPORTTRIG6_INVERT_A_7 (1<<7)
#define OAREPORTTRIG6_INVERT_A_8 (1<<8)
#define OAREPORTTRIG6_INVERT_A_9 (1<<9)
#define OAREPORTTRIG6_INVERT_A_10 (1<<10)
#define OAREPORTTRIG6_INVERT_A_11 (1<<11)
#define OAREPORTTRIG6_INVERT_A_12 (1<<12)
#define OAREPORTTRIG6_INVERT_A_13 (1<<13)
#define OAREPORTTRIG6_INVERT_A_14 (1<<14)
#define OAREPORTTRIG6_INVERT_A_15 (1<<15)
#define OAREPORTTRIG6_INVERT_B_0 (1<<16)
#define OAREPORTTRIG6_INVERT_B_1 (1<<17)
#define OAREPORTTRIG6_INVERT_B_2 (1<<18)
#define OAREPORTTRIG6_INVERT_B_3 (1<<19)
#define OAREPORTTRIG6_INVERT_C_0 (1<<20)
#define OAREPORTTRIG6_INVERT_C_1 (1<<21)
#define OAREPORTTRIG6_INVERT_D_0 (1<<22)
#define OAREPORTTRIG6_THRESHOLD_ENABLE (1<<23)
#define OAREPORTTRIG6_REPORT_TRIGGER_ENABLE (1<<31)
#define OAREPORTTRIG7 _MMIO(0x2758)
#define OAREPORTTRIG7_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG7_NOA_SELECT_8_SHIFT 0
#define OAREPORTTRIG7_NOA_SELECT_9_SHIFT 4
#define OAREPORTTRIG7_NOA_SELECT_10_SHIFT 8
#define OAREPORTTRIG7_NOA_SELECT_11_SHIFT 12
#define OAREPORTTRIG7_NOA_SELECT_12_SHIFT 16
#define OAREPORTTRIG7_NOA_SELECT_13_SHIFT 20
#define OAREPORTTRIG7_NOA_SELECT_14_SHIFT 24
#define OAREPORTTRIG7_NOA_SELECT_15_SHIFT 28
#define OAREPORTTRIG8 _MMIO(0x275c)
#define OAREPORTTRIG8_NOA_SELECT_MASK 0xf
#define OAREPORTTRIG8_NOA_SELECT_0_SHIFT 0
#define OAREPORTTRIG8_NOA_SELECT_1_SHIFT 4
#define OAREPORTTRIG8_NOA_SELECT_2_SHIFT 8
#define OAREPORTTRIG8_NOA_SELECT_3_SHIFT 12
#define OAREPORTTRIG8_NOA_SELECT_4_SHIFT 16
#define OAREPORTTRIG8_NOA_SELECT_5_SHIFT 20
#define OAREPORTTRIG8_NOA_SELECT_6_SHIFT 24
#define OAREPORTTRIG8_NOA_SELECT_7_SHIFT 28
/* CECX_0 */
#define OACEC_COMPARE_LESS_OR_EQUAL 6
#define OACEC_COMPARE_NOT_EQUAL 5
@ -994,6 +1104,51 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define OACEC7_0 _MMIO(0x27a8)
#define OACEC7_1 _MMIO(0x27ac)
/* OA perf counters */
#define OA_PERFCNT1_LO _MMIO(0x91B8)
#define OA_PERFCNT1_HI _MMIO(0x91BC)
#define OA_PERFCNT2_LO _MMIO(0x91C0)
#define OA_PERFCNT2_HI _MMIO(0x91C4)
#define OA_PERFMATRIX_LO _MMIO(0x91C8)
#define OA_PERFMATRIX_HI _MMIO(0x91CC)
/* RPM unit config (Gen8+) */
#define RPM_CONFIG0 _MMIO(0x0D00)
#define RPM_CONFIG1 _MMIO(0x0D04)
/* RPC unit config (Gen8+) */
#define RPM_CONFIG _MMIO(0x0D08)
/* NOA (Gen8+) */
#define NOA_CONFIG(i) _MMIO(0x0D0C + (i) * 4)
#define MICRO_BP0_0 _MMIO(0x9800)
#define MICRO_BP0_2 _MMIO(0x9804)
#define MICRO_BP0_1 _MMIO(0x9808)
#define MICRO_BP1_0 _MMIO(0x980C)
#define MICRO_BP1_2 _MMIO(0x9810)
#define MICRO_BP1_1 _MMIO(0x9814)
#define MICRO_BP2_0 _MMIO(0x9818)
#define MICRO_BP2_2 _MMIO(0x981C)
#define MICRO_BP2_1 _MMIO(0x9820)
#define MICRO_BP3_0 _MMIO(0x9824)
#define MICRO_BP3_2 _MMIO(0x9828)
#define MICRO_BP3_1 _MMIO(0x982C)
#define MICRO_BP_TRIGGER _MMIO(0x9830)
#define MICRO_BP3_COUNT_STATUS01 _MMIO(0x9834)
#define MICRO_BP3_COUNT_STATUS23 _MMIO(0x9838)
#define MICRO_BP_FIRED_ARMED _MMIO(0x983C)
#define GDT_CHICKEN_BITS _MMIO(0x9840)
#define GT_NOA_ENABLE 0x00000080
#define NOA_DATA _MMIO(0x986C)
#define NOA_WRITE _MMIO(0x9888)
#define _GEN7_PIPEA_DE_LOAD_SL 0x70068
#define _GEN7_PIPEB_DE_LOAD_SL 0x71068
@ -1063,9 +1218,26 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
#define DP_SSS_RESET(pipe) _DP_SSS(0x2, (pipe))
#define DP_SSS_PWR_GATE(pipe) _DP_SSS(0x3, (pipe))
/* See the PUNIT HAS v0.8 for the below bits */
enum punit_power_well {
/* These numbers are fixed and must match the position of the pw bits */
/*
* i915_power_well_id:
*
* Platform specific IDs used to look up power wells and - except for custom
* power wells - to define request/status register flag bit positions. As such
* the set of IDs on a given platform must be unique and except for custom
* power wells their value must stay fixed.
*/
enum i915_power_well_id {
/*
* I830
* - custom power well
*/
I830_DISP_PW_PIPES = 0,
/*
* VLV/CHV
* - PUNIT_REG_PWRGT_CTRL (bit: id*2),
* PUNIT_REG_PWRGT_STATUS (bit: id*2) (PUNIT HAS v0.8)
*/
PUNIT_POWER_WELL_RENDER = 0,
PUNIT_POWER_WELL_MEDIA = 1,
PUNIT_POWER_WELL_DISP2D = 3,
@ -1077,14 +1249,20 @@ enum punit_power_well {
PUNIT_POWER_WELL_DPIO_RX0 = 10,
PUNIT_POWER_WELL_DPIO_RX1 = 11,
PUNIT_POWER_WELL_DPIO_CMN_D = 12,
/* - custom power well */
CHV_DISP_PW_PIPE_A, /* 13 */
/* Not actual bit groups. Used as IDs for lookup_power_well() */
PUNIT_POWER_WELL_ALWAYS_ON,
};
/*
* HSW/BDW
* - HSW_PWR_WELL_CTL_DRIVER(0) (status bit: id*2, req bit: id*2+1)
*/
HSW_DISP_PW_GLOBAL = 15,
enum skl_disp_power_wells {
/* These numbers are fixed and must match the position of the pw bits */
SKL_DISP_PW_MISC_IO,
/*
* GEN9+
* - HSW_PWR_WELL_CTL_DRIVER(0) (status bit: id*2, req bit: id*2+1)
*/
SKL_DISP_PW_MISC_IO = 0,
SKL_DISP_PW_DDI_A_E,
GLK_DISP_PW_DDI_A = SKL_DISP_PW_DDI_A_E,
CNL_DISP_PW_DDI_A = SKL_DISP_PW_DDI_A_E,
@ -1103,17 +1281,19 @@ enum skl_disp_power_wells {
SKL_DISP_PW_1 = 14,
SKL_DISP_PW_2,
/* Not actual bit groups. Used as IDs for lookup_power_well() */
SKL_DISP_PW_ALWAYS_ON,
/* - custom power wells */
SKL_DISP_PW_DC_OFF,
BXT_DPIO_CMN_A,
BXT_DPIO_CMN_BC,
GLK_DPIO_CMN_C,
};
GLK_DPIO_CMN_C, /* 19 */
#define SKL_POWER_WELL_STATE(pw) (1 << ((pw) * 2))
#define SKL_POWER_WELL_REQ(pw) (1 << (((pw) * 2) + 1))
/*
* Multiple platforms.
* Must start following the highest ID of any platform.
* - custom power wells
*/
I915_DISP_PW_ALWAYS_ON = 20,
};
#define PUNIT_REG_PWRGT_CTRL 0x60
#define PUNIT_REG_PWRGT_STATUS 0x61
@ -2156,6 +2336,7 @@ enum skl_disp_power_wells {
#define DONE_REG _MMIO(0x40b0)
#define GEN8_PRIVATE_PAT_LO _MMIO(0x40e0)
#define GEN8_PRIVATE_PAT_HI _MMIO(0x40e0 + 4)
#define GEN10_PAT_INDEX(index) _MMIO(0x40e0 + index*4)
#define BSD_HWS_PGA_GEN7 _MMIO(0x04180)
#define BLT_HWS_PGA_GEN7 _MMIO(0x04280)
#define VEBOX_HWS_PGA_GEN7 _MMIO(0x04380)
@ -3783,6 +3964,7 @@ enum {
#define EDP_PSR_CTL _MMIO(dev_priv->psr_mmio_base + 0)
#define EDP_PSR_ENABLE (1<<31)
#define BDW_PSR_SINGLE_FRAME (1<<30)
#define EDP_PSR_RESTORE_PSR_ACTIVE_CTX_MASK (1<<29) /* SW can't modify */
#define EDP_PSR_LINK_STANDBY (1<<27)
#define EDP_PSR_MIN_LINK_ENTRY_TIME_MASK (3<<25)
#define EDP_PSR_MIN_LINK_ENTRY_TIME_8_LINES (0<<25)
@ -5227,6 +5409,9 @@ enum {
#define _PIPE_MISC_A 0x70030
#define _PIPE_MISC_B 0x71030
#define PIPEMISC_YUV420_ENABLE (1<<27)
#define PIPEMISC_YUV420_MODE_FULL_BLEND (1<<26)
#define PIPEMISC_OUTPUT_COLORSPACE_YUV (1<<11)
#define PIPEMISC_DITHER_BPC_MASK (7<<5)
#define PIPEMISC_DITHER_8_BPC (0<<5)
#define PIPEMISC_DITHER_10_BPC (1<<5)
@ -6106,6 +6291,10 @@ enum {
#define _PLANE_KEYMSK_2_A 0x70298
#define _PLANE_KEYMAX_1_A 0x701a0
#define _PLANE_KEYMAX_2_A 0x702a0
#define _PLANE_AUX_DIST_1_A 0x701c0
#define _PLANE_AUX_DIST_2_A 0x702c0
#define _PLANE_AUX_OFFSET_1_A 0x701c4
#define _PLANE_AUX_OFFSET_2_A 0x702c4
#define _PLANE_COLOR_CTL_1_A 0x701CC /* GLK+ */
#define _PLANE_COLOR_CTL_2_A 0x702CC /* GLK+ */
#define _PLANE_COLOR_CTL_3_A 0x703CC /* GLK+ */
@ -6212,6 +6401,24 @@ enum {
#define PLANE_NV12_BUF_CFG(pipe, plane) \
_MMIO_PLANE(plane, _PLANE_NV12_BUF_CFG_1(pipe), _PLANE_NV12_BUF_CFG_2(pipe))
#define _PLANE_AUX_DIST_1_B 0x711c0
#define _PLANE_AUX_DIST_2_B 0x712c0
#define _PLANE_AUX_DIST_1(pipe) \
_PIPE(pipe, _PLANE_AUX_DIST_1_A, _PLANE_AUX_DIST_1_B)
#define _PLANE_AUX_DIST_2(pipe) \
_PIPE(pipe, _PLANE_AUX_DIST_2_A, _PLANE_AUX_DIST_2_B)
#define PLANE_AUX_DIST(pipe, plane) \
_MMIO_PLANE(plane, _PLANE_AUX_DIST_1(pipe), _PLANE_AUX_DIST_2(pipe))
#define _PLANE_AUX_OFFSET_1_B 0x711c4
#define _PLANE_AUX_OFFSET_2_B 0x712c4
#define _PLANE_AUX_OFFSET_1(pipe) \
_PIPE(pipe, _PLANE_AUX_OFFSET_1_A, _PLANE_AUX_OFFSET_1_B)
#define _PLANE_AUX_OFFSET_2(pipe) \
_PIPE(pipe, _PLANE_AUX_OFFSET_2_A, _PLANE_AUX_OFFSET_2_B)
#define PLANE_AUX_OFFSET(pipe, plane) \
_MMIO_PLANE(plane, _PLANE_AUX_OFFSET_1(pipe), _PLANE_AUX_OFFSET_2(pipe))
#define _PLANE_COLOR_CTL_1_B 0x711CC
#define _PLANE_COLOR_CTL_2_B 0x712CC
#define _PLANE_COLOR_CTL_3_B 0x713CC
@ -6695,6 +6902,7 @@ enum {
# define CHICKEN3_DGMG_DONE_FIX_DISABLE (1 << 2)
#define CHICKEN_PAR1_1 _MMIO(0x42080)
#define SKL_RC_HASH_OUTSIDE (1 << 15)
#define DPA_MASK_VBLANK_SRD (1 << 15)
#define FORCE_ARB_IDLE_PLANES (1 << 14)
#define SKL_EDP_PSR_FIX_RDWRAP (1 << 3)
@ -6703,12 +6911,10 @@ enum {
#define KVM_CONFIG_CHANGE_NOTIFICATION_SELECT (1 << 14)
#define CHICKEN_MISC_2 _MMIO(0x42084)
#define GLK_CL0_PWR_DOWN (1 << 10)
#define GLK_CL1_PWR_DOWN (1 << 11)
#define CNL_COMP_PWR_DOWN (1 << 23)
#define GLK_CL2_PWR_DOWN (1 << 12)
#define CHICKEN_MISC_2 _MMIO(0x42084)
#define COMP_PWR_DOWN (1 << 23)
#define GLK_CL1_PWR_DOWN (1 << 11)
#define GLK_CL0_PWR_DOWN (1 << 10)
#define _CHICKEN_PIPESL_1_A 0x420b0
#define _CHICKEN_PIPESL_1_B 0x420b4
@ -7984,12 +8190,31 @@ enum {
#define SKL_AUD_CODEC_WAKE_SIGNAL (1 << 15)
/* HSW Power Wells */
#define HSW_PWR_WELL_BIOS _MMIO(0x45400) /* CTL1 */
#define HSW_PWR_WELL_DRIVER _MMIO(0x45404) /* CTL2 */
#define HSW_PWR_WELL_KVMR _MMIO(0x45408) /* CTL3 */
#define HSW_PWR_WELL_DEBUG _MMIO(0x4540C) /* CTL4 */
#define HSW_PWR_WELL_ENABLE_REQUEST (1<<31)
#define HSW_PWR_WELL_STATE_ENABLED (1<<30)
#define _HSW_PWR_WELL_CTL1 0x45400
#define _HSW_PWR_WELL_CTL2 0x45404
#define _HSW_PWR_WELL_CTL3 0x45408
#define _HSW_PWR_WELL_CTL4 0x4540C
/*
* Each power well control register contains up to 16 (request, status) HW
* flag tuples. The register index and HW flag shift is determined by the
* power well ID (see i915_power_well_id). There are 4 possible sources of
* power well requests each source having its own set of control registers:
* BIOS, DRIVER, KVMR, DEBUG.
*/
#define _HSW_PW_REG_IDX(pw) ((pw) >> 4)
#define _HSW_PW_SHIFT(pw) (((pw) & 0xf) * 2)
/* TODO: Add all PWR_WELL_CTL registers below for new platforms */
#define HSW_PWR_WELL_CTL_BIOS(pw) _MMIO(_PICK(_HSW_PW_REG_IDX(pw), \
_HSW_PWR_WELL_CTL1))
#define HSW_PWR_WELL_CTL_DRIVER(pw) _MMIO(_PICK(_HSW_PW_REG_IDX(pw), \
_HSW_PWR_WELL_CTL2))
#define HSW_PWR_WELL_CTL_KVMR _MMIO(_HSW_PWR_WELL_CTL3)
#define HSW_PWR_WELL_CTL_DEBUG(pw) _MMIO(_PICK(_HSW_PW_REG_IDX(pw), \
_HSW_PWR_WELL_CTL4))
#define HSW_PWR_WELL_CTL_REQ(pw) (1 << (_HSW_PW_SHIFT(pw) + 1))
#define HSW_PWR_WELL_CTL_STATE(pw) (1 << _HSW_PW_SHIFT(pw))
#define HSW_PWR_WELL_CTL5 _MMIO(0x45410)
#define HSW_PWR_WELL_ENABLE_SINGLE_STEP (1<<31)
#define HSW_PWR_WELL_PWR_GATE_OVERRIDE (1<<20)
@ -7997,11 +8222,17 @@ enum {
#define HSW_PWR_WELL_CTL6 _MMIO(0x45414)
/* SKL Fuse Status */
enum skl_power_gate {
SKL_PG0,
SKL_PG1,
SKL_PG2,
};
#define SKL_FUSE_STATUS _MMIO(0x42000)
#define SKL_FUSE_DOWNLOAD_STATUS (1<<31)
#define SKL_FUSE_PG0_DIST_STATUS (1<<27)
#define SKL_FUSE_PG1_DIST_STATUS (1<<26)
#define SKL_FUSE_PG2_DIST_STATUS (1<<25)
#define SKL_FUSE_DOWNLOAD_STATUS (1<<31)
/* PG0 (HW control->no power well ID), PG1..PG2 (SKL_DISP_PW1..SKL_DISP_PW2) */
#define SKL_PW_TO_PG(pw) ((pw) - SKL_DISP_PW_1 + SKL_PG1)
#define SKL_FUSE_PG_DIST_STATUS(pg) (1 << (27 - (pg)))
/* Per-pipe DDI Function Control */
#define _TRANS_DDI_FUNC_CTL_A 0x60400

2
drivers/gpu/drm/i915/i915_selftest.h
View File

@ -101,6 +101,4 @@ bool __igt_timeout(unsigned long timeout, const char *fmt, ...);
#define igt_timeout(t, fmt, ...) \
__igt_timeout((t), KERN_WARNING pr_fmt(fmt), ##__VA_ARGS__)
#define igt_can_mi_store_dword_imm(D) (INTEL_GEN(D) > 2)
#endif /* !__I915_SELFTEST_H__ */

6
drivers/gpu/drm/i915/i915_sysfs.c
View File

@ -220,7 +220,7 @@ out:
return ret;
}
static struct bin_attribute dpf_attrs = {
static const struct bin_attribute dpf_attrs = {
.attr = {.name = "l3_parity", .mode = (S_IRUSR | S_IWUSR)},
.size = GEN7_L3LOG_SIZE,
.read = i915_l3_read,
@ -229,7 +229,7 @@ static struct bin_attribute dpf_attrs = {
.private = (void *)0
};
static struct bin_attribute dpf_attrs_1 = {
static const struct bin_attribute dpf_attrs_1 = {
.attr = {.name = "l3_parity_slice_1", .mode = (S_IRUSR | S_IWUSR)},
.size = GEN7_L3LOG_SIZE,
.read = i915_l3_read,
@ -532,7 +532,7 @@ static ssize_t error_state_write(struct file *file, struct kobject *kobj,
return count;
}
static struct bin_attribute error_state_attr = {
static const struct bin_attribute error_state_attr = {
.attr.name = "error",
.attr.mode = S_IRUSR | S_IWUSR,
.size = 0,

7
drivers/gpu/drm/i915/i915_vgpu.c
View File

@ -75,10 +75,17 @@ void i915_check_vgpu(struct drm_i915_private *dev_priv)
return;
}
dev_priv->vgpu.caps = __raw_i915_read32(dev_priv, vgtif_reg(vgt_caps));
dev_priv->vgpu.active = true;
DRM_INFO("Virtual GPU for Intel GVT-g detected.\n");
}
bool intel_vgpu_has_full_48bit_ppgtt(struct drm_i915_private *dev_priv)
{
return dev_priv->vgpu.caps & VGT_CAPS_FULL_48BIT_PPGTT;
}
struct _balloon_info_ {
/*
* There are up to 2 regions per mappable/unmappable graphic

3
drivers/gpu/drm/i915/i915_vgpu.h
View File

@ -27,6 +27,9 @@
#include "i915_pvinfo.h"
void i915_check_vgpu(struct drm_i915_private *dev_priv);
bool intel_vgpu_has_full_48bit_ppgtt(struct drm_i915_private *dev_priv);
int intel_vgt_balloon(struct drm_i915_private *dev_priv);
void intel_vgt_deballoon(struct drm_i915_private *dev_priv);

22
drivers/gpu/drm/i915/i915_vma.c
View File

@ -597,33 +597,11 @@ static void i915_vma_destroy(struct i915_vma *vma)
kmem_cache_free(to_i915(vma->obj->base.dev)->vmas, vma);
}
void i915_vma_unlink_ctx(struct i915_vma *vma)
{
struct i915_gem_context *ctx = vma->ctx;
if (ctx->vma_lut.ht_size & I915_CTX_RESIZE_IN_PROGRESS) {
cancel_work_sync(&ctx->vma_lut.resize);
ctx->vma_lut.ht_size &= ~I915_CTX_RESIZE_IN_PROGRESS;
}
__hlist_del(&vma->ctx_node);
ctx->vma_lut.ht_count--;
if (i915_vma_is_ggtt(vma))
vma->obj->vma_hashed = NULL;
vma->ctx = NULL;
i915_vma_put(vma);
}
void i915_vma_close(struct i915_vma *vma)
{
GEM_BUG_ON(i915_vma_is_closed(vma));
vma->flags |= I915_VMA_CLOSED;
if (vma->ctx)
i915_vma_unlink_ctx(vma);
list_del(&vma->obj_link);
rb_erase(&vma->obj_node, &vma->obj->vma_tree);

6
drivers/gpu/drm/i915/i915_vma.h
View File

@ -112,13 +112,9 @@ struct i915_vma {
/**
* Used for performing relocations during execbuffer insertion.
*/
struct drm_i915_gem_exec_object2 *exec_entry;
unsigned int *exec_flags;
struct hlist_node exec_node;
u32 exec_handle;
struct i915_gem_context *ctx;
struct hlist_node ctx_node;
u32 ctx_handle;
};
struct i915_vma *

15
drivers/gpu/drm/i915/intel_bios.c
View File

@ -1120,8 +1120,8 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
bool is_dvi, is_hdmi, is_dp, is_edp, is_crt;
uint8_t aux_channel, ddc_pin;
/* Each DDI port can have more than one value on the "DVO Port" field,
* so look for all the possible values for each port and abort if more
* than one is found. */
* so look for all the possible values for each port.
*/
int dvo_ports[][3] = {
{DVO_PORT_HDMIA, DVO_PORT_DPA, -1},
{DVO_PORT_HDMIB, DVO_PORT_DPB, -1},
@ -1130,7 +1130,10 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
{DVO_PORT_CRT, DVO_PORT_HDMIE, DVO_PORT_DPE},
};
/* Find the child device to use, abort if more than one found. */
/*
* Find the first child device to reference the port, report if more
* than one found.
*/
for (i = 0; i < dev_priv->vbt.child_dev_num; i++) {
it = dev_priv->vbt.child_dev + i;
@ -1140,11 +1143,11 @@ static void parse_ddi_port(struct drm_i915_private *dev_priv, enum port port,
if (it->common.dvo_port == dvo_ports[port][j]) {
if (child) {
DRM_DEBUG_KMS("More than one child device for port %c in VBT.\n",
DRM_DEBUG_KMS("More than one child device for port %c in VBT, using the first.\n",
port_name(port));
return;
} else {
child = it;
}
child = it;
}
}
}

45
drivers/gpu/drm/i915/intel_color.c
View File

@ -41,6 +41,22 @@
#define LEGACY_LUT_LENGTH (sizeof(struct drm_color_lut) * 256)
/* Post offset values for RGB->YCBCR conversion */
#define POSTOFF_RGB_TO_YUV_HI 0x800
#define POSTOFF_RGB_TO_YUV_ME 0x100
#define POSTOFF_RGB_TO_YUV_LO 0x800
/*
* These values are direct register values specified in the Bspec,
* for RGB->YUV conversion matrix (colorspace BT709)
*/
#define CSC_RGB_TO_YUV_RU_GU 0x2ba809d8
#define CSC_RGB_TO_YUV_BU 0x37e80000
#define CSC_RGB_TO_YUV_RY_GY 0x1e089cc0
#define CSC_RGB_TO_YUV_BY 0xb5280000
#define CSC_RGB_TO_YUV_RV_GV 0xbce89ad8
#define CSC_RGB_TO_YUV_BV 0x1e080000
/*
* Extract the CSC coefficient from a CTM coefficient (in U32.32 fixed point
* format). This macro takes the coefficient we want transformed and the
@ -91,6 +107,30 @@ static void ctm_mult_by_limited(uint64_t *result, int64_t *input)
}
}
void i9xx_load_ycbcr_conversion_matrix(struct intel_crtc *intel_crtc)
{
int pipe = intel_crtc->pipe;
struct drm_i915_private *dev_priv = to_i915(intel_crtc->base.dev);
I915_WRITE(PIPE_CSC_PREOFF_HI(pipe), 0);
I915_WRITE(PIPE_CSC_PREOFF_ME(pipe), 0);
I915_WRITE(PIPE_CSC_PREOFF_LO(pipe), 0);
I915_WRITE(PIPE_CSC_COEFF_RU_GU(pipe), CSC_RGB_TO_YUV_RU_GU);
I915_WRITE(PIPE_CSC_COEFF_BU(pipe), CSC_RGB_TO_YUV_BU);
I915_WRITE(PIPE_CSC_COEFF_RY_GY(pipe), CSC_RGB_TO_YUV_RY_GY);
I915_WRITE(PIPE_CSC_COEFF_BY(pipe), CSC_RGB_TO_YUV_BY);
I915_WRITE(PIPE_CSC_COEFF_RV_GV(pipe), CSC_RGB_TO_YUV_RV_GV);
I915_WRITE(PIPE_CSC_COEFF_BV(pipe), CSC_RGB_TO_YUV_BV);
I915_WRITE(PIPE_CSC_POSTOFF_HI(pipe), POSTOFF_RGB_TO_YUV_HI);
I915_WRITE(PIPE_CSC_POSTOFF_ME(pipe), POSTOFF_RGB_TO_YUV_ME);
I915_WRITE(PIPE_CSC_POSTOFF_LO(pipe), POSTOFF_RGB_TO_YUV_LO);
I915_WRITE(PIPE_CSC_MODE(pipe), 0);
}
/* Set up the pipe CSC unit. */
static void i9xx_load_csc_matrix(struct drm_crtc_state *crtc_state)
{
@ -101,7 +141,10 @@ static void i9xx_load_csc_matrix(struct drm_crtc_state *crtc_state)
uint16_t coeffs[9] = { 0, };
struct intel_crtc_state *intel_crtc_state = to_intel_crtc_state(crtc_state);
if (crtc_state->ctm) {
if (intel_crtc_state->ycbcr420) {
i9xx_load_ycbcr_conversion_matrix(intel_crtc);
return;
} else if (crtc_state->ctm) {
struct drm_color_ctm *ctm =
(struct drm_color_ctm *)crtc_state->ctm->data;
uint64_t input[9] = { 0, };

5
drivers/gpu/drm/i915/intel_ddi.c
View File

@ -1174,6 +1174,9 @@ static void ddi_dotclock_get(struct intel_crtc_state *pipe_config)
else
dotclock = pipe_config->port_clock;
if (pipe_config->ycbcr420)
dotclock *= 2;
if (pipe_config->pixel_multiplier)
dotclock /= pipe_config->pixel_multiplier;
@ -1873,7 +1876,7 @@ cnl_get_buf_trans_edp(struct drm_i915_private *dev_priv,
if (dev_priv->vbt.edp.low_vswing) {
if (voltage == VOLTAGE_INFO_0_85V) {
*n_entries = ARRAY_SIZE(cnl_ddi_translations_edp_0_85V);
return cnl_ddi_translations_dp_0_85V;
return cnl_ddi_translations_edp_0_85V;
} else if (voltage == VOLTAGE_INFO_0_95V) {
*n_entries = ARRAY_SIZE(cnl_ddi_translations_edp_0_95V);
return cnl_ddi_translations_edp_0_95V;

1671
drivers/gpu/drm/i915/intel_display.c
View File

File diff suppressed because it is too large Load Diff

108
drivers/gpu/drm/i915/intel_dp.c
View File

@ -97,6 +97,9 @@ static const int bxt_rates[] = { 162000, 216000, 243000, 270000,
324000, 432000, 540000 };
static const int skl_rates[] = { 162000, 216000, 270000,
324000, 432000, 540000 };
static const int cnl_rates[] = { 162000, 216000, 270000,
324000, 432000, 540000,
648000, 810000 };
static const int default_rates[] = { 162000, 270000, 540000 };
/**
@ -229,8 +232,10 @@ intel_dp_set_source_rates(struct intel_dp *intel_dp)
{
struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp);
struct drm_i915_private *dev_priv = to_i915(dig_port->base.base.dev);
enum port port = dig_port->port;
const int *source_rates;
int size;
u32 voltage;
/* This should only be done once */
WARN_ON(intel_dp->source_rates || intel_dp->num_source_rates);
@ -238,6 +243,13 @@ intel_dp_set_source_rates(struct intel_dp *intel_dp)
if (IS_GEN9_LP(dev_priv)) {
source_rates = bxt_rates;
size = ARRAY_SIZE(bxt_rates);
} else if (IS_CANNONLAKE(dev_priv)) {
source_rates = cnl_rates;
size = ARRAY_SIZE(cnl_rates);
voltage = I915_READ(CNL_PORT_COMP_DW3) & VOLTAGE_INFO_MASK;
if (port == PORT_A || port == PORT_D ||
voltage == VOLTAGE_INFO_0_85V)
size -= 2;
} else if (IS_GEN9_BC(dev_priv)) {
source_rates = skl_rates;
size = ARRAY_SIZE(skl_rates);
@ -322,19 +334,20 @@ static int intel_dp_common_len_rate_limit(struct intel_dp *intel_dp,
return 0;
}
static bool intel_dp_link_params_valid(struct intel_dp *intel_dp)
static bool intel_dp_link_params_valid(struct intel_dp *intel_dp, int link_rate,
uint8_t lane_count)
{
/*
* FIXME: we need to synchronize the current link parameters with
* hardware readout. Currently fast link training doesn't work on
* boot-up.
*/
if (intel_dp->link_rate == 0 ||
intel_dp->link_rate > intel_dp->max_link_rate)
if (link_rate == 0 ||
link_rate > intel_dp->max_link_rate)
return false;
if (intel_dp->lane_count == 0 ||
intel_dp->lane_count > intel_dp_max_lane_count(intel_dp))
if (lane_count == 0 ||
lane_count > intel_dp_max_lane_count(intel_dp))
return false;
return true;
@ -1606,6 +1619,23 @@ static int intel_dp_compute_bpp(struct intel_dp *intel_dp,
return bpp;
}
static bool intel_edp_compare_alt_mode(struct drm_display_mode *m1,
struct drm_display_mode *m2)
{
bool bres = false;
if (m1 && m2)
bres = (m1->hdisplay == m2->hdisplay &&
m1->hsync_start == m2->hsync_start &&
m1->hsync_end == m2->hsync_end &&
m1->htotal == m2->htotal &&
m1->vdisplay == m2->vdisplay &&
m1->vsync_start == m2->vsync_start &&
m1->vsync_end == m2->vsync_end &&
m1->vtotal == m2->vtotal);
return bres;
}
bool
intel_dp_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
@ -1652,8 +1682,16 @@ intel_dp_compute_config(struct intel_encoder *encoder,
pipe_config->has_audio = intel_conn_state->force_audio == HDMI_AUDIO_ON;
if (is_edp(intel_dp) && intel_connector->panel.fixed_mode) {
intel_fixed_panel_mode(intel_connector->panel.fixed_mode,
adjusted_mode);
struct drm_display_mode *panel_mode =
intel_connector->panel.alt_fixed_mode;
struct drm_display_mode *req_mode = &pipe_config->base.mode;
if (!intel_edp_compare_alt_mode(req_mode, panel_mode))
panel_mode = intel_connector->panel.fixed_mode;
drm_mode_debug_printmodeline(panel_mode);
intel_fixed_panel_mode(panel_mode, adjusted_mode);
if (INTEL_GEN(dev_priv) >= 9) {
int ret;
@ -1677,12 +1715,18 @@ intel_dp_compute_config(struct intel_encoder *encoder,
if (intel_dp->compliance.test_type == DP_TEST_LINK_TRAINING) {
int index;
index = intel_dp_rate_index(intel_dp->common_rates,
intel_dp->num_common_rates,
intel_dp->compliance.test_link_rate);
if (index >= 0)
min_clock = max_clock = index;
min_lane_count = max_lane_count = intel_dp->compliance.test_lane_count;
/* Validate the compliance test data since max values
* might have changed due to link train fallback.
*/
if (intel_dp_link_params_valid(intel_dp, intel_dp->compliance.test_link_rate,
intel_dp->compliance.test_lane_count)) {
index = intel_dp_rate_index(intel_dp->common_rates,
intel_dp->num_common_rates,
intel_dp->compliance.test_link_rate);
if (index >= 0)
min_clock = max_clock = index;
min_lane_count = max_lane_count = intel_dp->compliance.test_lane_count;
}
}
DRM_DEBUG_KMS("DP link computation with max lane count %i "
"max bw %d pixel clock %iKHz\n",
@ -3963,8 +4007,7 @@ intel_dp_get_sink_irq_esi(struct intel_dp *intel_dp, u8 *sink_irq_vector)
static uint8_t intel_dp_autotest_link_training(struct intel_dp *intel_dp)
{
int status = 0;
int min_lane_count = 1;
int link_rate_index, test_link_rate;
int test_link_rate;
uint8_t test_lane_count, test_link_bw;
/* (DP CTS 1.2)
* 4.3.1.11
@ -3978,10 +4021,6 @@ static uint8_t intel_dp_autotest_link_training(struct intel_dp *intel_dp)
return DP_TEST_NAK;
}
test_lane_count &= DP_MAX_LANE_COUNT_MASK;
/* Validate the requested lane count */
if (test_lane_count < min_lane_count ||
test_lane_count > intel_dp->max_link_lane_count)
return DP_TEST_NAK;
status = drm_dp_dpcd_readb(&intel_dp->aux, DP_TEST_LINK_RATE,
&test_link_bw);
@ -3989,12 +4028,11 @@ static uint8_t intel_dp_autotest_link_training(struct intel_dp *intel_dp)
DRM_DEBUG_KMS("Link Rate read failed\n");
return DP_TEST_NAK;
}
/* Validate the requested link rate */
test_link_rate = drm_dp_bw_code_to_link_rate(test_link_bw);
link_rate_index = intel_dp_rate_index(intel_dp->common_rates,
intel_dp->num_common_rates,
test_link_rate);
if (link_rate_index < 0)
/* Validate the requested link rate and lane count */
if (!intel_dp_link_params_valid(intel_dp, test_link_rate,
test_lane_count))
return DP_TEST_NAK;
intel_dp->compliance.test_lane_count = test_lane_count;
@ -4263,7 +4301,8 @@ intel_dp_check_link_status(struct intel_dp *intel_dp)
* Validate the cached values of intel_dp->link_rate and
* intel_dp->lane_count before attempting to retrain.
*/
if (!intel_dp_link_params_valid(intel_dp))
if (!intel_dp_link_params_valid(intel_dp, intel_dp->link_rate,
intel_dp->lane_count))
return;
/* Retrain if Channel EQ or CR not ok */
@ -4566,7 +4605,7 @@ static bool bxt_digital_port_connected(struct drm_i915_private *dev_priv,
enum port port;
u32 bit;
intel_hpd_pin_to_port(intel_encoder->hpd_pin, &port);
port = intel_hpd_pin_to_port(intel_encoder->hpd_pin);
switch (port) {
case PORT_A:
bit = BXT_DE_PORT_HP_DDIA;
@ -5780,6 +5819,7 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp,
struct drm_device *dev = intel_encoder->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_display_mode *fixed_mode = NULL;
struct drm_display_mode *alt_fixed_mode = NULL;
struct drm_display_mode *downclock_mode = NULL;
bool has_dpcd;
struct drm_display_mode *scan;
@ -5835,13 +5875,14 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp,
}
intel_connector->edid = edid;
/* prefer fixed mode from EDID if available */
/* prefer fixed mode from EDID if available, save an alt mode also */
list_for_each_entry(scan, &connector->probed_modes, head) {
if ((scan->type & DRM_MODE_TYPE_PREFERRED)) {
fixed_mode = drm_mode_duplicate(dev, scan);
downclock_mode = intel_dp_drrs_init(
intel_connector, fixed_mode);
break;
} else if (!alt_fixed_mode) {
alt_fixed_mode = drm_mode_duplicate(dev, scan);
}
}
@ -5878,7 +5919,8 @@ static bool intel_edp_init_connector(struct intel_dp *intel_dp,
pipe_name(pipe));
}
intel_panel_init(&intel_connector->panel, fixed_mode, downclock_mode);
intel_panel_init(&intel_connector->panel, fixed_mode, alt_fixed_mode,
downclock_mode);
intel_connector->panel.backlight.power = intel_edp_backlight_power;
intel_panel_setup_backlight(connector, pipe);
@ -5904,26 +5946,22 @@ intel_dp_init_connector_port_info(struct intel_digital_port *intel_dig_port)
struct intel_encoder *encoder = &intel_dig_port->base;
struct intel_dp *intel_dp = &intel_dig_port->dp;
encoder->hpd_pin = intel_hpd_pin(intel_dig_port->port);
switch (intel_dig_port->port) {
case PORT_A:
encoder->hpd_pin = HPD_PORT_A;
intel_dp->aux_power_domain = POWER_DOMAIN_AUX_A;
break;
case PORT_B:
encoder->hpd_pin = HPD_PORT_B;
intel_dp->aux_power_domain = POWER_DOMAIN_AUX_B;
break;
case PORT_C:
encoder->hpd_pin = HPD_PORT_C;
intel_dp->aux_power_domain = POWER_DOMAIN_AUX_C;
break;
case PORT_D:
encoder->hpd_pin = HPD_PORT_D;
intel_dp->aux_power_domain = POWER_DOMAIN_AUX_D;
break;
case PORT_E:
encoder->hpd_pin = HPD_PORT_E;
/* FIXME: Check VBT for actual wiring of PORT E */
intel_dp->aux_power_domain = POWER_DOMAIN_AUX_D;
break;

87
drivers/gpu/drm/i915/intel_dp_aux_backlight.c
View File

@ -173,24 +173,6 @@ static bool intel_dp_aux_set_pwm_freq(struct intel_connector *connector)
return true;
}
/*
* Set minimum / maximum dynamic brightness percentage. This value is expressed
* as the percentage of normal brightness in 5% increments.
*/
static bool
intel_dp_aux_set_dynamic_backlight_percent(struct intel_dp *intel_dp,
u32 min, u32 max)
{
u8 dbc[] = { DIV_ROUND_CLOSEST(min, 5), DIV_ROUND_CLOSEST(max, 5) };
if (drm_dp_dpcd_write(&intel_dp->aux, DP_EDP_DBC_MINIMUM_BRIGHTNESS_SET,
dbc, sizeof(dbc)) < 0) {
DRM_DEBUG_KMS("Failed to write aux DBC brightness level\n");
return false;
}
return true;
}
static void intel_dp_aux_enable_backlight(const struct intel_crtc_state *crtc_state,
const struct drm_connector_state *conn_state)
{
@ -226,14 +208,6 @@ static void intel_dp_aux_enable_backlight(const struct intel_crtc_state *crtc_st
if (intel_dp_aux_set_pwm_freq(connector))
new_dpcd_buf |= DP_EDP_BACKLIGHT_FREQ_AUX_SET_ENABLE;
if (i915.enable_dbc &&
(intel_dp->edp_dpcd[2] & DP_EDP_DYNAMIC_BACKLIGHT_CAP)) {
if(intel_dp_aux_set_dynamic_backlight_percent(intel_dp, 0, 100)) {
new_dpcd_buf |= DP_EDP_DYNAMIC_BACKLIGHT_ENABLE;
DRM_DEBUG_KMS("Enable dynamic brightness.\n");
}
}
if (new_dpcd_buf != dpcd_buf) {
if (drm_dp_dpcd_writeb(&intel_dp->aux,
DP_EDP_BACKLIGHT_MODE_SET_REGISTER, new_dpcd_buf) < 0) {
@ -277,66 +251,15 @@ intel_dp_aux_display_control_capable(struct intel_connector *connector)
/* Check the eDP Display control capabilities registers to determine if
* the panel can support backlight control over the aux channel
*/
if ((intel_dp->edp_dpcd[1] & DP_EDP_TCON_BACKLIGHT_ADJUSTMENT_CAP) &&
(intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_AUX_SET_CAP)) {
if (intel_dp->edp_dpcd[1] & DP_EDP_TCON_BACKLIGHT_ADJUSTMENT_CAP &&
(intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_AUX_SET_CAP) &&
!(intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_PWM_PIN_CAP)) {
DRM_DEBUG_KMS("AUX Backlight Control Supported!\n");
return true;
}
return false;
}
/*
* Heuristic function whether we should use AUX for backlight adjustment or not.
*
* We should use AUX for backlight brightness adjustment if panel doesn't this
* via PWM pin or using AUX is better than using PWM pin.
*
* The heuristic to determine that using AUX pin is better than using PWM pin is
* that the panel support any of the feature list here.
* - Regional backlight brightness adjustment
* - Backlight PWM frequency set
* - More than 8 bits resolution of brightness level
* - Backlight enablement via AUX and not by BL_ENABLE pin
*
* If all above are not true, assume that using PWM pin is better.
*/
static bool
intel_dp_aux_display_control_heuristic(struct intel_connector *connector)
{
struct intel_dp *intel_dp = enc_to_intel_dp(&connector->encoder->base);
uint8_t reg_val;
/* Panel doesn't support adjusting backlight brightness via PWN pin */
if (!(intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_PWM_PIN_CAP))
return true;
/* Panel supports regional backlight brightness adjustment */
if (drm_dp_dpcd_readb(&intel_dp->aux, DP_EDP_GENERAL_CAP_3,
&reg_val) != 1) {
DRM_DEBUG_KMS("Failed to read DPCD register 0x%x\n",
DP_EDP_GENERAL_CAP_3);
return false;
}
if (reg_val > 0)
return true;
/* Panel supports backlight PWM frequency set */
if (intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_FREQ_AUX_SET_CAP)
return true;
/* Panel supports more than 8 bits resolution of brightness level */
if (intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_BYTE_COUNT)
return true;
/* Panel supports enabling backlight via AUX but not by BL_ENABLE pin */
if ((intel_dp->edp_dpcd[1] & DP_EDP_BACKLIGHT_AUX_ENABLE_CAP) &&
!(intel_dp->edp_dpcd[1] & DP_EDP_BACKLIGHT_PIN_ENABLE_CAP))
return true;
return false;
}
int intel_dp_aux_init_backlight_funcs(struct intel_connector *intel_connector)
{
struct intel_panel *panel = &intel_connector->panel;
@ -347,10 +270,6 @@ int intel_dp_aux_init_backlight_funcs(struct intel_connector *intel_connector)
if (!intel_dp_aux_display_control_capable(intel_connector))
return -ENODEV;
if (i915.enable_dpcd_backlight == -1 &&
!intel_dp_aux_display_control_heuristic(intel_connector))
return -ENODEV;
panel->backlight.setup = intel_dp_aux_setup_backlight;
panel->backlight.enable = intel_dp_aux_enable_backlight;
panel->backlight.disable = intel_dp_aux_disable_backlight;

8
drivers/gpu/drm/i915/intel_dp_link_training.c
View File

@ -321,12 +321,16 @@ intel_dp_start_link_train(struct intel_dp *intel_dp)
if (!intel_dp_link_training_channel_equalization(intel_dp))
goto failure_handling;
DRM_DEBUG_KMS("Link Training Passed at Link Rate = %d, Lane count = %d",
DRM_DEBUG_KMS("[CONNECTOR:%d:%s] Link Training Passed at Link Rate = %d, Lane count = %d",
intel_connector->base.base.id,
intel_connector->base.name,
intel_dp->link_rate, intel_dp->lane_count);
return;
failure_handling:
DRM_DEBUG_KMS("Link Training failed at link rate = %d, lane count = %d",
DRM_DEBUG_KMS("[CONNECTOR:%d:%s] Link Training failed at link rate = %d, lane count = %d",
intel_connector->base.base.id,
intel_connector->base.name,
intel_dp->link_rate, intel_dp->lane_count);
if (!intel_dp_get_link_train_fallback_values(intel_dp,
intel_dp->link_rate,

3
drivers/gpu/drm/i915/intel_dp_mst.c
View File

@ -370,6 +370,9 @@ intel_dp_mst_mode_valid(struct drm_connector *connector,
int bpp = 24; /* MST uses fixed bpp */
int max_rate, mode_rate, max_lanes, max_link_clock;
if (!intel_dp)
return MODE_ERROR;
max_link_clock = intel_dp_max_link_rate(intel_dp);
max_lanes = intel_dp_max_lane_count(intel_dp);

11
drivers/gpu/drm/i915/intel_dpll_mgr.c
View File

@ -2379,6 +2379,15 @@ cnl_get_dpll(struct intel_crtc *crtc, struct intel_crtc_state *crtc_state,
return pll;
}
static void cnl_dump_hw_state(struct drm_i915_private *dev_priv,
struct intel_dpll_hw_state *hw_state)
{
DRM_DEBUG_KMS("dpll_hw_state: "
"cfgcr0: 0x%x, cfgcr1: 0x%x\n",
hw_state->cfgcr0,
hw_state->cfgcr1);
}
static const struct intel_shared_dpll_funcs cnl_ddi_pll_funcs = {
.enable = cnl_ddi_pll_enable,
.disable = cnl_ddi_pll_disable,
@ -2395,7 +2404,7 @@ static const struct dpll_info cnl_plls[] = {
static const struct intel_dpll_mgr cnl_pll_mgr = {
.dpll_info = cnl_plls,
.get_dpll = cnl_get_dpll,
.dump_hw_state = skl_dump_hw_state,
.dump_hw_state = cnl_dump_hw_state,
};
/**

51
drivers/gpu/drm/i915/intel_drv.h
View File

@ -265,6 +265,7 @@ struct intel_encoder {
struct intel_panel {
struct drm_display_mode *fixed_mode;
struct drm_display_mode *alt_fixed_mode;
struct drm_display_mode *downclock_mode;
/* backlight */
@ -780,6 +781,9 @@ struct intel_crtc_state {
/* HDMI High TMDS char rate ratio */
bool hdmi_high_tmds_clock_ratio;
/* output format is YCBCR 4:2:0 */
bool ycbcr420;
};
struct intel_crtc {
@ -796,9 +800,6 @@ struct intel_crtc {
u8 plane_ids_mask;
unsigned long long enabled_power_domains;
struct intel_overlay *overlay;
struct intel_flip_work *flip_work;
atomic_t unpin_work_count;
/* Display surface base address adjustement for pageflips. Note that on
* gen4+ this only adjusts up to a tile, offsets within a tile are
@ -1131,24 +1132,6 @@ intel_get_crtc_for_plane(struct drm_i915_private *dev_priv, enum plane plane)
return dev_priv->plane_to_crtc_mapping[plane];
}
struct intel_flip_work {
struct work_struct unpin_work;
struct work_struct mmio_work;
struct drm_crtc *crtc;
struct i915_vma *old_vma;
struct drm_framebuffer *old_fb;
struct drm_i915_gem_object *pending_flip_obj;
struct drm_pending_vblank_event *event;
atomic_t pending;
u32 flip_count;
u32 gtt_offset;
struct drm_i915_gem_request *flip_queued_req;
u32 flip_queued_vblank;
u32 flip_ready_vblank;
unsigned int rotation;
};
struct intel_load_detect_pipe {
struct drm_atomic_state *restore_state;
};
@ -1210,12 +1193,12 @@ hdmi_to_dig_port(struct intel_hdmi *intel_hdmi)
bool intel_set_cpu_fifo_underrun_reporting(struct drm_i915_private *dev_priv,
enum pipe pipe, bool enable);
bool intel_set_pch_fifo_underrun_reporting(struct drm_i915_private *dev_priv,
enum transcoder pch_transcoder,
enum pipe pch_transcoder,
bool enable);
void intel_cpu_fifo_underrun_irq_handler(struct drm_i915_private *dev_priv,
enum pipe pipe);
void intel_pch_fifo_underrun_irq_handler(struct drm_i915_private *dev_priv,
enum transcoder pch_transcoder);
enum pipe pch_transcoder);
void intel_check_cpu_fifo_underruns(struct drm_i915_private *dev_priv);
void intel_check_pch_fifo_underruns(struct drm_i915_private *dev_priv);
@ -1250,9 +1233,9 @@ static inline bool intel_irqs_enabled(struct drm_i915_private *dev_priv)
int intel_get_crtc_scanline(struct intel_crtc *crtc);
void gen8_irq_power_well_post_enable(struct drm_i915_private *dev_priv,
unsigned int pipe_mask);
u8 pipe_mask);
void gen8_irq_power_well_pre_disable(struct drm_i915_private *dev_priv,
unsigned int pipe_mask);
u8 pipe_mask);
void gen9_reset_guc_interrupts(struct drm_i915_private *dev_priv);
void gen9_enable_guc_interrupts(struct drm_i915_private *dev_priv);
void gen9_disable_guc_interrupts(struct drm_i915_private *dev_priv);
@ -1325,7 +1308,7 @@ void intel_set_cdclk(struct drm_i915_private *dev_priv,
/* intel_display.c */
void i830_enable_pipe(struct drm_i915_private *dev_priv, enum pipe pipe);
void i830_disable_pipe(struct drm_i915_private *dev_priv, enum pipe pipe);
enum transcoder intel_crtc_pch_transcoder(struct intel_crtc *crtc);
enum pipe intel_crtc_pch_transcoder(struct intel_crtc *crtc);
void intel_update_rawclk(struct drm_i915_private *dev_priv);
int vlv_get_hpll_vco(struct drm_i915_private *dev_priv);
int vlv_get_cck_clock(struct drm_i915_private *dev_priv,
@ -1334,7 +1317,6 @@ int vlv_get_cck_clock_hpll(struct drm_i915_private *dev_priv,
const char *name, u32 reg);
void lpt_disable_pch_transcoder(struct drm_i915_private *dev_priv);
void lpt_disable_iclkip(struct drm_i915_private *dev_priv);
extern const struct drm_plane_funcs intel_plane_funcs;
void intel_init_display_hooks(struct drm_i915_private *dev_priv);
unsigned int intel_fb_xy_to_linear(int x, int y,
const struct intel_plane_state *state,
@ -1407,9 +1389,6 @@ void intel_unpin_fb_vma(struct i915_vma *vma);
struct drm_framebuffer *
intel_framebuffer_create(struct drm_i915_gem_object *obj,
struct drm_mode_fb_cmd2 *mode_cmd);
void intel_finish_page_flip_cs(struct drm_i915_private *dev_priv, int pipe);
void intel_finish_page_flip_mmio(struct drm_i915_private *dev_priv, int pipe);
void intel_check_page_flip(struct drm_i915_private *dev_priv, int pipe);
int intel_prepare_plane_fb(struct drm_plane *plane,
struct drm_plane_state *new_state);
void intel_cleanup_plane_fb(struct drm_plane *plane,
@ -1596,7 +1575,8 @@ void intel_hpd_poll_init(struct drm_i915_private *dev_priv);
#ifdef CONFIG_DRM_FBDEV_EMULATION
extern int intel_fbdev_init(struct drm_device *dev);
extern void intel_fbdev_initial_config_async(struct drm_device *dev);
extern void intel_fbdev_fini(struct drm_device *dev);
extern void intel_fbdev_unregister(struct drm_i915_private *dev_priv);
extern void intel_fbdev_fini(struct drm_i915_private *dev_priv);
extern void intel_fbdev_set_suspend(struct drm_device *dev, int state, bool synchronous);
extern void intel_fbdev_output_poll_changed(struct drm_device *dev);
extern void intel_fbdev_restore_mode(struct drm_device *dev);
@ -1610,7 +1590,11 @@ static inline void intel_fbdev_initial_config_async(struct drm_device *dev)
{
}
static inline void intel_fbdev_fini(struct drm_device *dev)
static inline void intel_fbdev_unregister(struct drm_i915_private *dev_priv)
{
}
static inline void intel_fbdev_fini(struct drm_i915_private *dev_priv)
{
}
@ -1695,6 +1679,7 @@ void intel_overlay_reset(struct drm_i915_private *dev_priv);
/* intel_panel.c */
int intel_panel_init(struct intel_panel *panel,
struct drm_display_mode *fixed_mode,
struct drm_display_mode *alt_fixed_mode,
struct drm_display_mode *downclock_mode);
void intel_panel_fini(struct intel_panel *panel);
void intel_fixed_panel_mode(const struct drm_display_mode *fixed_mode,
@ -1900,7 +1885,7 @@ struct intel_plane *intel_sprite_plane_create(struct drm_i915_private *dev_priv,
int intel_sprite_set_colorkey(struct drm_device *dev, void *data,
struct drm_file *file_priv);
void intel_pipe_update_start(struct intel_crtc *crtc);
void intel_pipe_update_end(struct intel_crtc *crtc, struct intel_flip_work *work);
void intel_pipe_update_end(struct intel_crtc *crtc);
/* intel_tv.c */
void intel_tv_init(struct drm_i915_private *dev_priv);

2
drivers/gpu/drm/i915/intel_dsi.c
View File

@ -1849,7 +1849,7 @@ void intel_dsi_init(struct drm_i915_private *dev_priv)
connector->display_info.width_mm = fixed_mode->width_mm;
connector->display_info.height_mm = fixed_mode->height_mm;
intel_panel_init(&intel_connector->panel, fixed_mode, NULL);
intel_panel_init(&intel_connector->panel, fixed_mode, NULL, NULL);
intel_panel_setup_backlight(connector, INVALID_PIPE);
intel_dsi_add_properties(intel_connector);

2
drivers/gpu/drm/i915/intel_dsi_dcs_backlight.c
View File

@ -46,7 +46,7 @@ static u32 dcs_get_backlight(struct intel_connector *connector)
struct intel_encoder *encoder = connector->encoder;
struct intel_dsi *intel_dsi = enc_to_intel_dsi(&encoder->base);
struct mipi_dsi_device *dsi_device;
u8 data;
u8 data = 0;
enum port port;
/* FIXME: Need to take care of 16 bit brightness level */

2
drivers/gpu/drm/i915/intel_dsi_vbt.c
View File

@ -306,7 +306,7 @@ static void bxt_exec_gpio(struct drm_i915_private *dev_priv,
if (!gpio_desc) {
gpio_desc = devm_gpiod_get_index(dev_priv->drm.dev,
"panel", gpio_index,
NULL, gpio_index,
value ? GPIOD_OUT_LOW :
GPIOD_OUT_HIGH);

2
drivers/gpu/drm/i915/intel_dvo.c
View File

@ -552,7 +552,7 @@ void intel_dvo_init(struct drm_i915_private *dev_priv)
*/
intel_panel_init(&intel_connector->panel,
intel_dvo_get_current_mode(connector),
NULL);
NULL, NULL);
intel_dvo->panel_wants_dither = true;
}

7
drivers/gpu/drm/i915/intel_engine_cs.c
View File

@ -337,9 +337,6 @@ void intel_engine_init_global_seqno(struct intel_engine_cs *engine, u32 seqno)
{
struct drm_i915_private *dev_priv = engine->i915;
GEM_BUG_ON(!intel_engine_is_idle(engine));
GEM_BUG_ON(i915_gem_active_isset(&engine->timeline->last_request));
/* Our semaphore implementation is strictly monotonic (i.e. we proceed
* so long as the semaphore value in the register/page is greater
* than the sync value), so whenever we reset the seqno,
@ -1283,6 +1280,10 @@ bool intel_engine_is_idle(struct intel_engine_cs *engine)
if (port_request(&engine->execlist_port[0]))
return false;
/* ELSP is empty, but there are ready requests? */
if (READ_ONCE(engine->execlist_first))
return false;
/* Ring stopped? */
if (!ring_is_idle(engine))
return false;

4
drivers/gpu/drm/i915/intel_fbc.c
View File

@ -461,6 +461,8 @@ static void intel_fbc_schedule_activation(struct intel_crtc *crtc)
struct intel_fbc_work *work = &fbc->work;
WARN_ON(!mutex_is_locked(&fbc->lock));
if (WARN_ON(!fbc->enabled))
return;
if (drm_crtc_vblank_get(&crtc->base)) {
DRM_ERROR("vblank not available for FBC on pipe %c\n",
@ -1216,7 +1218,7 @@ static void intel_fbc_underrun_work_fn(struct work_struct *work)
mutex_lock(&fbc->lock);
/* Maybe we were scheduled twice. */
if (fbc->underrun_detected)
if (fbc->underrun_detected || !fbc->enabled)
goto out;
DRM_DEBUG_KMS("Disabling FBC due to FIFO underrun.\n");

22
drivers/gpu/drm/i915/intel_fbdev.c
View File

@ -507,8 +507,6 @@ static void intel_fbdev_destroy(struct intel_fbdev *ifbdev)
* trying to rectify all the possible error paths leading here.
*/
drm_fb_helper_unregister_fbi(&ifbdev->helper);
drm_fb_helper_fini(&ifbdev->helper);
if (ifbdev->vma) {
@ -696,8 +694,10 @@ static void intel_fbdev_initial_config(void *data, async_cookie_t cookie)
/* Due to peculiar init order wrt to hpd handling this is separate. */
if (drm_fb_helper_initial_config(&ifbdev->helper,
ifbdev->preferred_bpp))
intel_fbdev_fini(ifbdev->helper.dev);
ifbdev->preferred_bpp)) {
intel_fbdev_unregister(to_i915(ifbdev->helper.dev));
intel_fbdev_fini(to_i915(ifbdev->helper.dev));
}
}
void intel_fbdev_initial_config_async(struct drm_device *dev)
@ -720,9 +720,8 @@ static void intel_fbdev_sync(struct intel_fbdev *ifbdev)
ifbdev->cookie = 0;
}
void intel_fbdev_fini(struct drm_device *dev)
void intel_fbdev_unregister(struct drm_i915_private *dev_priv)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_fbdev *ifbdev = dev_priv->fbdev;
if (!ifbdev)
@ -732,8 +731,17 @@ void intel_fbdev_fini(struct drm_device *dev)
if (!current_is_async())
intel_fbdev_sync(ifbdev);
drm_fb_helper_unregister_fbi(&ifbdev->helper);
}
void intel_fbdev_fini(struct drm_i915_private *dev_priv)
{
struct intel_fbdev *ifbdev = fetch_and_zero(&dev_priv->fbdev);
if (!ifbdev)
return;
intel_fbdev_destroy(ifbdev);
dev_priv->fbdev = NULL;
}
void intel_fbdev_set_suspend(struct drm_device *dev, int state, bool synchronous)

6
drivers/gpu/drm/i915/intel_fifo_underrun.c
View File

@ -313,11 +313,11 @@ bool intel_set_cpu_fifo_underrun_reporting(struct drm_i915_private *dev_priv,
* Returns the previous state of underrun reporting.
*/
bool intel_set_pch_fifo_underrun_reporting(struct drm_i915_private *dev_priv,
enum transcoder pch_transcoder,
enum pipe pch_transcoder,
bool enable)
{
struct intel_crtc *crtc =
intel_get_crtc_for_pipe(dev_priv, (enum pipe) pch_transcoder);
intel_get_crtc_for_pipe(dev_priv, pch_transcoder);
unsigned long flags;
bool old;
@ -390,7 +390,7 @@ void intel_cpu_fifo_underrun_irq_handler(struct drm_i915_private *dev_priv,
* interrupt to avoid an irq storm.
*/
void intel_pch_fifo_underrun_irq_handler(struct drm_i915_private *dev_priv,
enum transcoder pch_transcoder)
enum pipe pch_transcoder)
{
if (intel_set_pch_fifo_underrun_reporting(dev_priv, pch_transcoder,
false)) {

2
drivers/gpu/drm/i915/intel_hangcheck.c
View File

@ -324,7 +324,7 @@ hangcheck_get_action(struct intel_engine_cs *engine,
if (engine->hangcheck.seqno != hc->seqno)
return ENGINE_ACTIVE_SEQNO;
if (i915_seqno_passed(hc->seqno, intel_engine_last_submit(engine)))
if (intel_engine_is_idle(engine))
return ENGINE_IDLE;
return engine_stuck(engine, hc->acthd);

203
drivers/gpu/drm/i915/intel_hdmi.c
View File

@ -472,12 +472,18 @@ static void intel_hdmi_set_avi_infoframe(struct drm_encoder *encoder,
return;
}
if (crtc_state->ycbcr420)
frame.avi.colorspace = HDMI_COLORSPACE_YUV420;
else
frame.avi.colorspace = HDMI_COLORSPACE_RGB;
drm_hdmi_avi_infoframe_quant_range(&frame.avi, adjusted_mode,
crtc_state->limited_color_range ?
HDMI_QUANTIZATION_RANGE_LIMITED :
HDMI_QUANTIZATION_RANGE_FULL,
intel_hdmi->rgb_quant_range_selectable);
/* TODO: handle pixel repetition for YCBCR420 outputs */
intel_write_infoframe(encoder, crtc_state, &frame);
}
@ -1295,6 +1301,9 @@ intel_hdmi_mode_valid(struct drm_connector *connector,
if (mode->flags & DRM_MODE_FLAG_DBLCLK)
clock *= 2;
if (drm_mode_is_420_only(&connector->display_info, mode))
clock /= 2;
/* check if we can do 8bpc */
status = hdmi_port_clock_valid(hdmi, clock, true, force_dvi);
@ -1330,8 +1339,15 @@ static bool hdmi_12bpc_possible(struct intel_crtc_state *crtc_state)
if (connector_state->crtc != crtc_state->base.crtc)
continue;
if ((info->edid_hdmi_dc_modes & DRM_EDID_HDMI_DC_36) == 0)
return false;
if (crtc_state->ycbcr420) {
const struct drm_hdmi_info *hdmi = &info->hdmi;
if (!(hdmi->y420_dc_modes & DRM_EDID_YCBCR420_DC_36))
return false;
} else {
if (!(info->edid_hdmi_dc_modes & DRM_EDID_HDMI_DC_36))
return false;
}
}
/* Display Wa #1139 */
@ -1342,6 +1358,36 @@ static bool hdmi_12bpc_possible(struct intel_crtc_state *crtc_state)
return true;
}
static bool
intel_hdmi_ycbcr420_config(struct drm_connector *connector,
struct intel_crtc_state *config,
int *clock_12bpc, int *clock_8bpc)
{
struct intel_crtc *intel_crtc = to_intel_crtc(config->base.crtc);
if (!connector->ycbcr_420_allowed) {
DRM_ERROR("Platform doesn't support YCBCR420 output\n");
return false;
}
/* YCBCR420 TMDS rate requirement is half the pixel clock */
config->port_clock /= 2;
*clock_12bpc /= 2;
*clock_8bpc /= 2;
config->ycbcr420 = true;
/* YCBCR 420 output conversion needs a scaler */
if (skl_update_scaler_crtc(config)) {
DRM_DEBUG_KMS("Scaler allocation for output failed\n");
return false;
}
intel_pch_panel_fitting(intel_crtc, config,
DRM_MODE_SCALE_FULLSCREEN);
return true;
}
bool intel_hdmi_compute_config(struct intel_encoder *encoder,
struct intel_crtc_state *pipe_config,
struct drm_connector_state *conn_state)
@ -1349,7 +1395,8 @@ bool intel_hdmi_compute_config(struct intel_encoder *encoder,
struct intel_hdmi *intel_hdmi = enc_to_intel_hdmi(&encoder->base);
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct drm_display_mode *adjusted_mode = &pipe_config->base.adjusted_mode;
struct drm_scdc *scdc = &conn_state->connector->display_info.hdmi.scdc;
struct drm_connector *connector = conn_state->connector;
struct drm_scdc *scdc = &connector->display_info.hdmi.scdc;
struct intel_digital_connector_state *intel_conn_state =
to_intel_digital_connector_state(conn_state);
int clock_8bpc = pipe_config->base.adjusted_mode.crtc_clock;
@ -1379,6 +1426,14 @@ bool intel_hdmi_compute_config(struct intel_encoder *encoder,
clock_12bpc *= 2;
}
if (drm_mode_is_420_only(&connector->display_info, adjusted_mode)) {
if (!intel_hdmi_ycbcr420_config(connector, pipe_config,
&clock_12bpc, &clock_8bpc)) {
DRM_ERROR("Can't support YCBCR420 output\n");
return false;
}
}
if (HAS_PCH_SPLIT(dev_priv) && !HAS_DDI(dev_priv))
pipe_config->has_pch_encoder = true;
@ -1788,6 +1843,93 @@ void intel_hdmi_handle_sink_scrambling(struct intel_encoder *encoder,
DRM_DEBUG_KMS("sink scrambling handled\n");
}
static u8 chv_port_to_ddc_pin(struct drm_i915_private *dev_priv, enum port port)
{
u8 ddc_pin;
switch (port) {
case PORT_B:
ddc_pin = GMBUS_PIN_DPB;
break;
case PORT_C:
ddc_pin = GMBUS_PIN_DPC;
break;
case PORT_D:
ddc_pin = GMBUS_PIN_DPD_CHV;
break;
default:
MISSING_CASE(port);
ddc_pin = GMBUS_PIN_DPB;
break;
}
return ddc_pin;
}
static u8 bxt_port_to_ddc_pin(struct drm_i915_private *dev_priv, enum port port)
{
u8 ddc_pin;
switch (port) {
case PORT_B:
ddc_pin = GMBUS_PIN_1_BXT;
break;
case PORT_C:
ddc_pin = GMBUS_PIN_2_BXT;
break;
default:
MISSING_CASE(port);
ddc_pin = GMBUS_PIN_1_BXT;
break;
}
return ddc_pin;
}
static u8 cnp_port_to_ddc_pin(struct drm_i915_private *dev_priv,
enum port port)
{
u8 ddc_pin;
switch (port) {
case PORT_B:
ddc_pin = GMBUS_PIN_1_BXT;
break;
case PORT_C:
ddc_pin = GMBUS_PIN_2_BXT;
break;
case PORT_D:
ddc_pin = GMBUS_PIN_4_CNP;
break;
default:
MISSING_CASE(port);
ddc_pin = GMBUS_PIN_1_BXT;
break;
}
return ddc_pin;
}
static u8 g4x_port_to_ddc_pin(struct drm_i915_private *dev_priv,
enum port port)
{
u8 ddc_pin;
switch (port) {
case PORT_B:
ddc_pin = GMBUS_PIN_DPB;
break;
case PORT_C:
ddc_pin = GMBUS_PIN_DPC;
break;
case PORT_D:
ddc_pin = GMBUS_PIN_DPD;
break;
default:
MISSING_CASE(port);
ddc_pin = GMBUS_PIN_DPB;
break;
}
return ddc_pin;
}
static u8 intel_hdmi_ddc_pin(struct drm_i915_private *dev_priv,
enum port port)
{
@ -1801,32 +1943,14 @@ static u8 intel_hdmi_ddc_pin(struct drm_i915_private *dev_priv,
return info->alternate_ddc_pin;
}
switch (port) {
case PORT_B:
if (IS_GEN9_LP(dev_priv) || HAS_PCH_CNP(dev_priv))
ddc_pin = GMBUS_PIN_1_BXT;
else
ddc_pin = GMBUS_PIN_DPB;
break;
case PORT_C:
if (IS_GEN9_LP(dev_priv) || HAS_PCH_CNP(dev_priv))
ddc_pin = GMBUS_PIN_2_BXT;
else
ddc_pin = GMBUS_PIN_DPC;
break;
case PORT_D:
if (HAS_PCH_CNP(dev_priv))
ddc_pin = GMBUS_PIN_4_CNP;
else if (IS_CHERRYVIEW(dev_priv))
ddc_pin = GMBUS_PIN_DPD_CHV;
else
ddc_pin = GMBUS_PIN_DPD;
break;
default:
MISSING_CASE(port);
ddc_pin = GMBUS_PIN_DPB;
break;
}
if (IS_CHERRYVIEW(dev_priv))
ddc_pin = chv_port_to_ddc_pin(dev_priv, port);
else if (IS_GEN9_LP(dev_priv))
ddc_pin = bxt_port_to_ddc_pin(dev_priv, port);
else if (HAS_PCH_CNP(dev_priv))
ddc_pin = cnp_port_to_ddc_pin(dev_priv, port);
else
ddc_pin = g4x_port_to_ddc_pin(dev_priv, port);
DRM_DEBUG_KMS("Using DDC pin 0x%x for port %c (platform default)\n",
ddc_pin, port_name(port));
@ -1860,25 +1984,14 @@ void intel_hdmi_init_connector(struct intel_digital_port *intel_dig_port,
connector->doublescan_allowed = 0;
connector->stereo_allowed = 1;
if (IS_GEMINILAKE(dev_priv))
connector->ycbcr_420_allowed = true;
intel_hdmi->ddc_bus = intel_hdmi_ddc_pin(dev_priv, port);
switch (port) {
case PORT_B:
intel_encoder->hpd_pin = HPD_PORT_B;
break;
case PORT_C:
intel_encoder->hpd_pin = HPD_PORT_C;
break;
case PORT_D:
intel_encoder->hpd_pin = HPD_PORT_D;
break;
case PORT_E:
intel_encoder->hpd_pin = HPD_PORT_E;
break;
default:
MISSING_CASE(port);
if (WARN_ON(port == PORT_A))
return;
}
intel_encoder->hpd_pin = intel_hpd_pin(port);
if (IS_VALLEYVIEW(dev_priv) || IS_CHERRYVIEW(dev_priv)) {
intel_hdmi->write_infoframe = vlv_write_infoframe;

57
drivers/gpu/drm/i915/intel_hotplug.c
View File

@ -76,26 +76,54 @@
* it will use i915_hotplug_work_func where this logic is handled.
*/
bool intel_hpd_pin_to_port(enum hpd_pin pin, enum port *port)
/**
* intel_hpd_port - return port hard associated with certain pin.
* @pin: the hpd pin to get associated port
*
* Return port that is associatade with @pin and PORT_NONE if no port is
* hard associated with that @pin.
*/
enum port intel_hpd_pin_to_port(enum hpd_pin pin)
{
switch (pin) {
case HPD_PORT_A:
*port = PORT_A;
return true;
return PORT_A;
case HPD_PORT_B:
*port = PORT_B;
return true;
return PORT_B;
case HPD_PORT_C:
*port = PORT_C;
return true;
return PORT_C;
case HPD_PORT_D:
*port = PORT_D;
return true;
return PORT_D;
case HPD_PORT_E:
*port = PORT_E;
return true;
return PORT_E;
default:
return false; /* no hpd */
return PORT_NONE; /* no port for this pin */
}
}
/**
* intel_hpd_pin - return pin hard associated with certain port.
* @port: the hpd port to get associated pin
*
* Return pin that is associatade with @port and HDP_NONE if no pin is
* hard associated with that @port.
*/
enum hpd_pin intel_hpd_pin(enum port port)
{
switch (port) {
case PORT_A:
return HPD_PORT_A;
case PORT_B:
return HPD_PORT_B;
case PORT_C:
return HPD_PORT_C;
case PORT_D:
return HPD_PORT_D;
case PORT_E:
return HPD_PORT_E;
default:
MISSING_CASE(port);
return HPD_NONE;
}
}
@ -389,8 +417,9 @@ void intel_hpd_irq_handler(struct drm_i915_private *dev_priv,
if (!(BIT(i) & pin_mask))
continue;
is_dig_port = intel_hpd_pin_to_port(i, &port) &&
dev_priv->hotplug.irq_port[port];
port = intel_hpd_pin_to_port(i);
is_dig_port = port != PORT_NONE &&
dev_priv->hotplug.irq_port[port];
if (is_dig_port) {
bool long_hpd = long_mask & BIT(i);

36
drivers/gpu/drm/i915/intel_i2c.c
View File

@ -592,7 +592,6 @@ gmbus_xfer(struct i2c_adapter *adapter, struct i2c_msg *msgs, int num)
int ret;
intel_display_power_get(dev_priv, POWER_DOMAIN_GMBUS);
mutex_lock(&dev_priv->gmbus_mutex);
if (bus->force_bit) {
ret = i2c_bit_algo.master_xfer(adapter, msgs, num);
@ -604,7 +603,6 @@ gmbus_xfer(struct i2c_adapter *adapter, struct i2c_msg *msgs, int num)
bus->force_bit |= GMBUS_FORCE_BIT_RETRY;
}
mutex_unlock(&dev_priv->gmbus_mutex);
intel_display_power_put(dev_priv, POWER_DOMAIN_GMBUS);
return ret;
@ -624,6 +622,39 @@ static const struct i2c_algorithm gmbus_algorithm = {
.functionality = gmbus_func
};
static void gmbus_lock_bus(struct i2c_adapter *adapter,
unsigned int flags)
{
struct intel_gmbus *bus = to_intel_gmbus(adapter);
struct drm_i915_private *dev_priv = bus->dev_priv;
mutex_lock(&dev_priv->gmbus_mutex);
}
static int gmbus_trylock_bus(struct i2c_adapter *adapter,
unsigned int flags)
{
struct intel_gmbus *bus = to_intel_gmbus(adapter);
struct drm_i915_private *dev_priv = bus->dev_priv;
return mutex_trylock(&dev_priv->gmbus_mutex);
}
static void gmbus_unlock_bus(struct i2c_adapter *adapter,
unsigned int flags)
{
struct intel_gmbus *bus = to_intel_gmbus(adapter);
struct drm_i915_private *dev_priv = bus->dev_priv;
mutex_unlock(&dev_priv->gmbus_mutex);
}
const struct i2c_lock_operations gmbus_lock_ops = {
.lock_bus = gmbus_lock_bus,
.trylock_bus = gmbus_trylock_bus,
.unlock_bus = gmbus_unlock_bus,
};
/**
* intel_gmbus_setup - instantiate all Intel i2c GMBuses
* @dev_priv: i915 device private
@ -665,6 +696,7 @@ int intel_setup_gmbus(struct drm_i915_private *dev_priv)
bus->dev_priv = dev_priv;
bus->adapter.algo = &gmbus_algorithm;
bus->adapter.lock_ops = &gmbus_lock_ops;
/*
* We wish to retry with bit banging

36
drivers/gpu/drm/i915/intel_lrc.c
View File

@ -1306,6 +1306,31 @@ static void reset_common_ring(struct intel_engine_cs *engine,
{
struct execlist_port *port = engine->execlist_port;
struct intel_context *ce;
unsigned int n;
/*
* Catch up with any missed context-switch interrupts.
*
* Ideally we would just read the remaining CSB entries now that we
* know the gpu is idle. However, the CSB registers are sometimes^W
* often trashed across a GPU reset! Instead we have to rely on
* guessing the missed context-switch events by looking at what
* requests were completed.
*/
if (!request) {
for (n = 0; n < ARRAY_SIZE(engine->execlist_port); n++)
i915_gem_request_put(port_request(&port[n]));
memset(engine->execlist_port, 0, sizeof(engine->execlist_port));
return;
}
if (request->ctx != port_request(port)->ctx) {
i915_gem_request_put(port_request(port));
port[0] = port[1];
memset(&port[1], 0, sizeof(port[1]));
}
GEM_BUG_ON(request->ctx != port_request(port)->ctx);
/* If the request was innocent, we leave the request in the ELSP
* and will try to replay it on restarting. The context image may
@ -1317,7 +1342,7 @@ static void reset_common_ring(struct intel_engine_cs *engine,
* and have to at least restore the RING register in the context
* image back to the expected values to skip over the guilty request.
*/
if (!request || request->fence.error != -EIO)
if (request->fence.error != -EIO)
return;
/* We want a simple context + ring to execute the breadcrumb update.
@ -1339,15 +1364,6 @@ static void reset_common_ring(struct intel_engine_cs *engine,
request->ring->head = request->postfix;
intel_ring_update_space(request->ring);
/* Catch up with any missed context-switch interrupts */
if (request->ctx != port_request(port)->ctx) {
i915_gem_request_put(port_request(port));
port[0] = port[1];
memset(&port[1], 0, sizeof(port[1]));
}
GEM_BUG_ON(request->ctx != port_request(port)->ctx);
/* Reset WaIdleLiteRestore:bdw,skl as well */
request->tail =
intel_ring_wrap(request->ring,

1
drivers/gpu/drm/i915/intel_lrc.h
View File

@ -63,7 +63,6 @@ enum {
};
/* Logical Rings */
void intel_logical_ring_stop(struct intel_engine_cs *engine);
void intel_logical_ring_cleanup(struct intel_engine_cs *engine);
int logical_render_ring_init(struct intel_engine_cs *engine);
int logical_xcs_ring_init(struct intel_engine_cs *engine);

4
drivers/gpu/drm/i915/intel_lspcon.c
View File

@ -210,8 +210,8 @@ bool lspcon_init(struct intel_digital_port *intel_dig_port)
struct drm_device *dev = intel_dig_port->base.base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
if (!IS_GEN9(dev_priv)) {
DRM_ERROR("LSPCON is supported on GEN9 only\n");
if (!HAS_LSPCON(dev_priv)) {
DRM_ERROR("LSPCON is not supported on this platform\n");
return false;
}

3
drivers/gpu/drm/i915/intel_lvds.c
View File

@ -1138,7 +1138,8 @@ void intel_lvds_init(struct drm_i915_private *dev_priv)
out:
mutex_unlock(&dev->mode_config.mutex);
intel_panel_init(&intel_connector->panel, fixed_mode, downclock_mode);
intel_panel_init(&intel_connector->panel, fixed_mode, NULL,
downclock_mode);
intel_panel_setup_backlight(connector, INVALID_PIPE);
lvds_encoder->is_dual_link = compute_is_dual_link_lvds(lvds_encoder);

45
drivers/gpu/drm/i915/intel_opregion.c
View File

@ -27,6 +27,7 @@
#include <linux/acpi.h>
#include <linux/dmi.h>
#include <linux/firmware.h>
#include <acpi/video.h>
#include <drm/drmP.h>
@ -829,6 +830,10 @@ void intel_opregion_unregister(struct drm_i915_private *dev_priv)
memunmap(opregion->rvda);
opregion->rvda = NULL;
}
if (opregion->vbt_firmware) {
kfree(opregion->vbt_firmware);
opregion->vbt_firmware = NULL;
}
opregion->header = NULL;
opregion->acpi = NULL;
opregion->swsci = NULL;
@ -912,6 +917,43 @@ static const struct dmi_system_id intel_no_opregion_vbt[] = {
{ }
};
static int intel_load_vbt_firmware(struct drm_i915_private *dev_priv)
{
struct intel_opregion *opregion = &dev_priv->opregion;
const struct firmware *fw = NULL;
const char *name = i915.vbt_firmware;
int ret;
if (!name || !*name)
return -ENOENT;
ret = request_firmware(&fw, name, &dev_priv->drm.pdev->dev);
if (ret) {
DRM_ERROR("Requesting VBT firmware \"%s\" failed (%d)\n",
name, ret);
return ret;
}
if (intel_bios_is_valid_vbt(fw->data, fw->size)) {
opregion->vbt_firmware = kmemdup(fw->data, fw->size, GFP_KERNEL);
if (opregion->vbt_firmware) {
DRM_DEBUG_KMS("Found valid VBT firmware \"%s\"\n", name);
opregion->vbt = opregion->vbt_firmware;
opregion->vbt_size = fw->size;
ret = 0;
} else {
ret = -ENOMEM;
}
} else {
DRM_DEBUG_KMS("Invalid VBT firmware \"%s\"\n", name);
ret = -EINVAL;
}
release_firmware(fw);
return ret;
}
int intel_opregion_setup(struct drm_i915_private *dev_priv)
{
struct intel_opregion *opregion = &dev_priv->opregion;
@ -974,6 +1016,9 @@ int intel_opregion_setup(struct drm_i915_private *dev_priv)
if (mboxes & MBOX_ASLE_EXT)
DRM_DEBUG_DRIVER("ASLE extension supported\n");
if (intel_load_vbt_firmware(dev_priv) == 0)
goto out;
if (dmi_check_system(intel_no_opregion_vbt))
goto out;

11
drivers/gpu/drm/i915/intel_overlay.c
View File

@ -799,9 +799,13 @@ static int intel_overlay_do_put_image(struct intel_overlay *overlay,
if (ret != 0)
return ret;
atomic_inc(&dev_priv->gpu_error.pending_fb_pin);
vma = i915_gem_object_pin_to_display_plane(new_bo, 0, NULL);
if (IS_ERR(vma))
return PTR_ERR(vma);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
goto out_pin_section;
}
ret = i915_vma_put_fence(vma);
if (ret)
@ -886,6 +890,9 @@ static int intel_overlay_do_put_image(struct intel_overlay *overlay,
out_unpin:
i915_gem_object_unpin_from_display_plane(vma);
out_pin_section:
atomic_dec(&dev_priv->gpu_error.pending_fb_pin);
return ret;
}

9
drivers/gpu/drm/i915/intel_panel.c
View File

@ -110,7 +110,8 @@ intel_pch_panel_fitting(struct intel_crtc *intel_crtc,
/* Native modes don't need fitting */
if (adjusted_mode->crtc_hdisplay == pipe_config->pipe_src_w &&
adjusted_mode->crtc_vdisplay == pipe_config->pipe_src_h)
adjusted_mode->crtc_vdisplay == pipe_config->pipe_src_h &&
!pipe_config->ycbcr420)
goto done;
switch (fitting_mode) {
@ -1919,11 +1920,13 @@ intel_panel_init_backlight_funcs(struct intel_panel *panel)
int intel_panel_init(struct intel_panel *panel,
struct drm_display_mode *fixed_mode,
struct drm_display_mode *alt_fixed_mode,
struct drm_display_mode *downclock_mode)
{
intel_panel_init_backlight_funcs(panel);
panel->fixed_mode = fixed_mode;
panel->alt_fixed_mode = alt_fixed_mode;
panel->downclock_mode = downclock_mode;
return 0;
@ -1937,6 +1940,10 @@ void intel_panel_fini(struct intel_panel *panel)
if (panel->fixed_mode)
drm_mode_destroy(intel_connector->base.dev, panel->fixed_mode);
if (panel->alt_fixed_mode)
drm_mode_destroy(intel_connector->base.dev,
panel->alt_fixed_mode);
if (panel->downclock_mode)
drm_mode_destroy(intel_connector->base.dev,
panel->downclock_mode);

97
drivers/gpu/drm/i915/intel_pm.c
View File

@ -62,6 +62,20 @@ static void gen9_init_clock_gating(struct drm_i915_private *dev_priv)
I915_WRITE(CHICKEN_PAR1_1,
I915_READ(CHICKEN_PAR1_1) | SKL_EDP_PSR_FIX_RDWRAP);
/*
* Display WA#0390: skl,bxt,kbl,glk
*
* Must match Sampler, Pixel Back End, and Media
* (0xE194 bit 8, 0x7014 bit 13, 0x4DDC bits 27 and 31).
*
* Including bits outside the page in the hash would
* require 2 (or 4?) MiB alignment of resources. Just
* assume the defaul hashing mode which only uses bits
* within the page.
*/
I915_WRITE(CHICKEN_PAR1_1,
I915_READ(CHICKEN_PAR1_1) & ~SKL_RC_HASH_OUTSIDE);
I915_WRITE(GEN8_CONFIG0,
I915_READ(GEN8_CONFIG0) | GEN9_DEFAULT_FIXES);
@ -78,6 +92,12 @@ static void gen9_init_clock_gating(struct drm_i915_private *dev_priv)
/* WaFbcHighMemBwCorruptionAvoidance:skl,bxt,kbl,cfl */
I915_WRITE(ILK_DPFC_CHICKEN, I915_READ(ILK_DPFC_CHICKEN) |
ILK_DPFC_DISABLE_DUMMY0);
if (IS_SKYLAKE(dev_priv)) {
/* WaDisableDopClockGating */
I915_WRITE(GEN7_MISCCPCTL, I915_READ(GEN7_MISCCPCTL)
& ~GEN7_DOP_CLOCK_GATE_ENABLE);
}
}
static void bxt_init_clock_gating(struct drm_i915_private *dev_priv)
@ -2758,7 +2778,7 @@ hsw_compute_linetime_wm(const struct intel_crtc_state *cstate)
static void intel_read_wm_latency(struct drm_i915_private *dev_priv,
uint16_t wm[8])
{
if (IS_GEN9(dev_priv)) {
if (INTEL_GEN(dev_priv) >= 9) {
uint32_t val;
int ret, i;
int level, max_level = ilk_wm_max_level(dev_priv);
@ -2818,7 +2838,7 @@ static void intel_read_wm_latency(struct drm_i915_private *dev_priv,
}
/*
* WaWmMemoryReadLatency:skl,glk
* WaWmMemoryReadLatency:skl+,glk
*
* punit doesn't take into account the read latency so we need
* to add 2us to the various latency levels we retrieve from the
@ -2857,6 +2877,8 @@ static void intel_read_wm_latency(struct drm_i915_private *dev_priv,
wm[0] = 7;
wm[1] = (mltr >> MLTR_WM1_SHIFT) & ILK_SRLT_MASK;
wm[2] = (mltr >> MLTR_WM2_SHIFT) & ILK_SRLT_MASK;
} else {
MISSING_CASE(INTEL_DEVID(dev_priv));
}
}
@ -2912,7 +2934,7 @@ static void intel_print_wm_latency(struct drm_i915_private *dev_priv,
* - latencies are in us on gen9.
* - before then, WM1+ latency values are in 0.5us units
*/
if (IS_GEN9(dev_priv))
if (INTEL_GEN(dev_priv) >= 9)
latency *= 10;
else if (level > 0)
latency *= 5;
@ -3530,8 +3552,6 @@ bool ilk_disable_lp_wm(struct drm_device *dev)
return _ilk_disable_lp_wm(dev_priv, WM_DIRTY_LP_ALL);
}
#define SKL_SAGV_BLOCK_TIME 30 /* µs */
/*
* FIXME: We still don't have the proper code detect if we need to apply the WA,
* so assume we'll always need it in order to avoid underruns.
@ -3549,7 +3569,8 @@ static bool skl_needs_memory_bw_wa(struct intel_atomic_state *state)
static bool
intel_has_sagv(struct drm_i915_private *dev_priv)
{
if (IS_KABYLAKE(dev_priv) || IS_COFFEELAKE(dev_priv))
if (IS_KABYLAKE(dev_priv) || IS_COFFEELAKE(dev_priv) ||
IS_CANNONLAKE(dev_priv))
return true;
if (IS_SKYLAKE(dev_priv) &&
@ -3655,12 +3676,13 @@ bool intel_can_enable_sagv(struct drm_atomic_state *state)
struct intel_crtc_state *cstate;
enum pipe pipe;
int level, latency;
int sagv_block_time_us = IS_GEN9(dev_priv) ? 30 : 20;
if (!intel_has_sagv(dev_priv))
return false;
/*
* SKL workaround: bspec recommends we disable the SAGV when we have
* SKL+ workaround: bspec recommends we disable the SAGV when we have
* more then one pipe enabled
*
* If there are no active CRTCs, no additional checks need be performed
@ -3699,11 +3721,11 @@ bool intel_can_enable_sagv(struct drm_atomic_state *state)
latency += 15;
/*
* If any of the planes on this pipe don't enable wm levels
* that incur memory latencies higher then 30µs we can't enable
* the SAGV
* If any of the planes on this pipe don't enable wm levels that
* incur memory latencies higher than sagv_block_time_us we
* can't enable the SAGV.
*/
if (latency < SKL_SAGV_BLOCK_TIME)
if (latency < sagv_block_time_us)
return false;
}
@ -4071,7 +4093,9 @@ skl_ddb_min_alloc(const struct drm_plane_state *pstate,
/* For Non Y-tile return 8-blocks */
if (fb->modifier != I915_FORMAT_MOD_Y_TILED &&
fb->modifier != I915_FORMAT_MOD_Yf_TILED)
fb->modifier != I915_FORMAT_MOD_Yf_TILED &&
fb->modifier != I915_FORMAT_MOD_Y_TILED_CCS &&
fb->modifier != I915_FORMAT_MOD_Yf_TILED_CCS)
return 8;
/*
@ -4266,8 +4290,9 @@ skl_allocate_pipe_ddb(struct intel_crtc_state *cstate,
* should allow pixel_rate up to ~2 GHz which seems sufficient since max
* 2xcdclk is 1350 MHz and the pixel rate should never exceed that.
*/
static uint_fixed_16_16_t skl_wm_method1(uint32_t pixel_rate, uint8_t cpp,
uint32_t latency)
static uint_fixed_16_16_t
skl_wm_method1(const struct drm_i915_private *dev_priv, uint32_t pixel_rate,
uint8_t cpp, uint32_t latency)
{
uint32_t wm_intermediate_val;
uint_fixed_16_16_t ret;
@ -4277,6 +4302,10 @@ static uint_fixed_16_16_t skl_wm_method1(uint32_t pixel_rate, uint8_t cpp,
wm_intermediate_val = latency * pixel_rate * cpp;
ret = div_fixed16(wm_intermediate_val, 1000 * 512);
if (INTEL_GEN(dev_priv) >= 10)
ret = add_fixed16_u32(ret, 1);
return ret;
}
@ -4377,7 +4406,9 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
}
y_tiled = fb->modifier == I915_FORMAT_MOD_Y_TILED ||
fb->modifier == I915_FORMAT_MOD_Yf_TILED;
fb->modifier == I915_FORMAT_MOD_Yf_TILED ||
fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS;
x_tiled = fb->modifier == I915_FORMAT_MOD_X_TILED;
/* Display WA #1141: kbl,cfl */
@ -4430,9 +4461,13 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
if (y_tiled) {
interm_pbpl = DIV_ROUND_UP(plane_bytes_per_line *
y_min_scanlines, 512);
if (INTEL_GEN(dev_priv) >= 10)
interm_pbpl++;
plane_blocks_per_line = div_fixed16(interm_pbpl,
y_min_scanlines);
} else if (x_tiled) {
} else if (x_tiled && INTEL_GEN(dev_priv) == 9) {
interm_pbpl = DIV_ROUND_UP(plane_bytes_per_line, 512);
plane_blocks_per_line = u32_to_fixed16(interm_pbpl);
} else {
@ -4440,7 +4475,7 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
plane_blocks_per_line = u32_to_fixed16(interm_pbpl);
}
method1 = skl_wm_method1(plane_pixel_rate, cpp, latency);
method1 = skl_wm_method1(dev_priv, plane_pixel_rate, cpp, latency);
method2 = skl_wm_method2(plane_pixel_rate,
cstate->base.adjusted_mode.crtc_htotal,
latency,
@ -4472,6 +4507,13 @@ static int skl_compute_plane_wm(const struct drm_i915_private *dev_priv,
res_lines = div_round_up_fixed16(selected_result,
plane_blocks_per_line);
/* Display WA #1125: skl,bxt,kbl,glk */
if (level == 0 &&
(fb->modifier == I915_FORMAT_MOD_Y_TILED_CCS ||
fb->modifier == I915_FORMAT_MOD_Yf_TILED_CCS))
res_blocks += fixed16_to_u32_round_up(y_tile_minimum);
/* Display WA #1126: skl,bxt,kbl,glk */
if (level >= 1 && level <= 7) {
if (y_tiled) {
res_blocks += fixed16_to_u32_round_up(y_tile_minimum);
@ -8831,6 +8873,7 @@ static inline int gen6_check_mailbox_status(struct drm_i915_private *dev_priv)
case GEN6_PCODE_SUCCESS:
return 0;
case GEN6_PCODE_UNIMPLEMENTED_CMD:
return -ENODEV;
case GEN6_PCODE_ILLEGAL_CMD:
return -ENXIO;
case GEN6_PCODE_MIN_FREQ_TABLE_GT_RATIO_OUT_OF_RANGE:
@ -8878,7 +8921,8 @@ int sandybridge_pcode_read(struct drm_i915_private *dev_priv, u32 mbox, u32 *val
*/
if (I915_READ_FW(GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) {
DRM_DEBUG_DRIVER("warning: pcode (read) mailbox access failed\n");
DRM_DEBUG_DRIVER("warning: pcode (read from mbox %x) mailbox access failed for %ps\n",
mbox, __builtin_return_address(0));
return -EAGAIN;
}
@ -8889,7 +8933,8 @@ int sandybridge_pcode_read(struct drm_i915_private *dev_priv, u32 mbox, u32 *val
if (__intel_wait_for_register_fw(dev_priv,
GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0,
500, 0, NULL)) {
DRM_ERROR("timeout waiting for pcode read (%d) to finish\n", mbox);
DRM_ERROR("timeout waiting for pcode read (from mbox %x) to finish for %ps\n",
mbox, __builtin_return_address(0));
return -ETIMEDOUT;
}
@ -8902,8 +8947,8 @@ int sandybridge_pcode_read(struct drm_i915_private *dev_priv, u32 mbox, u32 *val
status = gen6_check_mailbox_status(dev_priv);
if (status) {
DRM_DEBUG_DRIVER("warning: pcode (read) mailbox access failed: %d\n",
status);
DRM_DEBUG_DRIVER("warning: pcode (read from mbox %x) mailbox access failed for %ps: %d\n",
mbox, __builtin_return_address(0), status);
return status;
}
@ -8923,7 +8968,8 @@ int sandybridge_pcode_write(struct drm_i915_private *dev_priv,
*/
if (I915_READ_FW(GEN6_PCODE_MAILBOX) & GEN6_PCODE_READY) {
DRM_DEBUG_DRIVER("warning: pcode (write) mailbox access failed\n");
DRM_DEBUG_DRIVER("warning: pcode (write of 0x%08x to mbox %x) mailbox access failed for %ps\n",
val, mbox, __builtin_return_address(0));
return -EAGAIN;
}
@ -8934,7 +8980,8 @@ int sandybridge_pcode_write(struct drm_i915_private *dev_priv,
if (__intel_wait_for_register_fw(dev_priv,
GEN6_PCODE_MAILBOX, GEN6_PCODE_READY, 0,
500, 0, NULL)) {
DRM_ERROR("timeout waiting for pcode write (%d) to finish\n", mbox);
DRM_ERROR("timeout waiting for pcode write of 0x%08x to mbox %x to finish for %ps\n",
val, mbox, __builtin_return_address(0));
return -ETIMEDOUT;
}
@ -8946,8 +8993,8 @@ int sandybridge_pcode_write(struct drm_i915_private *dev_priv,
status = gen6_check_mailbox_status(dev_priv);
if (status) {
DRM_DEBUG_DRIVER("warning: pcode (write) mailbox access failed: %d\n",
status);
DRM_DEBUG_DRIVER("warning: pcode (write of 0x%08x to mbox %x) mailbox access failed for %ps: %d\n",
val, mbox, __builtin_return_address(0), status);
return status;
}

Some files were not shown because too many files have changed in this diff Show More