deepcrayon
/
tinyrocs
1
0
Fork 0
Code Issues Activity
tinyrocs/docs/_source/toolchain-6.rst

505 lines
13 KiB
ReStructuredText
Raw Permalink Blame History

=========
Toolchain
=========
The main GPU toolchain is built from AMD upstream ROCm sources.
This is still sadness and is "95%" done.
* LLVM.
* Clang.
* ROCm.
* ``amdgpu`` modules.
* GPU control and monitoring software.
AMD's ROCm source repositories contain most of the toolchain software
that needs to be built. Note, there are many equivalent packages in
Debian's main free repository, but they are older versions than needed
(including in unstable sid).
All of the replacements have yet to be built. ROCm version 6.0.0 is being
used, which was released in Q4 2023.
There seems to be a bit of a chicken and egg issue to get the full toolchain
going with all the various libraries.
The optimal package build order has yet to be determined.
rocm-core
---------
Perhaps start by building ``rocm-core``.
.. literalinclude:: _static/toolchain/rocm-6/build-rocm-core.sh
:language: bash
rocm-cmake
----------
Build ``rocm-cmake`` thusly.
.. literalinclude:: _static/toolchain/rocm-6/build-rocm-cmake.sh
:language: bash
At this point, applications like ``amdsmi`` can be built.
amdsmi
------
``amd-smi`` is probably the best application for displaying information
and controlling the Radeon 7900 XTX hardware.
.. literalinclude:: _static/toolchain/rocm-6/build-amd-smi.sh
:language: bash
rocm_smi_lib
------------
Not sure which is best here. One repo is listed as deprecated, but has
``rocm-6.0.0`` tag. The new repo lacks that tag. Building is foobar.
Some of the packages that had their own repo are now merged in a
branch in the ROCm LLVM repo. meh.
Maybe:
.. literalinclude:: _static/toolchain/rocm-6/build-rocm-smi-lib.sh
:language: bash
ROCm-Device-Libs
----------------
This is deprecated, but still probably best to use for ROCm version ``6.0.0``.
The newer version is in the ``amd-stg`` branches in AMD's LLVM fork,
but that has other issues with ``6.0.0`` release.
Needs llvm-project installed first.
Perhaps just copy the llvm-project in to place, but don't build (yet).
Maybe save one rebuild.
.. literalinclude:: _static/toolchain/rocm-6/build-rocm-device-libs.sh
:language: bash
The ``lld`` here isn't supported in this build.
Maybe need to rebuild this package after seconda pass of LLVM.
LLVM First Pass
---------------
Build first pass thusly.
LLVM is the main compiler project for ROCm.
Building LLVM is one of the key components.
It includes ``clang``, libraries, etc.
AMD has their own fork of the ``llvm-project`` for their hardware.
It is kind of a pain to build it all....
There are also other repositories that have similar libraries,
and it isn't always clear which should be used.
Some packages fail to build if a package maintainer or contact isn't specified.
Update as needed.
Configure LLVM thusly. This may be missing bits or have unneeded bits. TBD.
Install the new package and mark it hold in apt so Debian won't upgrade
over top of it.
Compiling LLVM will take two passes.
Once with a more minimal configuration.
Then other dependencies are compiled with that version of LLVM (``clang``).
Once those dependencies (e.g. amd-libs ?) are compiled,
LLVM can be recompiled with more options, such as AMDGPU support.
XXX Still borkenopolis.
After LLVM is built, build a few other packages.
Then LLVM gets re-built, using the new packages.
There is a circular dependency is why it is this way, afaict.
.. literalinclude:: _static/toolchain/rocm-6/build-llvm-pass-1.sh
:language: bash
ROCm-CompilerSupport ``comgr``
------------------------------
See LLVM for another option to build ``comgr``.
.. literalinclude:: _static/toolchain/rocm-6/build-rocm-compilersupport-comgr.sh
:language: bash
ROCT-Thunk-Interface
--------------------
Perhaps this next.
.. literalinclude:: _static/toolchain/rocm-6/build-roct-thunk-interface.sh
:language: bash
ROCR-Runtime
------------
Builds ``hsa-rocr-dev``.
This has ``TARGET_DEVICES`` of many cards by default, but not ``gfx1100``.
It has ``hsakmt_DIR``, maybe set to wrong one (needs rebuild?).
.. literalinclude:: _static/toolchain/rocm-6/build-rocr-runtime.sh
:language: bash
ROCm-Device-Libs Rebuild
------------------------
The main LLVM build is still foobar. Hmm.
Rebuild this thing.
.. literalinclude:: _static/toolchain/rocm-6/build-rocm-device-libs-pass-2.sh
:language: bash
The ``lld`` here isn't supported in this build.
Maybe need to rebuild this package after seconda pass of LLVM.
LLVM Second Pass
----------------
Now rebuild LLVM again, with above libraries now in place...
I guess move the old one out of the way, and move the
new one into place...
Second pass, rebuild with more options...
.. literalinclude:: _static/toolchain/rocm-6/build-llvm-pass-2.sh
:language: bash
HIPCC
-----
Build goes further off rails here.
HIP docs describe a building like below, but has old repos, broken names.
While ``hipcc`` can be built, the ``hip`` package itself remains
a problem to package.
Maybe:
.. code-block:: sh
export ROCM_BRANCH="6.0.0"
git clone -b "$ROCM_BRANCH" https://github.com/ROCm-Developer-Tools/clr.git
git clone -b "$ROCM_BRANCH" https://github.com/ROCm-Developer-Tools/HIP.git
git clone -b "$ROCM_BRANCH" https://github.com/ROCm-Developer-Tools/HIPCC.git
mv HIP hip
mv HIPCC hipcc
export CLR_DIR="$(readlink -f clr)"
export HIP_DIR="$(readlink -f hip)"
export HIPCC_DIR="$(readlink -f hipcc)"
# HIP
cd "$HIP_DIR"
export ROCM_BRANCH=rocm-6.0.x
git checkout rocm-6.0.0
# XXX Cruft, unused package names are hard-coded.
# grep -v CPACK_DEBIAN_PACKAGE_DEPENDS CMakeLists.txt > a ; mv a CMakeLists.txt
rm -rf build
# HIPCC
cd "$HIPCC_DIR"
git checkout rocm-6.0.0
# meh, perhaps put below....
cd ../clr
git checkout rocm-6.0.0
cd ../hipcc
git checkout rocm-6.0.0
# ROCM_DIR /opt/rocm/share/rocm/cmake
# in amd/hipcc
# XXX Cruft, unused package names are hard-coded.
grep -v -e CPACK_DEBIAN_AMD_PACKAGE_DEPENDS -e CPACK_DEBIAN_PACKAGE_DEPENDS CMakeLists.txt > a ; mv a CMakeLists.txt
rm -rf build
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DROCM_DIR=/opt/rocm \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TXZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-DCPACK_GENERATOR=DEB \
-DCPACK_DEBIAN_PACKAGE_DEPENDS="" \
-DCPACK_PACKAGE_CONTACT="Jeff Moe <moe@parrot.codes>" \
ninja -C build package
sudo dpkg -i build/hipcc_1.0.0.99999-local_amd64.deb
HIP
---
At present this is broken, but needed for some other packages.
.. code-block:: sh
cd "$CLR_DIR"
export ROCM_BRANCH=rocm-6.0.x
rm -rf build
# FAILS, needs amd_comgr_DIR
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DROCM_DIR=/opt/rocm \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-DCPACK_GENERATOR=DEB \
-DCPACK_DEBIAN_PACKAGE_DEPENDS="" \
-DCPACK_PACKAGE_CONTACT="Jeff Moe <moe@parrot.codes>" \
-DHIP_COMMON_DIR=$HIP_DIR \
-DHIP_PLATFORM=amd \
-DCMAKE_PREFIX_PATH="/opt/rocm/" \
-DCMAKE_INSTALL_PREFIX=$PWD/install \
-DHIPCC_BIN_DIR=$HIPCC_DIR/build \
-DHIP_CATCH_TEST=0 \
-DCLR_BUILD_HIP=ON \
-DCLR_BUILD_OCL=OFF \
ninja -C build package
HIPCC
-----
Another variation on ``hipcc`` installation, see above.
Not sure which is best practice.
.. code-block:: sh
git clone https://github.com/ROCm/HIPCC
cd HIPCC/
export ROCM_BRANCH=rocm-6.0.x
git checkout rocm-6.0.0
rm -rf build
# meh XXX
grep -v CPACK_DEBIAN_PACKAGE_DEPENDS CMakeLists.txt > a
mv a CMakeLists.txt
cmake -B build -G Ninja \
-DLLVM_TARGET_ARCH=host \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-DROCM_DIR=/opt/rocm \
-DCPACK_GENERATOR=DEB \
-DROCM_CCACHE_BUILD=ON \
-DCPACK_DEBIAN_PACKAGE_DEPENDS="" \
-DROCM_DEP_ROCMCORE=OFF
ninja -C build package
sudo dpkg -i build/hipcc_1.0.0.99999-local_amd64.deb
ROCm-CompilerSupport
--------------------
Perhaps build earlier.
See also various ``comgr``...
.. code-block:: sh
git clone https://github.com/ROCm/ROCm-CompilerSupport
cd ROCm-CompilerSupport/lib/comgr
export ROCM_BRANCH=rocm-6.0.x
git checkout rocm-6.0.0
rm -rf build
cmake -B build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-DROCM_DIR=/opt/rocm \
-DCPACK_GENERATOR=DEB \
-DROCM_CCACHE_BUILD=ON \
-DAMDDeviceLibs_DIR=/usr/lib/cmake/AMDDeviceLibs \
# -DAMDDeviceLibs_DIR=/usr/lib/x86_64-linux-gnu/cmake/AMDDeviceLibs \
ninja -C build package
sudo dpkg -i build/comgr_2.6.0.99999-local_amd64.deb
clr
---
Chaos.
opencl hip etc
.. code-block:: sh
git clone https://github.com/ROCm/clr
cd clr/
export ROCM_BRANCH=rocm-6.0.x
git checkout rocm-6.0.0
export ROCM_BRANCH=rocm-6.0.x
# export CLR_DIR="$(readlink -f clr)"
# export HIP_DIR="$(readlink -f hip)"
# export HIPCC_DIR="$(readlink -f hipcc)"
# meh
virtualenv venv
source venv/bin/activate
pip install -U setuptools pip wheel
pip install CppHeaderParser
# cmake -DHIP_COMMON_DIR=$HIP_DIR -DHIP_PLATFORM=amd -DCMAKE_PREFIX_PATH="/opt/rocm/" -DCMAKE_INSTALL_PREFIX=$PWD/install -DHIPCC_BIN_DIR=$HIPCC_DIR/build -DHIP_CATCH_TEST=0 -DCLR_BUILD_HIP=ON -DCLR_BUILD_OCL=OFF ..
rm -rf build
cmake -B build -G Ninja \
-DHIP_CATCH_TEST=0 \
-DCLR_BUILD_HIP=ON \
-DCLR_BUILD_OCL=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DHIP_COMMON_DIR=/home/jebba/devel/ROCm/hip \
-DCMAKE_PREFIX_PATH="/opt/rocm/" \
-DHIP_PLATFORM=amd \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-DCPACK_GENERATOR=DEB \
-DCPACK_PACKAGING_INSTALL_PREFIX=/opt/rocm \
-DCLR_BUILD_OCL=ON \
-DCLR_BUILD_HIP=ON \
-DROCM_PATH=/opt/rocm \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DHIPCC_BIN_DIR=/usr/hip/bin \
-DCLR_BUILD_HIP=ON \
-DUSE_PROF_API=OFF \
-DOpenGL_GL_PREFERENCE=GLVND \
ninja -C build package
-DOpenGL_GL_PREFERENCE=LEGACY \
# build fails?
-DROCCLR_ENABLE_HSAIL=ON \
-D__HIP_ENABLE_PCH=OFF \
# Need HIP to build hipBLAS etc
# -DCLR_BUILD_HIP=ON \
# WORKAROUND_ISSUE_HIPRTC_TRUE_TYPE
# -DHIP_COMMON_DIR=~/devel/ROCm/HIP/HIP \
# -DWORKAROUND_ISSUE_HIPRTC_TRUE_TYPE=ON \
clang-ocl
---------
opencl
.. code-block:: sh
git clone https://github.com/ROCm/clang-ocl
cd clang-ocl/
export ROCM_BRANCH=rocm-6.0.x
git checkout rocm-6.0.0
grep -v CPACK_DEBIAN_PACKAGE_DEPENDS CMakeLists.txt > a
mv a CMakeLists.txt
rm -rf build
cmake -B build -G Ninja \
-DLLVM_TARGET_ARCH=host \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-DROCM_DIR=/opt/rocm \
-DROCM_DEP_ROCMCORE=OFF \
-DCPACK_DEBIAN_PACKAGE_DEPENDS="" \
# Triple: amdgcn-amd-amdhsa
ninja -C build package
sudo dpkg -i build/rocm-clang-ocl_0.5.0-894802a~dirty_amd64.deb
hipBLAS
-------
Needed by other applications, like ``llama.cpp``.
.. code-block:: sh
git clone https://github.com/ROCm/hipBLAS
cd hipBLAS/
export ROCM_BRANCH=rocm-6.0.x
git checkout rocm-6.0.0
rm -rf build
cmake -B build -G Ninja \
-DLLVM_TARGET_ARCH=host \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-Dhip_DIR=/opt/rocm/share/rocm/cmake
rocminfo
--------
``rocminfo`` shows ROCm information.
.. code-block:: sh
git clone https://github.com/ROCm/rocminfo
cd rocminfo/
export ROCM_BRANCH=rocm-6.0.x
git checkout rocm-6.0.0
rm -rf build
cmake -B build -G Ninja \
-DLLVM_TARGET_ARCH=host \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/opt/rocm \
-DCPACK_BINARY_DEB=ON \
-DCPACK_BINARY_STGZ=OFF \
-DCPACK_BINARY_TGZ=OFF \
-DCPACK_BINARY_TZ=OFF \
-DCPACK_SOURCE_TBZ2=OFF \
-DCPACK_SOURCE_TGZ=OFF \
-DCPACK_SOURCE_TZ=OFF \
-DROCM_DIR=/opt/rocm \
-DCPACK_GENERATOR=DEB \
-DROCM_CCACHE_BUILD=ON
ninja -C build package
sudo dpkg -i build/rocminfo_1.0.0.99999-local_amd64.deb
rocminfo
Reference in New Issue View Git Blame Copy Permalink