Compare commits
No commits in common. "master" and "master-ci" have entirely different histories.
19
.clang-tidy
19
.clang-tidy
|
|
@ -1,19 +0,0 @@
|
|||
---
|
||||
Checks: '
|
||||
bugprone-*,
|
||||
-bugprone-integer-division,
|
||||
-bugprone-narrowing-conversions,
|
||||
performance-*,
|
||||
clang-analyzer-*,
|
||||
misc-*,
|
||||
-misc-unused-parameters,
|
||||
modernize-*,
|
||||
-modernize-avoid-c-arrays,
|
||||
-modernize-deprecated-headers,
|
||||
-modernize-use-auto,
|
||||
-modernize-use-using,
|
||||
-modernize-use-nullptr,
|
||||
-modernize-use-trailing-return-type,
|
||||
'
|
||||
CheckOptions:
|
||||
...
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
((c++-mode (flycheck-gcc-language-standard . "c++11")
|
||||
(flycheck-clang-language-standard . "c++11")
|
||||
))
|
||||
|
|
@ -1,38 +0,0 @@
|
|||
**/.git
|
||||
.DS_Store
|
||||
*.dylib
|
||||
*.DSYM
|
||||
*.d
|
||||
*.pyc
|
||||
*.pyo
|
||||
.*.swp
|
||||
.*.swo
|
||||
.*.un~
|
||||
*.tmp
|
||||
*.o
|
||||
*.o-*
|
||||
*.os
|
||||
*.os-*
|
||||
*.so
|
||||
*.a
|
||||
|
||||
notebooks
|
||||
phone
|
||||
massivemap
|
||||
neos
|
||||
installer
|
||||
chffr/app2
|
||||
chffr/backend/env
|
||||
selfdrive/nav
|
||||
selfdrive/baseui
|
||||
chffr/lib/vidindex/vidindex
|
||||
selfdrive/test/simulator2
|
||||
**/cache_data
|
||||
xx/chffr/lib/vidindex/vidindex
|
||||
xx/plus
|
||||
xx/community
|
||||
xx/projects
|
||||
!xx/projects/eon_testing_master
|
||||
!xx/projects/map3d
|
||||
xx/ops
|
||||
xx/junk
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
root = true
|
||||
|
||||
[*]
|
||||
end_of_line = lf
|
||||
insert_final_newline = true
|
||||
trim_trailing_whitespace = true
|
||||
|
||||
[{*.py, *.pyx, *.pxd}]
|
||||
charset = utf-8
|
||||
indent_style = space
|
||||
indent_size = 2
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
*.dlc filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
|
|
@ -1,58 +0,0 @@
|
|||
name: Bug report
|
||||
description: For issues with running openpilot on your comma device
|
||||
labels: ["bug"]
|
||||
body:
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Before creating a **bug report**, please check the following:
|
||||
* If the issue likely only affects your car model or make, go back and open a **car bug report** instead.
|
||||
* Ensure you're running the latest openpilot release.
|
||||
* Ensure you're using officially supported hardware. Issues running on PCs have a different issue template.
|
||||
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
|
||||
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
|
||||
|
||||
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Describe the bug
|
||||
description: Also include a description of how to reproduce the bug
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: hw
|
||||
attributes:
|
||||
label: What hardware does this issue affect?
|
||||
multiple: true
|
||||
options:
|
||||
- comma three
|
||||
- comma two
|
||||
- EON Gold
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: route
|
||||
attributes:
|
||||
label: Provide a route where the issue occurs
|
||||
description: Ensure the route is fully uploaded at https://useradmin.comma.ai
|
||||
placeholder: 77611a1fac303767|2020-05-11--16-37-07
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: openpilot version
|
||||
description: If you're not on release, provide the commit hash
|
||||
placeholder: 0.8.10
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Additional info
|
||||
|
||||
|
|
@ -1,65 +0,0 @@
|
|||
name: Car bug report
|
||||
description: For issues with a particular car make or model
|
||||
labels: ["car bug"]
|
||||
body:
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Before creating a **bug report**, please check the following:
|
||||
* Ensure you're running the latest openpilot release.
|
||||
* Ensure you're using officially supported hardware. Issues running on PCs have a different issue template.
|
||||
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
|
||||
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
|
||||
|
||||
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Describe the bug
|
||||
description: Also include a description of how to reproduce the bug
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: hw
|
||||
attributes:
|
||||
label: What hardware does this issue affect?
|
||||
multiple: true
|
||||
options:
|
||||
- comma three
|
||||
- comma two
|
||||
- EON Gold
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: car
|
||||
attributes:
|
||||
label: Which car does this affect?
|
||||
placeholder: Toyota Prius 2017
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: route
|
||||
attributes:
|
||||
label: Provide a route where the issue occurs
|
||||
description: Ensure the route is fully uploaded at https://useradmin.comma.ai
|
||||
placeholder: 77611a1fac303767|2020-05-11--16-37-07
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: openpilot version
|
||||
description: If you're not on release, provide the commit hash
|
||||
placeholder: 0.8.10
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Additional info
|
||||
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Discussions
|
||||
url: https://github.com/commaai/openpilot/discussions
|
||||
about: For questions and discussion about openpilot
|
||||
- name: Community Wiki
|
||||
url: https://github.com/commaai/openpilot/wiki
|
||||
about: Check out our community wiki
|
||||
- name: Community Discord
|
||||
url: https://discord.comma.ai
|
||||
about: Check out our community discord
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
---
|
||||
name: Enhancement
|
||||
about: For openpilot enhancement suggestions
|
||||
title: ''
|
||||
labels: 'enhancement'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
name: PC bug report
|
||||
description: For issues with running openpilot on PC
|
||||
labels: ["PC"]
|
||||
body:
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: >
|
||||
Before creating a **bug report**, please check the following:
|
||||
* Ensure you're running the latest openpilot release.
|
||||
* Ensure there isn't an existing issue for your bug. If there is, leave a comment on the existing issue.
|
||||
* Ensure you're running stock openpilot. We cannot look into bug reports from forks.
|
||||
|
||||
If you're unsure whether you've hit a bug, check out the #installation-help channel in the [community Discord server](https://discord.comma.ai).
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Describe the bug
|
||||
description: Also include a description of how to reproduce the bug
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: os-version
|
||||
attributes:
|
||||
label: OS Version
|
||||
placeholder: Ubuntu 20.04
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: openpilot version or commit
|
||||
placeholder: bd36f2ec8d3559909678eff2690c10a520938367
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Additional info
|
||||
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
---
|
||||
name: Bug fix
|
||||
about: For openpilot bug fixes
|
||||
title: ''
|
||||
labels: 'bugfix'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
**Description**
|
||||
|
||||
<!-- A description of the bug and the fix. Also link the issue if it exists. -->
|
||||
|
||||
**Verification**
|
||||
|
||||
<!-- Explain how you tested this bug fix. -->
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
---
|
||||
name: Car Bug fix
|
||||
about: For vehicle/brand specifc bug fixes
|
||||
title: ''
|
||||
labels: 'car bug fix'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
**Description**
|
||||
|
||||
<!-- A description of the bug and the fix. Also link the issue if it exists. -->
|
||||
|
||||
**Verification**
|
||||
|
||||
<!-- Explain how you tested this bug fix. -->
|
||||
|
||||
**Route**
|
||||
|
||||
Route: [a route with the bug fix]
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
---
|
||||
name: Car port
|
||||
about: For new car ports
|
||||
title: ''
|
||||
labels: 'car port'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
**Checklist**
|
||||
|
||||
- [ ] added to README
|
||||
- [ ] test route added to [routes.py](https://github.com/commaai/openpilot/blob/master/selfdrive/car/tests/routes.py)
|
||||
- [ ] route with openpilot:
|
||||
- [ ] route with stock system:
|
||||
- [ ] car harness used (if comma doesn't sell it, put N/A):
|
||||
|
|
@ -1,11 +0,0 @@
|
|||
---
|
||||
name: Fingerprint
|
||||
about: For adding fingerprints to existing cars
|
||||
title: ''
|
||||
labels: 'fingerprint'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
Discord username: []
|
||||
|
||||
Route: []
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
---
|
||||
name: Refactor
|
||||
about: For code refactors
|
||||
title: ''
|
||||
labels: 'refactor'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
**Description**
|
||||
|
||||
<!-- A description of the refactor, including the goals it accomplishes. -->
|
||||
|
||||
**Verification**
|
||||
|
||||
<!-- Explain how you tested the refactor for regressions. -->
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
---
|
||||
name: Tuning
|
||||
about: For openpilot tuning changes
|
||||
title: ''
|
||||
labels: 'tuning'
|
||||
assignees: ''
|
||||
---
|
||||
|
||||
**Description**
|
||||
|
||||
<!-- A description of what is wrong with the current tuning and how the PR addresses this. -->
|
||||
|
||||
**Verification**
|
||||
|
||||
<!-- To verify tuning, capture the following scenarios (broadly, not exactly), with current tune and this tune.
|
||||
Use the PlotJuggler tuning layout to compare planned versus actual behavior.
|
||||
|
||||
Run ./juggle.py <route> --layout layouts/tuning.xml , screenshot the full tab of interest, and paste into this PR.
|
||||
|
||||
Longitudinal:
|
||||
* Maintaining speed at 25, 40, 65mph
|
||||
* Driving up and down hills
|
||||
* Accelerating from a stop
|
||||
* Decelerating to a stop
|
||||
* Following large changes in set speed
|
||||
* Coming to a stop behind a lead car
|
||||
|
||||
Lateral:
|
||||
* Straight driving at ~25, ~45 and ~65mph
|
||||
* Turns driving at ~25, ~45 and ~65mph
|
||||
|
||||
-->
|
||||
|
|
@ -1,38 +0,0 @@
|
|||
<!-- Please copy and paste the relevant template -->
|
||||
|
||||
<!--- ***** Template: Car bug fix *****
|
||||
|
||||
**Description** [](A description of the bug and the fix. Also link any relevant issues.)
|
||||
|
||||
**Verification** [](Explain how you tested this bug fix.)
|
||||
|
||||
**Route**
|
||||
Route: [a route with the bug fix]
|
||||
|
||||
-->
|
||||
|
||||
<!--- ***** Template: Bug fix *****
|
||||
|
||||
**Description** [](A description of the bug and the fix. Also link any relevant issues.)
|
||||
|
||||
**Verification** [](Explain how you tested this bug fix.)
|
||||
|
||||
-->
|
||||
|
||||
<!--- ***** Template: Car port *****
|
||||
|
||||
**Checklist**
|
||||
- [ ] added entry to CarInfo in selfdrive/car/*/values.py and ran `selfdrive/car/docs.py` to generate new docs
|
||||
- [ ] test route added to [routes.py](https://github.com/commaai/openpilot/blob/master/selfdrive/car/tests/routes.py)
|
||||
- [ ] route with openpilot:
|
||||
- [ ] route with stock system:
|
||||
|
||||
-->
|
||||
|
||||
<!--- ***** Template: Refactor *****
|
||||
|
||||
**Description** [](A description of the refactor, including the goals it accomplishes.)
|
||||
|
||||
**Verification** [](Explain how you tested the refactor for regressions.)
|
||||
|
||||
-->
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
name: prebuilt
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 * * * *'
|
||||
|
||||
env:
|
||||
BASE_IMAGE: openpilot-base
|
||||
DOCKER_REGISTRY: ghcr.io/commaai
|
||||
|
||||
DOCKER_LOGIN: docker login ghcr.io -u adeebshihadeh -p ${{ secrets.CONTAINER_TOKEN }}
|
||||
BUILD: |
|
||||
docker pull $(grep -iohP '(?<=^from)\s+\S+' Dockerfile.openpilot_base) || true
|
||||
docker pull $DOCKER_REGISTRY/$BASE_IMAGE:latest || true
|
||||
docker build --cache-from $DOCKER_REGISTRY/$BASE_IMAGE:latest -t $DOCKER_REGISTRY/$BASE_IMAGE:latest -t $BASE_IMAGE:latest -f Dockerfile.openpilot_base .
|
||||
|
||||
jobs:
|
||||
build_prebuilt:
|
||||
name: build prebuilt
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 60
|
||||
if: github.repository == 'commaai/openpilot'
|
||||
env:
|
||||
IMAGE_NAME: openpilot-prebuilt
|
||||
steps:
|
||||
- name: Wait for green check mark
|
||||
uses: lewagon/wait-on-check-action@v0.2
|
||||
with:
|
||||
ref: master
|
||||
wait-interval: 30
|
||||
running-workflow-name: 'build prebuilt'
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Build Docker image
|
||||
run: |
|
||||
eval "$BUILD"
|
||||
docker pull $DOCKER_REGISTRY/$IMAGE_NAME:latest || true
|
||||
docker build --cache-from $DOCKER_REGISTRY/$IMAGE_NAME:latest -t $DOCKER_REGISTRY/$IMAGE_NAME:latest -f Dockerfile.openpilot .
|
||||
- name: Push to container registry
|
||||
run: |
|
||||
$DOCKER_LOGIN
|
||||
docker push $DOCKER_REGISTRY/$IMAGE_NAME:latest
|
||||
|
|
@ -1,431 +0,0 @@
|
|||
name: selfdrive
|
||||
on:
|
||||
push:
|
||||
branches-ignore:
|
||||
- 'testing-closet*'
|
||||
pull_request:
|
||||
|
||||
env:
|
||||
BASE_IMAGE: openpilot-base
|
||||
DOCKER_REGISTRY: ghcr.io/commaai
|
||||
|
||||
DOCKER_LOGIN: docker login ghcr.io -u adeebshihadeh -p ${{ secrets.CONTAINER_TOKEN }}
|
||||
BUILD: |
|
||||
docker pull $(grep -iohP '(?<=^from)\s+\S+' Dockerfile.openpilot_base) || true
|
||||
docker pull $DOCKER_REGISTRY/$BASE_IMAGE:latest || true
|
||||
docker build --cache-from $DOCKER_REGISTRY/$BASE_IMAGE:latest -t $DOCKER_REGISTRY/$BASE_IMAGE:latest -t $BASE_IMAGE:latest -f Dockerfile.openpilot_base .
|
||||
|
||||
RUN: docker run --shm-size 1G -v $PWD:/tmp/openpilot -w /tmp/openpilot -e PYTHONPATH=/tmp/openpilot -e NUM_JOBS -e JOB_ID -e GITHUB_ACTION -e GITHUB_REF -e GITHUB_HEAD_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_RUN_ID -v /tmp/scons_cache:/tmp/scons_cache -v /tmp/comma_download_cache:/tmp/comma_download_cache $BASE_IMAGE /bin/sh -c
|
||||
UNIT_TEST: coverage run --append -m unittest discover
|
||||
|
||||
jobs:
|
||||
# TODO: once actions/cache supports read only mode, use the cache for all jobs
|
||||
build_release:
|
||||
name: build release
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
env:
|
||||
STRIPPED_DIR: tmppilot
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Check submodules
|
||||
if: github.ref == 'refs/heads/master' && github.repository == 'commaai/openpilot'
|
||||
run: release/check-submodules.sh
|
||||
- name: Cache scons
|
||||
id: scons-cache
|
||||
# TODO: Change the version to the released version when https://github.com/actions/cache/pull/489 (or 571) is merged.
|
||||
uses: actions/cache@03e00da99d75a2204924908e1cca7902cafce66b
|
||||
env:
|
||||
CACHE_SKIP_SAVE: true
|
||||
with:
|
||||
path: /tmp/scons_cache
|
||||
key: scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
restore-keys: |
|
||||
scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
scons-
|
||||
- name: Strip non-release files
|
||||
run: |
|
||||
mkdir $STRIPPED_DIR
|
||||
cp -pR --parents $(cat release/files_common) $STRIPPED_DIR
|
||||
cp -pR --parents $(cat release/files_tici) $STRIPPED_DIR
|
||||
cp -pR --parents $(cat release/files_pc) $STRIPPED_DIR
|
||||
cp Dockerfile.openpilot_base $STRIPPED_DIR
|
||||
|
||||
# need this to build on x86
|
||||
cp -pR --parents third_party/libyuv third_party/snpe selfdrive/modeld/runners $STRIPPED_DIR
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Build openpilot and run checks
|
||||
run: |
|
||||
cd $STRIPPED_DIR
|
||||
${{ env.RUN }} "CI=1 python selfdrive/manager/build.py && \
|
||||
python -m unittest discover selfdrive/car"
|
||||
- name: Cleanup scons cache
|
||||
run: |
|
||||
cd $STRIPPED_DIR
|
||||
${{ env.RUN }} "scons -j$(nproc) && \
|
||||
rm -rf /tmp/scons_cache/* && \
|
||||
scons -j$(nproc) --cache-populate"
|
||||
|
||||
#build_mac:
|
||||
# name: build macos
|
||||
# runs-on: macos-latest
|
||||
# timeout-minutes: 60
|
||||
# steps:
|
||||
# - uses: actions/checkout@v2
|
||||
# with:
|
||||
# submodules: true
|
||||
# - name: Determine pre-existing Homebrew packages
|
||||
# if: steps.dependency-cache.outputs.cache-hit != 'true'
|
||||
# run: |
|
||||
# echo 'EXISTING_CELLAR<<EOF' >> $GITHUB_ENV
|
||||
# ls -1 /usr/local/Cellar >> $GITHUB_ENV
|
||||
# echo 'EOF' >> $GITHUB_ENV
|
||||
# - name: Cache dependencies
|
||||
# id: dependency-cache
|
||||
# uses: actions/cache@v2
|
||||
# with:
|
||||
# path: |
|
||||
# ~/.pyenv
|
||||
# ~/.local/share/virtualenvs/
|
||||
# /usr/local/Cellar
|
||||
# ~/github_brew_cache_entries.txt
|
||||
# /tmp/scons_cache
|
||||
# key: macos-${{ hashFiles('tools/mac_setup.sh', 'update_requirements.sh', 'Pipfile*') }}
|
||||
# restore-keys: macos-
|
||||
# - name: Brew link restored dependencies
|
||||
# run: |
|
||||
# if [ -f ~/github_brew_cache_entries.txt ]; then
|
||||
# while read pkg; do
|
||||
# brew link --force "$pkg" # `--force` for keg-only packages
|
||||
# done < ~/github_brew_cache_entries.txt
|
||||
# else
|
||||
# echo "Cache entries not found"
|
||||
# fi
|
||||
# - name: Install dependencies
|
||||
# run: ./tools/mac_setup.sh
|
||||
# - name: Build openpilot
|
||||
# run: |
|
||||
# source tools/openpilot_env.sh
|
||||
# pipenv run selfdrive/manager/build.py
|
||||
#
|
||||
# # cleanup scons cache
|
||||
# rm -rf /tmp/scons_cache/
|
||||
# pipenv run scons -j$(nproc) --cache-populate
|
||||
# - name: Remove pre-existing Homebrew packages for caching
|
||||
# if: steps.dependency-cache.outputs.cache-hit != 'true'
|
||||
# run: |
|
||||
# cd /usr/local/Cellar
|
||||
# new_cellar=$(ls -1)
|
||||
# comm -12 <(echo "$EXISTING_CELLAR") <(echo "$new_cellar") | while read pkg; do
|
||||
# if [[ $pkg != "zstd" ]]; then # caching step needs zstd
|
||||
# rm -rf "$pkg"
|
||||
# fi
|
||||
# done
|
||||
# comm -13 <(echo "$EXISTING_CELLAR") <(echo "$new_cellar") | tee ~/github_brew_cache_entries.txt
|
||||
|
||||
build_webcam:
|
||||
name: build webcam
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 90
|
||||
env:
|
||||
IMAGE_NAME: openpilotwebcamci
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Build Docker image
|
||||
run: |
|
||||
eval "$BUILD"
|
||||
docker pull $DOCKER_REGISTRY/$IMAGE_NAME:latest || true
|
||||
docker build --cache-from $DOCKER_REGISTRY/$IMAGE_NAME:latest -t $DOCKER_REGISTRY/$IMAGE_NAME:latest -f tools/webcam/Dockerfile .
|
||||
- name: Build openpilot
|
||||
run: docker run --shm-size 1G --rm -v $PWD:/tmp/openpilot -e PYTHONPATH=/tmp/openpilot $DOCKER_REGISTRY/$IMAGE_NAME /bin/sh -c "cd /tmp/openpilot && USE_WEBCAM=1 scons -j$(nproc)"
|
||||
- name: Push to container registry
|
||||
if: github.ref == 'refs/heads/master' && github.repository == 'commaai/openpilot'
|
||||
run: |
|
||||
$DOCKER_LOGIN
|
||||
docker push $DOCKER_REGISTRY/$IMAGE_NAME:latest
|
||||
|
||||
docker_push:
|
||||
name: docker push
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
if: github.ref == 'refs/heads/master' && github.event_name != 'pull_request' && github.repository == 'commaai/openpilot'
|
||||
needs: static_analysis # hack to ensure slow tests run first since this and static_analysis are fast
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Push to container registry
|
||||
run: |
|
||||
$DOCKER_LOGIN
|
||||
docker push $DOCKER_REGISTRY/$BASE_IMAGE:latest
|
||||
|
||||
static_analysis:
|
||||
name: static analysis
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: pre-commit
|
||||
run: ${{ env.RUN }} "git init && git add -A && pre-commit run --all"
|
||||
|
||||
valgrind:
|
||||
name: valgrind
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Cache dependencies
|
||||
id: dependency-cache
|
||||
uses: actions/cache@v2
|
||||
with:
|
||||
path: /tmp/comma_download_cache
|
||||
key: ${{ hashFiles('.github/workflows/selfdrive_tests.yaml', 'selfdrive/test/test_valgrind_replay.py') }}
|
||||
- name: Cache scons
|
||||
id: scons-cache
|
||||
# TODO: Change the version to the released version when https://github.com/actions/cache/pull/489 (or 571) is merged.
|
||||
uses: actions/cache@03e00da99d75a2204924908e1cca7902cafce66b
|
||||
env:
|
||||
CACHE_SKIP_SAVE: true
|
||||
with:
|
||||
path: /tmp/scons_cache
|
||||
key: scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
restore-keys: |
|
||||
scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
scons-
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Run valgrind
|
||||
run: |
|
||||
${{ env.RUN }} "scons -j$(nproc) && \
|
||||
FILEREADER_CACHE=1 python selfdrive/test/test_valgrind_replay.py"
|
||||
- name: Print logs
|
||||
if: always()
|
||||
run: cat selfdrive/test/valgrind_logs.txt
|
||||
|
||||
unit_tests:
|
||||
name: unit tests
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
steps:
|
||||
- name: Get current date
|
||||
id: date
|
||||
run: echo "::set-output name=time::$(date +'%s')"
|
||||
- name: Output timestamp
|
||||
run: echo $TIMESTAMP
|
||||
env:
|
||||
TIMESTAMP: ${{ steps.date.outputs.time }}
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Cache scons
|
||||
id: scons-cache
|
||||
# TODO: Change the version to the released version when https://github.com/actions/cache/pull/489 (or 571) is merged.
|
||||
uses: actions/cache@03e00da99d75a2204924908e1cca7902cafce66b
|
||||
env:
|
||||
CACHE_SKIP_SAVE: ${{ github.ref != 'refs/heads/master' || github.repository != 'commaai/openpilot' }}
|
||||
with:
|
||||
path: /tmp/scons_cache
|
||||
key: scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-${{ steps.date.outputs.time }}
|
||||
restore-keys: scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Run unit tests
|
||||
run: |
|
||||
${{ env.RUN }} "export SKIP_LONG_TESTS=1 && \
|
||||
scons -j$(nproc) --test && \
|
||||
$UNIT_TEST common && \
|
||||
$UNIT_TEST opendbc/can && \
|
||||
$UNIT_TEST selfdrive/boardd && \
|
||||
$UNIT_TEST selfdrive/controls && \
|
||||
$UNIT_TEST selfdrive/monitoring && \
|
||||
$UNIT_TEST selfdrive/loggerd && \
|
||||
$UNIT_TEST selfdrive/car && \
|
||||
$UNIT_TEST selfdrive/locationd && \
|
||||
$UNIT_TEST selfdrive/athena && \
|
||||
$UNIT_TEST selfdrive/thermald && \
|
||||
$UNIT_TEST selfdrive/hardware/tici && \
|
||||
$UNIT_TEST tools/lib/tests && \
|
||||
./selfdrive/boardd/tests/test_boardd_usbprotocol && \
|
||||
./selfdrive/common/tests/test_util && \
|
||||
./selfdrive/common/tests/test_swaglog && \
|
||||
./selfdrive/loggerd/tests/test_logger &&\
|
||||
./selfdrive/proclogd/tests/test_proclog && \
|
||||
./selfdrive/ui/replay/tests/test_replay && \
|
||||
./selfdrive/camerad/test/ae_gray_test && \
|
||||
coverage xml"
|
||||
- name: "Upload coverage to Codecov"
|
||||
uses: codecov/codecov-action@v2
|
||||
|
||||
process_replay:
|
||||
name: process replay
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Cache dependencies
|
||||
id: dependency-cache
|
||||
uses: actions/cache@v2
|
||||
with:
|
||||
path: /tmp/comma_download_cache
|
||||
key: ${{ hashFiles('.github/workflows/selfdrive_tests.yaml', 'selfdrive/test/process_replay/test_processes.py') }}
|
||||
- name: Cache scons
|
||||
id: scons-cache
|
||||
# TODO: Change the version to the released version when https://github.com/actions/cache/pull/489 (or 571) is merged.
|
||||
uses: actions/cache@03e00da99d75a2204924908e1cca7902cafce66b
|
||||
env:
|
||||
CACHE_SKIP_SAVE: true
|
||||
with:
|
||||
path: /tmp/scons_cache
|
||||
key: scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
restore-keys: |
|
||||
scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
scons-
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Run replay
|
||||
run: |
|
||||
${{ env.RUN }} "scons -j$(nproc) && \
|
||||
FILEREADER_CACHE=1 CI=1 coverage run selfdrive/test/process_replay/test_processes.py && \
|
||||
coverage xml"
|
||||
- name: "Upload coverage to Codecov"
|
||||
uses: codecov/codecov-action@v2
|
||||
- name: Print diff
|
||||
if: always()
|
||||
run: cat selfdrive/test/process_replay/diff.txt
|
||||
- uses: actions/upload-artifact@v2
|
||||
if: always()
|
||||
continue-on-error: true
|
||||
with:
|
||||
name: process_replay_diff.txt
|
||||
path: selfdrive/test/process_replay/diff.txt
|
||||
|
||||
#model_replay:
|
||||
# name: model replay
|
||||
# runs-on: ubuntu-20.04
|
||||
# timeout-minutes: 50
|
||||
# steps:
|
||||
# - uses: actions/checkout@v2
|
||||
# with:
|
||||
# submodules: true
|
||||
# - name: Build Docker image
|
||||
# run: eval "$BUILD"
|
||||
# - name: Run replay
|
||||
# run: |
|
||||
# ${{ env.RUN }} "scons -j$(nproc) && \
|
||||
# selfdrive/test/process_replay/model_replay.py"
|
||||
|
||||
test_longitudinal:
|
||||
name: longitudinal
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Cache scons
|
||||
id: scons-cache
|
||||
# TODO: Change the version to the released version when https://github.com/actions/cache/pull/489 (or 571) is merged.
|
||||
uses: actions/cache@03e00da99d75a2204924908e1cca7902cafce66b
|
||||
env:
|
||||
CACHE_SKIP_SAVE: true
|
||||
with:
|
||||
path: /tmp/scons_cache
|
||||
key: scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
restore-keys: |
|
||||
scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
scons-
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Test longitudinal
|
||||
run: |
|
||||
${{ env.RUN }} "mkdir -p selfdrive/test/out && \
|
||||
scons -j$(nproc) && \
|
||||
cd selfdrive/test/longitudinal_maneuvers && \
|
||||
coverage run ./test_longitudinal.py && \
|
||||
coverage xml"
|
||||
- name: "Upload coverage to Codecov"
|
||||
uses: codecov/codecov-action@v2
|
||||
- uses: actions/upload-artifact@v2
|
||||
if: always()
|
||||
continue-on-error: true
|
||||
with:
|
||||
name: longitudinal
|
||||
path: selfdrive/test/longitudinal_maneuvers/out/longitudinal/
|
||||
|
||||
test_cars:
|
||||
name: cars
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
job: [0, 1, 2, 3]
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Cache dependencies
|
||||
id: dependency-cache
|
||||
uses: actions/cache@v2
|
||||
with:
|
||||
path: /tmp/comma_download_cache
|
||||
key: car_models-${{ hashFiles('selfdrive/car/tests/test_models.py', 'selfdrive/car/tests/routes.py') }}-${{ matrix.job }}
|
||||
- name: Cache scons
|
||||
id: scons-cache
|
||||
# TODO: Change the version to the released version when https://github.com/actions/cache/pull/489 (or 571) is merged.
|
||||
uses: actions/cache@03e00da99d75a2204924908e1cca7902cafce66b
|
||||
env:
|
||||
CACHE_SKIP_SAVE: true
|
||||
with:
|
||||
path: /tmp/scons_cache
|
||||
key: scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
restore-keys: |
|
||||
scons-${{ hashFiles('.github/workflows/selfdrive_tests.yaml') }}-
|
||||
scons-
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Test car models
|
||||
run: |
|
||||
${{ env.RUN }} "scons -j$(nproc) --test && \
|
||||
FILEREADER_CACHE=1 coverage run -m pytest selfdrive/car/tests/test_models.py && \
|
||||
coverage xml && \
|
||||
chmod -R 777 /tmp/comma_download_cache"
|
||||
env:
|
||||
NUM_JOBS: 4
|
||||
JOB_ID: ${{ matrix.job }}
|
||||
- name: "Upload coverage to Codecov"
|
||||
uses: codecov/codecov-action@v2
|
||||
|
||||
docs:
|
||||
name: build docs
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Build docker container
|
||||
run: |
|
||||
docker pull $DOCKER_REGISTRY/$BASE_IMAGE:latest || true
|
||||
docker pull $DOCKER_REGISTRY/openpilot-docs:latest || true
|
||||
DOCKER_BUILDKIT=1 docker build --cache-from $DOCKER_REGISTRY/openpilot-docs:latest -t $DOCKER_REGISTRY/openpilot-docs:latest -f docs/docker/Dockerfile .
|
||||
- name: Push docker container
|
||||
if: github.ref == 'refs/heads/master' && github.event_name != 'pull_request' && github.repository == 'commaai/openpilot'
|
||||
run: |
|
||||
$DOCKER_LOGIN
|
||||
docker push $DOCKER_REGISTRY/openpilot-docs:latest
|
||||
|
|
@ -1,71 +0,0 @@
|
|||
name: tools
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
|
||||
env:
|
||||
BASE_IMAGE: openpilot-base
|
||||
DOCKER_REGISTRY: ghcr.io/commaai
|
||||
DOCKER_LOGIN: docker login ghcr.io -u adeebshihadeh -p ${{ secrets.CONTAINER_TOKEN }}
|
||||
|
||||
BUILD: |
|
||||
docker pull $(grep -iohP '(?<=^from)\s+\S+' Dockerfile.openpilot_base) || true
|
||||
docker pull $DOCKER_REGISTRY/$BASE_IMAGE:latest || true
|
||||
docker build --cache-from $DOCKER_REGISTRY/$BASE_IMAGE:latest -t $DOCKER_REGISTRY/$BASE_IMAGE:latest -t $BASE_IMAGE:latest -f Dockerfile.openpilot_base .
|
||||
RUN: docker run --shm-size 1G -v $PWD:/tmp/openpilot -e PYTHONPATH=/tmp/openpilot -e GITHUB_ACTION -e GITHUB_REF -e GITHUB_HEAD_REF -e GITHUB_SHA -e \
|
||||
GITHUB_REPOSITORY -e GITHUB_RUN_ID -v /tmp/comma_download_cache:/tmp/comma_download_cache $BASE_IMAGE /bin/sh -c
|
||||
|
||||
jobs:
|
||||
plotjuggler:
|
||||
name: plotjuggler
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
- name: Build Docker image
|
||||
run: eval "$BUILD"
|
||||
- name: Unit test
|
||||
run: |
|
||||
${{ env.RUN }} "scons -j$(nproc) --directory=/tmp/openpilot/cereal && \
|
||||
apt-get update && \
|
||||
apt-get install -y libdw-dev libqt5svg5-dev libqt5x11extras5-dev && \
|
||||
cd /tmp/openpilot/tools/plotjuggler && \
|
||||
./test_plotjuggler.py"
|
||||
|
||||
simulator:
|
||||
name: simulator
|
||||
runs-on: ubuntu-20.04
|
||||
timeout-minutes: 50
|
||||
env:
|
||||
IMAGE_NAME: openpilot-sim
|
||||
if: github.repository == 'commaai/openpilot'
|
||||
steps:
|
||||
- uses: actions/checkout@v2
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
# HACK: cache LFS objects since they count against our quota
|
||||
# https://github.com/actions/checkout/issues/165#issuecomment-657673315
|
||||
- name: Create LFS file list
|
||||
run: git lfs ls-files -l | cut -d' ' -f1 | sort > .lfs-assets-id
|
||||
- name: Restore LFS cache
|
||||
uses: actions/cache@v2
|
||||
id: lfs-cache
|
||||
with:
|
||||
path: .git/lfs
|
||||
key: ${{ runner.os }}-lfs-${{ hashFiles('.lfs-assets-id') }}
|
||||
- name: Git LFS Pull
|
||||
run: git lfs pull
|
||||
|
||||
- name: Build Docker image
|
||||
run: |
|
||||
eval "$BUILD"
|
||||
docker pull $DOCKER_REGISTRY/$IMAGE_NAME:latest || true
|
||||
docker build --cache-from $DOCKER_REGISTRY/$IMAGE_NAME:latest -t $DOCKER_REGISTRY/$IMAGE_NAME:latest -f tools/sim/Dockerfile.sim .
|
||||
- name: Push to container registry
|
||||
if: github.ref == 'refs/heads/master' && github.repository == 'commaai/openpilot'
|
||||
run: |
|
||||
$DOCKER_LOGIN
|
||||
docker push $DOCKER_REGISTRY/$IMAGE_NAME:latest
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
[submodule "panda"]
|
||||
path = panda
|
||||
url = ../../commaai/panda.git
|
||||
[submodule "opendbc"]
|
||||
path = opendbc
|
||||
url = ../../commaai/opendbc.git
|
||||
[submodule "laika_repo"]
|
||||
path = laika_repo
|
||||
url = ../../commaai/laika.git
|
||||
[submodule "cereal"]
|
||||
path = cereal
|
||||
url = ../../commaai/cereal.git
|
||||
[submodule "rednose_repo"]
|
||||
path = rednose_repo
|
||||
url = ../../commaai/rednose.git
|
||||
|
|
@ -1,4 +0,0 @@
|
|||
[lfs]
|
||||
url = https://gitlab.com/commaai/openpilot-lfs.git/info/lfs
|
||||
pushurl = ssh://git@gitlab.com/commaai/openpilot-lfs.git
|
||||
locksverify = false
|
||||
|
|
@ -1,58 +0,0 @@
|
|||
repos:
|
||||
- repo: meta
|
||||
hooks:
|
||||
- id: check-hooks-apply
|
||||
- id: check-useless-excludes
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.1.0
|
||||
hooks:
|
||||
- id: check-ast
|
||||
exclude: '^(pyextra)/'
|
||||
- id: check-json
|
||||
- id: check-xml
|
||||
- id: check-yaml
|
||||
- id: check-merge-conflict
|
||||
- id: check-symlinks
|
||||
- id: check-added-large-files
|
||||
args: ['--maxkb=100']
|
||||
- repo: https://github.com/pre-commit/mirrors-mypy
|
||||
rev: v0.931
|
||||
hooks:
|
||||
- id: mypy
|
||||
exclude: '^(pyextra/)|(cereal/)|(rednose/)|(panda/)|(laika/)|(opendbc/)|(laika_repo/)|(rednose_repo/)/'
|
||||
additional_dependencies: ['git+https://github.com/numpy/numpy-stubs', 'types-requests', 'types-atomicwrites',
|
||||
'types-pycurl']
|
||||
- repo: https://github.com/PyCQA/flake8
|
||||
rev: 4.0.1
|
||||
hooks:
|
||||
- id: flake8
|
||||
exclude: '^(pyextra/)|(cereal/)|(rednose/)|(panda/)|(laika/)|(opendbc/)|(laika_repo/)|(rednose_repo/)|(selfdrive/debug/)/'
|
||||
additional_dependencies: ['flake8-no-implicit-concat']
|
||||
args:
|
||||
- --indent-size=2
|
||||
- --enable-extensions=NIC
|
||||
- --select=F,E112,E113,E304,E502,E701,E702,E703,E71,E72,E731,W191,W6
|
||||
- --statistics
|
||||
- -j4
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: pylint
|
||||
name: pylint
|
||||
entry: pylint
|
||||
language: system
|
||||
types: [python]
|
||||
exclude: '^(pyextra/)|(cereal/)|(rednose/)|(panda/)|(laika/)|(laika_repo/)|(rednose_repo/)'
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: cppcheck
|
||||
name: cppcheck
|
||||
entry: cppcheck
|
||||
language: system
|
||||
types: [c++]
|
||||
exclude: '^(third_party/)|(pyextra/)|(cereal/)|(opendbc/)|(panda/)|(tools/)|(selfdrive/modeld/thneed/debug/)|(selfdrive/modeld/test/)|(selfdrive/camerad/test/)/|(installer/)'
|
||||
args:
|
||||
- --error-exitcode=1
|
||||
- --language=c++
|
||||
- --quiet
|
||||
- --force
|
||||
- -j8
|
||||
476
.pylintrc
476
.pylintrc
|
|
@ -1,476 +0,0 @@
|
|||
[MASTER]
|
||||
|
||||
# A comma-separated list of package or module names from where C extensions may
|
||||
# be loaded. Extensions are loading into the active Python interpreter and may
|
||||
# run arbitrary code
|
||||
extension-pkg-whitelist=scipy cereal.messaging.messaging_pyx
|
||||
|
||||
# Add files or directories to the blacklist. They should be base names, not
|
||||
# paths.
|
||||
ignore=CVS
|
||||
|
||||
# Add files or directories matching the regex patterns to the blacklist. The
|
||||
# regex matches against base names, not paths.
|
||||
ignore-patterns=
|
||||
|
||||
# Python code to execute, usually for sys.path manipulation such as
|
||||
# pygtk.require().
|
||||
#init-hook=
|
||||
|
||||
# Use multiple processes to speed up Pylint.
|
||||
jobs=4
|
||||
|
||||
# List of plugins (as comma separated values of python modules names) to load,
|
||||
# usually to register additional checkers.
|
||||
load-plugins=
|
||||
|
||||
# Pickle collected data for later comparisons.
|
||||
persistent=yes
|
||||
|
||||
# Specify a configuration file.
|
||||
#rcfile=
|
||||
|
||||
# When enabled, pylint would attempt to guess common misconfiguration and emit
|
||||
# user-friendly hints instead of false-positive error messages
|
||||
suggestion-mode=yes
|
||||
|
||||
# Allow loading of arbitrary C extensions. Extensions are imported into the
|
||||
# active Python interpreter and may run arbitrary code.
|
||||
unsafe-load-any-extension=no
|
||||
|
||||
|
||||
[MESSAGES CONTROL]
|
||||
|
||||
# Only show warnings with the listed confidence levels. Leave empty to show
|
||||
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
|
||||
confidence=
|
||||
|
||||
# Disable the message, report, category or checker with the given id(s). You
|
||||
# can either give multiple identifiers separated by comma (,) or put this
|
||||
# option multiple times (only on the command line, not in the configuration
|
||||
# file where it should appear only once).You can also use "--disable=all" to
|
||||
# disable everything first and then reenable specific checks. For example, if
|
||||
# you want to run only the similarities checker, you can use "--disable=all
|
||||
# --enable=similarities". If you want to run only the classes checker, but have
|
||||
# no Warning level messages displayed, use"--disable=all --enable=classes
|
||||
# --disable=W"
|
||||
disable=C,R,W0613,W0511,W0212,W0201,W0106,W0603,W0621,W0703,W1201,W1203,E1136,W1514
|
||||
|
||||
|
||||
# Enable the message, report, category or checker with the given id(s). You can
|
||||
# either give multiple identifier separated by comma (,) or put this option
|
||||
# multiple time (only on the command line, not in the configuration file where
|
||||
# it should appear only once). See also the "--disable" option for examples.
|
||||
enable=c-extension-no-member
|
||||
|
||||
|
||||
[REPORTS]
|
||||
|
||||
# Python expression which should return a note less than 10 (10 is the highest
|
||||
# note). You have access to the variables errors warning, statement which
|
||||
# respectively contain the number of errors / warnings messages and the total
|
||||
# number of statements analyzed. This is used by the global evaluation report
|
||||
# (RP0004).
|
||||
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
|
||||
|
||||
# Template used to display messages. This is a python new-style format string
|
||||
# used to format the message information. See doc for all details
|
||||
#msg-template=
|
||||
|
||||
# Set the output format. Available formats are text, parseable, colorized, json
|
||||
# and msvs (visual studio).You can also give a reporter class, eg
|
||||
# mypackage.mymodule.MyReporterClass.
|
||||
output-format=text
|
||||
|
||||
# Tells whether to display a full report or only the messages
|
||||
reports=no
|
||||
|
||||
# Activate the evaluation score.
|
||||
score=yes
|
||||
|
||||
|
||||
[REFACTORING]
|
||||
|
||||
# Maximum number of nested blocks for function / method body
|
||||
max-nested-blocks=5
|
||||
|
||||
# Complete name of functions that never returns. When checking for
|
||||
# inconsistent-return-statements if a never returning function is called then
|
||||
# it will be considered as an explicit return statement and no message will be
|
||||
# printed.
|
||||
never-returning-functions=optparse.Values,sys.exit
|
||||
|
||||
|
||||
[LOGGING]
|
||||
|
||||
# Logging modules to check that the string format arguments are in logging
|
||||
# function parameter format
|
||||
logging-modules=logging
|
||||
|
||||
|
||||
[SPELLING]
|
||||
|
||||
# Limits count of emitted suggestions for spelling mistakes
|
||||
max-spelling-suggestions=4
|
||||
|
||||
# Spelling dictionary name. Available dictionaries: none. To make it working
|
||||
# install python-enchant package.
|
||||
spelling-dict=
|
||||
|
||||
# List of comma separated words that should not be checked.
|
||||
spelling-ignore-words=
|
||||
|
||||
# A path to a file that contains private dictionary; one word per line.
|
||||
spelling-private-dict-file=
|
||||
|
||||
# Tells whether to store unknown words to indicated private dictionary in
|
||||
# --spelling-private-dict-file option instead of raising a message.
|
||||
spelling-store-unknown-words=no
|
||||
|
||||
|
||||
[MISCELLANEOUS]
|
||||
|
||||
# List of note tags to take in consideration, separated by a comma.
|
||||
notes=FIXME,
|
||||
XXX,
|
||||
TODO
|
||||
|
||||
|
||||
[SIMILARITIES]
|
||||
|
||||
# Ignore comments when computing similarities.
|
||||
ignore-comments=yes
|
||||
|
||||
# Ignore docstrings when computing similarities.
|
||||
ignore-docstrings=yes
|
||||
|
||||
# Ignore imports when computing similarities.
|
||||
ignore-imports=no
|
||||
|
||||
# Minimum lines number of a similarity.
|
||||
min-similarity-lines=4
|
||||
|
||||
|
||||
[TYPECHECK]
|
||||
|
||||
# List of decorators that produce context managers, such as
|
||||
# contextlib.contextmanager. Add to this list to register other decorators that
|
||||
# produce valid context managers.
|
||||
contextmanager-decorators=contextlib.contextmanager
|
||||
|
||||
# List of members which are set dynamically and missed by pylint inference
|
||||
# system, and so shouldn't trigger E1101 when accessed. Python regular
|
||||
# expressions are accepted.
|
||||
generated-members=capnp.* cereal.* pygame.* zmq.* setproctitle.* smbus2.* usb1.* serial.* cv2.* ft4222.* carla.*
|
||||
|
||||
# Tells whether missing members accessed in mixin class should be ignored. A
|
||||
# mixin class is detected if its name ends with "mixin" (case insensitive).
|
||||
ignore-mixin-members=yes
|
||||
|
||||
# This flag controls whether pylint should warn about no-member and similar
|
||||
# checks whenever an opaque object is returned when inferring. The inference
|
||||
# can return multiple potential results while evaluating a Python object, but
|
||||
# some branches might not be evaluated, which results in partial inference. In
|
||||
# that case, it might be useful to still emit no-member and other checks for
|
||||
# the rest of the inferred objects.
|
||||
ignore-on-opaque-inference=yes
|
||||
|
||||
# List of class names for which member attributes should not be checked (useful
|
||||
# for classes with dynamically set attributes). This supports the use of
|
||||
# qualified names.
|
||||
ignored-classes=optparse.Values,thread._local,_thread._local
|
||||
|
||||
# List of module names for which member attributes should not be checked
|
||||
# (useful for modules/projects where namespaces are manipulated during runtime
|
||||
# and thus existing member attributes cannot be deduced by static analysis. It
|
||||
# supports qualified module names, as well as Unix pattern matching.
|
||||
ignored-modules=flask setproctitle usb1 flask.ext.socketio smbus2 usb1.*
|
||||
|
||||
# Show a hint with possible names when a member name was not found. The aspect
|
||||
# of finding the hint is based on edit distance.
|
||||
missing-member-hint=yes
|
||||
|
||||
# The minimum edit distance a name should have in order to be considered a
|
||||
# similar match for a missing member name.
|
||||
missing-member-hint-distance=1
|
||||
|
||||
# The total number of similar names that should be taken in consideration when
|
||||
# showing a hint for a missing member.
|
||||
missing-member-max-choices=1
|
||||
|
||||
|
||||
[VARIABLES]
|
||||
|
||||
# List of additional names supposed to be defined in builtins. Remember that
|
||||
# you should avoid to define new builtins when possible.
|
||||
additional-builtins=
|
||||
|
||||
# Tells whether unused global variables should be treated as a violation.
|
||||
allow-global-unused-variables=yes
|
||||
|
||||
# List of strings which can identify a callback function by name. A callback
|
||||
# name must start or end with one of those strings.
|
||||
callbacks=cb_,
|
||||
_cb
|
||||
|
||||
# A regular expression matching the name of dummy variables (i.e. expectedly
|
||||
# not used).
|
||||
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
|
||||
|
||||
# Argument names that match this expression will be ignored. Default to name
|
||||
# with leading underscore
|
||||
ignored-argument-names=_.*|^ignored_|^unused_
|
||||
|
||||
# Tells whether we should check for unused import in __init__ files.
|
||||
init-import=no
|
||||
|
||||
# List of qualified module names which can have objects that can redefine
|
||||
# builtins.
|
||||
redefining-builtins-modules=six.moves,past.builtins,future.builtins
|
||||
|
||||
|
||||
[FORMAT]
|
||||
|
||||
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
|
||||
expected-line-ending-format=
|
||||
|
||||
# Regexp for a line that is allowed to be longer than the limit.
|
||||
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
|
||||
|
||||
# Number of spaces of indent required inside a hanging or continued line.
|
||||
indent-after-paren=4
|
||||
|
||||
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
|
||||
# tab).
|
||||
indent-string=' '
|
||||
|
||||
# Maximum number of characters on a single line.
|
||||
max-line-length=100
|
||||
|
||||
# Maximum number of lines in a module
|
||||
max-module-lines=1000
|
||||
|
||||
# List of optional constructs for which whitespace checking is disabled. `dict-
|
||||
# separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
|
||||
# `trailing-comma` allows a space between comma and closing bracket: (a, ).
|
||||
# `empty-line` allows space-only lines.
|
||||
no-space-check=trailing-comma,
|
||||
dict-separator
|
||||
|
||||
# Allow the body of a class to be on the same line as the declaration if body
|
||||
# contains single statement.
|
||||
single-line-class-stmt=no
|
||||
|
||||
# Allow the body of an if to be on the same line as the test if there is no
|
||||
# else.
|
||||
single-line-if-stmt=no
|
||||
|
||||
|
||||
[BASIC]
|
||||
|
||||
# Naming style matching correct argument names
|
||||
argument-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct argument names. Overrides argument-
|
||||
# naming-style
|
||||
#argument-rgx=
|
||||
|
||||
# Naming style matching correct attribute names
|
||||
attr-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct attribute names. Overrides attr-naming-
|
||||
# style
|
||||
#attr-rgx=
|
||||
|
||||
# Bad variable names which should always be refused, separated by a comma
|
||||
bad-names=foo,
|
||||
bar,
|
||||
baz,
|
||||
toto,
|
||||
tutu,
|
||||
tata
|
||||
|
||||
# Naming style matching correct class attribute names
|
||||
class-attribute-naming-style=any
|
||||
|
||||
# Regular expression matching correct class attribute names. Overrides class-
|
||||
# attribute-naming-style
|
||||
#class-attribute-rgx=
|
||||
|
||||
# Naming style matching correct class names
|
||||
class-naming-style=PascalCase
|
||||
|
||||
# Regular expression matching correct class names. Overrides class-naming-style
|
||||
#class-rgx=
|
||||
|
||||
# Naming style matching correct constant names
|
||||
const-naming-style=UPPER_CASE
|
||||
|
||||
# Regular expression matching correct constant names. Overrides const-naming-
|
||||
# style
|
||||
#const-rgx=
|
||||
|
||||
# Minimum line length for functions/classes that require docstrings, shorter
|
||||
# ones are exempt.
|
||||
docstring-min-length=-1
|
||||
|
||||
# Naming style matching correct function names
|
||||
function-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct function names. Overrides function-
|
||||
# naming-style
|
||||
#function-rgx=
|
||||
|
||||
# Good variable names which should always be accepted, separated by a comma
|
||||
good-names=i,
|
||||
j,
|
||||
k,
|
||||
ex,
|
||||
Run,
|
||||
_
|
||||
|
||||
# Include a hint for the correct naming format with invalid-name
|
||||
include-naming-hint=no
|
||||
|
||||
# Naming style matching correct inline iteration names
|
||||
inlinevar-naming-style=any
|
||||
|
||||
# Regular expression matching correct inline iteration names. Overrides
|
||||
# inlinevar-naming-style
|
||||
#inlinevar-rgx=
|
||||
|
||||
# Naming style matching correct method names
|
||||
method-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct method names. Overrides method-naming-
|
||||
# style
|
||||
#method-rgx=
|
||||
|
||||
# Naming style matching correct module names
|
||||
module-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct module names. Overrides module-naming-
|
||||
# style
|
||||
#module-rgx=
|
||||
|
||||
# Colon-delimited sets of names that determine each other's naming style when
|
||||
# the name regexes allow several styles.
|
||||
name-group=
|
||||
|
||||
# Regular expression which should only match function or class names that do
|
||||
# not require a docstring.
|
||||
no-docstring-rgx=^_
|
||||
|
||||
# List of decorators that produce properties, such as abc.abstractproperty. Add
|
||||
# to this list to register other decorators that produce valid properties.
|
||||
property-classes=abc.abstractproperty
|
||||
|
||||
# Naming style matching correct variable names
|
||||
variable-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct variable names. Overrides variable-
|
||||
# naming-style
|
||||
#variable-rgx=
|
||||
|
||||
|
||||
[DESIGN]
|
||||
|
||||
# Maximum number of arguments for function / method
|
||||
max-args=5
|
||||
|
||||
# Maximum number of attributes for a class (see R0902).
|
||||
max-attributes=7
|
||||
|
||||
# Maximum number of boolean expressions in a if statement
|
||||
max-bool-expr=5
|
||||
|
||||
# Maximum number of branch for function / method body
|
||||
max-branches=12
|
||||
|
||||
# Maximum number of locals for function / method body
|
||||
max-locals=15
|
||||
|
||||
# Maximum number of parents for a class (see R0901).
|
||||
max-parents=7
|
||||
|
||||
# Maximum number of public methods for a class (see R0904).
|
||||
max-public-methods=20
|
||||
|
||||
# Maximum number of return / yield for function / method body
|
||||
max-returns=6
|
||||
|
||||
# Maximum number of statements in function / method body
|
||||
max-statements=50
|
||||
|
||||
# Minimum number of public methods for a class (see R0903).
|
||||
min-public-methods=2
|
||||
|
||||
|
||||
[CLASSES]
|
||||
|
||||
# List of method names used to declare (i.e. assign) instance attributes.
|
||||
defining-attr-methods=__init__,
|
||||
__new__,
|
||||
setUp
|
||||
|
||||
# List of member names, which should be excluded from the protected access
|
||||
# warning.
|
||||
exclude-protected=_asdict,
|
||||
_fields,
|
||||
_replace,
|
||||
_source,
|
||||
_make
|
||||
|
||||
# List of valid names for the first argument in a class method.
|
||||
valid-classmethod-first-arg=cls
|
||||
|
||||
# List of valid names for the first argument in a metaclass class method.
|
||||
valid-metaclass-classmethod-first-arg=mcs
|
||||
|
||||
|
||||
[IMPORTS]
|
||||
|
||||
# Allow wildcard imports from modules that define __all__.
|
||||
allow-wildcard-with-all=no
|
||||
|
||||
# Analyse import fallback blocks. This can be used to support both Python 2 and
|
||||
# 3 compatible code, which means that the block might have code that exists
|
||||
# only in one or another interpreter, leading to false positives when analysed.
|
||||
analyse-fallback-blocks=no
|
||||
|
||||
# Deprecated modules which should not be used, separated by a comma
|
||||
deprecated-modules=regsub,
|
||||
TERMIOS,
|
||||
Bastion,
|
||||
rexec
|
||||
|
||||
# Create a graph of external dependencies in the given file (report RP0402 must
|
||||
# not be disabled)
|
||||
ext-import-graph=
|
||||
|
||||
# Create a graph of every (i.e. internal and external) dependencies in the
|
||||
# given file (report RP0402 must not be disabled)
|
||||
import-graph=
|
||||
|
||||
# Create a graph of internal dependencies in the given file (report RP0402 must
|
||||
# not be disabled)
|
||||
int-import-graph=
|
||||
|
||||
# Force import order to recognize a module as part of the standard
|
||||
# compatibility libraries.
|
||||
known-standard-library=
|
||||
|
||||
# Force import order to recognize a module as part of a third party library.
|
||||
known-third-party=enchant
|
||||
|
||||
[STRING]
|
||||
|
||||
# This flag controls whether the implicit-str-concat should generate a warning
|
||||
# on implicit string concatenation in sequences defined over several lines.
|
||||
check-str-concat-over-line-jumps=yes
|
||||
|
||||
[EXCEPTIONS]
|
||||
|
||||
# Exceptions that will emit a warning when being caught. Defaults to
|
||||
# "Exception"
|
||||
overgeneral-exceptions=Exception
|
||||
|
|
@ -1 +0,0 @@
|
|||
3.8.10
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
FROM ghcr.io/commaai/openpilot-base:latest
|
||||
|
||||
ENV PYTHONUNBUFFERED 1
|
||||
|
||||
ENV OPENPILOT_PATH /home/batman/openpilot/
|
||||
ENV PYTHONPATH ${OPENPILOT_PATH}:${PYTHONPATH}
|
||||
|
||||
RUN mkdir -p ${OPENPILOT_PATH}
|
||||
WORKDIR ${OPENPILOT_PATH}
|
||||
|
||||
COPY Pipfile Pipfile.lock $OPENPILOT_PATH
|
||||
RUN pip install --no-cache-dir pipenv==2021.5.29 pip==21.3.1 && \
|
||||
pipenv install --system --deploy --dev --clear && \
|
||||
pip uninstall -y pipenv
|
||||
|
||||
COPY SConstruct ${OPENPILOT_PATH}
|
||||
|
||||
COPY ./pyextra ${OPENPILOT_PATH}/pyextra
|
||||
COPY ./third_party ${OPENPILOT_PATH}/third_party
|
||||
COPY ./site_scons ${OPENPILOT_PATH}/site_scons
|
||||
COPY ./laika ${OPENPILOT_PATH}/laika
|
||||
COPY ./laika_repo ${OPENPILOT_PATH}/laika_repo
|
||||
COPY ./rednose ${OPENPILOT_PATH}/rednose
|
||||
COPY ./tools ${OPENPILOT_PATH}/tools
|
||||
COPY ./release ${OPENPILOT_PATH}/release
|
||||
COPY ./common ${OPENPILOT_PATH}/common
|
||||
COPY ./opendbc ${OPENPILOT_PATH}/opendbc
|
||||
COPY ./cereal ${OPENPILOT_PATH}/cereal
|
||||
COPY ./panda ${OPENPILOT_PATH}/panda
|
||||
COPY ./selfdrive ${OPENPILOT_PATH}/selfdrive
|
||||
|
||||
RUN scons -j$(nproc)
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
FROM ubuntu:20.04
|
||||
|
||||
ENV PYTHONUNBUFFERED 1
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends sudo tzdata locales && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen
|
||||
ENV LANG en_US.UTF-8
|
||||
ENV LANGUAGE en_US:en
|
||||
ENV LC_ALL en_US.UTF-8
|
||||
|
||||
ENV PIPENV_SYSTEM=1
|
||||
ENV PYENV_VERSION=3.8.10
|
||||
ENV PYENV_ROOT="/root/.pyenv"
|
||||
ENV PATH="$PYENV_ROOT/bin:$PYENV_ROOT/shims:$PATH"
|
||||
|
||||
COPY Pipfile Pipfile.lock .python-version update_requirements.sh /tmp/
|
||||
COPY tools/ubuntu_setup.sh /tmp/tools/
|
||||
RUN cd /tmp && \
|
||||
tools/ubuntu_setup.sh && \
|
||||
rm -rf /var/lib/apt/lists/* && \
|
||||
rm -rf /tmp/* && \
|
||||
rm -rf /root/.cache && \
|
||||
pip uninstall -y pipenv && \
|
||||
|
||||
# remove unused architectures from gcc for panda
|
||||
cd /usr/lib/gcc/arm-none-eabi/9.2.1 && \
|
||||
rm -rf arm/ && \
|
||||
rm -rf thumb/nofp thumb/v6* thumb/v8* thumb/v7+fp thumb/v7-r+fp.sp
|
||||
83
Pipfile
83
Pipfile
|
|
@ -1,83 +0,0 @@
|
|||
[[source]]
|
||||
name = "pypi"
|
||||
url = "https://pypi.org/simple"
|
||||
verify_ssl = true
|
||||
|
||||
[dev-packages]
|
||||
control = "*"
|
||||
coverage = "*"
|
||||
dictdiffer = "*"
|
||||
fastcluster = "*"
|
||||
hexdump = "*"
|
||||
hypothesis = "*"
|
||||
inputs = "*"
|
||||
lru-dict = "*"
|
||||
markdown-it-py = "*"
|
||||
matplotlib = "*"
|
||||
mypy = "*"
|
||||
myst-parser = "*"
|
||||
numpy = "*"
|
||||
opencv-python-headless = "*"
|
||||
parameterized = "*"
|
||||
paramiko = "*"
|
||||
pprofile = "*"
|
||||
pre-commit = "*"
|
||||
pycurl = "*"
|
||||
pygame = "*"
|
||||
pyprof2calltree = "*"
|
||||
pytest = "*"
|
||||
pytest-xdist = "*"
|
||||
reverse_geocoder = "*"
|
||||
scipy = "*"
|
||||
sphinx = "*"
|
||||
sphinx-sitemap = "*"
|
||||
sphinx-rtd-theme = "*"
|
||||
breathe = "*"
|
||||
subprocess32 = "*"
|
||||
tenacity = "*"
|
||||
|
||||
[packages]
|
||||
atomicwrites = "*"
|
||||
casadi = {version = "*", markers="platform_system != 'Darwin'"}
|
||||
cffi = "*"
|
||||
crcmod = "*"
|
||||
cryptography = "*"
|
||||
Cython = "*"
|
||||
flake8 = "*"
|
||||
Flask = "*"
|
||||
future-fstrings = "*" # for acados
|
||||
gunicorn = "*"
|
||||
hexdump = "*"
|
||||
Jinja2 = "*"
|
||||
json-rpc = "*"
|
||||
libusb1 = "*"
|
||||
nose = "*"
|
||||
numpy = "*"
|
||||
onnx = "*"
|
||||
onnxruntime-gpu = {version = "*", markers="platform_system != 'Darwin'"}
|
||||
pillow = "*"
|
||||
psutil = "*"
|
||||
pycapnp = "==1.1.0"
|
||||
pycryptodome = "*"
|
||||
PyJWT = "*"
|
||||
pylint = "*"
|
||||
pyopencl = "*"
|
||||
pyserial = "*"
|
||||
python-dateutil = "*"
|
||||
PyYAML = "*"
|
||||
pyzmq = "*"
|
||||
requests = "*"
|
||||
scons = "*"
|
||||
sentry-sdk = "*"
|
||||
setproctitle = "*"
|
||||
six = "*"
|
||||
smbus2 = "*"
|
||||
sympy = "!=1.6.1"
|
||||
timezonefinder = "*"
|
||||
tqdm = "*"
|
||||
urllib3 = "*"
|
||||
utm = "*"
|
||||
websocket_client = "*"
|
||||
|
||||
[requires]
|
||||
python_version = "3.8"
|
||||
File diff suppressed because it is too large
Load Diff
1
cereal
1
cereal
|
|
@ -1 +0,0 @@
|
|||
Subproject commit a818779f1a55c6bf53fa0692e31111585662b722
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
gen
|
||||
node_modules
|
||||
package-lock.json
|
||||
*.tmp
|
||||
*.pyc
|
||||
__pycache__
|
||||
.*.swp
|
||||
.*.swo
|
||||
*.os
|
||||
*.o
|
||||
*.a
|
||||
|
||||
test_runner
|
||||
|
||||
libmessaging.*
|
||||
libmessaging_shared.*
|
||||
services.h
|
||||
.sconsign.dblite
|
||||
libcereal_shared.*
|
||||
.mypy_cache/
|
||||
catch2/
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
Import('env', 'envCython', 'arch', 'common')
|
||||
|
||||
import shutil
|
||||
|
||||
cereal_dir = Dir('.')
|
||||
gen_dir = Dir('gen')
|
||||
messaging_dir = Dir('messaging')
|
||||
|
||||
# Build cereal
|
||||
|
||||
schema_files = ['log.capnp', 'car.capnp', 'legacy.capnp']
|
||||
env.Command(["gen/c/include/c++.capnp.h"], [], "mkdir -p " + gen_dir.path + "/c/include && touch $TARGETS")
|
||||
env.Command([f'gen/cpp/{s}.c++' for s in schema_files] + [f'gen/cpp/{s}.h' for s in schema_files],
|
||||
schema_files,
|
||||
f"capnpc --src-prefix={cereal_dir.path} $SOURCES -o c++:{gen_dir.path}/cpp/")
|
||||
|
||||
# TODO: remove non shared cereal and messaging
|
||||
cereal_objects = env.SharedObject([f'gen/cpp/{s}.c++' for s in schema_files])
|
||||
|
||||
env.Library('cereal', cereal_objects)
|
||||
env.SharedLibrary('cereal_shared', cereal_objects)
|
||||
|
||||
# Build messaging
|
||||
|
||||
services_h = env.Command(['services.h'], ['services.py'], 'python3 ' + cereal_dir.path + '/services.py > $TARGET')
|
||||
|
||||
messaging_objects = env.SharedObject([
|
||||
'messaging/messaging.cc',
|
||||
'messaging/impl_zmq.cc',
|
||||
'messaging/impl_msgq.cc',
|
||||
'messaging/msgq.cc',
|
||||
'messaging/socketmaster.cc',
|
||||
])
|
||||
|
||||
messaging_lib = env.Library('messaging', messaging_objects)
|
||||
Depends('messaging/impl_zmq.cc', services_h)
|
||||
|
||||
env.Program('messaging/bridge', ['messaging/bridge.cc'], LIBS=[messaging_lib, 'zmq', common])
|
||||
Depends('messaging/bridge.cc', services_h)
|
||||
|
||||
envCython.Program('messaging/messaging_pyx.so', 'messaging/messaging_pyx.pyx', LIBS=envCython["LIBS"]+[messaging_lib, "zmq", common])
|
||||
|
||||
|
||||
# Build Vision IPC
|
||||
vipc_sources = [
|
||||
'visionipc/ipc.cc',
|
||||
'visionipc/visionipc_server.cc',
|
||||
'visionipc/visionipc_client.cc',
|
||||
'visionipc/visionbuf.cc',
|
||||
]
|
||||
|
||||
if arch in ["aarch64", "larch64"]:
|
||||
vipc_sources += ['visionipc/visionbuf_ion.cc']
|
||||
else:
|
||||
vipc_sources += ['visionipc/visionbuf_cl.cc']
|
||||
|
||||
vipc_objects = env.SharedObject(vipc_sources)
|
||||
vipc = env.Library('visionipc', vipc_objects)
|
||||
|
||||
|
||||
vipc_frameworks = []
|
||||
vipc_libs = envCython["LIBS"] + [vipc, messaging_lib, common, "zmq"]
|
||||
if arch == "Darwin":
|
||||
vipc_frameworks.append('OpenCL')
|
||||
else:
|
||||
if arch == "aarch64":
|
||||
vipc_libs.append('adreno_utils')
|
||||
vipc_libs.append('OpenCL')
|
||||
envCython.Program('visionipc/visionipc_pyx.so', 'visionipc/visionipc_pyx.pyx',
|
||||
LIBS=vipc_libs, FRAMEWORKS=vipc_frameworks)
|
||||
|
||||
if GetOption('test'):
|
||||
env.Program('messaging/test_runner', ['messaging/test_runner.cc', 'messaging/msgq_tests.cc'], LIBS=[messaging_lib, common])
|
||||
|
||||
env.Program('visionipc/test_runner', ['visionipc/test_runner.cc', 'visionipc/visionipc_tests.cc'],
|
||||
LIBS=['pthread'] + vipc_libs, FRAMEWORKS=vipc_frameworks)
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
# pylint: skip-file
|
||||
import os
|
||||
import capnp
|
||||
|
||||
CEREAL_PATH = os.path.dirname(os.path.abspath(__file__))
|
||||
capnp.remove_import_hook()
|
||||
|
||||
log = capnp.load(os.path.join(CEREAL_PATH, "log.capnp"))
|
||||
car = capnp.load(os.path.join(CEREAL_PATH, "car.capnp"))
|
||||
|
|
@ -0,0 +1,616 @@
|
|||
using Cxx = import "./include/c++.capnp";
|
||||
$Cxx.namespace("cereal");
|
||||
|
||||
@0x8e2af1e708af8b8d;
|
||||
|
||||
# ******* events causing controls state machine transition *******
|
||||
|
||||
struct CarEvent @0x9b1657f34caf3ad3 {
|
||||
name @0 :EventName;
|
||||
|
||||
# event types
|
||||
enable @1 :Bool;
|
||||
noEntry @2 :Bool;
|
||||
warning @3 :Bool; # alerts presented only when enabled or soft disabling
|
||||
userDisable @4 :Bool;
|
||||
softDisable @5 :Bool;
|
||||
immediateDisable @6 :Bool;
|
||||
preEnable @7 :Bool;
|
||||
permanent @8 :Bool; # alerts presented regardless of openpilot state
|
||||
|
||||
enum EventName @0xbaa8c5d505f727de {
|
||||
canError @0;
|
||||
steerUnavailable @1;
|
||||
brakeUnavailable @2;
|
||||
wrongGear @4;
|
||||
doorOpen @5;
|
||||
seatbeltNotLatched @6;
|
||||
espDisabled @7;
|
||||
wrongCarMode @8;
|
||||
steerTempUnavailable @9;
|
||||
reverseGear @10;
|
||||
buttonCancel @11;
|
||||
buttonEnable @12;
|
||||
pedalPressed @13;
|
||||
cruiseDisabled @14;
|
||||
speedTooLow @17;
|
||||
outOfSpace @18;
|
||||
overheat @19;
|
||||
calibrationIncomplete @20;
|
||||
calibrationInvalid @21;
|
||||
controlsMismatch @22;
|
||||
pcmEnable @23;
|
||||
pcmDisable @24;
|
||||
noTarget @25;
|
||||
radarFault @26;
|
||||
brakeHold @28;
|
||||
parkBrake @29;
|
||||
manualRestart @30;
|
||||
lowSpeedLockout @31;
|
||||
plannerError @32;
|
||||
joystickDebug @34;
|
||||
steerTempUnavailableSilent @35;
|
||||
resumeRequired @36;
|
||||
preDriverDistracted @37;
|
||||
promptDriverDistracted @38;
|
||||
driverDistracted @39;
|
||||
preDriverUnresponsive @43;
|
||||
promptDriverUnresponsive @44;
|
||||
driverUnresponsive @45;
|
||||
belowSteerSpeed @46;
|
||||
lowBattery @48;
|
||||
vehicleModelInvalid @50;
|
||||
accFaulted @51;
|
||||
sensorDataInvalid @52;
|
||||
commIssue @53;
|
||||
tooDistracted @54;
|
||||
posenetInvalid @55;
|
||||
soundsUnavailable @56;
|
||||
preLaneChangeLeft @57;
|
||||
preLaneChangeRight @58;
|
||||
laneChange @59;
|
||||
lowMemory @63;
|
||||
stockAeb @64;
|
||||
ldw @65;
|
||||
carUnrecognized @66;
|
||||
invalidLkasSetting @69;
|
||||
speedTooHigh @70;
|
||||
laneChangeBlocked @71;
|
||||
relayMalfunction @72;
|
||||
gasPressed @73;
|
||||
stockFcw @74;
|
||||
startup @75;
|
||||
startupNoCar @76;
|
||||
startupNoControl @77;
|
||||
startupMaster @78;
|
||||
startupNoFw @104;
|
||||
fcw @79;
|
||||
steerSaturated @80;
|
||||
belowEngageSpeed @84;
|
||||
noGps @85;
|
||||
wrongCruiseMode @87;
|
||||
modeldLagging @89;
|
||||
deviceFalling @90;
|
||||
fanMalfunction @91;
|
||||
cameraMalfunction @92;
|
||||
gpsMalfunction @94;
|
||||
processNotRunning @95;
|
||||
dashcamMode @96;
|
||||
controlsInitializing @98;
|
||||
usbError @99;
|
||||
roadCameraError @100;
|
||||
driverCameraError @101;
|
||||
wideRoadCameraError @102;
|
||||
localizerMalfunction @103;
|
||||
highCpuUsage @105;
|
||||
cruiseMismatch @106;
|
||||
lkasDisabled @107;
|
||||
|
||||
radarCanErrorDEPRECATED @15;
|
||||
communityFeatureDisallowedDEPRECATED @62;
|
||||
radarCommIssueDEPRECATED @67;
|
||||
driverMonitorLowAccDEPRECATED @68;
|
||||
gasUnavailableDEPRECATED @3;
|
||||
dataNeededDEPRECATED @16;
|
||||
modelCommIssueDEPRECATED @27;
|
||||
ipasOverrideDEPRECATED @33;
|
||||
geofenceDEPRECATED @40;
|
||||
driverMonitorOnDEPRECATED @41;
|
||||
driverMonitorOffDEPRECATED @42;
|
||||
calibrationProgressDEPRECATED @47;
|
||||
invalidGiraffeHondaDEPRECATED @49;
|
||||
invalidGiraffeToyotaDEPRECATED @60;
|
||||
internetConnectivityNeededDEPRECATED @61;
|
||||
whitePandaUnsupportedDEPRECATED @81;
|
||||
commIssueWarningDEPRECATED @83;
|
||||
focusRecoverActiveDEPRECATED @86;
|
||||
neosUpdateRequiredDEPRECATED @88;
|
||||
modelLagWarningDEPRECATED @93;
|
||||
startupOneplusDEPRECATED @82;
|
||||
startupFuzzyFingerprintDEPRECATED @97;
|
||||
}
|
||||
}
|
||||
|
||||
# ******* main car state @ 100hz *******
|
||||
# all speeds in m/s
|
||||
|
||||
struct CarState {
|
||||
events @13 :List(CarEvent);
|
||||
|
||||
# car speed
|
||||
vEgo @1 :Float32; # best estimate of speed
|
||||
aEgo @16 :Float32; # best estimate of acceleration
|
||||
vEgoRaw @17 :Float32; # unfiltered speed from CAN sensors
|
||||
yawRate @22 :Float32; # best estimate of yaw rate
|
||||
standstill @18 :Bool;
|
||||
wheelSpeeds @2 :WheelSpeeds;
|
||||
|
||||
# gas pedal, 0.0-1.0
|
||||
gas @3 :Float32; # this is user pedal only
|
||||
gasPressed @4 :Bool; # this is user pedal only
|
||||
|
||||
# brake pedal, 0.0-1.0
|
||||
brake @5 :Float32; # this is user pedal only
|
||||
brakePressed @6 :Bool; # this is user pedal only
|
||||
parkingBrake @39 :Bool;
|
||||
brakeHoldActive @38 :Bool;
|
||||
|
||||
# steering wheel
|
||||
steeringAngleDeg @7 :Float32;
|
||||
steeringAngleOffsetDeg @37 :Float32; # Offset betweens sensors in case there multiple
|
||||
steeringRateDeg @15 :Float32;
|
||||
steeringTorque @8 :Float32; # TODO: standardize units
|
||||
steeringTorqueEps @27 :Float32; # TODO: standardize units
|
||||
steeringPressed @9 :Bool; # if the user is using the steering wheel
|
||||
steeringRateLimited @29 :Bool; # if the torque is limited by the rate limiter
|
||||
steerFaultTemporary @35 :Bool; # temporary EPS fault
|
||||
steerFaultPermanent @36 :Bool; # permanent EPS fault
|
||||
stockAeb @30 :Bool;
|
||||
stockFcw @31 :Bool;
|
||||
espDisabled @32 :Bool;
|
||||
|
||||
# cruise state
|
||||
cruiseState @10 :CruiseState;
|
||||
|
||||
# gear
|
||||
gearShifter @14 :GearShifter;
|
||||
|
||||
# button presses
|
||||
buttonEvents @11 :List(ButtonEvent);
|
||||
leftBlinker @20 :Bool;
|
||||
rightBlinker @21 :Bool;
|
||||
genericToggle @23 :Bool;
|
||||
|
||||
# lock info
|
||||
doorOpen @24 :Bool;
|
||||
seatbeltUnlatched @25 :Bool;
|
||||
canValid @26 :Bool;
|
||||
|
||||
# clutch (manual transmission only)
|
||||
clutchPressed @28 :Bool;
|
||||
|
||||
# which packets this state came from
|
||||
canMonoTimes @12: List(UInt64);
|
||||
|
||||
# blindspot sensors
|
||||
leftBlindspot @33 :Bool; # Is there something blocking the left lane change
|
||||
rightBlindspot @34 :Bool; # Is there something blocking the right lane change
|
||||
|
||||
struct WheelSpeeds {
|
||||
# optional wheel speeds
|
||||
fl @0 :Float32;
|
||||
fr @1 :Float32;
|
||||
rl @2 :Float32;
|
||||
rr @3 :Float32;
|
||||
}
|
||||
|
||||
struct CruiseState {
|
||||
enabled @0 :Bool;
|
||||
speed @1 :Float32;
|
||||
available @2 :Bool;
|
||||
speedOffset @3 :Float32;
|
||||
standstill @4 :Bool;
|
||||
nonAdaptive @5 :Bool;
|
||||
}
|
||||
|
||||
enum GearShifter {
|
||||
unknown @0;
|
||||
park @1;
|
||||
drive @2;
|
||||
neutral @3;
|
||||
reverse @4;
|
||||
sport @5;
|
||||
low @6;
|
||||
brake @7;
|
||||
eco @8;
|
||||
manumatic @9;
|
||||
}
|
||||
|
||||
# send on change
|
||||
struct ButtonEvent {
|
||||
pressed @0 :Bool;
|
||||
type @1 :Type;
|
||||
|
||||
enum Type {
|
||||
unknown @0;
|
||||
leftBlinker @1;
|
||||
rightBlinker @2;
|
||||
accelCruise @3;
|
||||
decelCruise @4;
|
||||
cancel @5;
|
||||
altButton1 @6;
|
||||
altButton2 @7;
|
||||
altButton3 @8;
|
||||
setCruise @9;
|
||||
resumeCruise @10;
|
||||
gapAdjustCruise @11;
|
||||
}
|
||||
}
|
||||
|
||||
errorsDEPRECATED @0 :List(CarEvent.EventName);
|
||||
brakeLightsDEPRECATED @19 :Bool;
|
||||
}
|
||||
|
||||
# ******* radar state @ 20hz *******
|
||||
|
||||
struct RadarData @0x888ad6581cf0aacb {
|
||||
errors @0 :List(Error);
|
||||
points @1 :List(RadarPoint);
|
||||
|
||||
# which packets this state came from
|
||||
canMonoTimes @2 :List(UInt64);
|
||||
|
||||
enum Error {
|
||||
canError @0;
|
||||
fault @1;
|
||||
wrongConfig @2;
|
||||
}
|
||||
|
||||
# similar to LiveTracks
|
||||
# is one timestamp valid for all? I think so
|
||||
struct RadarPoint {
|
||||
trackId @0 :UInt64; # no trackId reuse
|
||||
|
||||
# these 3 are the minimum required
|
||||
dRel @1 :Float32; # m from the front bumper of the car
|
||||
yRel @2 :Float32; # m
|
||||
vRel @3 :Float32; # m/s
|
||||
|
||||
# these are optional and valid if they are not NaN
|
||||
aRel @4 :Float32; # m/s^2
|
||||
yvRel @5 :Float32; # m/s
|
||||
|
||||
# some radars flag measurements VS estimates
|
||||
measured @6 :Bool;
|
||||
}
|
||||
}
|
||||
|
||||
# ******* car controls @ 100hz *******
|
||||
|
||||
struct CarControl {
|
||||
# must be true for any actuator commands to work
|
||||
enabled @0 :Bool;
|
||||
latActive @11: Bool;
|
||||
longActive @12: Bool;
|
||||
|
||||
# Actuator commands as computed by controlsd
|
||||
actuators @6 :Actuators;
|
||||
|
||||
# Any car specific rate limits or quirks applied by
|
||||
# the CarController are reflected in actuatorsOutput
|
||||
# and matches what is sent to the car
|
||||
actuatorsOutput @10 :Actuators;
|
||||
|
||||
orientationNED @13 :List(Float32);
|
||||
angularVelocity @14 :List(Float32);
|
||||
|
||||
cruiseControl @4 :CruiseControl;
|
||||
hudControl @5 :HUDControl;
|
||||
|
||||
struct Actuators {
|
||||
# range from 0.0 - 1.0
|
||||
gas @0: Float32;
|
||||
brake @1: Float32;
|
||||
# range from -1.0 - 1.0
|
||||
steer @2: Float32;
|
||||
steeringAngleDeg @3: Float32;
|
||||
|
||||
speed @6: Float32; # m/s
|
||||
accel @4: Float32; # m/s^2
|
||||
longControlState @5: LongControlState;
|
||||
|
||||
enum LongControlState @0xe40f3a917d908282{
|
||||
off @0;
|
||||
pid @1;
|
||||
stopping @2;
|
||||
|
||||
startingDEPRECATED @3;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
struct CruiseControl {
|
||||
cancel @0: Bool;
|
||||
override @1: Bool;
|
||||
speedOverride @2: Float32;
|
||||
accelOverride @3: Float32;
|
||||
}
|
||||
|
||||
struct HUDControl {
|
||||
speedVisible @0: Bool;
|
||||
setSpeed @1: Float32;
|
||||
lanesVisible @2: Bool;
|
||||
leadVisible @3: Bool;
|
||||
visualAlert @4: VisualAlert;
|
||||
audibleAlert @5: AudibleAlert;
|
||||
rightLaneVisible @6: Bool;
|
||||
leftLaneVisible @7: Bool;
|
||||
rightLaneDepart @8: Bool;
|
||||
leftLaneDepart @9: Bool;
|
||||
|
||||
enum VisualAlert {
|
||||
# these are the choices from the Honda
|
||||
# map as good as you can for your car
|
||||
none @0;
|
||||
fcw @1;
|
||||
steerRequired @2;
|
||||
brakePressed @3;
|
||||
wrongGear @4;
|
||||
seatbeltUnbuckled @5;
|
||||
speedTooHigh @6;
|
||||
ldw @7;
|
||||
}
|
||||
|
||||
enum AudibleAlert {
|
||||
none @0;
|
||||
|
||||
engage @1;
|
||||
disengage @2;
|
||||
refuse @3;
|
||||
|
||||
warningSoft @4;
|
||||
warningImmediate @5;
|
||||
|
||||
prompt @6;
|
||||
promptRepeat @7;
|
||||
promptDistracted @8;
|
||||
}
|
||||
}
|
||||
|
||||
gasDEPRECATED @1 :Float32;
|
||||
brakeDEPRECATED @2 :Float32;
|
||||
steeringTorqueDEPRECATED @3 :Float32;
|
||||
activeDEPRECATED @7 :Bool;
|
||||
rollDEPRECATED @8 :Float32;
|
||||
pitchDEPRECATED @9 :Float32;
|
||||
}
|
||||
|
||||
# ****** car param ******
|
||||
|
||||
struct CarParams {
|
||||
carName @0 :Text;
|
||||
carFingerprint @1 :Text;
|
||||
fuzzyFingerprint @55 :Bool;
|
||||
|
||||
enableGasInterceptor @2 :Bool;
|
||||
pcmCruise @3 :Bool; # is openpilot's state tied to the PCM's cruise state?
|
||||
enableDsu @5 :Bool; # driving support unit
|
||||
enableApgs @6 :Bool; # advanced parking guidance system
|
||||
enableBsm @56 :Bool; # blind spot monitoring
|
||||
flags @64 :UInt32; # flags for car specific quirks
|
||||
|
||||
minEnableSpeed @7 :Float32;
|
||||
minSteerSpeed @8 :Float32;
|
||||
maxSteeringAngleDeg @54 :Float32;
|
||||
safetyConfigs @62 :List(SafetyConfig);
|
||||
alternativeExperience @65 :Int16; # panda flag for features like no disengage on gas
|
||||
|
||||
steerMaxBP @11 :List(Float32);
|
||||
steerMaxV @12 :List(Float32);
|
||||
gasMaxBPDEPRECATED @13 :List(Float32);
|
||||
gasMaxVDEPRECATED @14 :List(Float32);
|
||||
brakeMaxBPDEPRECATED @15 :List(Float32);
|
||||
brakeMaxVDEPRECATED @16 :List(Float32);
|
||||
|
||||
# things about the car in the manual
|
||||
mass @17 :Float32; # [kg] curb weight: all fluids no cargo
|
||||
wheelbase @18 :Float32; # [m] distance from rear axle to front axle
|
||||
centerToFront @19 :Float32; # [m] distance from center of mass to front axle
|
||||
steerRatio @20 :Float32; # [] ratio of steering wheel angle to front wheel angle
|
||||
steerRatioRear @21 :Float32; # [] ratio of steering wheel angle to rear wheel angle (usually 0)
|
||||
|
||||
# things we can derive
|
||||
rotationalInertia @22 :Float32; # [kg*m2] body rotational inertia
|
||||
tireStiffnessFront @23 :Float32; # [N/rad] front tire coeff of stiff
|
||||
tireStiffnessRear @24 :Float32; # [N/rad] rear tire coeff of stiff
|
||||
|
||||
longitudinalTuning @25 :LongitudinalPIDTuning;
|
||||
lateralParams @48 :LateralParams;
|
||||
lateralTuning :union {
|
||||
pid @26 :LateralPIDTuning;
|
||||
indi @27 :LateralINDITuning;
|
||||
lqr @40 :LateralLQRTuning;
|
||||
}
|
||||
|
||||
steerLimitAlert @28 :Bool;
|
||||
steerLimitTimer @47 :Float32; # time before steerLimitAlert is issued
|
||||
|
||||
vEgoStopping @29 :Float32; # Speed at which the car goes into stopping state
|
||||
vEgoStarting @59 :Float32; # Speed at which the car goes into starting state
|
||||
directAccelControl @30 :Bool; # Does the car have direct accel control or just gas/brake
|
||||
stoppingControl @31 :Bool; # Does the car allows full control even at lows speeds when stopping
|
||||
stopAccel @60 :Float32; # Required acceleraton to keep vehicle stationary
|
||||
steerRateCost @33 :Float32; # Lateral MPC cost on steering rate
|
||||
steerControlType @34 :SteerControlType;
|
||||
radarOffCan @35 :Bool; # True when radar objects aren't visible on CAN
|
||||
stoppingDecelRate @52 :Float32; # m/s^2/s while trying to stop
|
||||
|
||||
steerActuatorDelay @36 :Float32; # Steering wheel actuator delay in seconds
|
||||
longitudinalActuatorDelayLowerBound @61 :Float32; # Gas/Brake actuator delay in seconds, lower bound
|
||||
longitudinalActuatorDelayUpperBound @58 :Float32; # Gas/Brake actuator delay in seconds, upper bound
|
||||
openpilotLongitudinalControl @37 :Bool; # is openpilot doing the longitudinal control?
|
||||
carVin @38 :Text; # VIN number queried during fingerprinting
|
||||
dashcamOnly @41: Bool;
|
||||
transmissionType @43 :TransmissionType;
|
||||
carFw @44 :List(CarFw);
|
||||
|
||||
radarTimeStep @45: Float32 = 0.05; # time delta between radar updates, 20Hz is very standard
|
||||
fingerprintSource @49: FingerprintSource;
|
||||
networkLocation @50 :NetworkLocation; # Where Panda/C2 is integrated into the car's CAN network
|
||||
|
||||
wheelSpeedFactor @63 :Float32; # Multiplier on wheels speeds to computer actual speeds
|
||||
|
||||
struct SafetyConfig {
|
||||
safetyModel @0 :SafetyModel;
|
||||
safetyParam @1 :Int16;
|
||||
}
|
||||
|
||||
struct LateralParams {
|
||||
torqueBP @0 :List(Int32);
|
||||
torqueV @1 :List(Int32);
|
||||
}
|
||||
|
||||
struct LateralPIDTuning {
|
||||
kpBP @0 :List(Float32);
|
||||
kpV @1 :List(Float32);
|
||||
kiBP @2 :List(Float32);
|
||||
kiV @3 :List(Float32);
|
||||
kf @4 :Float32;
|
||||
}
|
||||
|
||||
struct LongitudinalPIDTuning {
|
||||
kpBP @0 :List(Float32);
|
||||
kpV @1 :List(Float32);
|
||||
kiBP @2 :List(Float32);
|
||||
kiV @3 :List(Float32);
|
||||
kf @6 :Float32;
|
||||
deadzoneBP @4 :List(Float32);
|
||||
deadzoneV @5 :List(Float32);
|
||||
}
|
||||
|
||||
struct LateralINDITuning {
|
||||
outerLoopGainBP @4 :List(Float32);
|
||||
outerLoopGainV @5 :List(Float32);
|
||||
innerLoopGainBP @6 :List(Float32);
|
||||
innerLoopGainV @7 :List(Float32);
|
||||
timeConstantBP @8 :List(Float32);
|
||||
timeConstantV @9 :List(Float32);
|
||||
actuatorEffectivenessBP @10 :List(Float32);
|
||||
actuatorEffectivenessV @11 :List(Float32);
|
||||
|
||||
outerLoopGainDEPRECATED @0 :Float32;
|
||||
innerLoopGainDEPRECATED @1 :Float32;
|
||||
timeConstantDEPRECATED @2 :Float32;
|
||||
actuatorEffectivenessDEPRECATED @3 :Float32;
|
||||
}
|
||||
|
||||
struct LateralLQRTuning {
|
||||
scale @0 :Float32;
|
||||
ki @1 :Float32;
|
||||
dcGain @2 :Float32;
|
||||
|
||||
# State space system
|
||||
a @3 :List(Float32);
|
||||
b @4 :List(Float32);
|
||||
c @5 :List(Float32);
|
||||
|
||||
k @6 :List(Float32); # LQR gain
|
||||
l @7 :List(Float32); # Kalman gain
|
||||
}
|
||||
|
||||
enum SafetyModel {
|
||||
silent @0;
|
||||
hondaNidec @1;
|
||||
toyota @2;
|
||||
elm327 @3;
|
||||
gm @4;
|
||||
hondaBoschGiraffe @5;
|
||||
ford @6;
|
||||
cadillac @7;
|
||||
hyundai @8;
|
||||
chrysler @9;
|
||||
tesla @10;
|
||||
subaru @11;
|
||||
gmPassive @12;
|
||||
mazda @13;
|
||||
nissan @14;
|
||||
volkswagen @15;
|
||||
toyotaIpas @16;
|
||||
allOutput @17;
|
||||
gmAscm @18;
|
||||
noOutput @19; # like silent but without silent CAN TXs
|
||||
hondaBosch @20;
|
||||
volkswagenPq @21;
|
||||
subaruLegacy @22; # pre-Global platform
|
||||
hyundaiLegacy @23;
|
||||
hyundaiCommunity @24;
|
||||
stellantis @25;
|
||||
faw @26;
|
||||
}
|
||||
|
||||
enum SteerControlType {
|
||||
torque @0;
|
||||
angle @1;
|
||||
}
|
||||
|
||||
enum TransmissionType {
|
||||
unknown @0;
|
||||
automatic @1; # Traditional auto, including DSG
|
||||
manual @2; # True "stick shift" only
|
||||
direct @3; # Electric vehicle or other direct drive
|
||||
cvt @4;
|
||||
}
|
||||
|
||||
struct CarFw {
|
||||
ecu @0 :Ecu;
|
||||
fwVersion @1 :Data;
|
||||
address @2: UInt32;
|
||||
subAddress @3: UInt8;
|
||||
}
|
||||
|
||||
enum Ecu {
|
||||
eps @0;
|
||||
esp @1;
|
||||
fwdRadar @2;
|
||||
fwdCamera @3;
|
||||
engine @4;
|
||||
unknown @5;
|
||||
transmission @8; # Transmission Control Module
|
||||
srs @9; # airbag
|
||||
gateway @10; # can gateway
|
||||
hud @11; # heads up display
|
||||
combinationMeter @12; # instrument cluster
|
||||
|
||||
# Toyota only
|
||||
dsu @6;
|
||||
apgs @7;
|
||||
|
||||
# Honda only
|
||||
vsa @13; # Vehicle Stability Assist
|
||||
programmedFuelInjection @14;
|
||||
electricBrakeBooster @15;
|
||||
shiftByWire @16;
|
||||
}
|
||||
|
||||
enum FingerprintSource {
|
||||
can @0;
|
||||
fw @1;
|
||||
fixed @2;
|
||||
}
|
||||
|
||||
enum NetworkLocation {
|
||||
fwdCamera @0; # Standard/default integration at LKAS camera
|
||||
gateway @1; # Integration at vehicle's CAN gateway
|
||||
}
|
||||
|
||||
enableCameraDEPRECATED @4 :Bool;
|
||||
isPandaBlackDEPRECATED @39 :Bool;
|
||||
hasStockCameraDEPRECATED @57 :Bool;
|
||||
safetyParamDEPRECATED @10 :Int16;
|
||||
safetyModelDEPRECATED @9 :SafetyModel;
|
||||
safetyModelPassiveDEPRECATED @42 :SafetyModel = silent;
|
||||
minSpeedCanDEPRECATED @51 :Float32;
|
||||
startAccelDEPRECATED @32 :Float32;
|
||||
communityFeatureDEPRECATED @46: Bool;
|
||||
startingAccelRateDEPRECATED @53 :Float32;
|
||||
}
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
# Copyright (c) 2013-2014 Sandstorm Development Group, Inc. and contributors
|
||||
# Licensed under the MIT License:
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
# of this software and associated documentation files (the "Software"), to deal
|
||||
# in the Software without restriction, including without limitation the rights
|
||||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
# copies of the Software, and to permit persons to whom the Software is
|
||||
# furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# The above copyright notice and this permission notice shall be included in
|
||||
# all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
# THE SOFTWARE.
|
||||
|
||||
@0xbdf87d7bb8304e81;
|
||||
$namespace("capnp::annotations");
|
||||
|
||||
annotation namespace(file): Text;
|
||||
annotation name(field, enumerant, struct, enum, interface, method, param, group, union): Text;
|
||||
|
|
@ -0,0 +1,574 @@
|
|||
using Cxx = import "./include/c++.capnp";
|
||||
$Cxx.namespace("cereal");
|
||||
|
||||
@0x80ef1ec4889c2a63;
|
||||
|
||||
# legacy.capnp: a home for deprecated structs
|
||||
|
||||
struct LogRotate @0x9811e1f38f62f2d1 {
|
||||
segmentNum @0 :Int32;
|
||||
path @1 :Text;
|
||||
}
|
||||
|
||||
struct LiveUI @0xc08240f996aefced {
|
||||
rearViewCam @0 :Bool;
|
||||
alertText1 @1 :Text;
|
||||
alertText2 @2 :Text;
|
||||
awarenessStatus @3 :Float32;
|
||||
}
|
||||
|
||||
struct UiLayoutState @0x88dcce08ad29dda0 {
|
||||
activeApp @0 :App;
|
||||
sidebarCollapsed @1 :Bool;
|
||||
mapEnabled @2 :Bool;
|
||||
mockEngaged @3 :Bool;
|
||||
|
||||
enum App @0x9917470acf94d285 {
|
||||
home @0;
|
||||
music @1;
|
||||
nav @2;
|
||||
settings @3;
|
||||
none @4;
|
||||
}
|
||||
}
|
||||
|
||||
struct OrbslamCorrection @0x8afd33dc9b35e1aa {
|
||||
correctionMonoTime @0 :UInt64;
|
||||
prePositionECEF @1 :List(Float64);
|
||||
postPositionECEF @2 :List(Float64);
|
||||
prePoseQuatECEF @3 :List(Float32);
|
||||
postPoseQuatECEF @4 :List(Float32);
|
||||
numInliers @5 :UInt32;
|
||||
}
|
||||
|
||||
struct EthernetPacket @0xa99a9d5b33cf5859 {
|
||||
pkt @0 :Data;
|
||||
ts @1 :Float32;
|
||||
}
|
||||
|
||||
struct CellInfo @0xcff7566681c277ce {
|
||||
timestamp @0 :UInt64;
|
||||
repr @1 :Text; # android toString() for now
|
||||
}
|
||||
|
||||
struct WifiScan @0xd4df5a192382ba0b {
|
||||
bssid @0 :Text;
|
||||
ssid @1 :Text;
|
||||
capabilities @2 :Text;
|
||||
frequency @3 :Int32;
|
||||
level @4 :Int32;
|
||||
timestamp @5 :Int64;
|
||||
|
||||
centerFreq0 @6 :Int32;
|
||||
centerFreq1 @7 :Int32;
|
||||
channelWidth @8 :ChannelWidth;
|
||||
operatorFriendlyName @9 :Text;
|
||||
venueName @10 :Text;
|
||||
is80211mcResponder @11 :Bool;
|
||||
passpoint @12 :Bool;
|
||||
|
||||
distanceCm @13 :Int32;
|
||||
distanceSdCm @14 :Int32;
|
||||
|
||||
enum ChannelWidth @0xcb6a279f015f6b51 {
|
||||
w20Mhz @0;
|
||||
w40Mhz @1;
|
||||
w80Mhz @2;
|
||||
w160Mhz @3;
|
||||
w80Plus80Mhz @4;
|
||||
}
|
||||
}
|
||||
|
||||
struct LiveEventData @0x94b7baa90c5c321e {
|
||||
name @0 :Text;
|
||||
value @1 :Int32;
|
||||
}
|
||||
|
||||
struct ModelData @0xb8aad62cffef28a9 {
|
||||
frameId @0 :UInt32;
|
||||
frameAge @12 :UInt32;
|
||||
frameDropPerc @13 :Float32;
|
||||
timestampEof @9 :UInt64;
|
||||
modelExecutionTime @14 :Float32;
|
||||
gpuExecutionTime @16 :Float32;
|
||||
rawPred @15 :Data;
|
||||
|
||||
path @1 :PathData;
|
||||
leftLane @2 :PathData;
|
||||
rightLane @3 :PathData;
|
||||
lead @4 :LeadData;
|
||||
freePath @6 :List(Float32);
|
||||
|
||||
settings @5 :ModelSettings;
|
||||
leadFuture @7 :LeadData;
|
||||
speed @8 :List(Float32);
|
||||
meta @10 :MetaData;
|
||||
longitudinal @11 :LongitudinalData;
|
||||
|
||||
struct PathData @0x8817eeea389e9f08 {
|
||||
points @0 :List(Float32);
|
||||
prob @1 :Float32;
|
||||
std @2 :Float32;
|
||||
stds @3 :List(Float32);
|
||||
poly @4 :List(Float32);
|
||||
validLen @5 :Float32;
|
||||
}
|
||||
|
||||
struct LeadData @0xd1c9bef96d26fa91 {
|
||||
dist @0 :Float32;
|
||||
prob @1 :Float32;
|
||||
std @2 :Float32;
|
||||
relVel @3 :Float32;
|
||||
relVelStd @4 :Float32;
|
||||
relY @5 :Float32;
|
||||
relYStd @6 :Float32;
|
||||
relA @7 :Float32;
|
||||
relAStd @8 :Float32;
|
||||
}
|
||||
|
||||
struct ModelSettings @0xa26e3710efd3e914 {
|
||||
bigBoxX @0 :UInt16;
|
||||
bigBoxY @1 :UInt16;
|
||||
bigBoxWidth @2 :UInt16;
|
||||
bigBoxHeight @3 :UInt16;
|
||||
boxProjection @4 :List(Float32);
|
||||
yuvCorrection @5 :List(Float32);
|
||||
inputTransform @6 :List(Float32);
|
||||
}
|
||||
|
||||
struct MetaData @0x9744f25fb60f2bf8 {
|
||||
engagedProb @0 :Float32;
|
||||
desirePrediction @1 :List(Float32);
|
||||
brakeDisengageProb @2 :Float32;
|
||||
gasDisengageProb @3 :Float32;
|
||||
steerOverrideProb @4 :Float32;
|
||||
desireState @5 :List(Float32);
|
||||
}
|
||||
|
||||
struct LongitudinalData @0xf98f999c6a071122 {
|
||||
distances @2 :List(Float32);
|
||||
speeds @0 :List(Float32);
|
||||
accelerations @1 :List(Float32);
|
||||
}
|
||||
}
|
||||
|
||||
struct ECEFPoint @0xc25bbbd524983447 {
|
||||
x @0 :Float64;
|
||||
y @1 :Float64;
|
||||
z @2 :Float64;
|
||||
}
|
||||
|
||||
struct ECEFPointDEPRECATED @0xe10e21168db0c7f7 {
|
||||
x @0 :Float32;
|
||||
y @1 :Float32;
|
||||
z @2 :Float32;
|
||||
}
|
||||
|
||||
struct GPSPlannerPoints @0xab54c59699f8f9f3 {
|
||||
curPosDEPRECATED @0 :ECEFPointDEPRECATED;
|
||||
pointsDEPRECATED @1 :List(ECEFPointDEPRECATED);
|
||||
curPos @6 :ECEFPoint;
|
||||
points @7 :List(ECEFPoint);
|
||||
valid @2 :Bool;
|
||||
trackName @3 :Text;
|
||||
speedLimit @4 :Float32;
|
||||
accelTarget @5 :Float32;
|
||||
}
|
||||
|
||||
struct GPSPlannerPlan @0xf5ad1d90cdc1dd6b {
|
||||
valid @0 :Bool;
|
||||
poly @1 :List(Float32);
|
||||
trackName @2 :Text;
|
||||
speed @3 :Float32;
|
||||
acceleration @4 :Float32;
|
||||
pointsDEPRECATED @5 :List(ECEFPointDEPRECATED);
|
||||
points @6 :List(ECEFPoint);
|
||||
xLookahead @7 :Float32;
|
||||
}
|
||||
|
||||
struct UiNavigationEvent @0x90c8426c3eaddd3b {
|
||||
type @0: Type;
|
||||
status @1: Status;
|
||||
distanceTo @2: Float32;
|
||||
endRoadPointDEPRECATED @3: ECEFPointDEPRECATED;
|
||||
endRoadPoint @4: ECEFPoint;
|
||||
|
||||
enum Type @0xe8db07dcf8fcea05 {
|
||||
none @0;
|
||||
laneChangeLeft @1;
|
||||
laneChangeRight @2;
|
||||
mergeLeft @3;
|
||||
mergeRight @4;
|
||||
turnLeft @5;
|
||||
turnRight @6;
|
||||
}
|
||||
|
||||
enum Status @0xb9aa88c75ef99a1f {
|
||||
none @0;
|
||||
passive @1;
|
||||
approaching @2;
|
||||
active @3;
|
||||
}
|
||||
}
|
||||
|
||||
struct LiveLocationData @0xb99b2bc7a57e8128 {
|
||||
status @0 :UInt8;
|
||||
|
||||
# 3D fix
|
||||
lat @1 :Float64;
|
||||
lon @2 :Float64;
|
||||
alt @3 :Float32; # m
|
||||
|
||||
# speed
|
||||
speed @4 :Float32; # m/s
|
||||
|
||||
# NED velocity components
|
||||
vNED @5 :List(Float32);
|
||||
|
||||
# roll, pitch, heading (x,y,z)
|
||||
roll @6 :Float32; # WRT to center of earth?
|
||||
pitch @7 :Float32; # WRT to center of earth?
|
||||
heading @8 :Float32; # WRT to north?
|
||||
|
||||
# what are these?
|
||||
wanderAngle @9 :Float32;
|
||||
trackAngle @10 :Float32;
|
||||
|
||||
# car frame -- https://upload.wikimedia.org/wikipedia/commons/f/f5/RPY_angles_of_cars.png
|
||||
|
||||
# gyro, in car frame, deg/s
|
||||
gyro @11 :List(Float32);
|
||||
|
||||
# accel, in car frame, m/s^2
|
||||
accel @12 :List(Float32);
|
||||
|
||||
accuracy @13 :Accuracy;
|
||||
|
||||
source @14 :SensorSource;
|
||||
# if we are fixing a location in the past
|
||||
fixMonoTime @15 :UInt64;
|
||||
|
||||
gpsWeek @16 :Int32;
|
||||
timeOfWeek @17 :Float64;
|
||||
|
||||
positionECEF @18 :List(Float64);
|
||||
poseQuatECEF @19 :List(Float32);
|
||||
pitchCalibration @20 :Float32;
|
||||
yawCalibration @21 :Float32;
|
||||
imuFrame @22 :List(Float32);
|
||||
|
||||
struct Accuracy @0x943dc4625473b03f {
|
||||
pNEDError @0 :List(Float32);
|
||||
vNEDError @1 :List(Float32);
|
||||
rollError @2 :Float32;
|
||||
pitchError @3 :Float32;
|
||||
headingError @4 :Float32;
|
||||
ellipsoidSemiMajorError @5 :Float32;
|
||||
ellipsoidSemiMinorError @6 :Float32;
|
||||
ellipsoidOrientationError @7 :Float32;
|
||||
}
|
||||
|
||||
enum SensorSource @0xc871d3cc252af657 {
|
||||
applanix @0;
|
||||
kalman @1;
|
||||
orbslam @2;
|
||||
timing @3;
|
||||
dummy @4;
|
||||
}
|
||||
}
|
||||
|
||||
struct OrbOdometry @0xd7700859ed1f5b76 {
|
||||
# timing first
|
||||
startMonoTime @0 :UInt64;
|
||||
endMonoTime @1 :UInt64;
|
||||
|
||||
# fundamental matrix and error
|
||||
f @2: List(Float64);
|
||||
err @3: Float64;
|
||||
|
||||
# number of inlier points
|
||||
inliers @4: Int32;
|
||||
|
||||
# for debug only
|
||||
# indexed by endMonoTime features
|
||||
# value is startMonoTime feature match
|
||||
# -1 if no match
|
||||
matches @5: List(Int16);
|
||||
}
|
||||
|
||||
struct OrbFeatures @0xcd60164a8a0159ef {
|
||||
timestampEof @0 :UInt64;
|
||||
# transposed arrays of normalized image coordinates
|
||||
# len(xs) == len(ys) == len(descriptors) * 32
|
||||
xs @1 :List(Float32);
|
||||
ys @2 :List(Float32);
|
||||
descriptors @3 :Data;
|
||||
octaves @4 :List(Int8);
|
||||
|
||||
# match index to last OrbFeatures
|
||||
# -1 if no match
|
||||
timestampLastEof @5 :UInt64;
|
||||
matches @6: List(Int16);
|
||||
}
|
||||
|
||||
struct OrbFeaturesSummary @0xd500d30c5803fa4f {
|
||||
timestampEof @0 :UInt64;
|
||||
timestampLastEof @1 :UInt64;
|
||||
|
||||
featureCount @2 :UInt16;
|
||||
matchCount @3 :UInt16;
|
||||
computeNs @4 :UInt64;
|
||||
}
|
||||
|
||||
struct OrbKeyFrame @0xc8233c0345e27e24 {
|
||||
# this is a globally unique id for the KeyFrame
|
||||
id @0: UInt64;
|
||||
|
||||
# this is the location of the KeyFrame
|
||||
pos @1: ECEFPoint;
|
||||
|
||||
# these are the features in the world
|
||||
# len(dpos) == len(descriptors) * 32
|
||||
dpos @2 :List(ECEFPoint);
|
||||
descriptors @3 :Data;
|
||||
}
|
||||
|
||||
struct KalmanOdometry @0x92e21bb7ea38793a {
|
||||
trans @0 :List(Float32); # m/s in device frame
|
||||
rot @1 :List(Float32); # rad/s in device frame
|
||||
transStd @2 :List(Float32); # std m/s in device frame
|
||||
rotStd @3 :List(Float32); # std rad/s in device frame
|
||||
}
|
||||
|
||||
struct OrbObservation @0x9b326d4e436afec7 {
|
||||
observationMonoTime @0 :UInt64;
|
||||
normalizedCoordinates @1 :List(Float32);
|
||||
locationECEF @2 :List(Float64);
|
||||
matchDistance @3: UInt32;
|
||||
}
|
||||
|
||||
struct CalibrationFeatures @0x8fdfadb254ea867a {
|
||||
frameId @0 :UInt32;
|
||||
|
||||
p0 @1 :List(Float32);
|
||||
p1 @2 :List(Float32);
|
||||
status @3 :List(Int8);
|
||||
}
|
||||
|
||||
struct NavStatus @0xbd8822120928120c {
|
||||
isNavigating @0 :Bool;
|
||||
currentAddress @1 :Address;
|
||||
|
||||
struct Address @0xce7cd672cacc7814 {
|
||||
title @0 :Text;
|
||||
lat @1 :Float64;
|
||||
lng @2 :Float64;
|
||||
house @3 :Text;
|
||||
address @4 :Text;
|
||||
street @5 :Text;
|
||||
city @6 :Text;
|
||||
state @7 :Text;
|
||||
country @8 :Text;
|
||||
}
|
||||
}
|
||||
|
||||
struct NavUpdate @0xdb98be6565516acb {
|
||||
isNavigating @0 :Bool;
|
||||
curSegment @1 :Int32;
|
||||
segments @2 :List(Segment);
|
||||
|
||||
struct LatLng @0x9eaef9187cadbb9b {
|
||||
lat @0 :Float64;
|
||||
lng @1 :Float64;
|
||||
}
|
||||
|
||||
struct Segment @0xa5b39b4fc4d7da3f {
|
||||
from @0 :LatLng;
|
||||
to @1 :LatLng;
|
||||
updateTime @2 :Int32;
|
||||
distance @3 :Int32;
|
||||
crossTime @4 :Int32;
|
||||
exitNo @5 :Int32;
|
||||
instruction @6 :Instruction;
|
||||
|
||||
parts @7 :List(LatLng);
|
||||
|
||||
enum Instruction @0xc5417a637451246f {
|
||||
turnLeft @0;
|
||||
turnRight @1;
|
||||
keepLeft @2;
|
||||
keepRight @3;
|
||||
straight @4;
|
||||
roundaboutExitNumber @5;
|
||||
roundaboutExit @6;
|
||||
roundaboutTurnLeft @7;
|
||||
unkn8 @8;
|
||||
roundaboutStraight @9;
|
||||
unkn10 @10;
|
||||
roundaboutTurnRight @11;
|
||||
unkn12 @12;
|
||||
roundaboutUturn @13;
|
||||
unkn14 @14;
|
||||
arrive @15;
|
||||
exitLeft @16;
|
||||
exitRight @17;
|
||||
unkn18 @18;
|
||||
uturn @19;
|
||||
# ...
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct TrafficEvent @0xacfa74a094e62626 {
|
||||
type @0 :Type;
|
||||
distance @1 :Float32;
|
||||
action @2 :Action;
|
||||
resuming @3 :Bool;
|
||||
|
||||
enum Type @0xd85d75253435bf4b {
|
||||
stopSign @0;
|
||||
lightRed @1;
|
||||
lightYellow @2;
|
||||
lightGreen @3;
|
||||
stopLight @4;
|
||||
}
|
||||
|
||||
enum Action @0xa6f6ce72165ccb49 {
|
||||
none @0;
|
||||
yield @1;
|
||||
stop @2;
|
||||
resumeReady @3;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
struct AndroidGnss @0xdfdf30d03fc485bd {
|
||||
union {
|
||||
measurements @0 :Measurements;
|
||||
navigationMessage @1 :NavigationMessage;
|
||||
}
|
||||
|
||||
struct Measurements @0xa20710d4f428d6cd {
|
||||
clock @0 :Clock;
|
||||
measurements @1 :List(Measurement);
|
||||
|
||||
struct Clock @0xa0e27b453a38f450 {
|
||||
timeNanos @0 :Int64;
|
||||
hardwareClockDiscontinuityCount @1 :Int32;
|
||||
|
||||
hasTimeUncertaintyNanos @2 :Bool;
|
||||
timeUncertaintyNanos @3 :Float64;
|
||||
|
||||
hasLeapSecond @4 :Bool;
|
||||
leapSecond @5 :Int32;
|
||||
|
||||
hasFullBiasNanos @6 :Bool;
|
||||
fullBiasNanos @7 :Int64;
|
||||
|
||||
hasBiasNanos @8 :Bool;
|
||||
biasNanos @9 :Float64;
|
||||
|
||||
hasBiasUncertaintyNanos @10 :Bool;
|
||||
biasUncertaintyNanos @11 :Float64;
|
||||
|
||||
hasDriftNanosPerSecond @12 :Bool;
|
||||
driftNanosPerSecond @13 :Float64;
|
||||
|
||||
hasDriftUncertaintyNanosPerSecond @14 :Bool;
|
||||
driftUncertaintyNanosPerSecond @15 :Float64;
|
||||
}
|
||||
|
||||
struct Measurement @0xd949bf717d77614d {
|
||||
svId @0 :Int32;
|
||||
constellation @1 :Constellation;
|
||||
|
||||
timeOffsetNanos @2 :Float64;
|
||||
state @3 :Int32;
|
||||
receivedSvTimeNanos @4 :Int64;
|
||||
receivedSvTimeUncertaintyNanos @5 :Int64;
|
||||
cn0DbHz @6 :Float64;
|
||||
pseudorangeRateMetersPerSecond @7 :Float64;
|
||||
pseudorangeRateUncertaintyMetersPerSecond @8 :Float64;
|
||||
accumulatedDeltaRangeState @9 :Int32;
|
||||
accumulatedDeltaRangeMeters @10 :Float64;
|
||||
accumulatedDeltaRangeUncertaintyMeters @11 :Float64;
|
||||
|
||||
hasCarrierFrequencyHz @12 :Bool;
|
||||
carrierFrequencyHz @13 :Float32;
|
||||
hasCarrierCycles @14 :Bool;
|
||||
carrierCycles @15 :Int64;
|
||||
hasCarrierPhase @16 :Bool;
|
||||
carrierPhase @17 :Float64;
|
||||
hasCarrierPhaseUncertainty @18 :Bool;
|
||||
carrierPhaseUncertainty @19 :Float64;
|
||||
hasSnrInDb @20 :Bool;
|
||||
snrInDb @21 :Float64;
|
||||
|
||||
multipathIndicator @22 :MultipathIndicator;
|
||||
|
||||
enum Constellation @0x9ef1f3ff0deb5ffb {
|
||||
unknown @0;
|
||||
gps @1;
|
||||
sbas @2;
|
||||
glonass @3;
|
||||
qzss @4;
|
||||
beidou @5;
|
||||
galileo @6;
|
||||
}
|
||||
|
||||
enum State @0xcbb9490adce12d72 {
|
||||
unknown @0;
|
||||
codeLock @1;
|
||||
bitSync @2;
|
||||
subframeSync @3;
|
||||
towDecoded @4;
|
||||
msecAmbiguous @5;
|
||||
symbolSync @6;
|
||||
gloStringSync @7;
|
||||
gloTodDecoded @8;
|
||||
bdsD2BitSync @9;
|
||||
bdsD2SubframeSync @10;
|
||||
galE1bcCodeLock @11;
|
||||
galE1c2ndCodeLock @12;
|
||||
galE1bPageSync @13;
|
||||
sbasSync @14;
|
||||
}
|
||||
|
||||
enum MultipathIndicator @0xc04e7b6231d4caa8 {
|
||||
unknown @0;
|
||||
detected @1;
|
||||
notDetected @2;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct NavigationMessage @0xe2517b083095fd4e {
|
||||
type @0 :Int32;
|
||||
svId @1 :Int32;
|
||||
messageId @2 :Int32;
|
||||
submessageId @3 :Int32;
|
||||
data @4 :Data;
|
||||
status @5 :Status;
|
||||
|
||||
enum Status @0xec1ff7996b35366f {
|
||||
unknown @0;
|
||||
parityPassed @1;
|
||||
parityRebuilt @2;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct LidarPts @0xe3d6685d4e9d8f7a {
|
||||
r @0 :List(UInt16); # uint16 m*500.0
|
||||
theta @1 :List(UInt16); # uint16 deg*100.0
|
||||
reflect @2 :List(UInt8); # uint8 0-255
|
||||
|
||||
# For storing out of file.
|
||||
idx @3 :UInt64;
|
||||
|
||||
# For storing in file
|
||||
pkt @4 :Data;
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,21 @@
|
|||
#pragma once
|
||||
|
||||
#ifdef SWAGLOG
|
||||
// cppcheck-suppress preprocessorErrorDirective
|
||||
#include SWAGLOG
|
||||
#else
|
||||
|
||||
#define CLOUDLOG_DEBUG 10
|
||||
#define CLOUDLOG_INFO 20
|
||||
#define CLOUDLOG_WARNING 30
|
||||
#define CLOUDLOG_ERROR 40
|
||||
#define CLOUDLOG_CRITICAL 50
|
||||
|
||||
#define cloudlog(lvl, fmt, ...) printf(fmt "\n", ## __VA_ARGS__)
|
||||
|
||||
#define LOGD(fmt, ...) cloudlog(CLOUDLOG_DEBUG, fmt, ## __VA_ARGS__)
|
||||
#define LOG(fmt, ...) cloudlog(CLOUDLOG_INFO, fmt, ## __VA_ARGS__)
|
||||
#define LOGW(fmt, ...) cloudlog(CLOUDLOG_WARNING, fmt, ## __VA_ARGS__)
|
||||
#define LOGE(fmt, ...) cloudlog(CLOUDLOG_ERROR, fmt, ## __VA_ARGS__)
|
||||
|
||||
#endif
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
demo
|
||||
bridge
|
||||
test_runner
|
||||
*.o
|
||||
*.os
|
||||
*.d
|
||||
*.a
|
||||
*.so
|
||||
messaging_pyx.cpp
|
||||
build/
|
||||
|
|
@ -0,0 +1,251 @@
|
|||
# must be build with scons
|
||||
from .messaging_pyx import Context, Poller, SubSocket, PubSocket # pylint: disable=no-name-in-module, import-error
|
||||
from .messaging_pyx import MultiplePublishersError, MessagingError # pylint: disable=no-name-in-module, import-error
|
||||
import os
|
||||
import capnp
|
||||
|
||||
from typing import Optional, List, Union
|
||||
from collections import deque
|
||||
|
||||
from cereal import log
|
||||
from cereal.services import service_list
|
||||
|
||||
assert MultiplePublishersError
|
||||
assert MessagingError
|
||||
|
||||
NO_TRAVERSAL_LIMIT = 2**64-1
|
||||
AVG_FREQ_HISTORY = 100
|
||||
SIMULATION = "SIMULATION" in os.environ
|
||||
|
||||
# sec_since_boot is faster, but allow to run standalone too
|
||||
try:
|
||||
from common.realtime import sec_since_boot
|
||||
except ImportError:
|
||||
import time
|
||||
sec_since_boot = time.time
|
||||
print("Warning, using python time.time() instead of faster sec_since_boot")
|
||||
|
||||
context = Context()
|
||||
|
||||
def log_from_bytes(dat: bytes) -> capnp.lib.capnp._DynamicStructReader:
|
||||
return log.Event.from_bytes(dat, traversal_limit_in_words=NO_TRAVERSAL_LIMIT)
|
||||
|
||||
def new_message(service: Optional[str] = None, size: Optional[int] = None) -> capnp.lib.capnp._DynamicStructBuilder:
|
||||
dat = log.Event.new_message()
|
||||
dat.logMonoTime = int(sec_since_boot() * 1e9)
|
||||
dat.valid = True
|
||||
if service is not None:
|
||||
if size is None:
|
||||
dat.init(service)
|
||||
else:
|
||||
dat.init(service, size)
|
||||
return dat
|
||||
|
||||
def pub_sock(endpoint: str) -> PubSocket:
|
||||
sock = PubSocket()
|
||||
sock.connect(context, endpoint)
|
||||
return sock
|
||||
|
||||
def sub_sock(endpoint: str, poller: Optional[Poller] = None, addr: str = "127.0.0.1",
|
||||
conflate: bool = False, timeout: Optional[int] = None) -> SubSocket:
|
||||
sock = SubSocket()
|
||||
sock.connect(context, endpoint, addr.encode('utf8'), conflate)
|
||||
|
||||
if timeout is not None:
|
||||
sock.setTimeout(timeout)
|
||||
|
||||
if poller is not None:
|
||||
poller.registerSocket(sock)
|
||||
return sock
|
||||
|
||||
|
||||
def drain_sock_raw(sock: SubSocket, wait_for_one: bool = False) -> List[bytes]:
|
||||
"""Receive all message currently available on the queue"""
|
||||
ret: List[bytes] = []
|
||||
while 1:
|
||||
if wait_for_one and len(ret) == 0:
|
||||
dat = sock.receive()
|
||||
else:
|
||||
dat = sock.receive(non_blocking=True)
|
||||
|
||||
if dat is None:
|
||||
break
|
||||
|
||||
ret.append(dat)
|
||||
|
||||
return ret
|
||||
|
||||
def drain_sock(sock: SubSocket, wait_for_one: bool = False) -> List[capnp.lib.capnp._DynamicStructReader]:
|
||||
"""Receive all message currently available on the queue"""
|
||||
ret: List[capnp.lib.capnp._DynamicStructReader] = []
|
||||
while 1:
|
||||
if wait_for_one and len(ret) == 0:
|
||||
dat = sock.receive()
|
||||
else:
|
||||
dat = sock.receive(non_blocking=True)
|
||||
|
||||
if dat is None: # Timeout hit
|
||||
break
|
||||
|
||||
dat = log_from_bytes(dat)
|
||||
ret.append(dat)
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
# TODO: print when we drop packets?
|
||||
def recv_sock(sock: SubSocket, wait: bool = False) -> Optional[capnp.lib.capnp._DynamicStructReader]:
|
||||
"""Same as drain sock, but only returns latest message. Consider using conflate instead."""
|
||||
dat = None
|
||||
|
||||
while 1:
|
||||
if wait and dat is None:
|
||||
rcv = sock.receive()
|
||||
else:
|
||||
rcv = sock.receive(non_blocking=True)
|
||||
|
||||
if rcv is None: # Timeout hit
|
||||
break
|
||||
|
||||
dat = rcv
|
||||
|
||||
if dat is not None:
|
||||
dat = log_from_bytes(dat)
|
||||
|
||||
return dat
|
||||
|
||||
def recv_one(sock: SubSocket) -> Optional[capnp.lib.capnp._DynamicStructReader]:
|
||||
dat = sock.receive()
|
||||
if dat is not None:
|
||||
dat = log_from_bytes(dat)
|
||||
return dat
|
||||
|
||||
def recv_one_or_none(sock: SubSocket) -> Optional[capnp.lib.capnp._DynamicStructReader]:
|
||||
dat = sock.receive(non_blocking=True)
|
||||
if dat is not None:
|
||||
dat = log_from_bytes(dat)
|
||||
return dat
|
||||
|
||||
def recv_one_retry(sock: SubSocket) -> capnp.lib.capnp._DynamicStructReader:
|
||||
"""Keep receiving until we get a message"""
|
||||
while True:
|
||||
dat = sock.receive()
|
||||
if dat is not None:
|
||||
return log_from_bytes(dat)
|
||||
|
||||
class SubMaster:
|
||||
def __init__(self, services: List[str], poll: Optional[List[str]] = None,
|
||||
ignore_alive: Optional[List[str]] = None, ignore_avg_freq: Optional[List[str]] = None,
|
||||
addr: str = "127.0.0.1"):
|
||||
self.frame = -1
|
||||
self.updated = {s: False for s in services}
|
||||
self.rcv_time = {s: 0. for s in services}
|
||||
self.rcv_frame = {s: 0 for s in services}
|
||||
self.alive = {s: False for s in services}
|
||||
self.recv_dts = {s: deque([0.0] * AVG_FREQ_HISTORY, maxlen=AVG_FREQ_HISTORY) for s in services}
|
||||
self.sock = {}
|
||||
self.freq = {}
|
||||
self.data = {}
|
||||
self.valid = {}
|
||||
self.logMonoTime = {}
|
||||
|
||||
self.poller = Poller()
|
||||
self.non_polled_services = [s for s in services if poll is not None and
|
||||
len(poll) and s not in poll]
|
||||
|
||||
self.ignore_average_freq = [] if ignore_avg_freq is None else ignore_avg_freq
|
||||
self.ignore_alive = [] if ignore_alive is None else ignore_alive
|
||||
|
||||
for s in services:
|
||||
if addr is not None:
|
||||
p = self.poller if s not in self.non_polled_services else None
|
||||
self.sock[s] = sub_sock(s, poller=p, addr=addr, conflate=True)
|
||||
self.freq[s] = service_list[s].frequency
|
||||
|
||||
try:
|
||||
data = new_message(s)
|
||||
except capnp.lib.capnp.KjException: # pylint: disable=c-extension-no-member
|
||||
data = new_message(s, 0) # lists
|
||||
|
||||
self.data[s] = getattr(data, s)
|
||||
self.logMonoTime[s] = 0
|
||||
self.valid[s] = data.valid
|
||||
|
||||
def __getitem__(self, s: str) -> capnp.lib.capnp._DynamicStructReader:
|
||||
return self.data[s]
|
||||
|
||||
def update(self, timeout: int = 1000) -> None:
|
||||
msgs = []
|
||||
for sock in self.poller.poll(timeout):
|
||||
msgs.append(recv_one_or_none(sock))
|
||||
|
||||
# non-blocking receive for non-polled sockets
|
||||
for s in self.non_polled_services:
|
||||
msgs.append(recv_one_or_none(self.sock[s]))
|
||||
self.update_msgs(sec_since_boot(), msgs)
|
||||
|
||||
def update_msgs(self, cur_time: float, msgs: List[capnp.lib.capnp._DynamicStructReader]) -> None:
|
||||
self.frame += 1
|
||||
self.updated = dict.fromkeys(self.updated, False)
|
||||
for msg in msgs:
|
||||
if msg is None:
|
||||
continue
|
||||
|
||||
s = msg.which()
|
||||
self.updated[s] = True
|
||||
|
||||
if self.rcv_time[s] > 1e-5 and self.freq[s] > 1e-5 and (s not in self.non_polled_services) \
|
||||
and (s not in self.ignore_average_freq):
|
||||
self.recv_dts[s].append(cur_time - self.rcv_time[s])
|
||||
|
||||
self.rcv_time[s] = cur_time
|
||||
self.rcv_frame[s] = self.frame
|
||||
self.data[s] = getattr(msg, s)
|
||||
self.logMonoTime[s] = msg.logMonoTime
|
||||
self.valid[s] = msg.valid
|
||||
|
||||
if SIMULATION:
|
||||
self.alive[s] = True
|
||||
|
||||
if not SIMULATION:
|
||||
for s in self.data:
|
||||
# arbitrary small number to avoid float comparison. If freq is 0, we can skip the check
|
||||
if self.freq[s] > 1e-5:
|
||||
# alive if delay is within 10x the expected frequency
|
||||
self.alive[s] = (cur_time - self.rcv_time[s]) < (10. / self.freq[s])
|
||||
|
||||
# alive if average frequency is higher than 90% of expected frequency
|
||||
avg_dt = sum(self.recv_dts[s]) / AVG_FREQ_HISTORY
|
||||
expected_dt = 1 / (self.freq[s] * 0.90)
|
||||
self.alive[s] = self.alive[s] and (avg_dt < expected_dt)
|
||||
else:
|
||||
self.alive[s] = True
|
||||
|
||||
def all_alive(self, service_list=None) -> bool:
|
||||
if service_list is None: # check all
|
||||
service_list = self.alive.keys()
|
||||
return all(self.alive[s] for s in service_list if s not in self.ignore_alive)
|
||||
|
||||
def all_valid(self, service_list=None) -> bool:
|
||||
if service_list is None: # check all
|
||||
service_list = self.valid.keys()
|
||||
return all(self.valid[s] for s in service_list)
|
||||
|
||||
def all_alive_and_valid(self, service_list=None) -> bool:
|
||||
if service_list is None: # check all
|
||||
service_list = self.alive.keys()
|
||||
return self.all_alive(service_list=service_list) and self.all_valid(service_list=service_list)
|
||||
|
||||
class PubMaster:
|
||||
def __init__(self, services: List[str]):
|
||||
self.sock = {}
|
||||
for s in services:
|
||||
self.sock[s] = pub_sock(s)
|
||||
|
||||
def send(self, s: str, dat: Union[bytes, capnp.lib.capnp._DynamicStructBuilder]) -> None:
|
||||
if not isinstance(dat, bytes):
|
||||
dat = dat.to_bytes()
|
||||
self.sock[s].send(dat)
|
||||
|
||||
def all_readers_updated(self, s: str) -> bool:
|
||||
return self.sock[s].all_readers_updated()
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
#include <algorithm>
|
||||
#include <cassert>
|
||||
#include <csignal>
|
||||
#include <iostream>
|
||||
#include <map>
|
||||
#include <string>
|
||||
|
||||
typedef void (*sighandler_t)(int sig);
|
||||
|
||||
#include "impl_msgq.h"
|
||||
#include "impl_zmq.h"
|
||||
#include "services.h"
|
||||
|
||||
void sigpipe_handler(int sig) {
|
||||
assert(sig == SIGPIPE);
|
||||
std::cout << "SIGPIPE received" << std::endl;
|
||||
}
|
||||
|
||||
static std::vector<std::string> get_services(std::string whitelist_str, bool zmq_to_msgq) {
|
||||
std::vector<std::string> service_list;
|
||||
for (const auto& it : services) {
|
||||
std::string name = it.name;
|
||||
bool in_whitelist = whitelist_str.find(name) != std::string::npos;
|
||||
if (name == "plusFrame" || name == "uiLayoutState" || (zmq_to_msgq && !in_whitelist)) {
|
||||
continue;
|
||||
}
|
||||
service_list.push_back(name);
|
||||
}
|
||||
return service_list;
|
||||
}
|
||||
|
||||
int main(int argc, char** argv) {
|
||||
signal(SIGPIPE, (sighandler_t)sigpipe_handler);
|
||||
|
||||
bool zmq_to_msgq = argc > 2;
|
||||
std::string ip = zmq_to_msgq ? argv[1] : "127.0.0.1";
|
||||
std::string whitelist_str = zmq_to_msgq ? std::string(argv[2]) : "";
|
||||
|
||||
Poller *poller;
|
||||
Context *pub_context;
|
||||
Context *sub_context;
|
||||
if (zmq_to_msgq) { // republishes zmq debugging messages as msgq
|
||||
poller = new ZMQPoller();
|
||||
pub_context = new MSGQContext();
|
||||
sub_context = new ZMQContext();
|
||||
} else {
|
||||
poller = new MSGQPoller();
|
||||
pub_context = new ZMQContext();
|
||||
sub_context = new MSGQContext();
|
||||
}
|
||||
|
||||
std::map<SubSocket*, PubSocket*> sub2pub;
|
||||
for (auto endpoint: get_services(whitelist_str, zmq_to_msgq)) {
|
||||
PubSocket * pub_sock;
|
||||
SubSocket * sub_sock;
|
||||
if (zmq_to_msgq) {
|
||||
pub_sock = new MSGQPubSocket();
|
||||
sub_sock = new ZMQSubSocket();
|
||||
} else {
|
||||
pub_sock = new ZMQPubSocket();
|
||||
sub_sock = new MSGQSubSocket();
|
||||
}
|
||||
pub_sock->connect(pub_context, endpoint);
|
||||
sub_sock->connect(sub_context, endpoint, ip, false);
|
||||
|
||||
poller->registerSocket(sub_sock);
|
||||
sub2pub[sub_sock] = pub_sock;
|
||||
}
|
||||
|
||||
while (true) {
|
||||
for (auto sub_sock : poller->poll(100)) {
|
||||
Message * msg = sub_sock->receive();
|
||||
if (msg == NULL) continue;
|
||||
sub2pub[sub_sock]->sendMessage(msg);
|
||||
delete msg;
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -0,0 +1,230 @@
|
|||
#include <cassert>
|
||||
#include <cstring>
|
||||
#include <iostream>
|
||||
#include <cstdlib>
|
||||
#include <csignal>
|
||||
#include <cerrno>
|
||||
|
||||
#include "services.h"
|
||||
#include "impl_msgq.h"
|
||||
|
||||
|
||||
volatile sig_atomic_t msgq_do_exit = 0;
|
||||
|
||||
void sig_handler(int signal) {
|
||||
assert(signal == SIGINT || signal == SIGTERM);
|
||||
msgq_do_exit = 1;
|
||||
}
|
||||
|
||||
static bool service_exists(std::string path){
|
||||
for (const auto& it : services) {
|
||||
if (it.name == path) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
static size_t get_size(std::string endpoint){
|
||||
size_t sz = DEFAULT_SEGMENT_SIZE;
|
||||
|
||||
if (endpoint == "roadCameraState" || endpoint == "driverCameraState" || endpoint == "wideRoadCameraState"){
|
||||
sz *= 10;
|
||||
}
|
||||
|
||||
return sz;
|
||||
}
|
||||
|
||||
|
||||
MSGQContext::MSGQContext() {
|
||||
}
|
||||
|
||||
MSGQContext::~MSGQContext() {
|
||||
}
|
||||
|
||||
void MSGQMessage::init(size_t sz) {
|
||||
size = sz;
|
||||
data = new char[size];
|
||||
}
|
||||
|
||||
void MSGQMessage::init(char * d, size_t sz) {
|
||||
size = sz;
|
||||
data = new char[size];
|
||||
memcpy(data, d, size);
|
||||
}
|
||||
|
||||
void MSGQMessage::takeOwnership(char * d, size_t sz) {
|
||||
size = sz;
|
||||
data = d;
|
||||
}
|
||||
|
||||
void MSGQMessage::close() {
|
||||
if (size > 0){
|
||||
delete[] data;
|
||||
}
|
||||
size = 0;
|
||||
}
|
||||
|
||||
MSGQMessage::~MSGQMessage() {
|
||||
this->close();
|
||||
}
|
||||
|
||||
int MSGQSubSocket::connect(Context *context, std::string endpoint, std::string address, bool conflate, bool check_endpoint){
|
||||
assert(context);
|
||||
assert(address == "127.0.0.1");
|
||||
|
||||
if (check_endpoint && !service_exists(std::string(endpoint))){
|
||||
std::cout << "Warning, " << std::string(endpoint) << " is not in service list." << std::endl;
|
||||
}
|
||||
|
||||
q = new msgq_queue_t;
|
||||
int r = msgq_new_queue(q, endpoint.c_str(), get_size(endpoint));
|
||||
if (r != 0){
|
||||
return r;
|
||||
}
|
||||
|
||||
msgq_init_subscriber(q);
|
||||
|
||||
if (conflate){
|
||||
q->read_conflate = true;
|
||||
}
|
||||
|
||||
timeout = -1;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
Message * MSGQSubSocket::receive(bool non_blocking){
|
||||
msgq_do_exit = 0;
|
||||
|
||||
void (*prev_handler_sigint)(int);
|
||||
void (*prev_handler_sigterm)(int);
|
||||
if (!non_blocking){
|
||||
prev_handler_sigint = std::signal(SIGINT, sig_handler);
|
||||
prev_handler_sigterm = std::signal(SIGTERM, sig_handler);
|
||||
}
|
||||
|
||||
msgq_msg_t msg;
|
||||
|
||||
MSGQMessage *r = NULL;
|
||||
|
||||
int rc = msgq_msg_recv(&msg, q);
|
||||
|
||||
// Hack to implement blocking read with a poller. Don't use this
|
||||
while (!non_blocking && rc == 0 && msgq_do_exit == 0){
|
||||
msgq_pollitem_t items[1];
|
||||
items[0].q = q;
|
||||
|
||||
int t = (timeout != -1) ? timeout : 100;
|
||||
|
||||
int n = msgq_poll(items, 1, t);
|
||||
rc = msgq_msg_recv(&msg, q);
|
||||
|
||||
// The poll indicated a message was ready, but the receive failed. Try again
|
||||
if (n == 1 && rc == 0){
|
||||
continue;
|
||||
}
|
||||
|
||||
if (timeout != -1){
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
if (!non_blocking){
|
||||
std::signal(SIGINT, prev_handler_sigint);
|
||||
std::signal(SIGTERM, prev_handler_sigterm);
|
||||
}
|
||||
|
||||
errno = msgq_do_exit ? EINTR : 0;
|
||||
|
||||
if (rc > 0){
|
||||
if (msgq_do_exit){
|
||||
msgq_msg_close(&msg); // Free unused message on exit
|
||||
} else {
|
||||
r = new MSGQMessage;
|
||||
r->takeOwnership(msg.data, msg.size);
|
||||
}
|
||||
}
|
||||
|
||||
return (Message*)r;
|
||||
}
|
||||
|
||||
void MSGQSubSocket::setTimeout(int t){
|
||||
timeout = t;
|
||||
}
|
||||
|
||||
MSGQSubSocket::~MSGQSubSocket(){
|
||||
if (q != NULL){
|
||||
msgq_close_queue(q);
|
||||
delete q;
|
||||
}
|
||||
}
|
||||
|
||||
int MSGQPubSocket::connect(Context *context, std::string endpoint, bool check_endpoint){
|
||||
assert(context);
|
||||
|
||||
if (check_endpoint && !service_exists(std::string(endpoint))){
|
||||
std::cout << "Warning, " << std::string(endpoint) << " is not in service list." << std::endl;
|
||||
}
|
||||
|
||||
q = new msgq_queue_t;
|
||||
int r = msgq_new_queue(q, endpoint.c_str(), get_size(endpoint));
|
||||
if (r != 0){
|
||||
return r;
|
||||
}
|
||||
|
||||
msgq_init_publisher(q);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int MSGQPubSocket::sendMessage(Message *message){
|
||||
msgq_msg_t msg;
|
||||
msg.data = message->getData();
|
||||
msg.size = message->getSize();
|
||||
|
||||
return msgq_msg_send(&msg, q);
|
||||
}
|
||||
|
||||
int MSGQPubSocket::send(char *data, size_t size){
|
||||
msgq_msg_t msg;
|
||||
msg.data = data;
|
||||
msg.size = size;
|
||||
|
||||
return msgq_msg_send(&msg, q);
|
||||
}
|
||||
|
||||
bool MSGQPubSocket::all_readers_updated() {
|
||||
return msgq_all_readers_updated(q);
|
||||
}
|
||||
|
||||
MSGQPubSocket::~MSGQPubSocket(){
|
||||
if (q != NULL){
|
||||
msgq_close_queue(q);
|
||||
delete q;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
void MSGQPoller::registerSocket(SubSocket * socket){
|
||||
assert(num_polls + 1 < MAX_POLLERS);
|
||||
polls[num_polls].q = (msgq_queue_t*)socket->getRawSocket();
|
||||
|
||||
sockets.push_back(socket);
|
||||
num_polls++;
|
||||
}
|
||||
|
||||
std::vector<SubSocket*> MSGQPoller::poll(int timeout){
|
||||
std::vector<SubSocket*> r;
|
||||
|
||||
msgq_poll(polls, num_polls, timeout);
|
||||
for (size_t i = 0; i < num_polls; i++){
|
||||
if (polls[i].revents){
|
||||
r.push_back(sockets[i]);
|
||||
}
|
||||
}
|
||||
|
||||
return r;
|
||||
}
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
#pragma once
|
||||
#include "messaging.h"
|
||||
#include "msgq.h"
|
||||
#include <zmq.h>
|
||||
#include <string>
|
||||
|
||||
#define MAX_POLLERS 128
|
||||
|
||||
class MSGQContext : public Context {
|
||||
private:
|
||||
void * context = NULL;
|
||||
public:
|
||||
MSGQContext();
|
||||
void * getRawContext() {return context;}
|
||||
~MSGQContext();
|
||||
};
|
||||
|
||||
class MSGQMessage : public Message {
|
||||
private:
|
||||
char * data;
|
||||
size_t size;
|
||||
public:
|
||||
void init(size_t size);
|
||||
void init(char *data, size_t size);
|
||||
void takeOwnership(char *data, size_t size);
|
||||
size_t getSize(){return size;}
|
||||
char * getData(){return data;}
|
||||
void close();
|
||||
~MSGQMessage();
|
||||
};
|
||||
|
||||
class MSGQSubSocket : public SubSocket {
|
||||
private:
|
||||
msgq_queue_t * q = NULL;
|
||||
int timeout;
|
||||
public:
|
||||
int connect(Context *context, std::string endpoint, std::string address, bool conflate=false, bool check_endpoint=true);
|
||||
void setTimeout(int timeout);
|
||||
void * getRawSocket() {return (void*)q;}
|
||||
Message *receive(bool non_blocking=false);
|
||||
~MSGQSubSocket();
|
||||
};
|
||||
|
||||
class MSGQPubSocket : public PubSocket {
|
||||
private:
|
||||
msgq_queue_t * q = NULL;
|
||||
public:
|
||||
int connect(Context *context, std::string endpoint, bool check_endpoint=true);
|
||||
int sendMessage(Message *message);
|
||||
int send(char *data, size_t size);
|
||||
bool all_readers_updated();
|
||||
~MSGQPubSocket();
|
||||
};
|
||||
|
||||
class MSGQPoller : public Poller {
|
||||
private:
|
||||
std::vector<SubSocket*> sockets;
|
||||
msgq_pollitem_t polls[MAX_POLLERS];
|
||||
size_t num_polls = 0;
|
||||
|
||||
public:
|
||||
void registerSocket(SubSocket *socket);
|
||||
std::vector<SubSocket*> poll(int timeout);
|
||||
~MSGQPoller(){};
|
||||
};
|
||||
|
|
@ -0,0 +1,168 @@
|
|||
#include <cassert>
|
||||
#include <cstring>
|
||||
#include <iostream>
|
||||
#include <cstdlib>
|
||||
#include <cerrno>
|
||||
|
||||
#include <zmq.h>
|
||||
|
||||
#include "services.h"
|
||||
#include "impl_zmq.h"
|
||||
|
||||
static int get_port(std::string endpoint) {
|
||||
int port = -1;
|
||||
for (const auto& it : services) {
|
||||
std::string name = it.name;
|
||||
if (name == endpoint) {
|
||||
port = it.port;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
assert(port >= 0);
|
||||
return port;
|
||||
}
|
||||
|
||||
ZMQContext::ZMQContext() {
|
||||
context = zmq_ctx_new();
|
||||
}
|
||||
|
||||
ZMQContext::~ZMQContext() {
|
||||
zmq_ctx_term(context);
|
||||
}
|
||||
|
||||
void ZMQMessage::init(size_t sz) {
|
||||
size = sz;
|
||||
data = new char[size];
|
||||
}
|
||||
|
||||
void ZMQMessage::init(char * d, size_t sz) {
|
||||
size = sz;
|
||||
data = new char[size];
|
||||
memcpy(data, d, size);
|
||||
}
|
||||
|
||||
void ZMQMessage::close() {
|
||||
if (size > 0){
|
||||
delete[] data;
|
||||
}
|
||||
size = 0;
|
||||
}
|
||||
|
||||
ZMQMessage::~ZMQMessage() {
|
||||
this->close();
|
||||
}
|
||||
|
||||
|
||||
int ZMQSubSocket::connect(Context *context, std::string endpoint, std::string address, bool conflate, bool check_endpoint){
|
||||
sock = zmq_socket(context->getRawContext(), ZMQ_SUB);
|
||||
if (sock == NULL){
|
||||
return -1;
|
||||
}
|
||||
|
||||
zmq_setsockopt(sock, ZMQ_SUBSCRIBE, "", 0);
|
||||
|
||||
if (conflate){
|
||||
int arg = 1;
|
||||
zmq_setsockopt(sock, ZMQ_CONFLATE, &arg, sizeof(int));
|
||||
}
|
||||
|
||||
int reconnect_ivl = 500;
|
||||
zmq_setsockopt(sock, ZMQ_RECONNECT_IVL_MAX, &reconnect_ivl, sizeof(reconnect_ivl));
|
||||
|
||||
full_endpoint = "tcp://" + address + ":";
|
||||
if (check_endpoint){
|
||||
full_endpoint += std::to_string(get_port(endpoint));
|
||||
} else {
|
||||
full_endpoint += endpoint;
|
||||
}
|
||||
|
||||
return zmq_connect(sock, full_endpoint.c_str());
|
||||
}
|
||||
|
||||
|
||||
Message * ZMQSubSocket::receive(bool non_blocking){
|
||||
zmq_msg_t msg;
|
||||
assert(zmq_msg_init(&msg) == 0);
|
||||
|
||||
int flags = non_blocking ? ZMQ_DONTWAIT : 0;
|
||||
int rc = zmq_msg_recv(&msg, sock, flags);
|
||||
Message *r = NULL;
|
||||
|
||||
if (rc >= 0){
|
||||
// Make a copy to ensure the data is aligned
|
||||
r = new ZMQMessage;
|
||||
r->init((char*)zmq_msg_data(&msg), zmq_msg_size(&msg));
|
||||
}
|
||||
|
||||
zmq_msg_close(&msg);
|
||||
return r;
|
||||
}
|
||||
|
||||
void ZMQSubSocket::setTimeout(int timeout){
|
||||
zmq_setsockopt(sock, ZMQ_RCVTIMEO, &timeout, sizeof(int));
|
||||
}
|
||||
|
||||
ZMQSubSocket::~ZMQSubSocket(){
|
||||
zmq_close(sock);
|
||||
}
|
||||
|
||||
int ZMQPubSocket::connect(Context *context, std::string endpoint, bool check_endpoint){
|
||||
sock = zmq_socket(context->getRawContext(), ZMQ_PUB);
|
||||
if (sock == NULL){
|
||||
return -1;
|
||||
}
|
||||
|
||||
full_endpoint = "tcp://*:";
|
||||
if (check_endpoint){
|
||||
full_endpoint += std::to_string(get_port(endpoint));
|
||||
} else {
|
||||
full_endpoint += endpoint;
|
||||
}
|
||||
|
||||
return zmq_bind(sock, full_endpoint.c_str());
|
||||
}
|
||||
|
||||
int ZMQPubSocket::sendMessage(Message *message){
|
||||
return zmq_send(sock, message->getData(), message->getSize(), ZMQ_DONTWAIT);
|
||||
}
|
||||
|
||||
int ZMQPubSocket::send(char *data, size_t size){
|
||||
return zmq_send(sock, data, size, ZMQ_DONTWAIT);
|
||||
}
|
||||
|
||||
bool ZMQPubSocket::all_readers_updated() {
|
||||
assert(false); // TODO not implemented
|
||||
return false;
|
||||
}
|
||||
|
||||
ZMQPubSocket::~ZMQPubSocket(){
|
||||
zmq_close(sock);
|
||||
}
|
||||
|
||||
|
||||
void ZMQPoller::registerSocket(SubSocket * socket){
|
||||
assert(num_polls + 1 < MAX_POLLERS);
|
||||
polls[num_polls].socket = socket->getRawSocket();
|
||||
polls[num_polls].events = ZMQ_POLLIN;
|
||||
|
||||
sockets.push_back(socket);
|
||||
num_polls++;
|
||||
}
|
||||
|
||||
std::vector<SubSocket*> ZMQPoller::poll(int timeout){
|
||||
std::vector<SubSocket*> r;
|
||||
|
||||
int rc = zmq_poll(polls, num_polls, timeout);
|
||||
if (rc < 0){
|
||||
return r;
|
||||
}
|
||||
|
||||
for (size_t i = 0; i < num_polls; i++){
|
||||
if (polls[i].revents){
|
||||
r.push_back(sockets[i]);
|
||||
}
|
||||
}
|
||||
|
||||
return r;
|
||||
}
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
#pragma once
|
||||
#include "messaging.h"
|
||||
#include <zmq.h>
|
||||
#include <string>
|
||||
|
||||
#define MAX_POLLERS 128
|
||||
|
||||
class ZMQContext : public Context {
|
||||
private:
|
||||
void * context = NULL;
|
||||
public:
|
||||
ZMQContext();
|
||||
void * getRawContext() {return context;}
|
||||
~ZMQContext();
|
||||
};
|
||||
|
||||
class ZMQMessage : public Message {
|
||||
private:
|
||||
char * data;
|
||||
size_t size;
|
||||
public:
|
||||
void init(size_t size);
|
||||
void init(char *data, size_t size);
|
||||
size_t getSize(){return size;}
|
||||
char * getData(){return data;}
|
||||
void close();
|
||||
~ZMQMessage();
|
||||
};
|
||||
|
||||
class ZMQSubSocket : public SubSocket {
|
||||
private:
|
||||
void * sock;
|
||||
std::string full_endpoint;
|
||||
public:
|
||||
int connect(Context *context, std::string endpoint, std::string address, bool conflate=false, bool check_endpoint=true);
|
||||
void setTimeout(int timeout);
|
||||
void * getRawSocket() {return sock;}
|
||||
Message *receive(bool non_blocking=false);
|
||||
~ZMQSubSocket();
|
||||
};
|
||||
|
||||
class ZMQPubSocket : public PubSocket {
|
||||
private:
|
||||
void * sock;
|
||||
std::string full_endpoint;
|
||||
public:
|
||||
int connect(Context *context, std::string endpoint, bool check_endpoint=true);
|
||||
int sendMessage(Message *message);
|
||||
int send(char *data, size_t size);
|
||||
bool all_readers_updated();
|
||||
~ZMQPubSocket();
|
||||
};
|
||||
|
||||
class ZMQPoller : public Poller {
|
||||
private:
|
||||
std::vector<SubSocket*> sockets;
|
||||
zmq_pollitem_t polls[MAX_POLLERS];
|
||||
size_t num_polls = 0;
|
||||
|
||||
public:
|
||||
void registerSocket(SubSocket *socket);
|
||||
std::vector<SubSocket*> poll(int timeout);
|
||||
~ZMQPoller(){};
|
||||
};
|
||||
|
|
@ -0,0 +1,103 @@
|
|||
#include "messaging.h"
|
||||
#include "impl_zmq.h"
|
||||
#include "impl_msgq.h"
|
||||
|
||||
#ifdef __APPLE__
|
||||
const bool MUST_USE_ZMQ = true;
|
||||
#else
|
||||
const bool MUST_USE_ZMQ = false;
|
||||
#endif
|
||||
|
||||
bool messaging_use_zmq(){
|
||||
return std::getenv("ZMQ") || MUST_USE_ZMQ;
|
||||
}
|
||||
|
||||
Context * Context::create(){
|
||||
Context * c;
|
||||
if (messaging_use_zmq()){
|
||||
c = new ZMQContext();
|
||||
} else {
|
||||
c = new MSGQContext();
|
||||
}
|
||||
return c;
|
||||
}
|
||||
|
||||
SubSocket * SubSocket::create(){
|
||||
SubSocket * s;
|
||||
if (messaging_use_zmq()){
|
||||
s = new ZMQSubSocket();
|
||||
} else {
|
||||
s = new MSGQSubSocket();
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
SubSocket * SubSocket::create(Context * context, std::string endpoint, std::string address, bool conflate, bool check_endpoint){
|
||||
SubSocket *s = SubSocket::create();
|
||||
int r = s->connect(context, endpoint, address, conflate, check_endpoint);
|
||||
|
||||
if (r == 0) {
|
||||
return s;
|
||||
} else {
|
||||
delete s;
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
PubSocket * PubSocket::create(){
|
||||
PubSocket * s;
|
||||
if (messaging_use_zmq()){
|
||||
s = new ZMQPubSocket();
|
||||
} else {
|
||||
s = new MSGQPubSocket();
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
PubSocket * PubSocket::create(Context * context, std::string endpoint, bool check_endpoint){
|
||||
PubSocket *s = PubSocket::create();
|
||||
int r = s->connect(context, endpoint, check_endpoint);
|
||||
|
||||
if (r == 0) {
|
||||
return s;
|
||||
} else {
|
||||
delete s;
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
Poller * Poller::create(){
|
||||
Poller * p;
|
||||
if (messaging_use_zmq()){
|
||||
p = new ZMQPoller();
|
||||
} else {
|
||||
p = new MSGQPoller();
|
||||
}
|
||||
return p;
|
||||
}
|
||||
|
||||
Poller * Poller::create(std::vector<SubSocket*> sockets){
|
||||
Poller * p = Poller::create();
|
||||
|
||||
for (auto s : sockets){
|
||||
p->registerSocket(s);
|
||||
}
|
||||
return p;
|
||||
}
|
||||
|
||||
extern "C" Context * messaging_context_create() {
|
||||
return Context::create();
|
||||
}
|
||||
|
||||
extern "C" SubSocket * messaging_subsocket_create(Context* context, const char* endpoint) {
|
||||
return SubSocket::create(context, std::string(endpoint));
|
||||
}
|
||||
|
||||
extern "C" PubSocket * messaging_pubsocket_create(Context* context, const char* endpoint) {
|
||||
return PubSocket::create(context, std::string(endpoint));
|
||||
}
|
||||
|
||||
extern "C" Poller * messaging_poller_create(SubSocket** sockets, int size) {
|
||||
std::vector<SubSocket*> socketsVec(sockets, sockets + size);
|
||||
return Poller::create(socketsVec);
|
||||
}
|
||||
|
|
@ -0,0 +1,145 @@
|
|||
#pragma once
|
||||
#include <cstddef>
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include <capnp/serialize.h>
|
||||
#include "../gen/cpp/log.capnp.h"
|
||||
|
||||
#ifdef __APPLE__
|
||||
#define CLOCK_BOOTTIME CLOCK_MONOTONIC
|
||||
#endif
|
||||
|
||||
#define MSG_MULTIPLE_PUBLISHERS 100
|
||||
|
||||
bool messaging_use_zmq();
|
||||
|
||||
class Context {
|
||||
public:
|
||||
virtual void * getRawContext() = 0;
|
||||
static Context * create();
|
||||
virtual ~Context(){};
|
||||
};
|
||||
|
||||
class Message {
|
||||
public:
|
||||
virtual void init(size_t size) = 0;
|
||||
virtual void init(char * data, size_t size) = 0;
|
||||
virtual void close() = 0;
|
||||
virtual size_t getSize() = 0;
|
||||
virtual char * getData() = 0;
|
||||
virtual ~Message(){};
|
||||
};
|
||||
|
||||
|
||||
class SubSocket {
|
||||
public:
|
||||
virtual int connect(Context *context, std::string endpoint, std::string address, bool conflate=false, bool check_endpoint=true) = 0;
|
||||
virtual void setTimeout(int timeout) = 0;
|
||||
virtual Message *receive(bool non_blocking=false) = 0;
|
||||
virtual void * getRawSocket() = 0;
|
||||
static SubSocket * create();
|
||||
static SubSocket * create(Context * context, std::string endpoint, std::string address="127.0.0.1", bool conflate=false, bool check_endpoint=true);
|
||||
virtual ~SubSocket(){};
|
||||
};
|
||||
|
||||
class PubSocket {
|
||||
public:
|
||||
virtual int connect(Context *context, std::string endpoint, bool check_endpoint=true) = 0;
|
||||
virtual int sendMessage(Message *message) = 0;
|
||||
virtual int send(char *data, size_t size) = 0;
|
||||
virtual bool all_readers_updated() = 0;
|
||||
static PubSocket * create();
|
||||
static PubSocket * create(Context * context, std::string endpoint, bool check_endpoint=true);
|
||||
static PubSocket * create(Context * context, std::string endpoint, int port, bool check_endpoint=true);
|
||||
virtual ~PubSocket(){};
|
||||
};
|
||||
|
||||
class Poller {
|
||||
public:
|
||||
virtual void registerSocket(SubSocket *socket) = 0;
|
||||
virtual std::vector<SubSocket*> poll(int timeout) = 0;
|
||||
static Poller * create();
|
||||
static Poller * create(std::vector<SubSocket*> sockets);
|
||||
virtual ~Poller(){};
|
||||
};
|
||||
|
||||
class SubMaster {
|
||||
public:
|
||||
SubMaster(const std::vector<const char *> &service_list,
|
||||
const char *address = nullptr, const std::vector<const char *> &ignore_alive = {});
|
||||
void update(int timeout = 1000);
|
||||
void update_msgs(uint64_t current_time, const std::vector<std::pair<std::string, cereal::Event::Reader>> &messages);
|
||||
inline bool allAlive(const std::vector<const char *> &service_list = {}) { return all_(service_list, false, true); }
|
||||
inline bool allValid(const std::vector<const char *> &service_list = {}) { return all_(service_list, true, false); }
|
||||
inline bool allAliveAndValid(const std::vector<const char *> &service_list = {}) { return all_(service_list, true, true); }
|
||||
void drain();
|
||||
~SubMaster();
|
||||
|
||||
uint64_t frame = 0;
|
||||
bool updated(const char *name) const;
|
||||
bool alive(const char *name) const;
|
||||
bool valid(const char *name) const;
|
||||
uint64_t rcv_frame(const char *name) const;
|
||||
uint64_t rcv_time(const char *name) const;
|
||||
cereal::Event::Reader &operator[](const char *name) const;
|
||||
|
||||
private:
|
||||
bool all_(const std::vector<const char *> &service_list, bool valid, bool alive);
|
||||
Poller *poller_ = nullptr;
|
||||
struct SubMessage;
|
||||
std::map<SubSocket *, SubMessage *> messages_;
|
||||
std::map<std::string, SubMessage *> services_;
|
||||
};
|
||||
|
||||
class MessageBuilder : public capnp::MallocMessageBuilder {
|
||||
public:
|
||||
MessageBuilder() = default;
|
||||
|
||||
cereal::Event::Builder initEvent(bool valid = true) {
|
||||
cereal::Event::Builder event = initRoot<cereal::Event>();
|
||||
struct timespec t;
|
||||
clock_gettime(CLOCK_BOOTTIME, &t);
|
||||
uint64_t current_time = t.tv_sec * 1000000000ULL + t.tv_nsec;
|
||||
event.setLogMonoTime(current_time);
|
||||
event.setValid(valid);
|
||||
return event;
|
||||
}
|
||||
|
||||
kj::ArrayPtr<capnp::byte> toBytes() {
|
||||
heapArray_ = capnp::messageToFlatArray(*this);
|
||||
return heapArray_.asBytes();
|
||||
}
|
||||
|
||||
private:
|
||||
kj::Array<capnp::word> heapArray_;
|
||||
};
|
||||
|
||||
class PubMaster {
|
||||
public:
|
||||
PubMaster(const std::vector<const char *> &service_list);
|
||||
inline int send(const char *name, capnp::byte *data, size_t size) { return sockets_.at(name)->send((char *)data, size); }
|
||||
int send(const char *name, MessageBuilder &msg);
|
||||
~PubMaster();
|
||||
|
||||
private:
|
||||
std::map<std::string, PubSocket *> sockets_;
|
||||
};
|
||||
|
||||
class AlignedBuffer {
|
||||
public:
|
||||
kj::ArrayPtr<const capnp::word> align(const char *data, const size_t size) {
|
||||
words_size = size / sizeof(capnp::word) + 1;
|
||||
if (aligned_buf.size() < words_size) {
|
||||
aligned_buf = kj::heapArray<capnp::word>(words_size < 512 ? 512 : words_size);
|
||||
}
|
||||
memcpy(aligned_buf.begin(), data, size);
|
||||
return aligned_buf.slice(0, words_size);
|
||||
}
|
||||
inline kj::ArrayPtr<const capnp::word> align(Message *m) {
|
||||
return align(m->getData(), m->getSize());
|
||||
}
|
||||
private:
|
||||
kj::Array<capnp::word> aligned_buf;
|
||||
size_t words_size;
|
||||
};
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
# distutils: language = c++
|
||||
#cython: language_level=3
|
||||
|
||||
from libcpp.string cimport string
|
||||
from libcpp.vector cimport vector
|
||||
from libcpp cimport bool
|
||||
|
||||
|
||||
cdef extern from "messaging.h":
|
||||
cdef cppclass Context:
|
||||
@staticmethod
|
||||
Context * create()
|
||||
|
||||
cdef cppclass Message:
|
||||
void init(size_t)
|
||||
void init(char *, size_t)
|
||||
void close()
|
||||
size_t getSize()
|
||||
char *getData()
|
||||
|
||||
cdef cppclass SubSocket:
|
||||
@staticmethod
|
||||
SubSocket * create()
|
||||
int connect(Context *, string, string, bool)
|
||||
Message * receive(bool)
|
||||
void setTimeout(int)
|
||||
|
||||
cdef cppclass PubSocket:
|
||||
@staticmethod
|
||||
PubSocket * create()
|
||||
int connect(Context *, string)
|
||||
int sendMessage(Message *)
|
||||
int send(char *, size_t)
|
||||
bool all_readers_updated()
|
||||
|
||||
cdef cppclass Poller:
|
||||
@staticmethod
|
||||
Poller * create()
|
||||
void registerSocket(SubSocket *)
|
||||
vector[SubSocket*] poll(int) nogil
|
||||
|
|
@ -0,0 +1,154 @@
|
|||
# distutils: language = c++
|
||||
# cython: c_string_encoding=ascii, language_level=3
|
||||
|
||||
import sys
|
||||
from libcpp.string cimport string
|
||||
from libcpp cimport bool
|
||||
from libc cimport errno
|
||||
|
||||
|
||||
from .messaging cimport Context as cppContext
|
||||
from .messaging cimport SubSocket as cppSubSocket
|
||||
from .messaging cimport PubSocket as cppPubSocket
|
||||
from .messaging cimport Poller as cppPoller
|
||||
from .messaging cimport Message as cppMessage
|
||||
|
||||
|
||||
class MessagingError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class MultiplePublishersError(MessagingError):
|
||||
pass
|
||||
|
||||
|
||||
cdef class Context:
|
||||
cdef cppContext * context
|
||||
|
||||
def __cinit__(self):
|
||||
self.context = cppContext.create()
|
||||
|
||||
def term(self):
|
||||
del self.context
|
||||
self.context = NULL
|
||||
|
||||
def __dealloc__(self):
|
||||
pass
|
||||
# Deleting the context will hang if sockets are still active
|
||||
# TODO: Figure out a way to make sure the context is closed last
|
||||
# del self.context
|
||||
|
||||
|
||||
cdef class Poller:
|
||||
cdef cppPoller * poller
|
||||
cdef list sub_sockets
|
||||
|
||||
def __cinit__(self):
|
||||
self.sub_sockets = []
|
||||
self.poller = cppPoller.create()
|
||||
|
||||
def __dealloc__(self):
|
||||
del self.poller
|
||||
|
||||
def registerSocket(self, SubSocket socket):
|
||||
self.sub_sockets.append(socket)
|
||||
self.poller.registerSocket(socket.socket)
|
||||
|
||||
def poll(self, timeout):
|
||||
sockets = []
|
||||
cdef int t = timeout
|
||||
|
||||
with nogil:
|
||||
result = self.poller.poll(t)
|
||||
|
||||
for s in result:
|
||||
socket = SubSocket()
|
||||
socket.setPtr(s)
|
||||
sockets.append(socket)
|
||||
|
||||
return sockets
|
||||
|
||||
cdef class SubSocket:
|
||||
cdef cppSubSocket * socket
|
||||
cdef bool is_owner
|
||||
|
||||
def __cinit__(self):
|
||||
self.socket = cppSubSocket.create()
|
||||
self.is_owner = True
|
||||
|
||||
if self.socket == NULL:
|
||||
raise MessagingError
|
||||
|
||||
def __dealloc__(self):
|
||||
if self.is_owner:
|
||||
del self.socket
|
||||
|
||||
cdef setPtr(self, cppSubSocket * ptr):
|
||||
if self.is_owner:
|
||||
del self.socket
|
||||
|
||||
self.is_owner = False
|
||||
self.socket = ptr
|
||||
|
||||
def connect(self, Context context, string endpoint, string address=b"127.0.0.1", bool conflate=False):
|
||||
r = self.socket.connect(context.context, endpoint, address, conflate)
|
||||
|
||||
if r != 0:
|
||||
if errno.errno == errno.EADDRINUSE:
|
||||
raise MultiplePublishersError
|
||||
else:
|
||||
raise MessagingError
|
||||
|
||||
def setTimeout(self, int timeout):
|
||||
self.socket.setTimeout(timeout)
|
||||
|
||||
def receive(self, bool non_blocking=False):
|
||||
msg = self.socket.receive(non_blocking)
|
||||
|
||||
if msg == NULL:
|
||||
# If a blocking read returns no message check errno if SIGINT was caught in the C++ code
|
||||
if errno.errno == errno.EINTR:
|
||||
print("SIGINT received, exiting")
|
||||
sys.exit(1)
|
||||
|
||||
return None
|
||||
else:
|
||||
sz = msg.getSize()
|
||||
m = msg.getData()[:sz]
|
||||
del msg
|
||||
|
||||
return m
|
||||
|
||||
|
||||
cdef class PubSocket:
|
||||
cdef cppPubSocket * socket
|
||||
|
||||
def __cinit__(self):
|
||||
self.socket = cppPubSocket.create()
|
||||
if self.socket == NULL:
|
||||
raise MessagingError
|
||||
|
||||
def __dealloc__(self):
|
||||
del self.socket
|
||||
|
||||
def connect(self, Context context, string endpoint):
|
||||
r = self.socket.connect(context.context, endpoint)
|
||||
|
||||
if r != 0:
|
||||
if errno.errno == errno.EADDRINUSE:
|
||||
raise MultiplePublishersError
|
||||
else:
|
||||
raise MessagingError
|
||||
|
||||
def send(self, bytes data):
|
||||
length = len(data)
|
||||
r = self.socket.send(<char*>data, length)
|
||||
|
||||
if r != length:
|
||||
if errno.errno == errno.EADDRINUSE:
|
||||
raise MultiplePublishersError
|
||||
else:
|
||||
raise MessagingError
|
||||
|
||||
def all_readers_updated(self):
|
||||
return self.socket.all_readers_updated()
|
||||
|
|
@ -0,0 +1,464 @@
|
|||
#include <iostream>
|
||||
#include <cassert>
|
||||
#include <cerrno>
|
||||
#include <cmath>
|
||||
#include <cstring>
|
||||
#include <cstdint>
|
||||
#include <chrono>
|
||||
#include <algorithm>
|
||||
#include <cstdlib>
|
||||
#include <csignal>
|
||||
#include <random>
|
||||
|
||||
#include <poll.h>
|
||||
#include <sys/ioctl.h>
|
||||
#include <sys/mman.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/syscall.h>
|
||||
#include <fcntl.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <stdio.h>
|
||||
|
||||
#include "msgq.h"
|
||||
|
||||
void sigusr2_handler(int signal) {
|
||||
assert(signal == SIGUSR2);
|
||||
}
|
||||
|
||||
uint64_t msgq_get_uid(void){
|
||||
std::random_device rd("/dev/urandom");
|
||||
std::uniform_int_distribution<uint64_t> distribution(0,std::numeric_limits<uint32_t>::max());
|
||||
|
||||
#ifdef __APPLE__
|
||||
// TODO: this doesn't work
|
||||
uint64_t uid = distribution(rd) << 32 | getpid();
|
||||
#else
|
||||
uint64_t uid = distribution(rd) << 32 | syscall(SYS_gettid);
|
||||
#endif
|
||||
|
||||
return uid;
|
||||
}
|
||||
|
||||
int msgq_msg_init_size(msgq_msg_t * msg, size_t size){
|
||||
msg->size = size;
|
||||
msg->data = new(std::nothrow) char[size];
|
||||
|
||||
return (msg->data == NULL) ? -1 : 0;
|
||||
}
|
||||
|
||||
|
||||
int msgq_msg_init_data(msgq_msg_t * msg, char * data, size_t size) {
|
||||
int r = msgq_msg_init_size(msg, size);
|
||||
|
||||
if (r == 0)
|
||||
memcpy(msg->data, data, size);
|
||||
|
||||
return r;
|
||||
}
|
||||
|
||||
int msgq_msg_close(msgq_msg_t * msg){
|
||||
if (msg->size > 0)
|
||||
delete[] msg->data;
|
||||
|
||||
msg->size = 0;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
void msgq_reset_reader(msgq_queue_t * q){
|
||||
int id = q->reader_id;
|
||||
q->read_valids[id]->store(true);
|
||||
q->read_pointers[id]->store(*q->write_pointer);
|
||||
}
|
||||
|
||||
void msgq_wait_for_subscriber(msgq_queue_t *q){
|
||||
while (*q->num_readers == 0){
|
||||
;
|
||||
}
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
int msgq_new_queue(msgq_queue_t * q, const char * path, size_t size){
|
||||
assert(size < 0xFFFFFFFF); // Buffer must be smaller than 2^32 bytes
|
||||
std::signal(SIGUSR2, sigusr2_handler);
|
||||
|
||||
const char * prefix = "/dev/shm/";
|
||||
char * full_path = new char[strlen(path) + strlen(prefix) + 1];
|
||||
strcpy(full_path, prefix);
|
||||
strcat(full_path, path);
|
||||
|
||||
auto fd = open(full_path, O_RDWR | O_CREAT, 0664);
|
||||
if (fd < 0) {
|
||||
std::cout << "Warning, could not open: " << full_path << std::endl;
|
||||
delete[] full_path;
|
||||
return -1;
|
||||
}
|
||||
delete[] full_path;
|
||||
|
||||
int rc = ftruncate(fd, size + sizeof(msgq_header_t));
|
||||
if (rc < 0){
|
||||
close(fd);
|
||||
return -1;
|
||||
}
|
||||
char * mem = (char*)mmap(NULL, size + sizeof(msgq_header_t), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
|
||||
close(fd);
|
||||
|
||||
if (mem == NULL){
|
||||
return -1;
|
||||
}
|
||||
q->mmap_p = mem;
|
||||
|
||||
msgq_header_t *header = (msgq_header_t *)mem;
|
||||
|
||||
// Setup pointers to header segment
|
||||
q->num_readers = reinterpret_cast<std::atomic<uint64_t>*>(&header->num_readers);
|
||||
q->write_pointer = reinterpret_cast<std::atomic<uint64_t>*>(&header->write_pointer);
|
||||
q->write_uid = reinterpret_cast<std::atomic<uint64_t>*>(&header->write_uid);
|
||||
|
||||
for (size_t i = 0; i < NUM_READERS; i++){
|
||||
q->read_pointers[i] = reinterpret_cast<std::atomic<uint64_t>*>(&header->read_pointers[i]);
|
||||
q->read_valids[i] = reinterpret_cast<std::atomic<uint64_t>*>(&header->read_valids[i]);
|
||||
q->read_uids[i] = reinterpret_cast<std::atomic<uint64_t>*>(&header->read_uids[i]);
|
||||
}
|
||||
|
||||
q->data = mem + sizeof(msgq_header_t);
|
||||
q->size = size;
|
||||
q->reader_id = -1;
|
||||
|
||||
q->endpoint = path;
|
||||
q->read_conflate = false;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
void msgq_close_queue(msgq_queue_t *q){
|
||||
if (q->mmap_p != NULL){
|
||||
munmap(q->mmap_p, q->size + sizeof(msgq_header_t));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
void msgq_init_publisher(msgq_queue_t * q) {
|
||||
//std::cout << "Starting publisher" << std::endl;
|
||||
uint64_t uid = msgq_get_uid();
|
||||
|
||||
*q->write_uid = uid;
|
||||
*q->num_readers = 0;
|
||||
|
||||
for (size_t i = 0; i < NUM_READERS; i++){
|
||||
*q->read_valids[i] = false;
|
||||
*q->read_uids[i] = 0;
|
||||
}
|
||||
|
||||
q->write_uid_local = uid;
|
||||
}
|
||||
|
||||
static void thread_signal(uint32_t tid) {
|
||||
#ifndef SYS_tkill
|
||||
// TODO: this won't work for multithreaded programs
|
||||
kill(tid, SIGUSR2);
|
||||
#else
|
||||
syscall(SYS_tkill, tid, SIGUSR2);
|
||||
#endif
|
||||
}
|
||||
|
||||
void msgq_init_subscriber(msgq_queue_t * q) {
|
||||
assert(q != NULL);
|
||||
assert(q->num_readers != NULL);
|
||||
|
||||
uint64_t uid = msgq_get_uid();
|
||||
|
||||
// Get reader id
|
||||
while (true){
|
||||
uint64_t cur_num_readers = *q->num_readers;
|
||||
uint64_t new_num_readers = cur_num_readers + 1;
|
||||
|
||||
// No more slots available. Reset all subscribers to kick out inactive ones
|
||||
if (new_num_readers > NUM_READERS){
|
||||
std::cout << "Warning, evicting all subscribers!" << std::endl;
|
||||
*q->num_readers = 0;
|
||||
|
||||
for (size_t i = 0; i < NUM_READERS; i++){
|
||||
*q->read_valids[i] = false;
|
||||
|
||||
uint64_t old_uid = *q->read_uids[i];
|
||||
*q->read_uids[i] = 0;
|
||||
|
||||
// Wake up reader in case they are in a poll
|
||||
thread_signal(old_uid & 0xFFFFFFFF);
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
// Use atomic compare and swap to handle race condition
|
||||
// where two subscribers start at the same time
|
||||
if (std::atomic_compare_exchange_strong(q->num_readers,
|
||||
&cur_num_readers,
|
||||
new_num_readers)){
|
||||
q->reader_id = cur_num_readers;
|
||||
q->read_uid_local = uid;
|
||||
|
||||
// We start with read_valid = false,
|
||||
// on the first read the read pointer will be synchronized with the write pointer
|
||||
*q->read_valids[cur_num_readers] = false;
|
||||
*q->read_pointers[cur_num_readers] = 0;
|
||||
*q->read_uids[cur_num_readers] = uid;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
//std::cout << "New subscriber id: " << q->reader_id << " uid: " << q->read_uid_local << " " << q->endpoint << std::endl;
|
||||
msgq_reset_reader(q);
|
||||
}
|
||||
|
||||
int msgq_msg_send(msgq_msg_t * msg, msgq_queue_t *q){
|
||||
// Die if we are no longer the active publisher
|
||||
if (q->write_uid_local != *q->write_uid){
|
||||
std::cout << "Killing old publisher: " << q->endpoint << std::endl;
|
||||
errno = EADDRINUSE;
|
||||
return -1;
|
||||
}
|
||||
|
||||
uint64_t total_msg_size = ALIGN(msg->size + sizeof(int64_t));
|
||||
|
||||
// We need to fit at least three messages in the queue,
|
||||
// then we can always safely access the last message
|
||||
assert(3 * total_msg_size <= q->size);
|
||||
|
||||
uint64_t num_readers = *q->num_readers;
|
||||
|
||||
uint32_t write_cycles, write_pointer;
|
||||
UNPACK64(write_cycles, write_pointer, *q->write_pointer);
|
||||
|
||||
char *p = q->data + write_pointer; // add base offset
|
||||
|
||||
// Check remaining space
|
||||
// Always leave space for a wraparound tag for the next message, including alignment
|
||||
int64_t remaining_space = q->size - write_pointer - total_msg_size - sizeof(int64_t);
|
||||
if (remaining_space <= 0){
|
||||
// Write -1 size tag indicating wraparound
|
||||
*(int64_t*)p = -1;
|
||||
|
||||
// Invalidate all readers that are beyond the write pointer
|
||||
// TODO: should we handle the case where a new reader shows up while this is running?
|
||||
for (uint64_t i = 0; i < num_readers; i++){
|
||||
uint64_t read_pointer = *q->read_pointers[i];
|
||||
uint64_t read_cycles = read_pointer >> 32;
|
||||
read_pointer &= 0xFFFFFFFF;
|
||||
|
||||
if ((read_pointer > write_pointer) && (read_cycles != write_cycles)) {
|
||||
*q->read_valids[i] = false;
|
||||
}
|
||||
}
|
||||
|
||||
// Update global and local copies of write pointer and write_cycles
|
||||
write_pointer = 0;
|
||||
write_cycles = write_cycles + 1;
|
||||
PACK64(*q->write_pointer, write_cycles, write_pointer);
|
||||
|
||||
// Set actual pointer to the beginning of the data segment
|
||||
p = q->data;
|
||||
}
|
||||
|
||||
// Invalidate readers that are in the area that will be written
|
||||
uint64_t start = write_pointer;
|
||||
uint64_t end = ALIGN(start + sizeof(int64_t) + msg->size);
|
||||
|
||||
for (uint64_t i = 0; i < num_readers; i++){
|
||||
uint32_t read_cycles, read_pointer;
|
||||
UNPACK64(read_cycles, read_pointer, *q->read_pointers[i]);
|
||||
|
||||
if ((read_pointer >= start) && (read_pointer < end) && (read_cycles != write_cycles)) {
|
||||
*q->read_valids[i] = false;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
// Write size tag
|
||||
std::atomic<int64_t> *size_p = reinterpret_cast<std::atomic<int64_t>*>(p);
|
||||
*size_p = msg->size;
|
||||
|
||||
// Copy data
|
||||
memcpy(p + sizeof(int64_t), msg->data, msg->size);
|
||||
__sync_synchronize();
|
||||
|
||||
// Update write pointer
|
||||
uint32_t new_ptr = ALIGN(write_pointer + msg->size + sizeof(int64_t));
|
||||
PACK64(*q->write_pointer, write_cycles, new_ptr);
|
||||
|
||||
// Notify readers
|
||||
for (uint64_t i = 0; i < num_readers; i++){
|
||||
uint64_t reader_uid = *q->read_uids[i];
|
||||
thread_signal(reader_uid & 0xFFFFFFFF);
|
||||
}
|
||||
|
||||
return msg->size;
|
||||
}
|
||||
|
||||
|
||||
int msgq_msg_ready(msgq_queue_t * q){
|
||||
start:
|
||||
int id = q->reader_id;
|
||||
assert(id >= 0); // Make sure subscriber is initialized
|
||||
|
||||
if (q->read_uid_local != *q->read_uids[id]){
|
||||
std::cout << q->endpoint << ": Reader was evicted, reconnecting" << std::endl;
|
||||
msgq_init_subscriber(q);
|
||||
goto start;
|
||||
}
|
||||
|
||||
// Check valid
|
||||
if (!*q->read_valids[id]){
|
||||
msgq_reset_reader(q);
|
||||
goto start;
|
||||
}
|
||||
|
||||
uint32_t read_cycles, read_pointer;
|
||||
UNPACK64(read_cycles, read_pointer, *q->read_pointers[id]);
|
||||
|
||||
uint32_t write_cycles, write_pointer;
|
||||
UNPACK64(write_cycles, write_pointer, *q->write_pointer);
|
||||
|
||||
// Check if new message is available
|
||||
return (read_pointer != write_pointer);
|
||||
}
|
||||
|
||||
int msgq_msg_recv(msgq_msg_t * msg, msgq_queue_t * q){
|
||||
start:
|
||||
int id = q->reader_id;
|
||||
assert(id >= 0); // Make sure subscriber is initialized
|
||||
|
||||
if (q->read_uid_local != *q->read_uids[id]){
|
||||
std::cout << q->endpoint << ": Reader was evicted, reconnecting" << std::endl;
|
||||
msgq_init_subscriber(q);
|
||||
goto start;
|
||||
}
|
||||
|
||||
// Check valid
|
||||
if (!*q->read_valids[id]){
|
||||
msgq_reset_reader(q);
|
||||
goto start;
|
||||
}
|
||||
|
||||
uint32_t read_cycles, read_pointer;
|
||||
UNPACK64(read_cycles, read_pointer, *q->read_pointers[id]);
|
||||
|
||||
uint32_t write_cycles, write_pointer;
|
||||
UNPACK64(write_cycles, write_pointer, *q->write_pointer);
|
||||
|
||||
char * p = q->data + read_pointer;
|
||||
|
||||
// Check if new message is available
|
||||
if (read_pointer == write_pointer) {
|
||||
msg->size = 0;
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Read potential message size
|
||||
std::atomic<int64_t> *size_p = reinterpret_cast<std::atomic<int64_t>*>(p);
|
||||
std::int64_t size = *size_p;
|
||||
|
||||
// Check if the size that was read is valid
|
||||
if (!*q->read_valids[id]){
|
||||
msgq_reset_reader(q);
|
||||
goto start;
|
||||
}
|
||||
|
||||
// If size is -1 the buffer was full, and we need to wrap around
|
||||
if (size == -1){
|
||||
read_cycles++;
|
||||
PACK64(*q->read_pointers[id], read_cycles, 0);
|
||||
goto start;
|
||||
}
|
||||
|
||||
// crashing is better than passing garbage data to the consumer
|
||||
// the size will have weird value if it was overwritten by data accidentally
|
||||
assert((uint64_t)size < q->size);
|
||||
assert(size > 0);
|
||||
|
||||
uint32_t new_read_pointer = ALIGN(read_pointer + sizeof(std::int64_t) + size);
|
||||
|
||||
// If conflate is true, check if this is the latest message, else start over
|
||||
if (q->read_conflate){
|
||||
if (new_read_pointer != write_pointer){
|
||||
// Update read pointer
|
||||
PACK64(*q->read_pointers[id], read_cycles, new_read_pointer);
|
||||
goto start;
|
||||
}
|
||||
}
|
||||
|
||||
// Copy message
|
||||
if (msgq_msg_init_size(msg, size) < 0)
|
||||
return -1;
|
||||
|
||||
__sync_synchronize();
|
||||
memcpy(msg->data, p + sizeof(int64_t), size);
|
||||
__sync_synchronize();
|
||||
|
||||
// Update read pointer
|
||||
PACK64(*q->read_pointers[id], read_cycles, new_read_pointer);
|
||||
|
||||
// Check if the actual data that was copied is valid
|
||||
if (!*q->read_valids[id]){
|
||||
msgq_msg_close(msg);
|
||||
msgq_reset_reader(q);
|
||||
goto start;
|
||||
}
|
||||
|
||||
|
||||
return msg->size;
|
||||
}
|
||||
|
||||
|
||||
|
||||
int msgq_poll(msgq_pollitem_t * items, size_t nitems, int timeout){
|
||||
int num = 0;
|
||||
|
||||
// Check if messages ready
|
||||
for (size_t i = 0; i < nitems; i++) {
|
||||
items[i].revents = msgq_msg_ready(items[i].q);
|
||||
if (items[i].revents) num++;
|
||||
}
|
||||
|
||||
int ms = (timeout == -1) ? 100 : timeout;
|
||||
struct timespec ts;
|
||||
ts.tv_sec = ms / 1000;
|
||||
ts.tv_nsec = (ms % 1000) * 1000 * 1000;
|
||||
|
||||
|
||||
while (num == 0) {
|
||||
int ret;
|
||||
|
||||
ret = nanosleep(&ts, &ts);
|
||||
|
||||
// Check if messages ready
|
||||
for (size_t i = 0; i < nitems; i++) {
|
||||
if (items[i].revents == 0 && msgq_msg_ready(items[i].q)){
|
||||
num += 1;
|
||||
items[i].revents = 1;
|
||||
}
|
||||
}
|
||||
|
||||
// exit if we had a timeout and the sleep finished
|
||||
if (timeout != -1 && ret == 0){
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return num;
|
||||
}
|
||||
|
||||
bool msgq_all_readers_updated(msgq_queue_t *q) {
|
||||
uint64_t num_readers = *q->num_readers;
|
||||
for (uint64_t i = 0; i < num_readers; i++) {
|
||||
if (*q->read_valids[i] && *q->write_pointer != *q->read_pointers[i]) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return num_readers > 0;
|
||||
}
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
#pragma once
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
#include <string>
|
||||
#include <atomic>
|
||||
|
||||
#define DEFAULT_SEGMENT_SIZE (10 * 1024 * 1024)
|
||||
#define NUM_READERS 10
|
||||
#define ALIGN(n) ((n + (8 - 1)) & -8)
|
||||
|
||||
#define UNPACK64(higher, lower, input) do {uint64_t tmp = input; higher = tmp >> 32; lower = tmp & 0xFFFFFFFF;} while (0)
|
||||
#define PACK64(output, higher, lower) output = ((uint64_t)higher << 32 ) | ((uint64_t)lower & 0xFFFFFFFF)
|
||||
|
||||
struct msgq_header_t {
|
||||
uint64_t num_readers;
|
||||
uint64_t write_pointer;
|
||||
uint64_t write_uid;
|
||||
uint64_t read_pointers[NUM_READERS];
|
||||
uint64_t read_valids[NUM_READERS];
|
||||
uint64_t read_uids[NUM_READERS];
|
||||
};
|
||||
|
||||
struct msgq_queue_t {
|
||||
std::atomic<uint64_t> *num_readers;
|
||||
std::atomic<uint64_t> *write_pointer;
|
||||
std::atomic<uint64_t> *write_uid;
|
||||
std::atomic<uint64_t> *read_pointers[NUM_READERS];
|
||||
std::atomic<uint64_t> *read_valids[NUM_READERS];
|
||||
std::atomic<uint64_t> *read_uids[NUM_READERS];
|
||||
char * mmap_p;
|
||||
char * data;
|
||||
size_t size;
|
||||
int reader_id;
|
||||
uint64_t read_uid_local;
|
||||
uint64_t write_uid_local;
|
||||
|
||||
bool read_conflate;
|
||||
std::string endpoint;
|
||||
};
|
||||
|
||||
struct msgq_msg_t {
|
||||
size_t size;
|
||||
char * data;
|
||||
};
|
||||
|
||||
struct msgq_pollitem_t {
|
||||
msgq_queue_t *q;
|
||||
int revents;
|
||||
};
|
||||
|
||||
void msgq_wait_for_subscriber(msgq_queue_t *q);
|
||||
void msgq_reset_reader(msgq_queue_t *q);
|
||||
|
||||
int msgq_msg_init_size(msgq_msg_t *msg, size_t size);
|
||||
int msgq_msg_init_data(msgq_msg_t *msg, char * data, size_t size);
|
||||
int msgq_msg_close(msgq_msg_t *msg);
|
||||
|
||||
int msgq_new_queue(msgq_queue_t * q, const char * path, size_t size);
|
||||
void msgq_close_queue(msgq_queue_t *q);
|
||||
void msgq_init_publisher(msgq_queue_t * q);
|
||||
void msgq_init_subscriber(msgq_queue_t * q);
|
||||
|
||||
int msgq_msg_send(msgq_msg_t *msg, msgq_queue_t *q);
|
||||
int msgq_msg_recv(msgq_msg_t *msg, msgq_queue_t *q);
|
||||
int msgq_msg_ready(msgq_queue_t * q);
|
||||
int msgq_poll(msgq_pollitem_t * items, size_t nitems, int timeout);
|
||||
|
||||
bool msgq_all_readers_updated(msgq_queue_t *q);
|
||||
|
|
@ -0,0 +1,205 @@
|
|||
#include <time.h>
|
||||
#include <assert.h>
|
||||
#include <stdlib.h>
|
||||
#include <string>
|
||||
#include <mutex>
|
||||
|
||||
#include "services.h"
|
||||
#include "messaging.h"
|
||||
|
||||
const bool SIMULATION = (getenv("SIMULATION") != nullptr) && (std::string(getenv("SIMULATION")) == "1");
|
||||
|
||||
static inline uint64_t nanos_since_boot() {
|
||||
struct timespec t;
|
||||
clock_gettime(CLOCK_BOOTTIME, &t);
|
||||
return t.tv_sec * 1000000000ULL + t.tv_nsec;
|
||||
}
|
||||
|
||||
static const service *get_service(const char *name) {
|
||||
for (const auto &it : services) {
|
||||
if (strcmp(it.name, name) == 0) return ⁢
|
||||
}
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
static inline bool inList(const std::vector<const char *> &list, const char *value) {
|
||||
for (auto &v : list) {
|
||||
if (strcmp(value, v) == 0) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
class MessageContext {
|
||||
public:
|
||||
MessageContext() : ctx_(nullptr) {};
|
||||
~MessageContext() { delete ctx_; }
|
||||
inline Context *context() {
|
||||
std::call_once(init_flag, [=]() { ctx_ = Context::create(); });
|
||||
return ctx_;
|
||||
}
|
||||
private:
|
||||
Context *ctx_;
|
||||
std::once_flag init_flag;
|
||||
};
|
||||
|
||||
MessageContext message_context;
|
||||
|
||||
struct SubMaster::SubMessage {
|
||||
std::string name;
|
||||
SubSocket *socket = nullptr;
|
||||
int freq = 0;
|
||||
bool updated = false, alive = false, valid = true, ignore_alive;
|
||||
uint64_t rcv_time = 0, rcv_frame = 0;
|
||||
void *allocated_msg_reader = nullptr;
|
||||
capnp::FlatArrayMessageReader *msg_reader = nullptr;
|
||||
AlignedBuffer aligned_buf;
|
||||
cereal::Event::Reader event;
|
||||
};
|
||||
|
||||
SubMaster::SubMaster(const std::vector<const char *> &service_list, const char *address,
|
||||
const std::vector<const char *> &ignore_alive) {
|
||||
poller_ = Poller::create();
|
||||
for (auto name : service_list) {
|
||||
const service *serv = get_service(name);
|
||||
assert(serv != nullptr);
|
||||
SubSocket *socket = SubSocket::create(message_context.context(), name, address ? address : "127.0.0.1", true);
|
||||
assert(socket != 0);
|
||||
poller_->registerSocket(socket);
|
||||
SubMessage *m = new SubMessage{
|
||||
.name = name,
|
||||
.socket = socket,
|
||||
.freq = serv->frequency,
|
||||
.ignore_alive = inList(ignore_alive, name),
|
||||
.allocated_msg_reader = malloc(sizeof(capnp::FlatArrayMessageReader))};
|
||||
m->msg_reader = new (m->allocated_msg_reader) capnp::FlatArrayMessageReader({});
|
||||
messages_[socket] = m;
|
||||
services_[name] = m;
|
||||
}
|
||||
}
|
||||
|
||||
void SubMaster::update(int timeout) {
|
||||
for (auto &kv : messages_) kv.second->updated = false;
|
||||
|
||||
auto sockets = poller_->poll(timeout);
|
||||
uint64_t current_time = nanos_since_boot();
|
||||
|
||||
std::vector<std::pair<std::string, cereal::Event::Reader>> messages;
|
||||
|
||||
for (auto s : sockets) {
|
||||
Message *msg = s->receive(true);
|
||||
if (msg == nullptr) continue;
|
||||
|
||||
SubMessage *m = messages_.at(s);
|
||||
|
||||
m->msg_reader->~FlatArrayMessageReader();
|
||||
capnp::ReaderOptions options;
|
||||
options.traversalLimitInWords = kj::maxValue; // Don't limit
|
||||
m->msg_reader = new (m->allocated_msg_reader) capnp::FlatArrayMessageReader(m->aligned_buf.align(msg), options);
|
||||
delete msg;
|
||||
messages.push_back({m->name, m->msg_reader->getRoot<cereal::Event>()});
|
||||
}
|
||||
|
||||
update_msgs(current_time, messages);
|
||||
}
|
||||
|
||||
void SubMaster::update_msgs(uint64_t current_time, const std::vector<std::pair<std::string, cereal::Event::Reader>> &messages){
|
||||
if (++frame == UINT64_MAX) frame = 1;
|
||||
|
||||
for(auto &kv : messages) {
|
||||
auto m_find = services_.find(kv.first);
|
||||
if (m_find == services_.end()){
|
||||
continue;
|
||||
}
|
||||
SubMessage *m = m_find->second;
|
||||
m->event = kv.second;
|
||||
m->updated = true;
|
||||
m->rcv_time = current_time;
|
||||
m->rcv_frame = frame;
|
||||
m->valid = m->event.getValid();
|
||||
if (SIMULATION) m->alive = true;
|
||||
}
|
||||
|
||||
if (!SIMULATION) {
|
||||
for (auto &kv : messages_) {
|
||||
SubMessage *m = kv.second;
|
||||
m->alive = (m->freq <= (1e-5) || ((current_time - m->rcv_time) * (1e-9)) < (10.0 / m->freq));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bool SubMaster::all_(const std::vector<const char *> &service_list, bool valid, bool alive) {
|
||||
int found = 0;
|
||||
for (auto &kv : messages_) {
|
||||
SubMessage *m = kv.second;
|
||||
if (service_list.size() == 0 || inList(service_list, m->name.c_str())) {
|
||||
found += (!valid || m->valid) && (!alive || (m->alive || m->ignore_alive));
|
||||
}
|
||||
}
|
||||
return service_list.size() == 0 ? found == messages_.size() : found == service_list.size();
|
||||
}
|
||||
|
||||
void SubMaster::drain() {
|
||||
while (true) {
|
||||
auto polls = poller_->poll(0);
|
||||
if (polls.size() == 0)
|
||||
break;
|
||||
|
||||
for (auto sock : polls) {
|
||||
Message *msg = sock->receive(true);
|
||||
delete msg;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bool SubMaster::updated(const char *name) const {
|
||||
return services_.at(name)->updated;
|
||||
}
|
||||
|
||||
bool SubMaster::alive(const char *name) const {
|
||||
return services_.at(name)->alive;
|
||||
}
|
||||
|
||||
bool SubMaster::valid(const char *name) const {
|
||||
return services_.at(name)->valid;
|
||||
}
|
||||
|
||||
uint64_t SubMaster::rcv_frame(const char *name) const {
|
||||
return services_.at(name)->rcv_frame;
|
||||
}
|
||||
|
||||
uint64_t SubMaster::rcv_time(const char *name) const {
|
||||
return services_.at(name)->rcv_time;
|
||||
}
|
||||
|
||||
cereal::Event::Reader &SubMaster::operator[](const char *name) const {
|
||||
return services_.at(name)->event;
|
||||
};
|
||||
|
||||
SubMaster::~SubMaster() {
|
||||
delete poller_;
|
||||
for (auto &kv : messages_) {
|
||||
SubMessage *m = kv.second;
|
||||
m->msg_reader->~FlatArrayMessageReader();
|
||||
free(m->allocated_msg_reader);
|
||||
delete m->socket;
|
||||
delete m;
|
||||
}
|
||||
}
|
||||
|
||||
PubMaster::PubMaster(const std::vector<const char *> &service_list) {
|
||||
for (auto name : service_list) {
|
||||
assert(get_service(name) != nullptr);
|
||||
PubSocket *socket = PubSocket::create(message_context.context(), name);
|
||||
assert(socket);
|
||||
sockets_[name] = socket;
|
||||
}
|
||||
}
|
||||
|
||||
int PubMaster::send(const char *name, MessageBuilder &msg) {
|
||||
auto bytes = msg.toBytes();
|
||||
return send(name, bytes.begin(), bytes.size());
|
||||
}
|
||||
|
||||
PubMaster::~PubMaster() {
|
||||
for (auto s : sockets_) delete s.second;
|
||||
}
|
||||
|
|
@ -0,0 +1,96 @@
|
|||
#!/usr/bin/env python3
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
TICI = os.path.isfile('/TICI')
|
||||
RESERVED_PORT = 8022 # sshd
|
||||
STARTING_PORT = 8001
|
||||
|
||||
|
||||
def new_port(port: int):
|
||||
port += STARTING_PORT
|
||||
return port + 1 if port >= RESERVED_PORT else port
|
||||
|
||||
|
||||
class Service:
|
||||
def __init__(self, port: int, should_log: bool, frequency: float, decimation: Optional[int] = None):
|
||||
self.port = port
|
||||
self.should_log = should_log
|
||||
self.frequency = frequency
|
||||
self.decimation = decimation
|
||||
|
||||
DCAM_FREQ = 10. if not TICI else 20.
|
||||
|
||||
services = {
|
||||
# service: (should_log, frequency, qlog decimation (optional))
|
||||
"sensorEvents": (True, 100., 100),
|
||||
"gpsNMEA": (True, 9.),
|
||||
"deviceState": (True, 2., 1),
|
||||
"can": (True, 100.),
|
||||
"controlsState": (True, 100., 10),
|
||||
"pandaStates": (True, 2., 1),
|
||||
"peripheralState": (True, 2., 1),
|
||||
"radarState": (True, 20., 5),
|
||||
"roadEncodeIdx": (True, 20., 1),
|
||||
"liveTracks": (True, 20.),
|
||||
"sendcan": (True, 100., 139),
|
||||
"logMessage": (True, 0.),
|
||||
"errorLogMessage": (True, 0., 1),
|
||||
"liveCalibration": (True, 4., 4),
|
||||
"androidLog": (True, 0.),
|
||||
"carState": (True, 100., 10),
|
||||
"carControl": (True, 100., 10),
|
||||
"longitudinalPlan": (True, 20., 5),
|
||||
"procLog": (True, 0.5),
|
||||
"gpsLocationExternal": (True, 10., 10),
|
||||
"ubloxGnss": (True, 10.),
|
||||
"qcomGnss": (True, 2.),
|
||||
"clocks": (True, 1., 1),
|
||||
"ubloxRaw": (True, 20.),
|
||||
"liveLocationKalman": (True, 20., 5),
|
||||
"liveParameters": (True, 20., 5),
|
||||
"cameraOdometry": (True, 20., 5),
|
||||
"lateralPlan": (True, 20., 5),
|
||||
"thumbnail": (True, 0.2, 1),
|
||||
"carEvents": (True, 1., 1),
|
||||
"carParams": (True, 0.02, 1),
|
||||
"roadCameraState": (True, 20., 20),
|
||||
"driverCameraState": (True, DCAM_FREQ, DCAM_FREQ),
|
||||
"driverEncodeIdx": (True, DCAM_FREQ, 1),
|
||||
"driverState": (True, DCAM_FREQ, DCAM_FREQ / 2),
|
||||
"driverMonitoringState": (True, DCAM_FREQ, DCAM_FREQ / 2),
|
||||
"wideRoadEncodeIdx": (True, 20., 1),
|
||||
"wideRoadCameraState": (True, 20., 20),
|
||||
"modelV2": (True, 20., 40),
|
||||
"managerState": (True, 2., 1),
|
||||
"uploaderState": (True, 0., 1),
|
||||
"navInstruction": (True, 0.),
|
||||
"navRoute": (True, 0.),
|
||||
"navThumbnail": (True, 0.),
|
||||
|
||||
# debug
|
||||
"testJoystick": (False, 0.),
|
||||
}
|
||||
service_list = {name: Service(new_port(idx), *vals) for # type: ignore
|
||||
idx, (name, vals) in enumerate(services.items())}
|
||||
|
||||
|
||||
def build_header():
|
||||
h = ""
|
||||
h += "/* THIS IS AN AUTOGENERATED FILE, PLEASE EDIT services.py */\n"
|
||||
h += "#ifndef __SERVICES_H\n"
|
||||
h += "#define __SERVICES_H\n"
|
||||
h += "struct service { char name[0x100]; int port; bool should_log; int frequency; int decimation; };\n"
|
||||
h += "static struct service services[] = {\n"
|
||||
for k, v in service_list.items():
|
||||
should_log = "true" if v.should_log else "false"
|
||||
decimation = -1 if v.decimation is None else v.decimation
|
||||
h += ' { "%s", %d, %s, %d, %d },\n' % \
|
||||
(k, v.port, should_log, v.frequency, decimation)
|
||||
h += "};\n"
|
||||
h += "#endif\n"
|
||||
return h
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(build_header())
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
visionipc_pyx.cpp
|
||||
*.so
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <unistd.h>
|
||||
#include <assert.h>
|
||||
#include <errno.h>
|
||||
|
||||
#include <sys/mman.h>
|
||||
#include <sys/socket.h>
|
||||
#include <sys/un.h>
|
||||
|
||||
#ifdef __APPLE__
|
||||
#define getsocket() socket(AF_UNIX, SOCK_STREAM, 0)
|
||||
#else
|
||||
#define getsocket() socket(AF_UNIX, SOCK_SEQPACKET, 0)
|
||||
#endif
|
||||
|
||||
#include "ipc.h"
|
||||
|
||||
int ipc_connect(const char* socket_path) {
|
||||
int err;
|
||||
|
||||
int sock = getsocket();
|
||||
|
||||
if (sock < 0) return -1;
|
||||
struct sockaddr_un addr = {
|
||||
.sun_family = AF_UNIX,
|
||||
};
|
||||
snprintf(addr.sun_path, sizeof(addr.sun_path), "%s", socket_path);
|
||||
err = connect(sock, (struct sockaddr*)&addr, sizeof(addr));
|
||||
if (err != 0) {
|
||||
close(sock);
|
||||
return -1;
|
||||
}
|
||||
|
||||
return sock;
|
||||
}
|
||||
|
||||
int ipc_bind(const char* socket_path) {
|
||||
int err;
|
||||
|
||||
unlink(socket_path);
|
||||
|
||||
int sock = getsocket();
|
||||
|
||||
struct sockaddr_un addr = {
|
||||
.sun_family = AF_UNIX,
|
||||
};
|
||||
snprintf(addr.sun_path, sizeof(addr.sun_path), "%s", socket_path);
|
||||
err = bind(sock, (struct sockaddr *)&addr, sizeof(addr));
|
||||
assert(err == 0);
|
||||
|
||||
err = listen(sock, 3);
|
||||
assert(err == 0);
|
||||
|
||||
return sock;
|
||||
}
|
||||
|
||||
|
||||
int ipc_sendrecv_with_fds(bool send, int fd, void *buf, size_t buf_size, int* fds, int num_fds,
|
||||
int *out_num_fds) {
|
||||
char control_buf[CMSG_SPACE(sizeof(int) * num_fds)];
|
||||
memset(control_buf, 0, CMSG_SPACE(sizeof(int) * num_fds));
|
||||
|
||||
struct iovec iov = {
|
||||
.iov_base = buf,
|
||||
.iov_len = buf_size,
|
||||
};
|
||||
struct msghdr msg = {
|
||||
.msg_iov = &iov,
|
||||
.msg_iovlen = 1,
|
||||
};
|
||||
|
||||
if (num_fds > 0) {
|
||||
assert(fds);
|
||||
|
||||
msg.msg_control = control_buf;
|
||||
msg.msg_controllen = CMSG_SPACE(sizeof(int) * num_fds);
|
||||
}
|
||||
|
||||
if (send) {
|
||||
if (num_fds) {
|
||||
struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
|
||||
assert(cmsg);
|
||||
cmsg->cmsg_level = SOL_SOCKET;
|
||||
cmsg->cmsg_type = SCM_RIGHTS;
|
||||
cmsg->cmsg_len = CMSG_LEN(sizeof(int) * num_fds);
|
||||
memcpy(CMSG_DATA(cmsg), fds, sizeof(int) * num_fds);
|
||||
}
|
||||
return sendmsg(fd, &msg, 0);
|
||||
} else {
|
||||
int r = recvmsg(fd, &msg, 0);
|
||||
if (r < 0) return r;
|
||||
|
||||
int recv_fds = 0;
|
||||
if (msg.msg_controllen > 0) {
|
||||
struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
|
||||
assert(cmsg);
|
||||
assert(cmsg->cmsg_level == SOL_SOCKET && cmsg->cmsg_type == SCM_RIGHTS);
|
||||
recv_fds = (cmsg->cmsg_len - CMSG_LEN(0));
|
||||
assert(recv_fds > 0 && (recv_fds % sizeof(int)) == 0);
|
||||
recv_fds /= sizeof(int);
|
||||
|
||||
assert(fds && recv_fds <= num_fds);
|
||||
memcpy(fds, CMSG_DATA(cmsg), sizeof(int) * recv_fds);
|
||||
}
|
||||
|
||||
if (msg.msg_flags) {
|
||||
for (int i=0; i<recv_fds; i++) {
|
||||
close(fds[i]);
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (fds) {
|
||||
assert(out_num_fds);
|
||||
*out_num_fds = recv_fds;
|
||||
}
|
||||
return r;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
#pragma once
|
||||
#include <cstddef>
|
||||
|
||||
int ipc_connect(const char* socket_path);
|
||||
int ipc_bind(const char* socket_path);
|
||||
int ipc_sendrecv_with_fds(bool send, int fd, void *buf, size_t buf_size, int* fds, int num_fds,
|
||||
int *out_num_fds);
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
#include "visionbuf.h"
|
||||
|
||||
#define ALIGN(x, align) (((x) + (align)-1) & ~((align)-1))
|
||||
|
||||
#ifdef QCOM
|
||||
// from libadreno_utils.so
|
||||
extern "C" void compute_aligned_width_and_height(int width,
|
||||
int height,
|
||||
int bpp,
|
||||
int tile_mode,
|
||||
int raster_mode,
|
||||
int padding_threshold,
|
||||
int *aligned_w,
|
||||
int *aligned_h);
|
||||
#endif
|
||||
|
||||
void visionbuf_compute_aligned_width_and_height(int width, int height, int *aligned_w, int *aligned_h) {
|
||||
#ifdef QCOM
|
||||
compute_aligned_width_and_height(ALIGN(width, 32), ALIGN(height, 32), 3, 0, 0, 512, aligned_w, aligned_h);
|
||||
#else
|
||||
*aligned_w = width; *aligned_h = height;
|
||||
#endif
|
||||
}
|
||||
|
||||
void VisionBuf::init_rgb(size_t init_width, size_t init_height, size_t init_stride) {
|
||||
this->rgb = true;
|
||||
this->width = init_width;
|
||||
this->height = init_height;
|
||||
this->stride = init_stride;
|
||||
}
|
||||
|
||||
void VisionBuf::init_yuv(size_t init_width, size_t init_height){
|
||||
this->rgb = false;
|
||||
this->width = init_width;
|
||||
this->height = init_height;
|
||||
|
||||
this->y = (uint8_t *)this->addr;
|
||||
this->u = this->y + (this->width * this->height);
|
||||
this->v = this->u + (this->width / 2 * this->height / 2);
|
||||
}
|
||||
|
||||
|
||||
uint64_t VisionBuf::get_frame_id() {
|
||||
return *frame_id;
|
||||
}
|
||||
|
||||
void VisionBuf::set_frame_id(uint64_t id) {
|
||||
*frame_id = id;
|
||||
}
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
#pragma once
|
||||
#include "visionipc.h"
|
||||
|
||||
#define CL_USE_DEPRECATED_OPENCL_1_2_APIS
|
||||
#ifdef __APPLE__
|
||||
#include <OpenCL/cl.h>
|
||||
#else
|
||||
#include <CL/cl.h>
|
||||
#endif
|
||||
|
||||
#define VISIONBUF_SYNC_FROM_DEVICE 0
|
||||
#define VISIONBUF_SYNC_TO_DEVICE 1
|
||||
|
||||
enum VisionStreamType {
|
||||
VISION_STREAM_RGB_ROAD,
|
||||
VISION_STREAM_RGB_DRIVER,
|
||||
VISION_STREAM_RGB_WIDE_ROAD,
|
||||
|
||||
VISION_STREAM_ROAD,
|
||||
VISION_STREAM_DRIVER,
|
||||
VISION_STREAM_WIDE_ROAD,
|
||||
|
||||
VISION_STREAM_RGB_MAP,
|
||||
VISION_STREAM_MAX,
|
||||
};
|
||||
|
||||
class VisionBuf {
|
||||
public:
|
||||
size_t len = 0;
|
||||
size_t mmap_len = 0;
|
||||
void * addr = nullptr;
|
||||
uint64_t *frame_id;
|
||||
int fd = 0;
|
||||
|
||||
bool rgb = false;
|
||||
size_t width = 0;
|
||||
size_t height = 0;
|
||||
size_t stride = 0;
|
||||
|
||||
// YUV
|
||||
uint8_t * y = nullptr;
|
||||
uint8_t * u = nullptr;
|
||||
uint8_t * v = nullptr;
|
||||
|
||||
// Visionipc
|
||||
uint64_t server_id = 0;
|
||||
size_t idx = 0;
|
||||
VisionStreamType type;
|
||||
|
||||
// OpenCL
|
||||
cl_mem buf_cl = nullptr;
|
||||
cl_command_queue copy_q = nullptr;
|
||||
|
||||
// ion
|
||||
int handle = 0;
|
||||
|
||||
void allocate(size_t len);
|
||||
void import();
|
||||
void init_cl(cl_device_id device_id, cl_context ctx);
|
||||
void init_rgb(size_t width, size_t height, size_t stride);
|
||||
void init_yuv(size_t width, size_t height);
|
||||
int sync(int dir);
|
||||
int free();
|
||||
|
||||
void set_frame_id(uint64_t id);
|
||||
uint64_t get_frame_id();
|
||||
};
|
||||
|
||||
void visionbuf_compute_aligned_width_and_height(int width, int height, int *aligned_w, int *aligned_h);
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
#include "visionbuf.h"
|
||||
|
||||
#include <atomic>
|
||||
#include <stdio.h>
|
||||
#include <fcntl.h>
|
||||
#include <assert.h>
|
||||
#include <stdlib.h>
|
||||
#include <unistd.h>
|
||||
#include <sys/mman.h>
|
||||
#include <sys/types.h>
|
||||
|
||||
std::atomic<int> offset = 0;
|
||||
|
||||
static void *malloc_with_fd(size_t len, int *fd) {
|
||||
char full_path[0x100];
|
||||
|
||||
#ifdef __APPLE__
|
||||
snprintf(full_path, sizeof(full_path)-1, "/tmp/visionbuf_%d_%d", getpid(), offset++);
|
||||
#else
|
||||
snprintf(full_path, sizeof(full_path)-1, "/dev/shm/visionbuf_%d_%d", getpid(), offset++);
|
||||
#endif
|
||||
|
||||
*fd = open(full_path, O_RDWR | O_CREAT, 0664);
|
||||
assert(*fd >= 0);
|
||||
|
||||
unlink(full_path);
|
||||
|
||||
ftruncate(*fd, len);
|
||||
void *addr = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, *fd, 0);
|
||||
assert(addr != MAP_FAILED);
|
||||
|
||||
return addr;
|
||||
}
|
||||
|
||||
void VisionBuf::allocate(size_t length) {
|
||||
this->len = length;
|
||||
this->mmap_len = this->len;
|
||||
this->addr = malloc_with_fd(this->len, &this->fd);
|
||||
this->frame_id = (uint64_t*)((uint8_t*)this->addr + this->len);
|
||||
}
|
||||
|
||||
void VisionBuf::init_cl(cl_device_id device_id, cl_context ctx){
|
||||
int err;
|
||||
|
||||
this->copy_q = clCreateCommandQueue(ctx, device_id, 0, &err);
|
||||
assert(err == 0);
|
||||
|
||||
this->buf_cl = clCreateBuffer(ctx, CL_MEM_READ_WRITE | CL_MEM_USE_HOST_PTR, this->len, this->addr, &err);
|
||||
assert(err == 0);
|
||||
}
|
||||
|
||||
|
||||
void VisionBuf::import(){
|
||||
assert(this->fd >= 0);
|
||||
this->addr = mmap(NULL, this->mmap_len, PROT_READ | PROT_WRITE, MAP_SHARED, this->fd, 0);
|
||||
assert(this->addr != MAP_FAILED);
|
||||
|
||||
this->frame_id = (uint64_t*)((uint8_t*)this->addr + this->len);
|
||||
}
|
||||
|
||||
|
||||
int VisionBuf::sync(int dir) {
|
||||
int err = 0;
|
||||
if (!this->buf_cl) return 0;
|
||||
|
||||
if (dir == VISIONBUF_SYNC_FROM_DEVICE) {
|
||||
err = clEnqueueReadBuffer(this->copy_q, this->buf_cl, CL_FALSE, 0, this->len, this->addr, 0, NULL, NULL);
|
||||
} else {
|
||||
err = clEnqueueWriteBuffer(this->copy_q, this->buf_cl, CL_FALSE, 0, this->len, this->addr, 0, NULL, NULL);
|
||||
}
|
||||
|
||||
if (err == 0){
|
||||
err = clFinish(this->copy_q);
|
||||
}
|
||||
|
||||
return err;
|
||||
}
|
||||
|
||||
int VisionBuf::free() {
|
||||
int err = 0;
|
||||
if (this->buf_cl){
|
||||
err = clReleaseMemObject(this->buf_cl);
|
||||
if (err != 0) return err;
|
||||
|
||||
err = clReleaseCommandQueue(this->copy_q);
|
||||
if (err != 0) return err;
|
||||
}
|
||||
|
||||
err = munmap(this->addr, this->len);
|
||||
if (err != 0) return err;
|
||||
|
||||
err = close(this->fd);
|
||||
return err;
|
||||
}
|
||||
|
|
@ -0,0 +1,158 @@
|
|||
#include <stdlib.h>
|
||||
#include <stdio.h>
|
||||
#include <string.h>
|
||||
#include <assert.h>
|
||||
#include <errno.h>
|
||||
#include <sys/mman.h>
|
||||
#include <sys/ioctl.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/stat.h>
|
||||
#include <fcntl.h>
|
||||
#include <unistd.h>
|
||||
#include <linux/ion.h>
|
||||
#include <CL/cl_ext.h>
|
||||
|
||||
#include <msm_ion.h>
|
||||
|
||||
#include "visionbuf.h"
|
||||
|
||||
// keep trying if x gets interrupted by a signal
|
||||
#define HANDLE_EINTR(x) \
|
||||
({ \
|
||||
decltype(x) ret; \
|
||||
int try_cnt = 0; \
|
||||
do { \
|
||||
ret = (x); \
|
||||
} while (ret == -1 && errno == EINTR && try_cnt++ < 100); \
|
||||
ret; \
|
||||
})
|
||||
|
||||
// just hard-code these for convenience
|
||||
// size_t device_page_size = 0;
|
||||
// clGetDeviceInfo(device_id, CL_DEVICE_PAGE_SIZE_QCOM,
|
||||
// sizeof(device_page_size), &device_page_size,
|
||||
// NULL);
|
||||
|
||||
// size_t padding_cl = 0;
|
||||
// clGetDeviceInfo(device_id, CL_DEVICE_EXT_MEM_PADDING_IN_BYTES_QCOM,
|
||||
// sizeof(padding_cl), &padding_cl,
|
||||
// NULL);
|
||||
#define DEVICE_PAGE_SIZE_CL 4096
|
||||
#define PADDING_CL 0
|
||||
|
||||
static int ion_fd = -1;
|
||||
static void ion_init() {
|
||||
if (ion_fd == -1) {
|
||||
ion_fd = open("/dev/ion", O_RDWR | O_NONBLOCK);
|
||||
}
|
||||
}
|
||||
|
||||
void VisionBuf::allocate(size_t length) {
|
||||
int err;
|
||||
|
||||
ion_init();
|
||||
|
||||
struct ion_allocation_data ion_alloc = {0};
|
||||
ion_alloc.len = length + PADDING_CL + sizeof(uint64_t);
|
||||
ion_alloc.align = 4096;
|
||||
ion_alloc.heap_id_mask = 1 << ION_IOMMU_HEAP_ID;
|
||||
ion_alloc.flags = ION_FLAG_CACHED;
|
||||
|
||||
err = HANDLE_EINTR(ioctl(ion_fd, ION_IOC_ALLOC, &ion_alloc));
|
||||
assert(err == 0);
|
||||
|
||||
struct ion_fd_data ion_fd_data = {0};
|
||||
ion_fd_data.handle = ion_alloc.handle;
|
||||
err = HANDLE_EINTR(ioctl(ion_fd, ION_IOC_SHARE, &ion_fd_data));
|
||||
assert(err == 0);
|
||||
|
||||
void *mmap_addr = mmap(NULL, ion_alloc.len,
|
||||
PROT_READ | PROT_WRITE,
|
||||
MAP_SHARED, ion_fd_data.fd, 0);
|
||||
assert(mmap_addr != MAP_FAILED);
|
||||
|
||||
memset(mmap_addr, 0, ion_alloc.len);
|
||||
|
||||
this->len = length;
|
||||
this->mmap_len = ion_alloc.len;
|
||||
this->addr = mmap_addr;
|
||||
this->handle = ion_alloc.handle;
|
||||
this->fd = ion_fd_data.fd;
|
||||
this->frame_id = (uint64_t*)((uint8_t*)this->addr + this->len + PADDING_CL);
|
||||
}
|
||||
|
||||
void VisionBuf::import(){
|
||||
int err;
|
||||
assert(this->fd >= 0);
|
||||
|
||||
ion_init();
|
||||
|
||||
// Get handle
|
||||
struct ion_fd_data fd_data = {0};
|
||||
fd_data.fd = this->fd;
|
||||
err = HANDLE_EINTR(ioctl(ion_fd, ION_IOC_IMPORT, &fd_data));
|
||||
assert(err == 0);
|
||||
|
||||
this->handle = fd_data.handle;
|
||||
this->addr = mmap(NULL, this->mmap_len, PROT_READ | PROT_WRITE, MAP_SHARED, this->fd, 0);
|
||||
assert(this->addr != MAP_FAILED);
|
||||
|
||||
this->frame_id = (uint64_t*)((uint8_t*)this->addr + this->len + PADDING_CL);
|
||||
}
|
||||
|
||||
void VisionBuf::init_cl(cl_device_id device_id, cl_context ctx) {
|
||||
int err;
|
||||
|
||||
assert(((uintptr_t)this->addr % DEVICE_PAGE_SIZE_CL) == 0);
|
||||
|
||||
cl_mem_ion_host_ptr ion_cl = {0};
|
||||
ion_cl.ext_host_ptr.allocation_type = CL_MEM_ION_HOST_PTR_QCOM;
|
||||
ion_cl.ext_host_ptr.host_cache_policy = CL_MEM_HOST_UNCACHED_QCOM;
|
||||
ion_cl.ion_filedesc = this->fd;
|
||||
ion_cl.ion_hostptr = this->addr;
|
||||
|
||||
this->buf_cl = clCreateBuffer(ctx,
|
||||
CL_MEM_USE_HOST_PTR | CL_MEM_EXT_HOST_PTR_QCOM,
|
||||
this->len, &ion_cl, &err);
|
||||
assert(err == 0);
|
||||
}
|
||||
|
||||
|
||||
int VisionBuf::sync(int dir) {
|
||||
struct ion_flush_data flush_data = {0};
|
||||
flush_data.handle = this->handle;
|
||||
flush_data.vaddr = this->addr;
|
||||
flush_data.offset = 0;
|
||||
flush_data.length = this->len;
|
||||
|
||||
// ION_IOC_INV_CACHES ~= DMA_FROM_DEVICE
|
||||
// ION_IOC_CLEAN_CACHES ~= DMA_TO_DEVICE
|
||||
// ION_IOC_CLEAN_INV_CACHES ~= DMA_BIDIRECTIONAL
|
||||
|
||||
struct ion_custom_data custom_data = {0};
|
||||
|
||||
assert(dir == VISIONBUF_SYNC_FROM_DEVICE || dir == VISIONBUF_SYNC_TO_DEVICE);
|
||||
custom_data.cmd = (dir == VISIONBUF_SYNC_FROM_DEVICE) ?
|
||||
ION_IOC_INV_CACHES : ION_IOC_CLEAN_CACHES;
|
||||
|
||||
custom_data.arg = (unsigned long)&flush_data;
|
||||
return HANDLE_EINTR(ioctl(ion_fd, ION_IOC_CUSTOM, &custom_data));
|
||||
}
|
||||
|
||||
int VisionBuf::free() {
|
||||
int err = 0;
|
||||
|
||||
if (this->buf_cl){
|
||||
err = clReleaseMemObject(this->buf_cl);
|
||||
if (err != 0) return err;
|
||||
}
|
||||
|
||||
err = munmap(this->addr, this->mmap_len);
|
||||
if (err != 0) return err;
|
||||
|
||||
err = close(this->fd);
|
||||
if (err != 0) return err;
|
||||
|
||||
struct ion_handle_data handle_data = {.handle = this->handle};
|
||||
return HANDLE_EINTR(ioctl(ion_fd, ION_IOC_FREE, &handle_data));
|
||||
}
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <cstddef>
|
||||
|
||||
constexpr int VISIONIPC_MAX_FDS = 128;
|
||||
|
||||
struct VisionIpcBufExtra {
|
||||
uint32_t frame_id;
|
||||
uint64_t timestamp_sof;
|
||||
uint64_t timestamp_eof;
|
||||
};
|
||||
|
||||
struct VisionIpcPacket {
|
||||
uint64_t server_id;
|
||||
size_t idx;
|
||||
struct VisionIpcBufExtra extra;
|
||||
};
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
# distutils: language = c++
|
||||
#cython: language_level=3
|
||||
|
||||
from libcpp.string cimport string
|
||||
from libcpp.vector cimport vector
|
||||
from libc.stdint cimport uint32_t, uint64_t
|
||||
from libcpp cimport bool
|
||||
|
||||
cdef extern from "visionbuf.h":
|
||||
cdef enum VisionStreamType:
|
||||
pass
|
||||
|
||||
cdef cppclass VisionBuf:
|
||||
void * addr
|
||||
size_t len
|
||||
size_t width
|
||||
size_t height
|
||||
size_t stride
|
||||
|
||||
cdef extern from "visionipc.h":
|
||||
struct VisionIpcBufExtra:
|
||||
uint32_t frame_id
|
||||
uint64_t timestamp_sof
|
||||
uint64_t timestamp_eof
|
||||
|
||||
cdef extern from "visionipc_server.h":
|
||||
cdef cppclass VisionIpcServer:
|
||||
VisionIpcServer(string, void*, void*)
|
||||
void create_buffers(VisionStreamType, size_t, bool, size_t, size_t)
|
||||
VisionBuf * get_buffer(VisionStreamType)
|
||||
void send(VisionBuf *, VisionIpcBufExtra *, bool)
|
||||
void start_listener()
|
||||
|
||||
cdef extern from "visionipc_client.h":
|
||||
cdef cppclass VisionIpcClient:
|
||||
VisionIpcClient(string, VisionStreamType, bool, void*, void*)
|
||||
VisionBuf * recv(VisionIpcBufExtra *, int)
|
||||
bool connect(bool)
|
||||
bool is_connected()
|
||||
|
|
@ -0,0 +1,129 @@
|
|||
#include <chrono>
|
||||
#include <cassert>
|
||||
#include <iostream>
|
||||
#include <thread>
|
||||
|
||||
#include "visionipc/ipc.h"
|
||||
#include "visionipc/visionipc_client.h"
|
||||
#include "visionipc/visionipc_server.h"
|
||||
#include "logger/logger.h"
|
||||
|
||||
VisionIpcClient::VisionIpcClient(std::string name, VisionStreamType type, bool conflate, cl_device_id device_id, cl_context ctx) : name(name), type(type), device_id(device_id), ctx(ctx) {
|
||||
msg_ctx = Context::create();
|
||||
sock = SubSocket::create(msg_ctx, get_endpoint_name(name, type), "127.0.0.1", conflate, false);
|
||||
|
||||
poller = Poller::create();
|
||||
poller->registerSocket(sock);
|
||||
}
|
||||
|
||||
// Connect is not thread safe. Do not use the buffers while calling connect
|
||||
bool VisionIpcClient::connect(bool blocking){
|
||||
connected = false;
|
||||
|
||||
// Cleanup old buffers on reconnect
|
||||
for (size_t i = 0; i < num_buffers; i++){
|
||||
if (buffers[i].free() != 0) {
|
||||
LOGE("Failed to free buffer %zu", i);
|
||||
}
|
||||
}
|
||||
|
||||
num_buffers = 0;
|
||||
|
||||
// Connect to server socket and ask for all FDs of type
|
||||
std::string path = "/tmp/visionipc_" + name;
|
||||
|
||||
int socket_fd = -1;
|
||||
while (socket_fd < 0) {
|
||||
socket_fd = ipc_connect(path.c_str());
|
||||
|
||||
if (socket_fd < 0) {
|
||||
if (blocking){
|
||||
std::cout << "VisionIpcClient connecting" << std::endl;
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Send stream type to server to request FDs
|
||||
int r = ipc_sendrecv_with_fds(true, socket_fd, &type, sizeof(type), nullptr, 0, nullptr);
|
||||
assert(r == sizeof(type));
|
||||
|
||||
// Get FDs
|
||||
int fds[VISIONIPC_MAX_FDS];
|
||||
VisionBuf bufs[VISIONIPC_MAX_FDS];
|
||||
r = ipc_sendrecv_with_fds(false, socket_fd, &bufs, sizeof(bufs), fds, VISIONIPC_MAX_FDS, &num_buffers);
|
||||
|
||||
assert(num_buffers >= 0);
|
||||
assert(r == sizeof(VisionBuf) * num_buffers);
|
||||
|
||||
// Import buffers
|
||||
for (size_t i = 0; i < num_buffers; i++){
|
||||
buffers[i] = bufs[i];
|
||||
buffers[i].fd = fds[i];
|
||||
buffers[i].import();
|
||||
if (buffers[i].rgb) {
|
||||
buffers[i].init_rgb(buffers[i].width, buffers[i].height, buffers[i].stride);
|
||||
} else {
|
||||
buffers[i].init_yuv(buffers[i].width, buffers[i].height);
|
||||
}
|
||||
|
||||
if (device_id) buffers[i].init_cl(device_id, ctx);
|
||||
}
|
||||
|
||||
close(socket_fd);
|
||||
connected = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
VisionBuf * VisionIpcClient::recv(VisionIpcBufExtra * extra, const int timeout_ms){
|
||||
auto p = poller->poll(timeout_ms);
|
||||
|
||||
if (!p.size()){
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
Message * r = sock->receive(true);
|
||||
if (r == nullptr){
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// Get buffer
|
||||
assert(r->getSize() == sizeof(VisionIpcPacket));
|
||||
VisionIpcPacket *packet = (VisionIpcPacket*)r->getData();
|
||||
|
||||
assert(packet->idx < num_buffers);
|
||||
VisionBuf * buf = &buffers[packet->idx];
|
||||
|
||||
if (buf->server_id != packet->server_id){
|
||||
connected = false;
|
||||
delete r;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
if (extra) {
|
||||
*extra = packet->extra;
|
||||
}
|
||||
|
||||
if (buf->sync(VISIONBUF_SYNC_TO_DEVICE) != 0) {
|
||||
LOGE("Failed to sync buffer");
|
||||
}
|
||||
|
||||
delete r;
|
||||
return buf;
|
||||
}
|
||||
|
||||
|
||||
|
||||
VisionIpcClient::~VisionIpcClient(){
|
||||
for (size_t i = 0; i < num_buffers; i++){
|
||||
if (buffers[i].free() != 0) {
|
||||
LOGE("Failed to free buffer %zu", i);
|
||||
}
|
||||
}
|
||||
|
||||
delete sock;
|
||||
delete poller;
|
||||
delete msg_ctx;
|
||||
}
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
#pragma once
|
||||
#include <vector>
|
||||
#include <string>
|
||||
#include <unistd.h>
|
||||
|
||||
#include "messaging/messaging.h"
|
||||
#include "visionipc/visionipc.h"
|
||||
#include "visionipc/visionbuf.h"
|
||||
|
||||
class VisionIpcClient {
|
||||
private:
|
||||
std::string name;
|
||||
Context * msg_ctx;
|
||||
SubSocket * sock;
|
||||
Poller * poller;
|
||||
|
||||
VisionStreamType type;
|
||||
|
||||
cl_device_id device_id = nullptr;
|
||||
cl_context ctx = nullptr;
|
||||
|
||||
void init_msgq(bool conflate);
|
||||
|
||||
public:
|
||||
bool connected = false;
|
||||
int num_buffers = 0;
|
||||
VisionBuf buffers[VISIONIPC_MAX_FDS];
|
||||
VisionIpcClient(std::string name, VisionStreamType type, bool conflate, cl_device_id device_id=nullptr, cl_context ctx=nullptr);
|
||||
~VisionIpcClient();
|
||||
VisionBuf * recv(VisionIpcBufExtra * extra=nullptr, const int timeout_ms=100);
|
||||
bool connect(bool blocking=true);
|
||||
bool is_connected() { return connected; }
|
||||
};
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
# distutils: language = c++
|
||||
# cython: c_string_encoding=ascii, language_level=3
|
||||
|
||||
import sys
|
||||
import numpy as np
|
||||
cimport numpy as cnp
|
||||
from cython.view cimport array
|
||||
from libc.string cimport memcpy
|
||||
from libc.stdint cimport uint32_t, uint64_t
|
||||
from libcpp cimport bool
|
||||
from libcpp.string cimport string
|
||||
|
||||
from .visionipc cimport VisionIpcServer as cppVisionIpcServer
|
||||
from .visionipc cimport VisionIpcClient as cppVisionIpcClient
|
||||
from .visionipc cimport VisionBuf as cppVisionBuf
|
||||
from .visionipc cimport VisionIpcBufExtra
|
||||
|
||||
cpdef enum VisionStreamType:
|
||||
VISION_STREAM_RGB_ROAD
|
||||
VISION_STREAM_RGB_DRIVER
|
||||
VISION_STREAM_RGB_WIDE_ROAD
|
||||
VISION_STREAM_ROAD
|
||||
VISION_STREAM_DRIVER
|
||||
VISION_STREAM_WIDE_ROAD
|
||||
VISION_STREAM_RGB_MAP
|
||||
|
||||
|
||||
cdef class VisionIpcServer:
|
||||
cdef cppVisionIpcServer * server
|
||||
|
||||
def __init__(self, string name):
|
||||
self.server = new cppVisionIpcServer(name, NULL, NULL)
|
||||
|
||||
def create_buffers(self, VisionStreamType tp, size_t num_buffers, bool rgb, size_t width, size_t height):
|
||||
self.server.create_buffers(tp, num_buffers, rgb, width, height)
|
||||
|
||||
def send(self, VisionStreamType tp, bytes data, uint32_t frame_id=0, uint64_t timestamp_sof=0, uint64_t timestamp_eof=0):
|
||||
cdef cppVisionBuf * buf = self.server.get_buffer(tp)
|
||||
|
||||
# Populate buffer
|
||||
assert buf.len == len(data)
|
||||
memcpy(buf.addr, <char*>data, len(data))
|
||||
|
||||
cdef VisionIpcBufExtra extra
|
||||
extra.frame_id = frame_id
|
||||
extra.timestamp_sof = timestamp_sof
|
||||
extra.timestamp_eof = timestamp_eof
|
||||
|
||||
self.server.send(buf, &extra, False)
|
||||
|
||||
def start_listener(self):
|
||||
self.server.start_listener()
|
||||
|
||||
def __dealloc__(self):
|
||||
del self.server
|
||||
|
||||
|
||||
cdef class VisionIpcClient:
|
||||
cdef cppVisionBuf * buf
|
||||
cdef cppVisionIpcClient * client
|
||||
|
||||
def __cinit__(self, string name, VisionStreamType stream, bool conflate):
|
||||
self.client = new cppVisionIpcClient(name, stream, conflate, NULL, NULL)
|
||||
self.buf = NULL
|
||||
|
||||
def __dealloc__(self):
|
||||
del self.client
|
||||
|
||||
@property
|
||||
def width(self):
|
||||
return None if not self.buf else self.buf.width
|
||||
|
||||
@property
|
||||
def height(self):
|
||||
return None if not self.buf else self.buf.height
|
||||
|
||||
@property
|
||||
def stride(self):
|
||||
return None if not self.buf else self.buf.stride
|
||||
|
||||
def recv(self, int timeout_ms=100):
|
||||
self.buf = self.client.recv(NULL, timeout_ms)
|
||||
if not self.buf:
|
||||
return None
|
||||
cdef cnp.ndarray dat = np.empty(self.buf.len, dtype=np.uint8)
|
||||
cdef char[:] dat_view = dat
|
||||
memcpy(&dat_view[0], self.buf.addr, self.buf.len)
|
||||
return dat
|
||||
|
||||
def connect(self, bool blocking):
|
||||
return self.client.connect(blocking)
|
||||
|
||||
def is_connected(self):
|
||||
return self.client.is_connected()
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
#include <iostream>
|
||||
#include <chrono>
|
||||
#include <cassert>
|
||||
#include <random>
|
||||
|
||||
#include <poll.h>
|
||||
#include <sys/socket.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include "messaging/messaging.h"
|
||||
#include "visionipc/ipc.h"
|
||||
#include "visionipc/visionipc_server.h"
|
||||
#include "logger/logger.h"
|
||||
|
||||
std::string get_endpoint_name(std::string name, VisionStreamType type){
|
||||
if (messaging_use_zmq()){
|
||||
assert(name == "camerad" || name == "navd");
|
||||
return std::to_string(9000 + static_cast<int>(type));
|
||||
} else {
|
||||
return "visionipc_" + name + "_" + std::to_string(type);
|
||||
}
|
||||
}
|
||||
|
||||
VisionIpcServer::VisionIpcServer(std::string name, cl_device_id device_id, cl_context ctx) : name(name), device_id(device_id), ctx(ctx) {
|
||||
msg_ctx = Context::create();
|
||||
|
||||
std::random_device rd("/dev/urandom");
|
||||
std::uniform_int_distribution<uint64_t> distribution(0,std::numeric_limits<uint64_t>::max());
|
||||
server_id = distribution(rd);
|
||||
}
|
||||
|
||||
void VisionIpcServer::create_buffers(VisionStreamType type, size_t num_buffers, bool rgb, size_t width, size_t height){
|
||||
// TODO: assert that this type is not created yet
|
||||
assert(num_buffers < VISIONIPC_MAX_FDS);
|
||||
int aligned_w = 0, aligned_h = 0;
|
||||
|
||||
size_t size = 0;
|
||||
size_t stride = 0; // Only used for RGB
|
||||
|
||||
if (rgb) {
|
||||
visionbuf_compute_aligned_width_and_height(width, height, &aligned_w, &aligned_h);
|
||||
size = (size_t)aligned_w * (size_t)aligned_h * 3;
|
||||
stride = aligned_w * 3;
|
||||
} else {
|
||||
size = width * height * 3 / 2;
|
||||
}
|
||||
|
||||
// Create map + alloc requested buffers
|
||||
for (size_t i = 0; i < num_buffers; i++){
|
||||
VisionBuf* buf = new VisionBuf();
|
||||
buf->allocate(size);
|
||||
buf->idx = i;
|
||||
buf->type = type;
|
||||
|
||||
if (device_id) buf->init_cl(device_id, ctx);
|
||||
|
||||
rgb ? buf->init_rgb(width, height, stride) : buf->init_yuv(width, height);
|
||||
|
||||
buffers[type].push_back(buf);
|
||||
}
|
||||
|
||||
cur_idx[type] = 0;
|
||||
|
||||
// Create msgq publisher for each of the `name` + type combos
|
||||
// TODO: compute port number directly if using zmq
|
||||
sockets[type] = PubSocket::create(msg_ctx, get_endpoint_name(name, type), false);
|
||||
}
|
||||
|
||||
|
||||
void VisionIpcServer::start_listener(){
|
||||
listener_thread = std::thread(&VisionIpcServer::listener, this);
|
||||
}
|
||||
|
||||
|
||||
void VisionIpcServer::listener(){
|
||||
std::cout << "Starting listener for: " << name << std::endl;
|
||||
|
||||
std::string path = "/tmp/visionipc_" + name;
|
||||
int sock = ipc_bind(path.c_str());
|
||||
assert(sock >= 0);
|
||||
|
||||
while (!should_exit){
|
||||
// Wait for incoming connection
|
||||
struct pollfd polls[1] = {{0}};
|
||||
polls[0].fd = sock;
|
||||
polls[0].events = POLLIN;
|
||||
|
||||
int ret = poll(polls, 1, 100);
|
||||
if (ret < 0) {
|
||||
if (errno == EINTR || errno == EAGAIN) continue;
|
||||
std::cout << "poll failed, stopping listener" << std::endl;
|
||||
break;
|
||||
}
|
||||
|
||||
if (should_exit) break;
|
||||
if (!polls[0].revents) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Handle incoming request
|
||||
int fd = accept(sock, NULL, NULL);
|
||||
assert(fd >= 0);
|
||||
|
||||
VisionStreamType type = VisionStreamType::VISION_STREAM_MAX;
|
||||
int r = ipc_sendrecv_with_fds(false, fd, &type, sizeof(type), nullptr, 0, nullptr);
|
||||
assert(r == sizeof(type));
|
||||
if (buffers.count(type) <= 0) {
|
||||
std::cout << "got request for invalid buffer type: " << type << std::endl;
|
||||
close(fd);
|
||||
continue;
|
||||
}
|
||||
|
||||
int fds[VISIONIPC_MAX_FDS];
|
||||
int num_fds = buffers[type].size();
|
||||
VisionBuf bufs[VISIONIPC_MAX_FDS];
|
||||
|
||||
for (int i = 0; i < num_fds; i++){
|
||||
fds[i] = buffers[type][i]->fd;
|
||||
bufs[i] = *buffers[type][i];
|
||||
|
||||
// Remove some private openCL/ion metadata
|
||||
bufs[i].buf_cl = 0;
|
||||
bufs[i].copy_q = 0;
|
||||
bufs[i].handle = 0;
|
||||
|
||||
bufs[i].server_id = server_id;
|
||||
}
|
||||
|
||||
r = ipc_sendrecv_with_fds(true, fd, &bufs, sizeof(VisionBuf) * num_fds, fds, num_fds, nullptr);
|
||||
|
||||
close(fd);
|
||||
}
|
||||
|
||||
std::cout << "Stopping listener for: " << name << std::endl;
|
||||
close(sock);
|
||||
}
|
||||
|
||||
|
||||
|
||||
VisionBuf * VisionIpcServer::get_buffer(VisionStreamType type){
|
||||
// Do we want to keep track if the buffer has been sent out yet and warn user?
|
||||
assert(buffers.count(type));
|
||||
auto b = buffers[type];
|
||||
return b[cur_idx[type]++ % b.size()];
|
||||
}
|
||||
|
||||
void VisionIpcServer::send(VisionBuf * buf, VisionIpcBufExtra * extra, bool sync){
|
||||
if (sync) {
|
||||
if (buf->sync(VISIONBUF_SYNC_FROM_DEVICE) != 0) {
|
||||
LOGE("Failed to sync buffer");
|
||||
}
|
||||
}
|
||||
assert(buffers.count(buf->type));
|
||||
assert(buf->idx < buffers[buf->type].size());
|
||||
|
||||
// Send over correct msgq socket
|
||||
VisionIpcPacket packet = {0};
|
||||
packet.server_id = server_id;
|
||||
packet.idx = buf->idx;
|
||||
packet.extra = *extra;
|
||||
|
||||
sockets[buf->type]->send((char*)&packet, sizeof(packet));
|
||||
}
|
||||
|
||||
VisionIpcServer::~VisionIpcServer(){
|
||||
should_exit = true;
|
||||
listener_thread.join();
|
||||
|
||||
// VisionBuf cleanup
|
||||
for( auto const& [type, buf] : buffers ) {
|
||||
for (VisionBuf* b : buf){
|
||||
if (b->free() != 0) {
|
||||
LOGE("Failed to free buffer");
|
||||
}
|
||||
delete b;
|
||||
}
|
||||
}
|
||||
|
||||
// Messaging cleanup
|
||||
for( auto const& [type, sock] : sockets ) {
|
||||
delete sock;
|
||||
}
|
||||
delete msg_ctx;
|
||||
}
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
#pragma once
|
||||
#include <vector>
|
||||
#include <string>
|
||||
#include <thread>
|
||||
#include <atomic>
|
||||
#include <map>
|
||||
|
||||
#include "messaging/messaging.h"
|
||||
#include "visionipc/visionipc.h"
|
||||
#include "visionipc/visionbuf.h"
|
||||
|
||||
std::string get_endpoint_name(std::string name, VisionStreamType type);
|
||||
|
||||
class VisionIpcServer {
|
||||
private:
|
||||
cl_device_id device_id = nullptr;
|
||||
cl_context ctx = nullptr;
|
||||
uint64_t server_id;
|
||||
|
||||
std::atomic<bool> should_exit = false;
|
||||
std::string name;
|
||||
std::thread listener_thread;
|
||||
|
||||
std::map<VisionStreamType, std::atomic<size_t> > cur_idx;
|
||||
std::map<VisionStreamType, std::vector<VisionBuf*> > buffers;
|
||||
std::map<VisionStreamType, std::map<VisionBuf*, size_t> > idxs;
|
||||
|
||||
Context * msg_ctx;
|
||||
std::map<VisionStreamType, PubSocket*> sockets;
|
||||
|
||||
void listener(void);
|
||||
|
||||
public:
|
||||
VisionIpcServer(std::string name, cl_device_id device_id=nullptr, cl_context ctx=nullptr);
|
||||
~VisionIpcServer();
|
||||
|
||||
VisionBuf * get_buffer(VisionStreamType type);
|
||||
|
||||
void create_buffers(VisionStreamType type, size_t num_buffers, bool rgb, size_t width, size_t height);
|
||||
void send(VisionBuf * buf, VisionIpcBufExtra * extra, bool sync=true);
|
||||
void start_listener();
|
||||
};
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
#include <thread>
|
||||
#include <chrono>
|
||||
|
||||
#include "catch2/catch.hpp"
|
||||
#include "visionipc_server.h"
|
||||
#include "visionipc_client.h"
|
||||
|
||||
static void zmq_sleep(int milliseconds=1000){
|
||||
if (messaging_use_zmq()){
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(milliseconds));
|
||||
}
|
||||
}
|
||||
|
||||
TEST_CASE("Connecting"){
|
||||
VisionIpcServer server("camerad");
|
||||
server.create_buffers(VISION_STREAM_ROAD, 1, false, 100, 100);
|
||||
server.start_listener();
|
||||
|
||||
VisionIpcClient client = VisionIpcClient("camerad", VISION_STREAM_ROAD, false);
|
||||
REQUIRE(client.connect());
|
||||
|
||||
REQUIRE(client.connected);
|
||||
}
|
||||
|
||||
TEST_CASE("Check buffers"){
|
||||
size_t width = 100, height = 200, num_buffers = 5;
|
||||
VisionIpcServer server("camerad");
|
||||
server.create_buffers(VISION_STREAM_ROAD, num_buffers, false, width, height);
|
||||
server.start_listener();
|
||||
|
||||
VisionIpcClient client = VisionIpcClient("camerad", VISION_STREAM_ROAD, false);
|
||||
REQUIRE(client.connect());
|
||||
|
||||
REQUIRE(client.buffers[0].width == width);
|
||||
REQUIRE(client.buffers[0].height == height);
|
||||
REQUIRE(client.buffers[0].len);
|
||||
REQUIRE(client.num_buffers == num_buffers);
|
||||
}
|
||||
|
||||
TEST_CASE("Check yuv/rgb"){
|
||||
VisionIpcServer server("camerad");
|
||||
server.create_buffers(VISION_STREAM_ROAD, 1, false, 100, 100);
|
||||
server.create_buffers(VISION_STREAM_RGB_ROAD, 1, true, 100, 100);
|
||||
server.start_listener();
|
||||
|
||||
VisionIpcClient client_yuv = VisionIpcClient("camerad", VISION_STREAM_ROAD, false);
|
||||
VisionIpcClient client_rgb = VisionIpcClient("camerad", VISION_STREAM_RGB_ROAD, false);
|
||||
client_yuv.connect();
|
||||
client_rgb.connect();
|
||||
|
||||
REQUIRE(client_rgb.buffers[0].rgb == true);
|
||||
REQUIRE(client_yuv.buffers[0].rgb == false);
|
||||
}
|
||||
|
||||
TEST_CASE("Send single buffer"){
|
||||
VisionIpcServer server("camerad");
|
||||
server.create_buffers(VISION_STREAM_ROAD, 1, true, 100, 100);
|
||||
server.start_listener();
|
||||
|
||||
VisionIpcClient client = VisionIpcClient("camerad", VISION_STREAM_ROAD, false);
|
||||
REQUIRE(client.connect());
|
||||
zmq_sleep();
|
||||
|
||||
VisionBuf * buf = server.get_buffer(VISION_STREAM_ROAD);
|
||||
REQUIRE(buf != nullptr);
|
||||
|
||||
*((uint64_t*)buf->addr) = 1234;
|
||||
|
||||
VisionIpcBufExtra extra = {0};
|
||||
extra.frame_id = 1337;
|
||||
buf->set_frame_id(extra.frame_id);
|
||||
|
||||
server.send(buf, &extra);
|
||||
|
||||
VisionIpcBufExtra extra_recv = {0};
|
||||
VisionBuf * recv_buf = client.recv(&extra_recv);
|
||||
REQUIRE(recv_buf != nullptr);
|
||||
REQUIRE(*(uint64_t*)recv_buf->addr == 1234);
|
||||
REQUIRE(extra_recv.frame_id == extra.frame_id);
|
||||
REQUIRE(recv_buf->get_frame_id() == extra.frame_id);
|
||||
}
|
||||
|
||||
|
||||
TEST_CASE("Test no conflate"){
|
||||
VisionIpcServer server("camerad");
|
||||
server.create_buffers(VISION_STREAM_ROAD, 1, true, 100, 100);
|
||||
server.start_listener();
|
||||
|
||||
VisionIpcClient client = VisionIpcClient("camerad", VISION_STREAM_ROAD, false);
|
||||
REQUIRE(client.connect());
|
||||
zmq_sleep();
|
||||
|
||||
VisionBuf * buf = server.get_buffer(VISION_STREAM_ROAD);
|
||||
REQUIRE(buf != nullptr);
|
||||
|
||||
VisionIpcBufExtra extra = {0};
|
||||
extra.frame_id = 1;
|
||||
server.send(buf, &extra);
|
||||
extra.frame_id = 2;
|
||||
server.send(buf, &extra);
|
||||
|
||||
VisionIpcBufExtra extra_recv = {0};
|
||||
VisionBuf * recv_buf = client.recv(&extra_recv);
|
||||
REQUIRE(recv_buf != nullptr);
|
||||
REQUIRE(extra_recv.frame_id == 1);
|
||||
|
||||
recv_buf = client.recv(&extra_recv);
|
||||
REQUIRE(recv_buf != nullptr);
|
||||
REQUIRE(extra_recv.frame_id == 2);
|
||||
}
|
||||
|
||||
TEST_CASE("Test conflate"){
|
||||
VisionIpcServer server("camerad");
|
||||
server.create_buffers(VISION_STREAM_ROAD, 1, true, 100, 100);
|
||||
server.start_listener();
|
||||
|
||||
VisionIpcClient client = VisionIpcClient("camerad", VISION_STREAM_ROAD, true);
|
||||
REQUIRE(client.connect());
|
||||
zmq_sleep();
|
||||
|
||||
VisionBuf * buf = server.get_buffer(VISION_STREAM_ROAD);
|
||||
REQUIRE(buf != nullptr);
|
||||
|
||||
VisionIpcBufExtra extra = {0};
|
||||
extra.frame_id = 1;
|
||||
server.send(buf, &extra);
|
||||
extra.frame_id = 2;
|
||||
server.send(buf, &extra);
|
||||
|
||||
VisionIpcBufExtra extra_recv = {0};
|
||||
VisionBuf * recv_buf = client.recv(&extra_recv);
|
||||
REQUIRE(recv_buf != nullptr);
|
||||
REQUIRE(extra_recv.frame_id == 2);
|
||||
|
||||
recv_buf = client.recv(&extra_recv);
|
||||
REQUIRE(recv_buf == nullptr);
|
||||
}
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
comment: false
|
||||
coverage:
|
||||
status:
|
||||
project:
|
||||
default:
|
||||
informational: true
|
||||
patch: off
|
||||
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
class lazy_property():
|
||||
"""Defines a property whose value will be computed only once and as needed.
|
||||
|
||||
This can only be used on instance methods.
|
||||
"""
|
||||
def __init__(self, func):
|
||||
self._func = func
|
||||
|
||||
def __get__(self, obj_self, cls):
|
||||
value = self._func(obj_self)
|
||||
setattr(obj_self, self._func.__name__, value)
|
||||
return value
|
||||
|
|
@ -1,22 +0,0 @@
|
|||
import numpy as np
|
||||
|
||||
|
||||
def deep_interp_np(x, xp, fp, axis=None):
|
||||
if axis is not None:
|
||||
fp = fp.swapaxes(0,axis)
|
||||
x = np.atleast_1d(x)
|
||||
xp = np.array(xp)
|
||||
if len(xp) < 2:
|
||||
return np.repeat(fp, len(x), axis=0)
|
||||
if min(np.diff(xp)) < 0:
|
||||
raise RuntimeError('Bad x array for interpolation')
|
||||
j = np.searchsorted(xp, x) - 1
|
||||
j = np.clip(j, 0, len(xp)-2)
|
||||
d = np.divide(x - xp[j], xp[j + 1] - xp[j], out=np.ones_like(x, dtype=np.float64), where=xp[j + 1] - xp[j] != 0)
|
||||
vals_interp = (fp[j].T*(1 - d)).T + (fp[j + 1].T*d).T
|
||||
if axis is not None:
|
||||
vals_interp = vals_interp.swapaxes(0,axis)
|
||||
if len(vals_interp) == 1:
|
||||
return vals_interp[0]
|
||||
else:
|
||||
return vals_interp
|
||||
|
|
@ -1,6 +0,0 @@
|
|||
def replace_right(s, old, new, occurrence):
|
||||
# replace_right('1232425', '2', ' ', 1) -> '12324 5'
|
||||
# replace_right('1232425', '2', ' ', 2) -> '123 4 5'
|
||||
|
||||
split = s.rsplit(old, occurrence)
|
||||
return new.join(split)
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
import os
|
||||
import unittest
|
||||
from uuid import uuid4
|
||||
|
||||
from common.file_helpers import atomic_write_on_fs_tmp
|
||||
from common.file_helpers import atomic_write_in_dir
|
||||
|
||||
|
||||
class TestFileHelpers(unittest.TestCase):
|
||||
def run_atomic_write_func(self, atomic_write_func):
|
||||
path = f"/tmp/tmp{uuid4()}"
|
||||
with atomic_write_func(path) as f:
|
||||
f.write("test")
|
||||
|
||||
with open(path) as f:
|
||||
self.assertEqual(f.read(), "test")
|
||||
os.remove(path)
|
||||
|
||||
def test_atomic_write_on_fs_tmp(self):
|
||||
self.run_atomic_write_func(atomic_write_on_fs_tmp)
|
||||
|
||||
def test_atomic_write_in_dir(self):
|
||||
self.run_atomic_write_func(atomic_write_in_dir)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,26 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
from markdown_it import MarkdownIt
|
||||
import os
|
||||
import unittest
|
||||
|
||||
from common.basedir import BASEDIR
|
||||
from common.markdown import parse_markdown
|
||||
|
||||
|
||||
class TestMarkdown(unittest.TestCase):
|
||||
# validate that our simple markdown parser produces the same output as `markdown_it` from pip
|
||||
def test_current_release_notes(self):
|
||||
self.maxDiff = None
|
||||
|
||||
with open(os.path.join(BASEDIR, "RELEASES.md")) as f:
|
||||
for r in f.read().split("\n\n"):
|
||||
|
||||
# No hyperlink support is ok
|
||||
if '[' in r:
|
||||
continue
|
||||
|
||||
self.assertEqual(MarkdownIt().render(r), parse_markdown(r))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,26 +0,0 @@
|
|||
import numpy as np
|
||||
import unittest
|
||||
|
||||
from common.numpy_fast import interp
|
||||
|
||||
|
||||
class InterpTest(unittest.TestCase):
|
||||
def test_correctness_controls(self):
|
||||
_A_CRUISE_MIN_BP = np.asarray([0., 5., 10., 20., 40.])
|
||||
_A_CRUISE_MIN_V = np.asarray([-1.0, -.8, -.67, -.5, -.30])
|
||||
v_ego_arr = [-1, -1e-12, 0, 4, 5, 6, 7, 10, 11, 15.2, 20, 21, 39,
|
||||
39.999999, 40, 41]
|
||||
|
||||
expected = np.interp(v_ego_arr, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
|
||||
actual = interp(v_ego_arr, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
|
||||
|
||||
np.testing.assert_equal(actual, expected)
|
||||
|
||||
for v_ego in v_ego_arr:
|
||||
expected = np.interp(v_ego, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
|
||||
actual = interp(v_ego, _A_CRUISE_MIN_BP, _A_CRUISE_MIN_V)
|
||||
np.testing.assert_equal(actual, expected)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,94 +0,0 @@
|
|||
import threading
|
||||
import time
|
||||
import tempfile
|
||||
import shutil
|
||||
import unittest
|
||||
|
||||
from common.params import Params, ParamKeyType, UnknownKeyName, put_nonblocking
|
||||
|
||||
class TestParams(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmpdir = tempfile.mkdtemp()
|
||||
print("using", self.tmpdir)
|
||||
self.params = Params(self.tmpdir)
|
||||
|
||||
def tearDown(self):
|
||||
shutil.rmtree(self.tmpdir)
|
||||
|
||||
def test_params_put_and_get(self):
|
||||
self.params.put("DongleId", "cb38263377b873ee")
|
||||
assert self.params.get("DongleId") == b"cb38263377b873ee"
|
||||
|
||||
def test_params_non_ascii(self):
|
||||
st = b"\xe1\x90\xff"
|
||||
self.params.put("CarParams", st)
|
||||
assert self.params.get("CarParams") == st
|
||||
|
||||
def test_params_get_cleared_manager_start(self):
|
||||
self.params.put("CarParams", "test")
|
||||
self.params.put("DongleId", "cb38263377b873ee")
|
||||
assert self.params.get("CarParams") == b"test"
|
||||
self.params.clear_all(ParamKeyType.CLEAR_ON_MANAGER_START)
|
||||
assert self.params.get("CarParams") is None
|
||||
assert self.params.get("DongleId") is not None
|
||||
|
||||
def test_params_two_things(self):
|
||||
self.params.put("DongleId", "bob")
|
||||
self.params.put("AthenadPid", "123")
|
||||
assert self.params.get("DongleId") == b"bob"
|
||||
assert self.params.get("AthenadPid") == b"123"
|
||||
|
||||
def test_params_get_block(self):
|
||||
def _delayed_writer():
|
||||
time.sleep(0.1)
|
||||
self.params.put("CarParams", "test")
|
||||
threading.Thread(target=_delayed_writer).start()
|
||||
assert self.params.get("CarParams") is None
|
||||
assert self.params.get("CarParams", True) == b"test"
|
||||
|
||||
def test_params_unknown_key_fails(self):
|
||||
with self.assertRaises(UnknownKeyName):
|
||||
self.params.get("swag")
|
||||
|
||||
with self.assertRaises(UnknownKeyName):
|
||||
self.params.get_bool("swag")
|
||||
|
||||
with self.assertRaises(UnknownKeyName):
|
||||
self.params.put("swag", "abc")
|
||||
|
||||
with self.assertRaises(UnknownKeyName):
|
||||
self.params.put_bool("swag", True)
|
||||
|
||||
def test_delete_not_there(self):
|
||||
assert self.params.get("CarParams") is None
|
||||
self.params.delete("CarParams")
|
||||
assert self.params.get("CarParams") is None
|
||||
|
||||
def test_get_bool(self):
|
||||
self.params.delete("IsMetric")
|
||||
self.assertFalse(self.params.get_bool("IsMetric"))
|
||||
|
||||
self.params.put_bool("IsMetric", True)
|
||||
self.assertTrue(self.params.get_bool("IsMetric"))
|
||||
|
||||
self.params.put_bool("IsMetric", False)
|
||||
self.assertFalse(self.params.get_bool("IsMetric"))
|
||||
|
||||
self.params.put("IsMetric", "1")
|
||||
self.assertTrue(self.params.get_bool("IsMetric"))
|
||||
|
||||
self.params.put("IsMetric", "0")
|
||||
self.assertFalse(self.params.get_bool("IsMetric"))
|
||||
|
||||
def test_put_non_blocking_with_get_block(self):
|
||||
q = Params(self.tmpdir)
|
||||
def _delayed_writer():
|
||||
time.sleep(0.1)
|
||||
put_nonblocking("CarParams", "test", self.tmpdir)
|
||||
threading.Thread(target=_delayed_writer).start()
|
||||
assert q.get("CarParams") is None
|
||||
assert q.get("CarParams", True) == b"test"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,47 +0,0 @@
|
|||
import os
|
||||
import tempfile
|
||||
import shutil
|
||||
import unittest
|
||||
|
||||
from common.xattr import getxattr, setxattr, listxattr, removexattr
|
||||
|
||||
class TestParams(unittest.TestCase):
|
||||
USER_TEST='user.test'
|
||||
def setUp(self):
|
||||
self.tmpdir = tempfile.mkdtemp()
|
||||
self.tmpfn = os.path.join(self.tmpdir, 'test.txt')
|
||||
open(self.tmpfn, 'w').close()
|
||||
#print("using", self.tmpfn)
|
||||
|
||||
def tearDown(self):
|
||||
shutil.rmtree(self.tmpdir)
|
||||
|
||||
def test_getxattr_none(self):
|
||||
a = getxattr(self.tmpfn, TestParams.USER_TEST)
|
||||
assert a is None
|
||||
|
||||
def test_listxattr_none(self):
|
||||
l = listxattr(self.tmpfn)
|
||||
assert l == []
|
||||
|
||||
def test_setxattr(self):
|
||||
setxattr(self.tmpfn, TestParams.USER_TEST, b'123')
|
||||
a = getxattr(self.tmpfn, TestParams.USER_TEST)
|
||||
assert a == b'123'
|
||||
|
||||
def test_listxattr(self):
|
||||
setxattr(self.tmpfn, 'user.test1', b'123')
|
||||
setxattr(self.tmpfn, 'user.test2', b'123')
|
||||
l = listxattr(self.tmpfn)
|
||||
assert l == ['user.test1', 'user.test2']
|
||||
|
||||
def test_removexattr(self):
|
||||
setxattr(self.tmpfn, TestParams.USER_TEST, b'123')
|
||||
a = getxattr(self.tmpfn, TestParams.USER_TEST)
|
||||
assert a == b'123'
|
||||
removexattr(self.tmpfn, TestParams.USER_TEST)
|
||||
a = getxattr(self.tmpfn, TestParams.USER_TEST)
|
||||
assert a is None
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
transformations
|
||||
transformations.cpp
|
||||
|
|
@ -1,70 +0,0 @@
|
|||
|
||||
Reference Frames
|
||||
------
|
||||
Many reference frames are used throughout. This
|
||||
folder contains all helper functions needed to
|
||||
transform between them. Generally this is done
|
||||
by generating a rotation matrix and multiplying.
|
||||
|
||||
|
||||
| Name | [x, y, z] | Units | Notes |
|
||||
| :-------------: |:-------------:| :-----:| :----: |
|
||||
| Geodetic | [Latitude, Longitude, Altitude] | geodetic coordinates | Sometimes used as [lon, lat, alt], avoid this frame. |
|
||||
| ECEF | [x, y, z] | meters | We use **ITRF14 (IGS14)**, NOT NAD83. <br> This is the global Mesh3D frame. |
|
||||
| NED | [North, East, Down] | meters | Relative to earth's surface, useful for vizualizing. |
|
||||
| Device | [Forward, Right, Down] | meters | This is the Mesh3D local frame. <br> Relative to camera, **not imu.** <br> |
|
||||
| Calibrated | [Forward, Right, Down] | meters | This is the frame the model outputs are in. <br> More details below. <br>|
|
||||
| Car | [Forward, Right, Down] | meters | This is useful for estimating position of points on the road. <br> More details below. <br>|
|
||||
| View | [Right, Down, Forward] | meters | Like device frame, but according to camera conventions. |
|
||||
| Camera | [u, v, focal] | pixels | Like view frame, but 2d on the camera image.|
|
||||
| Normalized Camera | [u / focal, v / focal, 1] | / | |
|
||||
| Model | [u, v, focal] | pixels | The sampled rectangle of the full camera frame the model uses. |
|
||||
| Normalized Model | [u / focal, v / focal, 1] | / | |
|
||||
|
||||
|
||||
|
||||
|
||||
Orientation Conventations
|
||||
------
|
||||
Quaternions, rotation matrices and euler angles are three
|
||||
equivalent representations of orientation and all three are
|
||||
used throughout the code base.
|
||||
|
||||
For euler angles the preferred convention is [roll, pitch, yaw]
|
||||
which corresponds to rotations around the [x, y, z] axes. All
|
||||
euler angles should always be in radians or radians/s unless
|
||||
for plotting or display purposes. For quaternions the hamilton
|
||||
notations is preferred which is [q<sub>w</sub>, q<sub>x</sub>, q<sub>y</sub>, q<sub>z</sub>]. All quaternions
|
||||
should always be normalized with a strictly positive q<sub>w</sub>. **These
|
||||
quaternions are a unique representation of orientation whereas euler angles
|
||||
or rotation matrices are not.**
|
||||
|
||||
To rotate from one frame into another with euler angles the
|
||||
convention is to rotate around roll, then pitch and then yaw,
|
||||
while rotating around the rotated axes, not the original axes.
|
||||
|
||||
|
||||
Car frame
|
||||
------
|
||||
Device frame is aligned with the road-facing camera used by openpilot. However, when controlling the vehicle it is helpful to think in a reference frame aligned with the vehicle. These two reference frames can be different.
|
||||
|
||||
The orientation of car frame is defined to be aligned with the car's direction of travel and the road plane when the vehicle is driving on a flat road and not turning. The origin of car frame is defined to be directly below device frame (in car frame), such that it is on the road plane. The position and orientation of this frame is not necessarily always aligned with the direction of travel or the road plane due to suspension movements and other effects.
|
||||
|
||||
|
||||
Calibrated frame
|
||||
------
|
||||
It is helpful for openpilot's driving model to take in images that look similar when mounted differently in different cars. To achieve this we "calibrate" the images by transforming it into calibrated frame. Calibrated frame is defined to be aligned with car frame in pitch and yaw, and aligned with device frame in roll. It also has the same origin as device frame.
|
||||
|
||||
|
||||
Example
|
||||
------
|
||||
To transform global Mesh3D positions and orientations (positions_ecef, quats_ecef) into the local frame described by the
|
||||
first position and orientation from Mesh3D one would do:
|
||||
```
|
||||
ecef_from_local = rot_from_quat(quats_ecef[0])
|
||||
local_from_ecef = ecef_from_local.T
|
||||
positions_local = np.einsum('ij,kj->ki', local_from_ecef, postions_ecef - positions_ecef[0])
|
||||
rotations_global = rot_from_quat(quats_ecef)
|
||||
rotations_local = np.einsum('ij,kjl->kil', local_from_ecef, rotations_global)
|
||||
eulers_local = euler_from_rot(rotations_local)
|
||||
```
|
||||
|
|
@ -1,109 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
|
||||
import numpy as np
|
||||
import unittest
|
||||
|
||||
import common.transformations.coordinates as coord
|
||||
|
||||
geodetic_positions = np.array([[37.7610403, -122.4778699, 115],
|
||||
[27.4840915, -68.5867592, 2380],
|
||||
[32.4916858, -113.652821, -6],
|
||||
[15.1392514, 103.6976037, 24],
|
||||
[24.2302229, 44.2835412, 1650]])
|
||||
|
||||
ecef_positions = np.array([[-2711076.55270557, -4259167.14692758, 3884579.87669935],
|
||||
[ 2068042.69652729, -5273435.40316622, 2927004.89190746],
|
||||
[-2160412.60461669, -4932588.89873832, 3406542.29652851],
|
||||
[-1458247.92550567, 5983060.87496612, 1654984.6099885 ],
|
||||
[ 4167239.10867871, 4064301.90363223, 2602234.6065749 ]])
|
||||
|
||||
ecef_positions_offset = np.array([[-2711004.46961115, -4259099.33540613, 3884605.16002147],
|
||||
[ 2068074.30639499, -5273413.78835412, 2927012.48741131],
|
||||
[-2160344.53748176, -4932586.20092211, 3406636.2962545 ],
|
||||
[-1458211.98517094, 5983151.11161276, 1655077.02698447],
|
||||
[ 4167271.20055269, 4064398.22619263, 2602238.95265847]])
|
||||
|
||||
|
||||
ned_offsets = np.array([[78.722153649976391, 24.396208657446344, 60.343017506838436],
|
||||
[10.699003365155221, 37.319278617604269, 4.1084100025050407],
|
||||
[95.282646251726959, 61.266689955574428, -25.376506058505054],
|
||||
[68.535769283630003, -56.285970011848889, -100.54840137956515],
|
||||
[-33.066609321880179, 46.549821994306861, -84.062540548335591]])
|
||||
|
||||
ecef_init_batch = np.array([2068042.69652729, -5273435.40316622, 2927004.89190746])
|
||||
ecef_positions_offset_batch = np.array([[ 2068089.41454771, -5273434.46829148, 2927074.04783672],
|
||||
[ 2068103.31628647, -5273393.92275431, 2927102.08725987],
|
||||
[ 2068108.49939636, -5273359.27047121, 2927045.07091581],
|
||||
[ 2068075.12395611, -5273381.69432566, 2927041.08207992],
|
||||
[ 2068060.72033399, -5273430.6061505, 2927094.54928305]])
|
||||
|
||||
ned_offsets_batch = np.array([[ 53.88103168, 43.83445935, -46.27488057],
|
||||
[ 93.83378995, 71.57943024, -30.23113187],
|
||||
[ 57.26725796, 89.05602684, 23.02265814],
|
||||
[ 49.71775195, 49.79767572, 17.15351015],
|
||||
[ 78.56272609, 18.53100158, -43.25290759]])
|
||||
|
||||
|
||||
class TestNED(unittest.TestCase):
|
||||
def test_small_distances(self):
|
||||
start_geodetic = np.array([33.8042184, -117.888593, 0.0])
|
||||
local_coord = coord.LocalCoord.from_geodetic(start_geodetic)
|
||||
|
||||
start_ned = local_coord.geodetic2ned(start_geodetic)
|
||||
np.testing.assert_array_equal(start_ned, np.zeros(3,))
|
||||
|
||||
west_geodetic = start_geodetic + [0, -0.0005, 0]
|
||||
west_ned = local_coord.geodetic2ned(west_geodetic)
|
||||
self.assertLess(np.abs(west_ned[0]), 1e-3)
|
||||
self.assertLess(west_ned[1], 0)
|
||||
|
||||
southwest_geodetic = start_geodetic + [-0.0005, -0.002, 0]
|
||||
southwest_ned = local_coord.geodetic2ned(southwest_geodetic)
|
||||
self.assertLess(southwest_ned[0], 0)
|
||||
self.assertLess(southwest_ned[1], 0)
|
||||
|
||||
def test_ecef_geodetic(self):
|
||||
# testing single
|
||||
np.testing.assert_allclose(ecef_positions[0], coord.geodetic2ecef(geodetic_positions[0]), rtol=1e-9)
|
||||
np.testing.assert_allclose(geodetic_positions[0, :2], coord.ecef2geodetic(ecef_positions[0])[:2], rtol=1e-9)
|
||||
np.testing.assert_allclose(geodetic_positions[0, 2], coord.ecef2geodetic(ecef_positions[0])[2], rtol=1e-9, atol=1e-4)
|
||||
|
||||
np.testing.assert_allclose(geodetic_positions[:, :2], coord.ecef2geodetic(ecef_positions)[:, :2], rtol=1e-9)
|
||||
np.testing.assert_allclose(geodetic_positions[:, 2], coord.ecef2geodetic(ecef_positions)[:, 2], rtol=1e-9, atol=1e-4)
|
||||
np.testing.assert_allclose(ecef_positions, coord.geodetic2ecef(geodetic_positions), rtol=1e-9)
|
||||
|
||||
|
||||
def test_ned(self):
|
||||
for ecef_pos in ecef_positions:
|
||||
converter = coord.LocalCoord.from_ecef(ecef_pos)
|
||||
ecef_pos_moved = ecef_pos + [25, -25, 25]
|
||||
ecef_pos_moved_double_converted = converter.ned2ecef(converter.ecef2ned(ecef_pos_moved))
|
||||
np.testing.assert_allclose(ecef_pos_moved, ecef_pos_moved_double_converted, rtol=1e-9)
|
||||
|
||||
for geo_pos in geodetic_positions:
|
||||
converter = coord.LocalCoord.from_geodetic(geo_pos)
|
||||
geo_pos_moved = geo_pos + np.array([0, 0, 10])
|
||||
geo_pos_double_converted_moved = converter.ned2geodetic(converter.geodetic2ned(geo_pos) + np.array([0, 0, -10]))
|
||||
np.testing.assert_allclose(geo_pos_moved[:2], geo_pos_double_converted_moved[:2], rtol=1e-9, atol=1e-6)
|
||||
np.testing.assert_allclose(geo_pos_moved[2], geo_pos_double_converted_moved[2], rtol=1e-9, atol=1e-4)
|
||||
|
||||
def test_ned_saved_results(self):
|
||||
for i, ecef_pos in enumerate(ecef_positions):
|
||||
converter = coord.LocalCoord.from_ecef(ecef_pos)
|
||||
np.testing.assert_allclose(converter.ned2ecef(ned_offsets[i]),
|
||||
ecef_positions_offset[i],
|
||||
rtol=1e-9, atol=1e-4)
|
||||
np.testing.assert_allclose(converter.ecef2ned(ecef_positions_offset[i]),
|
||||
ned_offsets[i],
|
||||
rtol=1e-9, atol=1e-4)
|
||||
|
||||
def test_ned_batch(self):
|
||||
converter = coord.LocalCoord.from_ecef(ecef_init_batch)
|
||||
np.testing.assert_allclose(converter.ecef2ned(ecef_positions_offset_batch),
|
||||
ned_offsets_batch,
|
||||
rtol=1e-9, atol=1e-7)
|
||||
np.testing.assert_allclose(converter.ned2ecef(ned_offsets_batch),
|
||||
ecef_positions_offset_batch,
|
||||
rtol=1e-9, atol=1e-7)
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,68 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
|
||||
import numpy as np
|
||||
import unittest
|
||||
|
||||
from common.transformations.orientation import euler2quat, quat2euler, euler2rot, rot2euler, \
|
||||
rot2quat, quat2rot, \
|
||||
ned_euler_from_ecef
|
||||
|
||||
eulers = np.array([[ 1.46520501, 2.78688383, 2.92780854],
|
||||
[ 4.86909526, 3.60618161, 4.30648981],
|
||||
[ 3.72175965, 2.68763705, 5.43895988],
|
||||
[ 5.92306687, 5.69573614, 0.81100357],
|
||||
[ 0.67838374, 5.02402037, 2.47106426]])
|
||||
|
||||
quats = np.array([[ 0.66855182, -0.71500939, 0.19539353, 0.06017818],
|
||||
[ 0.43163717, 0.70013301, 0.28209145, 0.49389021],
|
||||
[ 0.44121991, -0.08252646, 0.34257534, 0.82532207],
|
||||
[ 0.88578382, -0.04515356, -0.32936046, 0.32383617],
|
||||
[ 0.06578165, 0.61282835, 0.07126891, 0.78424163]])
|
||||
|
||||
ecef_positions = np.array([[-2711076.55270557, -4259167.14692758, 3884579.87669935],
|
||||
[ 2068042.69652729, -5273435.40316622, 2927004.89190746],
|
||||
[-2160412.60461669, -4932588.89873832, 3406542.29652851],
|
||||
[-1458247.92550567, 5983060.87496612, 1654984.6099885 ],
|
||||
[ 4167239.10867871, 4064301.90363223, 2602234.6065749 ]])
|
||||
|
||||
ned_eulers = np.array([[ 0.46806039, -0.4881889 , 1.65697808],
|
||||
[-2.14525969, -0.36533066, 0.73813479],
|
||||
[-1.39523364, -0.58540761, -1.77376356],
|
||||
[-1.84220435, 0.61828016, -1.03310421],
|
||||
[ 2.50450101, 0.36304151, 0.33136365]])
|
||||
|
||||
|
||||
class TestOrientation(unittest.TestCase):
|
||||
def test_quat_euler(self):
|
||||
for i, eul in enumerate(eulers):
|
||||
np.testing.assert_allclose(quats[i], euler2quat(eul), rtol=1e-7)
|
||||
np.testing.assert_allclose(quats[i], euler2quat(quat2euler(quats[i])), rtol=1e-6)
|
||||
for i, eul in enumerate(eulers):
|
||||
np.testing.assert_allclose(quats[i], euler2quat(list(eul)), rtol=1e-7)
|
||||
np.testing.assert_allclose(quats[i], euler2quat(quat2euler(list(quats[i]))), rtol=1e-6)
|
||||
np.testing.assert_allclose(quats, euler2quat(eulers), rtol=1e-7)
|
||||
np.testing.assert_allclose(quats, euler2quat(quat2euler(quats)), rtol=1e-6)
|
||||
|
||||
def test_rot_euler(self):
|
||||
for eul in eulers:
|
||||
np.testing.assert_allclose(euler2quat(eul), euler2quat(rot2euler(euler2rot(eul))), rtol=1e-7)
|
||||
for eul in eulers:
|
||||
np.testing.assert_allclose(euler2quat(eul), euler2quat(rot2euler(euler2rot(list(eul)))), rtol=1e-7)
|
||||
np.testing.assert_allclose(euler2quat(eulers), euler2quat(rot2euler(euler2rot(eulers))), rtol=1e-7)
|
||||
|
||||
def test_rot_quat(self):
|
||||
for quat in quats:
|
||||
np.testing.assert_allclose(quat, rot2quat(quat2rot(quat)), rtol=1e-7)
|
||||
for quat in quats:
|
||||
np.testing.assert_allclose(quat, rot2quat(quat2rot(list(quat))), rtol=1e-7)
|
||||
np.testing.assert_allclose(quats, rot2quat(quat2rot(quats)), rtol=1e-7)
|
||||
|
||||
def test_euler_ned(self):
|
||||
for i in range(len(eulers)):
|
||||
np.testing.assert_allclose(ned_eulers[i], ned_euler_from_ecef(ecef_positions[i], eulers[i]), rtol=1e-7)
|
||||
#np.testing.assert_allclose(eulers[i], ecef_euler_from_ned(ecef_positions[i], ned_eulers[i]), rtol=1e-7)
|
||||
# np.testing.assert_allclose(ned_eulers, ned_euler_from_ecef(ecef_positions, eulers), rtol=1e-7)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -1,61 +0,0 @@
|
|||
import sys
|
||||
import pygame # pylint: disable=import-error
|
||||
import cv2 # pylint: disable=import-error
|
||||
|
||||
class Window():
|
||||
def __init__(self, w, h, caption="window", double=False, halve=False):
|
||||
self.w = w
|
||||
self.h = h
|
||||
pygame.display.init()
|
||||
pygame.display.set_caption(caption)
|
||||
self.double = double
|
||||
self.halve = halve
|
||||
if self.double:
|
||||
self.rw, self.rh = w*2, h*2
|
||||
elif self.halve:
|
||||
self.rw, self.rh = w//2, h//2
|
||||
else:
|
||||
self.rw, self.rh = w, h
|
||||
self.screen = pygame.display.set_mode((self.rw, self.rh))
|
||||
pygame.display.flip()
|
||||
|
||||
# hack for xmonad, it shrinks the window by 6 pixels after the display.flip
|
||||
if self.screen.get_width() != self.rw:
|
||||
self.screen = pygame.display.set_mode((self.rw+(self.rw-self.screen.get_width()), self.rh+(self.rh-self.screen.get_height())))
|
||||
pygame.display.flip()
|
||||
|
||||
def draw(self, out):
|
||||
pygame.event.pump()
|
||||
if self.double:
|
||||
out2 = cv2.resize(out, (self.w*2, self.h*2))
|
||||
pygame.surfarray.blit_array(self.screen, out2.swapaxes(0, 1))
|
||||
elif self.halve:
|
||||
out2 = cv2.resize(out, (self.w//2, self.h//2))
|
||||
pygame.surfarray.blit_array(self.screen, out2.swapaxes(0, 1))
|
||||
else:
|
||||
pygame.surfarray.blit_array(self.screen, out.swapaxes(0, 1))
|
||||
pygame.display.flip()
|
||||
|
||||
def getkey(self):
|
||||
while 1:
|
||||
event = pygame.event.wait()
|
||||
if event.type == pygame.QUIT:
|
||||
pygame.quit()
|
||||
sys.exit()
|
||||
if event.type == pygame.KEYDOWN:
|
||||
return event.key
|
||||
|
||||
def getclick(self):
|
||||
for event in pygame.event.get():
|
||||
if event.type == pygame.MOUSEBUTTONDOWN:
|
||||
mx, my = pygame.mouse.get_pos()
|
||||
return mx, my
|
||||
|
||||
if __name__ == "__main__":
|
||||
import numpy as np
|
||||
win = Window(200, 200, double=True)
|
||||
img = np.zeros((200, 200, 3), np.uint8)
|
||||
while 1:
|
||||
print("draw")
|
||||
img += 1
|
||||
win.draw(img)
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
# Minimal makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
OPENPILOT_ROOT = `git rev-parse --show-toplevel`
|
||||
|
||||
# You can set these variables from the command line, and also
|
||||
# from the environment for the first two.
|
||||
SPHINXOPTS ?=
|
||||
SPHINXBUILD ?= sphinx-build
|
||||
DOCSDIR = "$(OPENPILOT_ROOT)/docs"
|
||||
SOURCEDIR = "$(OPENPILOT_ROOT)/build/docs"
|
||||
DOCSBUILDDIR = "$(OPENPILOT_ROOT)/build/docs"
|
||||
BUILDDIR = "$(OPENPILOT_ROOT)/build"
|
||||
|
||||
# Put it first so that "make" without argument is like "make help".
|
||||
help:
|
||||
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(DOCSBUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
||||
clean:
|
||||
@echo "Cleaning build folder..."
|
||||
rm -rf "$(BUILDDIR)"
|
||||
|
||||
.PHONY: help Makefile
|
||||
|
||||
# Catch-all target: route all unknown targets to Sphinx using the new
|
||||
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
|
||||
%: Makefile
|
||||
@echo "Cleaning build folder..."
|
||||
rm -rf "$(BUILDDIR)"
|
||||
mkdir -p "$(DOCSBUILDDIR)"
|
||||
|
||||
@echo "Copying docs & config to build folder..."
|
||||
cp -a "$(DOCSDIR)" "$(BUILDDIR)"
|
||||
cd "$(OPENPILOT_ROOT)" && \
|
||||
find . -type f \( -name "*.md" -o -name "*.rst" -o -name "*.png" -o -name "*.jpg" \) \
|
||||
-not -path "*/.*" \
|
||||
-not -path "./build/*" \
|
||||
-not -path "./docs/*" \
|
||||
-not -path "./xx/*" \
|
||||
-exec cp --parents "{}" ./build/docs/ \;
|
||||
|
||||
@echo "Building rst files..."
|
||||
sphinx-apidoc -o "$(DOCSBUILDDIR)" ../ \
|
||||
../xx ../laika_repo ../rednose_repo ../pyextra ../notebooks ../panda_jungle \
|
||||
../third_party \
|
||||
../panda/examples \
|
||||
../scripts \
|
||||
../selfdrive/modeld \
|
||||
../selfdrive/debug \
|
||||
$(shell find .. -type d -name "*test*")
|
||||
|
||||
@echo "Building html files..."
|
||||
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(DOCSBUILDDIR)" $(SPHINXOPTS) $(O)
|
||||
|
|
@ -1,33 +0,0 @@
|
|||
# Safety
|
||||
|
||||
openpilot is an Adaptive Cruise Control (ACC) and Automated Lane Centering (ALC) system.
|
||||
Like other ACC and ALC systems, openpilot is a failsafe passive system and it requires the
|
||||
driver to be alert and to pay attention at all times.
|
||||
|
||||
In order to enforce driver alertness, openpilot includes a driver monitoring feature
|
||||
that alerts the driver when distracted.
|
||||
|
||||
However, even with an attentive driver, we must make further efforts for the system to be
|
||||
safe. We repeat, **driver alertness is necessary, but not sufficient, for openpilot to be
|
||||
used safely** and openpilot is provided with no warranty of fitness for any purpose.
|
||||
|
||||
openpilot is developed in good faith to be compliant with FMVSS requirements and to follow
|
||||
industry standards of safety for Level 2 Driver Assistance Systems. In particular, we observe
|
||||
ISO26262 guidelines, including those from [pertinent documents](https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/13498a_812_573_alcsystemreport.pdf)
|
||||
released by NHTSA. In addition, we impose strict coding guidelines (like [MISRA C : 2012](https://www.misra.org.uk/what-is-misra/))
|
||||
on parts of openpilot that are safety relevant. We also perform software-in-the-loop,
|
||||
hardware-in-the-loop and in-vehicle tests before each software release.
|
||||
|
||||
Following Hazard and Risk Analysis and FMEA, at a very high level, we have designed openpilot
|
||||
ensuring two main safety requirements.
|
||||
|
||||
1. The driver must always be capable to immediately retake manual control of the vehicle,
|
||||
by stepping on either pedal or by pressing the cancel button.
|
||||
2. The vehicle must not alter its trajectory too quickly for the driver to safely
|
||||
react. This means that while the system is engaged, the actuators are constrained
|
||||
to operate within reasonable limits.
|
||||
|
||||
For additional safety implementation details, refer to [panda safety model](https://github.com/commaai/panda#safety-model). For vehicle specific implementation of the safety concept, refer to [panda/board/safety/](https://github.com/commaai/panda/tree/master/board/safety).
|
||||
|
||||
**Extra note**: comma.ai strongly discourages the use of openpilot forks with safety code either missing or
|
||||
not fully meeting the above requirements.
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 15 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 KiB |
|
|
@ -1,2 +0,0 @@
|
|||
User-agent: *
|
||||
Sitemap: https://docs.comma.ai/sitemap.xml
|
||||
|
|
@ -1,3 +0,0 @@
|
|||
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M12.2584 16.8419L12 16.686L11.7416 16.8419L6.57613 19.9596L7.94693 14.0836L8.01549 13.7897L7.78739 13.5921L3.22308 9.63803L9.23227 9.12821L9.5327 9.10272L9.65036 8.82511L12 3.2813L14.3496 8.82511L14.4673 9.10272L14.7677 9.12821L20.7769 9.63803L16.2126 13.5921L15.9845 13.7897L16.0531 14.0836L17.4239 19.9596L12.2584 16.8419Z" stroke="#B7B7B7"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 457 B |
|
|
@ -1,3 +0,0 @@
|
|||
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M12.2584 16.8419L12 16.686L11.7416 16.8419L6.57613 19.9596L7.94693 14.0836L8.01549 13.7897L7.78739 13.5921L3.22308 9.63803L9.23227 9.12821L9.5327 9.10272L9.65036 8.82511L12 3.2813L14.3496 8.82511L14.4673 9.10272L14.7677 9.12821L20.7769 9.63803L16.2126 13.5921L15.9845 13.7897L16.0531 14.0836L17.4239 19.9596L12.2584 16.8419Z" fill="#F5C543" stroke="#F0A43B"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 472 B |
|
|
@ -1,5 +0,0 @@
|
|||
<svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M12 2V17.27L5.82 21L7.46 13.97L2 9.24L9.19 8.63L12 2Z" fill="#F5C543"/>
|
||||
<path d="M12.2584 16.8419L12 16.686L11.7416 16.8419L6.57613 19.9596L7.94693 14.0836L8.01549 13.7897L7.78739 13.5921L3.22308 9.63803L9.23227 9.12821L9.5327 9.10272L9.65036 8.82511L12 3.2813L14.3496 8.82511L14.4673 9.10272L14.7677 9.12821L20.7769 9.63803L16.2126 13.5921L15.9845 13.7897L16.0531 14.0836L17.4239 19.9596L12.2584 16.8419Z" stroke="#B7B7B7"/>
|
||||
<path d="M7.33226 18.9192L11 16.7055L12 16.1V17.27L5.82 21L7.46 13.97L2 9.24L9.19 8.63L12 2V4.6L11 6.92203L9.8754 9.57544L4.44617 10.0361L8.57098 13.6094L7.33226 18.9192Z" fill="#F0A43B"/>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 727 B |
105
docs/c_docs.rst
105
docs/c_docs.rst
|
|
@ -1,105 +0,0 @@
|
|||
openpilot
|
||||
==========
|
||||
|
||||
|
||||
opendbc
|
||||
------
|
||||
.. autodoxygenindex::
|
||||
:project: opendbc_can
|
||||
|
||||
|
||||
cereal
|
||||
------
|
||||
|
||||
messaging
|
||||
^^^^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: cereal_messaging
|
||||
|
||||
visionipc
|
||||
^^^^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: cereal_visionipc
|
||||
|
||||
|
||||
selfdrive
|
||||
---------
|
||||
|
||||
camerad
|
||||
^^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_camerad_cameras
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_camerad_transforms
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_camerad_imgproc
|
||||
|
||||
locationd
|
||||
^^^^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_locationd
|
||||
|
||||
ui
|
||||
^^
|
||||
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_ui
|
||||
|
||||
soundd
|
||||
""""""
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_ui_soundd
|
||||
|
||||
navd
|
||||
""""
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_ui_navd
|
||||
|
||||
replay
|
||||
""""""
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_ui_replay
|
||||
|
||||
qt
|
||||
""
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_ui_qt_offroad
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_ui_qt_maps
|
||||
|
||||
proclogd
|
||||
^^^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_proclogd
|
||||
|
||||
modeld
|
||||
^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_modeld_transforms
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_modeld_models
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_modeld_thneed
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_modeld_runners
|
||||
|
||||
common
|
||||
^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_common
|
||||
|
||||
sensorsd
|
||||
^^^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_sensord_sensors
|
||||
|
||||
boardd
|
||||
^^^^^^
|
||||
.. autodoxygenindex::
|
||||
:project: selfdrive_boardd
|
||||
|
||||
|
||||
rednose
|
||||
-------
|
||||
.. autodoxygenindex::
|
||||
:project: rednose_repo_rednose_helpers
|
||||
146
docs/conf.py
146
docs/conf.py
|
|
@ -1,146 +0,0 @@
|
|||
# type: ignore
|
||||
# pylint: skip-file
|
||||
|
||||
# Configuration file for the Sphinx documentation builder.
|
||||
#
|
||||
# This file only contains a selection of the most common options. For a full
|
||||
# list see the documentation:
|
||||
# https://www.sphinx-doc.org/en/master/usage/configuration.html
|
||||
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
#
|
||||
import os
|
||||
from os.path import exists
|
||||
import sys
|
||||
from selfdrive.version import get_version
|
||||
from common.basedir import BASEDIR
|
||||
sys.path.insert(0, os.path.abspath('.'))
|
||||
sys.path.insert(0, os.path.abspath('..'))
|
||||
|
||||
VERSION = get_version()
|
||||
|
||||
|
||||
# -- Project information -----------------------------------------------------
|
||||
|
||||
project = 'openpilot docs'
|
||||
copyright = '2021, comma.ai'
|
||||
author = 'comma.ai'
|
||||
version = VERSION
|
||||
release = VERSION
|
||||
language = 'en'
|
||||
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = [
|
||||
'sphinx.ext.autodoc', # Auto-generate docs
|
||||
'sphinx.ext.viewcode', # Add view code link to modules
|
||||
'sphinx_rtd_theme', # Read The Docs theme
|
||||
'myst_parser', # Markdown parsing
|
||||
'breathe', # Doxygen C/C++ integration
|
||||
'sphinx_sitemap', # sitemap generation for SEO
|
||||
]
|
||||
|
||||
myst_html_meta = {
|
||||
"description": "openpilot docs",
|
||||
"keywords": "op, openpilot, docs, documentation",
|
||||
"robots": "all,follow",
|
||||
"googlebot": "index,follow,snippet,archive",
|
||||
"property=og:locale": "en_US",
|
||||
"property=og:site_name": "docs.comma.ai",
|
||||
"property=og:url": "https://docs.comma.ai",
|
||||
"property=og:title": "openpilot Documentation",
|
||||
"property=og:type": "website",
|
||||
"property=og:image:type": "image/jpeg",
|
||||
"property=og:image:width": "400",
|
||||
"property=og:image": "https://docs.comma.ai/_static/logo.png",
|
||||
"property=og:image:url": "https://docs.comma.ai/_static/logo.png",
|
||||
"property=og:image:secure_url": "https://docs.comma.ai/_static/logo.png",
|
||||
"property=og:description": "openpilot Documentation",
|
||||
"property=twitter:card": "summary_large_image",
|
||||
"property=twitter:logo": "https://docs.comma.ai/_static/logo.png",
|
||||
"property=twitter:title": "openpilot Documentation",
|
||||
"property=twitter:description": "openpilot Documentation"
|
||||
}
|
||||
|
||||
html_baseurl = 'https://docs.comma.ai/'
|
||||
sitemap_filename = "sitemap.xml"
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
# This pattern also affects html_static_path and html_extra_path.
|
||||
exclude_patterns = []
|
||||
|
||||
|
||||
# -- c docs configuration ---------------------------------------------------
|
||||
|
||||
# Breathe Configuration
|
||||
# breathe_default_project = "c_docs"
|
||||
breathe_build_directory = f"{BASEDIR}/build/docs/html/xml"
|
||||
breathe_separate_member_pages = True
|
||||
breathe_default_members = ('members', 'private-members', 'undoc-members')
|
||||
breathe_domain_by_extension = {
|
||||
"h": "cc",
|
||||
}
|
||||
breathe_implementation_filename_extensions = ['.c', '.cc']
|
||||
breathe_doxygen_config_options = {}
|
||||
breathe_projects_source = {}
|
||||
|
||||
# only document files that have accompanying .cc files next to them
|
||||
print("searching for c_docs...")
|
||||
for root, dirs, files in os.walk(BASEDIR):
|
||||
found = False
|
||||
breath_src = {}
|
||||
breathe_srcs_list = []
|
||||
|
||||
for file in files:
|
||||
ccFile = os.path.join(root, file)[:-2] + ".cc"
|
||||
|
||||
if file.endswith(".h") and exists(ccFile):
|
||||
f = os.path.join(root, file)
|
||||
|
||||
parent_dir_abs = os.path.dirname(f)
|
||||
parent_dir = parent_dir_abs[len(BASEDIR) + 1:]
|
||||
parent_project = parent_dir.replace('/', '_')
|
||||
print(f"\tFOUND: {f} in {parent_project}")
|
||||
|
||||
breathe_srcs_list.append(file)
|
||||
found = True
|
||||
|
||||
if found:
|
||||
breath_src[parent_project] = (parent_dir_abs, breathe_srcs_list)
|
||||
breathe_projects_source.update(breath_src)
|
||||
|
||||
print(f"breathe_projects_source: {breathe_projects_source.keys()}")
|
||||
|
||||
# -- Options for HTML output -------------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
#
|
||||
html_theme = 'sphinx_rtd_theme'
|
||||
html_show_copyright = True
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
html_static_path = ['_static']
|
||||
html_logo = '_static/logo.png'
|
||||
html_favicon = '_static/favicon.ico'
|
||||
html_theme_options = {
|
||||
'logo_only': False,
|
||||
'display_version': True,
|
||||
'vcs_pageview_mode': 'blob',
|
||||
'style_nav_header_background': '#000000',
|
||||
}
|
||||
html_extra_path = ['_static']
|
||||
|
|
@ -1,45 +0,0 @@
|
|||
FROM ghcr.io/commaai/openpilot-base:latest
|
||||
|
||||
ENV PYTHONUNBUFFERED 1
|
||||
|
||||
ENV OPENPILOT_PATH /home/batman/openpilot/
|
||||
ENV PYTHONPATH ${OPENPILOT_PATH}:${PYTHONPATH}
|
||||
|
||||
RUN mkdir -p ${OPENPILOT_PATH}
|
||||
WORKDIR ${OPENPILOT_PATH}
|
||||
|
||||
COPY Pipfile Pipfile.lock $OPENPILOT_PATH
|
||||
RUN pip install --no-cache-dir pipenv==2021.5.29 pip==21.3.1 && \
|
||||
pipenv install --system --deploy --dev --clear && \
|
||||
pip uninstall -y pipenv
|
||||
|
||||
|
||||
COPY SConstruct ${OPENPILOT_PATH}
|
||||
|
||||
COPY ./pyextra ${OPENPILOT_PATH}/pyextra
|
||||
COPY ./third_party ${OPENPILOT_PATH}/third_party
|
||||
COPY ./site_scons ${OPENPILOT_PATH}/site_scons
|
||||
COPY ./laika ${OPENPILOT_PATH}/laika
|
||||
COPY ./laika_repo ${OPENPILOT_PATH}/laika_repo
|
||||
COPY ./rednose ${OPENPILOT_PATH}/rednose
|
||||
COPY ./rednose_repo ${OPENPILOT_PATH}/rednose_repo
|
||||
COPY ./tools ${OPENPILOT_PATH}/tools
|
||||
COPY ./release ${OPENPILOT_PATH}/release
|
||||
COPY ./common ${OPENPILOT_PATH}/common
|
||||
COPY ./opendbc ${OPENPILOT_PATH}/opendbc
|
||||
COPY ./cereal ${OPENPILOT_PATH}/cereal
|
||||
COPY ./panda ${OPENPILOT_PATH}/panda
|
||||
COPY ./selfdrive ${OPENPILOT_PATH}/selfdrive
|
||||
COPY ./*.md ${OPENPILOT_PATH}/
|
||||
|
||||
RUN scons -j$(nproc)
|
||||
|
||||
RUN apt update && apt install doxygen -y
|
||||
COPY ./docs ${OPENPILOT_PATH}/docs
|
||||
RUN git init .
|
||||
WORKDIR ${OPENPILOT_PATH}/docs
|
||||
RUN make html
|
||||
|
||||
FROM nginx:1.21
|
||||
COPY --from=0 /home/batman/openpilot/build/docs/html /usr/share/nginx/html
|
||||
COPY ./docs/docker/nginx.conf /etc/nginx/conf.d/default.conf
|
||||
|
|
@ -1,15 +0,0 @@
|
|||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name localhost;
|
||||
|
||||
gzip on;
|
||||
gzip_types text/html text/plain text/css text/xml text/javascript application/javascript application/x-javascript;
|
||||
gzip_min_length 1024;
|
||||
gzip_vary on;
|
||||
|
||||
root /usr/share/nginx/html;
|
||||
location / {
|
||||
try_files $uri $uri/ /index.html;
|
||||
}
|
||||
}
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
# openpilot Documentation
|
||||
|
||||
```{include} README.md
|
||||
```
|
||||
|
||||
```{toctree}
|
||||
:caption: 'General'
|
||||
:maxdepth: 4
|
||||
|
||||
CARS.md
|
||||
CONTRIBUTING.md
|
||||
INTEGRATION.md
|
||||
LIMITATIONS.md
|
||||
SAFETY.md
|
||||
```
|
||||
|
||||
```{toctree}
|
||||
:caption: 'Overview'
|
||||
:maxdepth: 2
|
||||
|
||||
overview.rst
|
||||
```
|
||||
|
||||
## API Documentation
|
||||
|
||||
- {ref}`genindex`
|
||||
- {ref}`modindex`
|
||||
- {ref}`search`
|
||||
|
||||
```{toctree}
|
||||
:caption: 'Python API'
|
||||
:maxdepth: 2
|
||||
|
||||
modules.rst
|
||||
```
|
||||
|
||||
```{toctree}
|
||||
:caption: 'C/C++ API'
|
||||
:maxdepth: 4
|
||||
|
||||
c_docs.rst
|
||||
```
|
||||
|
|
@ -1,79 +0,0 @@
|
|||
openpilot
|
||||
=========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
Debugging <selfdrive/debug/README.md>
|
||||
selfdrive/loggerd/README.md
|
||||
Driver Monitoring <selfdrive/monitoring/README.md>
|
||||
Process Replay <selfdrive/test/process_replay/README.md>
|
||||
|
||||
cereal
|
||||
=========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
cereal/README.md
|
||||
cereal/messaging/msgq.md
|
||||
|
||||
laika
|
||||
=========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
laika_repo/README.md
|
||||
|
||||
models
|
||||
=========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
models/README.md
|
||||
|
||||
opendbc
|
||||
=========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
opendbc/README.md
|
||||
|
||||
panda
|
||||
=========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
panda/README.md
|
||||
panda/UPDATING.md
|
||||
panda/board/README.md
|
||||
panda/drivers/linux/README.md
|
||||
panda/drivers/windows/README.md
|
||||
|
||||
|
||||
rednose
|
||||
=========
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
rednose_repo/README.md
|
||||
|
||||
|
||||
tools
|
||||
=========
|
||||
.. toctree::
|
||||
:maxdepth: 4
|
||||
|
||||
tools/CTF.md
|
||||
tools/joystick/README.md
|
||||
tools/lib/README.md
|
||||
tools/plotjuggler/README.md
|
||||
tools/replay/README.md
|
||||
tools/serial/README.md
|
||||
Simulator <tools/sim/README.md>
|
||||
tools/ssh/README.md
|
||||
Webcam <tools/webcam/README.md>
|
||||
|
|
@ -1 +0,0 @@
|
|||
Subproject commit 94066cb2b4ad5f2bcb8e33ce02fe15a73a00aace
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue