Commit Graph

49 Commits (spacecruft)

Author SHA1 Message Date
Willem Melching 33edb62967
hypothesis fuzz testing (#20818)
* add hypothesis testing

* cleanup

* some cleanup

* check for infinity too

* one-liner

* add more fields that are used

* no print

* add locationd testing

* sensor timestamp is signed

* recursive finite checking

* keep locationd inputs finite for now

* specify proces name on command line

* increase timeout and add raw speed

* abstract runner in function

* add unittest class
2021-05-05 13:25:29 +02:00
Willem Melching 7dcda0e461 remove raven from pip packages 2021-04-22 15:29:43 +02:00
robbederks 6a824d7651
Zookeeper support (#20000)
* zookeeper lib

* add ft4222 to dev pip packages and fix include error

* started on CI

* it's a file

* now it should be happy

* use docker for all on-device tests

* test scripts

* does this work?

* access to devices

* too broad. only usb enough?

* permissions for zookeeper usb

* as env var maybe?

* this?

* try this for now

* all devices

* move to correct location for impoerts

* right paths

* not running in the right agent?

* ofc not

* fix broken merge

* add ft4222 package again

* add timeout

* power monitor

* cleanup

Co-authored-by: Batman <batman@openpilot-ci.internal>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2021-04-13 15:08:00 -07:00
Greg Hogan 3d48bd934d
log to file and send through athena (#20250)
* log to file and send through athena

* rename logging level

* pass thru log formatter

* logMessage is TEXT

* send queue always strings

* switch to xattr and lower priority queue

* enable cloud logging for devices

* time or size based log rotation

* basename -> dirname

* remove HARDWARE.get_cloudlog_enabled

* fix errors

* fix another exception

* xattrs need to be bytes

* sending works

* cleanup files at start

* add id and adjust formatting

* do not send active log file

* better names

* separate log formatters

* fix formatter super init

* fix log file order

* ensure file always has file formatter

* i see why there was no formatter

* apply same formatting to cpp log msgs

* apply same formatting to cpp log msgs

* update queue names in tests

* strip deprecated keys in STATUS_PACKET

* strip DEPRECATED from dict recursively

* athena log queue test

* instanceof instead of type

* isinstance instead of type

* use super

* remove logentries

* last_scan param unused

* comment about special log msg attr names

* add dict_helpers.py to release files

* use monotonic time and counter for log rotation

* update for adjusted log file naming

* use monotonic clock for tracking last log file scan
2021-03-25 13:30:09 -07:00
Adeeb Shihadeh 20e6f6bd79
bump pycapnp version (#20326)
* bump pycapnp version

* bump cereal
2021-03-11 20:18:38 -08:00
Adeeb Shihadeh 3e70395e82
add new sentry client package (#20178) 2021-02-26 22:03:19 -08:00
Greg Hogan 5fbfaaef52 remove some pip packages 2021-02-23 11:38:05 -08:00
Greg Hogan a796294474 eliminate python-logstash-async 2021-02-20 10:16:46 -08:00
Willem Melching 4c04e220a1
Timezoned (#19960)
* untested code

* add to manager

* whitespace

* only save on fix

* cleanup

* fix typo

* add to ignored processes

* import tici

* fix param name

* cleanup

* only run offroad

* use itertools

* wait for thermald

* simpler

* fall back to api call

* add param to override timezone

Co-authored-by: Comma Device <device@comma.ai>
2021-02-06 11:07:37 +01:00
George Hotz bd9388c933 pyopencl pipenv 2021-01-08 10:47:19 -08:00
George Hotz cc4c34cc0b
smp_pipenv (#19687) 2021-01-07 08:52:51 -08:00
George Hotz bbe868d312 Revert "add two dev packages for training"
This reverts commit 3da368247a.
2020-12-09 21:48:38 -08:00
George Hotz 3da368247a add two dev packages for training 2020-12-09 11:07:18 -08:00
George Hotz 2df0f9ffb1 add onnx and bump versions in Pipfile.lock 2020-11-14 17:20:20 -08:00
HaraldSchafer 08846b5c0e
Torch model (#2452)
* refactor draw model

* rebase master

* correct valid_len

* rename function

* rename variables

* white space

* rebase to master

* e16c13ac-927d-455e-ae0a-81b482a2c787

* start rewriting

* save proress

* compiles!

* oops

* many fixes

* seems to work

* fix desires

* finally cleaned

* wrong std for ll

* dont pulse none

* compiles!

* ready to test

* WIP does not compile

* compiles

* various fixes

* does something!

* full 3d

* not needed

* draw up to 100m

* fix segfault

* wrong sign

* fix flicker

* add road edges

* finish v2 packet

* Added pytorch supercombo

* fix rebase

* no more keras

* Hacky solution to the NCHW/NHWC incompatibility between SNPE and our frame data

* dont break dmonitoringd, final model 229e3ce1-7259-412b-85e6-cc646d70f1d8/430

* fix hack

* Revert "fix hack"

This reverts commit 5550fc01a7881d065a5eddbbb42dac55ef7ec36c.

* Removed axis permutation hack

* Folded padding layers into conv layers

* Removed the last pad layer from the dlc

* Revert "Removed the last pad layer from the dlc"

This reverts commit b85f24b9e1d04abf64e85901a7ff49e00d82020a.

* Revert "Folded padding layers into conv layers"

This reverts commit b8d1773e4e76dea481acebbfad6a6235fbb58463.

* vision model: 5034ac8b-5703-4a49-948b-11c064d10880/780  temporal model: 229e3ce1-7259-412b-85e6-cc646d70f1d8/430  with permute + pool opt

* fix ui drawing with clips

* ./compile_torch.py 5034ac8b-5703-4a49-948b-11c064d10880/780 dfcd2375-81d8-49df-95bf-1d2d6ad86010/450 with variable history length

* std::clamp

* not sure how this compiled before

* 2895ace6-a296-47ac-86e6-17ea800a74e5/550

* db090195-8810-42de-ab38-bb835d775d87/601

* 5m is very little

* onnx runner

* add onnxruntime to pipfile

* run in real time without using the whole CPU

* bump cereal;

* add stds

* set road edge opacity based on stddev

* don't access the model packet in paint

* convert mat.h to a c++ header file (#2499)

* update tests

* safety first

Co-authored-by: deanlee <deanlee3@gmail.com>
Co-authored-by: mitchell <mitchell@comma.ai>
Co-authored-by: Comma Device <device@comma.ai>
Co-authored-by: George Hotz <george@comma.ai>
Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2020-11-11 20:31:46 -08:00
Greg Hogan 789a94932b Revert "pytorch cuda 11.0 (#2496)" (didn't work!)
This reverts commit 14c6de9094.
2020-11-06 19:38:23 -08:00
Greg Hogan 14c6de9094
pytorch cuda 11.0 (#2496) 2020-11-06 19:02:25 -08:00
George Hotz e63f434d5c
pin blosc (#2492) 2020-11-05 21:06:08 -08:00
George Hotz e927bd89ab Revert "pin blosc version"
This reverts commit 433714e377.
2020-11-05 16:41:07 -08:00
George Hotz 433714e377 pin blosc version 2020-11-05 16:24:41 -08:00
Greg Hogan 06f63d37fa
add blosc to pipfile (#2464) 2020-10-31 08:19:54 -07:00
Adeeb Shihadeh 719c5e3598
ubuntu 16.04 -> 20.04 (#2378) 2020-10-29 16:51:09 -07:00
Greg Hogan 05c16b1ff9 pytorch pip packages 2020-09-03 17:57:11 -07:00
Harald Schafer 270880c705 tf 2.2. 2020-08-08 11:17:41 -07:00
Harald Schafer b467f5777b not useful 2020-07-29 10:21:44 -07:00
Willem Melching 95b0c69c12
update pipfile.lock (#1896)
* update pipfile

* matrix is deprecated

* Numpy almost equal

* sympy 1.6
2020-07-21 21:15:09 +02:00
Adeeb Shihadeh b44569284a
test each model fingerprints with FW fingerprinting (#1769)
* test each model fingerprints with FW fingerprinting

* lock

* test for all
2020-06-23 15:33:04 -07:00
Willem Melching 6f46eed874 Fixed virtualenv https://github.com/pypa/virtualenv/issues/1857 2020-06-12 12:25:18 -07:00
Willem Melching 0f26c04a7b use older pre-commit version 2020-06-12 12:14:16 -07:00
Willem Melching b19098dbf8 remove pycocotools 2020-06-12 11:48:14 -07:00
Willem Melching 3d08dcc3b2
Run mypy commit hook (#1591)
* run mypy commit hook

* fix mypy errors
2020-05-28 15:05:04 -07:00
Willem Melching bd06434243
pre-commit pylint (#1580)
* "The commit-hook project sounds interesting though. I would definitely merge something that runs flake8 and pylint on the modified files!"
-  pd0wm, https://github.com/commaai/openpilot/pull/1575#issuecomment-634974344

* add pylint to pre-commit and make everything pass

* Remove uncommented stuff

Co-authored-by: J <user@4800.lan>
2020-05-27 20:00:14 -07:00
Ewout ter Hoeven b1795235f9
Add workflow to update Pipfile.lock weekly (#1542)
* Create workflow to update the Pipfile.lock weekly

* Update-pipfile: Also install wheel

* Install wheel in different step

* Pipfile: Only lock major versions

Assumes Semantic Versioning, and only sets the major versions of all packages

* Run update-pipfile on changes in pipfile and use tensorflow

Move from tensorflow-gpu to tensorflow, an excelent test for this PR

* [create-pull-request] automated change

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2020-05-20 13:16:02 -07:00
Harald Schafer 06d4b80792 breaks docker 2020-05-12 17:00:33 -07:00
George Hotz e3999495fa update tensorflow-gpu to 2.2.0 2020-05-08 11:02:11 -07:00
ZwX1616 aa460a1623 add TFmot to pipenv 2020-05-06 13:15:30 -07:00
Willem Melching b3e7d94303 run both deterministic and statistical profiler 2020-04-30 17:39:37 -07:00
Willem Melching 079c9e714b Write profiling result after running 2020-04-30 16:19:12 -07:00
Greg Hogan 3e78705761 tensorflow-gpu==2.2.0rc4 2020-04-30 12:10:58 -07:00
Willem Melching bb02a565b5 remove pyqt5 2020-04-29 13:11:12 -07:00
George Hotz 0a6d9a7bbd remove unused overpy, bump ubuntu_setup to 3.8.2 2020-04-28 15:23:39 -07:00
Harald Schafer 18ab25d919 python 3.82 2020-04-27 15:01:19 -07:00
Harald Schafer ba0d92de52 add that in for dlib 2020-04-24 13:32:58 -07:00
Harald Schafer 6a551dee7d new packages 2020-04-24 11:12:57 -07:00
HaraldSchafer 6840f96009
clean a little (#1416)
* clean a little

* those are needed

* put those back

* :Revert "those are needed"

This reverts commit ee7671af1990626eb803b74f2abd7ce926cf3b67.

* fix messed up commits

* need that for ublox test
2020-04-23 22:39:35 -07:00
Harald Schafer 88b4a727c2 tf 2.2 2020-04-22 19:20:53 -07:00
George Hotz 027deba07f upgrade to tensorflow 2.1 2020-03-27 11:45:23 -07:00
Harald Schafer df6b16c7f6 tf 2.1 has issues 2020-01-21 14:28:49 -08:00
George Hotz 012535a84e root directory non hidden files 2020-01-17 10:22:00 -08:00