Commit Graph

22 Commits (master)

Author SHA1 Message Date
Shane Smiskol e5005b8f44
scons: fix duplicate environments with test flag (#23702)
* fix duplicate environments when built with --test

* lib

Co-authored-by: Adeeb Shihadeh <adeebshihadeh@gmail.com>
2022-02-06 14:08:02 -08:00
Dean Lee b5eb02181c
loggerd: add test case for sync_encoders (#23020) 2021-11-24 13:34:16 -08:00
Dean Lee 75dd0d6296
loggerd: split loggerd.cc into three files 2021-11-23 17:53:49 -08:00
Dean Lee 2b4a477fbc
replay: improve segment download and merge (#22654)
* no-cache mode

* fix test cases build error

* space

* don't create cache dir in no-cache mode

* fix errors in test cases

* no_local_cache_

* set the number of connections by chunk_size

* use size_t instead of int64_t

* add test case for no-cache mode

* rename variables

* fix SIGSEGV

* cleanup

* faster decompressBZ2

* always decompress bz2

* add test cases

* prepare for python interface

* fix test cases build error

* continue

* camera_replay: cache remote file

* protected inheritance

* single option name

* TODO

* test_case for LogReader&FrameReader

* fix wrong require

* test case for FileReader

* cleanup test

* test:fix wrong filename

* check cached file's checksum

* fix mkdir permissions err

cleanup filereader

* remove initialize libav network libraries.

dd

* abort all loading if one failed

* cleanup tests

* use threadpool to limit concurrent downloads

* cache more segments

* merge 3 segments for replay

* one segment uses about 100M of memory

* use segments_need_merge.size()

* shutdown

* fix stuck if exit replay  before keyboard thread started

* load one segment at a time

* small cleanup

* cleanup filereader

* space

* tiny cleanup

* merge master

* cleanup test cases

* use util:create_directories

* cleanup framereader
2021-11-01 11:55:56 +01:00
Dean Lee d28b98c602
replay : move utility functions into separate file (#22414)
* move functions into util

* read bz2 into stream

* pre-decompress log in the download thread

* cleanup logreader

* cache sha256 path

* use readBZ2file in test_logger

* Revert "cache sha256 path"

This reverts commit 60459d3ea09a2c80f4560cf95b1ce7d6af59f06d.

* use macro

* use ostringstream

* cleanup readBZ2File

* move precise_nano_sleep into util
2021-10-04 15:39:59 +02:00
Dean Lee 935cbd3139
LoggerState: test cases (#21855)
* test multiple threads writing to log

* fix typo

* use util::getenv

* check INIT_DATA and SENTINEL

* test END_OF_SEGMENT

* add comment

* test multiple threads logging and rotation

* cleanup

* update

* LoggerHandle:quick fix

* cleanup

* revert test_loggerd.py

* refactor test case

* check lock file

* check refcnt after close

* test_runner.cc

d

* int eixt_signal
2021-08-30 17:12:09 -07:00
Adeeb Shihadeh 36d8c89ebf
enable --as-needed linking (#20619)
* only link as needed

* a few more

* need to fix order

* everywhre

* fix mac

* update spinner + text

* fix c2 build

* another fix

Co-authored-by: Comma Device <device@comma.ai>
2021-04-08 11:17:07 -07:00
Adeeb Shihadeh 2e64049ea4
fix loggerd after bootlog split (#19840)
* fix loggerd after bootlog split

* logger handles that

* unused

Co-authored-by: Comma Device <device@comma.ai>
2021-01-19 14:27:19 -08:00
Dean Lee f973c56a36
loggerd: split bootlog to a separate program (#19831)
* bootlog

* blank lines

* move file_exists to util.h

* src = ['loggerd.cc']

* log_init_data

* Revert "src = ['loggerd.cc']"

This reverts commit 18a587023d.

* add bootlog to .gitignore

* use s->has_qlog instead of true

* add blank lines & remove extern C
2021-01-19 12:16:12 -08:00
Adeeb Shihadeh 92e4f04873
Loggerd for PC (#19730) 2021-01-12 16:28:22 -08:00
Adeeb Shihadeh a1c7f19fe4
Video Encoder abstraction (#19729) 2021-01-12 00:03:31 -08:00
Adeeb Shihadeh 6b0890f128
loggerd: fix use after free and memory leaks (#19695)
* fix use after free and memory leaks

* cleanup

* fix typo

Co-authored-by: Comma Device <device@comma.ai>
2021-01-08 15:39:14 -08:00
Willem Melching fb496c692a
VisionIPC 2.0 (#19641)
* Squashed vipc

* Update release files

* Remove else

* add visionipc to release files

* use poller in vipc receive

* opencl framework instead of lib on macos

* Fix camera webcam

* Fix opencl on mac in ui

* more webcam fixes

* typo in ui sconsfile

* Use cur_yuv_buf

* visionbuf c++ class

* Camera qcom was still using visionbuf_allocate

* Turn loggerd back on

* fix snapshot

* No build needed

* update test camerad

* no more release callback

* make encoder c++

* Revert "no more release callback"

This reverts commit e5707b07002fee665d0483d90713154efc2d70d4.

* fix exit handlers

* No need to check errno

* move release callback call

* s/VIPCBufExtra/VisionIpcBufExtra/g

* use non blocking connect

* ui use non blocking connect

* Lower condition variable wait time

* Snapshot cleanup

* bump cereal

* bump cereal
2021-01-08 14:54:25 +01:00
Adeeb Shihadeh 206d072bb4
loggerd cleanup (#19668) 2021-01-07 23:08:40 -08:00
Adeeb Shihadeh 56146d84ce
remove raw logger (#2719)
* remove raw logger

* no raw_clips

* only remove raw clips

* cleanup
2020-12-08 15:23:06 -08:00
Willem Melching a4ccfcffe8
Remove czmq part 1 (#2332)
* remove czmq

* remove streaming mode from loggerd

* Add include for usleep
2020-10-13 16:01:07 +02:00
Willem Melching ab5af232b2
Use C++ version of SubMaster and PubMaster (#1548)
* add PubMaster & SubMaster

remove 'delete msg'

remove headers

* use constructor to initial all submster

* modify drain sockets

* fix typo in ssconscript.remove lines

no checkValid in loggerd

last modify

handle_message:event->&event

fix type

remove heads

event to auto

* new interface

* api changed

* Revert "use constructor to initial all submster"

This reverts commit 73be7ea46250a325ce41d3a0445e34395a2ae692.

* change to new api

* revert loggerd

* dd

* use new PubSub api

* update to new interface

don't modify loggerd

reset panda

reset opendbc

remove empty lines

* switch to new pubMaster

* update to the new inteface

change

remove error code

. to ->

merge paramsd.cc

update panda

fix typo

simplify

fix typo

* Fix build

* always conflate

Co-authored-by: deanlee <deanlee3@gmail.com>
2020-05-21 16:04:33 -07:00
Adeeb Shihadeh 9670f8fb9f these don't need json 2020-05-04 23:37:07 -07:00
George Hotz cfcf3f9368
tici: fix loggerd (#1302)
* loggerd builds

* codec

* add frame_id cams

* working

* no qcam on tici

* pthread always

Co-authored-by: Tici <robbe@comma.ai>
2020-03-31 18:25:05 -07:00
George Hotz 2f2b85581c
Add sentinels to logs and qlogs (#1205)
* sentinel WIP

* logger is now a cc, not a c

* simpler code doesn't touch loggerd.cc

* oops, double lock. and logreader is nicer
2020-03-05 14:29:34 -08:00
Adeeb 0ad5715255
use services.h in loggerd (#1056)
* loggerd: use services.h

* don't need yaml-cpp anymore

* forgot that
2020-02-06 10:56:01 -08:00
George Hotz da079d47d7 logcatd, loggerd, mapd, modeld, proclogd 2020-01-17 11:20:17 -08:00