Commit Graph

192 Commits (staging)

Author SHA1 Message Date
Connor Rigby 46d259202c
Add sequence start/complete logs to regimen and farm_events 2019-09-13 16:03:34 -07:00
Connor Rigby d22a390505
Update `if` logs to have more contextual information 2019-09-13 16:03:33 -07:00
Connor Rigby cf2877c76e
Ensure firmware/rpc logs always dispatch 2019-09-13 16:03:33 -07:00
Connor Rigby f400b2746f
Add new RPC for `toggle_pin`
This was previously implemented as a "macro" that just expanded
`toggle_pin` to a series of `read_pin` -> `write_pin` -> `read_pin` this
lead to three logs instead of just one. The implementation is still
mostly the same, but now there is less logging.

Also factored all pin control into it's own module
2019-09-13 16:03:32 -07:00
Connor Rigby 5373849f51
Update dependencies
* `elixir_make`
* `muontrap`
* `vintage_net`
* `busybox`
* `mdns_lite`
* `nerves_hub`
* `plug`
2019-09-13 16:03:32 -07:00
Connor Rigby 70c9035769
Fix non-string errors causing crashes 2019-09-13 16:03:32 -07:00
Connor Rigby 209ded96cb
Add Lua helpers for updating and getting resources 2019-09-13 16:03:32 -07:00
Connor Rigby a242cce830
Add new log type for assertion logs 2019-09-13 16:03:32 -07:00
Connor Rigby f756302fe6
Add option to continue and not recover from assertion 2019-09-13 16:03:32 -07:00
Connor Rigby 528acd119c
Fix execute block inside lua expression block 2019-09-13 16:03:32 -07:00
Gabriel Burnworth 2619586102
Change `update_farmware` fields to match corpus 2019-09-13 16:03:32 -07:00
Connor Rigby d3081fcc56
Change field names of `assert` to match corpus 2019-09-13 16:03:31 -07:00
Connor Rigby 7e1ceaf7be
Implement new AST: `assert`.
This is a new AST that will allow executing a
simple expression and conditionally pass/fail
and cleanup when it completes.
2019-09-13 16:03:31 -07:00
Connor Rigby a04ddf0159
Fix set_pin_io mode 2019-09-13 16:03:31 -07:00
Connor Rigby fcadeebe6e
Add `update_farmware` rpc back 2019-09-13 16:03:31 -07:00
connor rigby 3231651b5f
Add :set_pin_io_mode rpc back to known rpcs 2019-09-13 16:03:30 -07:00
Connor Rigby a6efc9df38
Remove scheduler debug logs 2019-09-13 16:03:28 -07:00
Connor Rigby 3569a21c72
Update to elixir 1.9 2019-09-13 16:03:27 -07:00
connor rigby d4157175d4
Increase grace period in scheduler to 15 minutes 2019-09-13 16:03:27 -07:00
gabrielburnworth 62a101dc9c
Fix some farmware bugs 2019-09-13 16:03:27 -07:00
Connor Rigby ed84cfeb2f
Add feature to subscribe to update to scheduled calendar 2019-09-13 16:03:27 -07:00
Connor Rigby 6c59d54244
Update logs for starting/completing a sequence 2019-09-13 16:03:26 -07:00
Connor Rigby 3d07c263ea
Fix scheduler not reindexing when an event is missed
* Add some breadcrumbs to see what the data that generated the scheduled
event looks like
* only monitor a process if it isn't monitored yet
2019-09-13 16:03:26 -07:00
connor rigby 55533675f5
Reenable sequence init and sequence complete logs
Sort of a hack, but store some metadata about a sequence in the AST
format, and if it exists, log it
2019-09-13 16:03:26 -07:00
Connor Rigby 6bb75de021
Allow flashing firmware at runtime
* Moves tty detector to farmbot_core
   * Update elixir config to reflect this
* Update `flash_firmware` rpc to not circularly flash firmware
2019-09-13 16:03:25 -07:00
Connor Rigby a160d6a41b
Fix factory_resetting the arduino-firmware
* Adds `api_migrated` field to FirmwareConfig
* Changes `factory_reset()` syscall to be `factory_reset(package)`
2019-09-13 16:03:25 -07:00
connor rigby 2a884f3b41
Fix typo in `If` 2019-09-13 16:03:25 -07:00
connor rigby e7fed8838d
Update home all log 2019-09-13 16:03:25 -07:00
connor rigby 5ca92090a1
Update more logs
* Any float is not formatted very specifically
   * if the remainder in the decimal is `0`, don't display decimal
   * if the remainder is not zero, truncate to only one decimal place

* Write/Read pin
   * yeah that got messy
2019-09-13 16:03:25 -07:00
connor rigby ce3480a0f5
Add log for relative movement 2019-09-13 16:03:25 -07:00
connor rigby d2239d76c3
Try add log channels more aggressively 2019-09-13 16:03:24 -07:00
connor rigby 059af23faf
Add parens to movement logs 2019-09-13 16:03:24 -07:00
connor rigby 28b43bebd3
Update more logs for clarity
* Add log for things that enumerate all axis
* change `homing` to `going home`..
2019-09-13 16:03:24 -07:00
connor rigby 0cbb4ba784
Fix named pins causing celeryscript errors
NamedPin values were encoded as tuples. This changes it to
pattern matchable structs, and implements a `to_string` behaviour
for those structs
2019-09-13 16:03:24 -07:00
Connor Rigby c92c79347f
Add back step logs 2019-09-13 16:03:24 -07:00
connor rigby 672c1cc5e6
Fix CeleryScript Compiler typo in find_home/1 2019-09-13 16:03:23 -07:00
Connor Rigby 60b075cf88
Normalize asset commans
Many clauses were not getting hit depending on if
`update/3` was called with a Module or a "String".
This normalizes everything early.
2019-09-13 16:03:23 -07:00
Connor Rigby e6b29ea559
Refactor CeleryScript execution
* FarmEvents now schedule all events up front
  * Executions are stored in the database
* Regimens now schedule all items up front
  * Executions are stored in the database
* fix Deleting a FarmEvent with a running RegimenInstance
* `execute`int and `schedule`ing a CeleryScript are not separate things
* Everywhere `scheduling` and `executing` CeleryScript updated to use
the new API
2019-09-13 16:03:23 -07:00
Connor Rigby 00d0897672
Dry up celery_script compilation
This adds much more type checking and error
handling in the compiled celery_script.
Also removes the hack/shortcut in the StubSysCalls
module.
2019-09-13 16:03:23 -07:00
Connor Rigby 697850b4b3
Implement process separation for celery scheduler
This should hopefuly prevent hanging forever and other weird
issues relating to executing celeryscript
2019-09-13 16:03:23 -07:00
Connor Rigby 5496efd390
Revert "Update scheduler to use ETS instead of storing data on a local state"
This reverts commit 4b2fa3f312.
2019-09-13 16:03:23 -07:00
Connor Rigby c0ddf0a32d
Revert "Update command_runner to not use entire scheduler"
This reverts commit e5d1e00c5f.
2019-09-13 16:03:22 -07:00
Connor Rigby 12640ae805
Revert "make sure to setup ets table for farmbot_celeryscript before starting it"
This reverts commit 2f689e4a1f.
2019-09-13 16:03:22 -07:00
Connor Rigby 14ff862176
Allow named_pin to return metadata
This will allow doing special thing contextually depending
on the type of pin being read.
2019-09-13 16:03:22 -07:00
Connor Rigby 2d66f09485
Enable `warnings_as_errors` for each app 2019-09-13 16:03:21 -07:00
Connor Rigby be1c6dc869
Add basic support for the express firmware target (#718)
* Initital setup for FarmbotExpress

* Update farmbot-arduino-firmware commit ref
2019-09-13 16:03:21 -07:00
connor rigby 3afed72bf5
Update scheduler_test to be less strict.
When the scheduler was split up into two different processes
for executing cellery_script, checking times doesn't really make
sense. Asserting that they do infact execute in a certain
amount of time is fine
2019-09-13 16:03:20 -07:00
Connor Rigby d7b43ffebe
Fix scheduler tests 2019-09-13 16:03:20 -07:00
Connor Rigby c653502f9a
Update command_runner to not use entire scheduler 2019-09-13 16:03:20 -07:00
Connor Rigby b9b222fe3c
make sure to setup ets table for farmbot_celeryscript before starting it 2019-09-13 16:03:20 -07:00
Connor Rigby 80b6e53569
Update scheduler to use ETS instead of storing data on a local state 2019-09-13 16:03:20 -07:00
Connor Rigby 3d036aded6
Fix zero expecting a speed paramater 2019-09-13 16:03:20 -07:00
connor rigby 38a1dac5ae
Change order of axis movements 2019-09-13 16:03:20 -07:00
Connor Rigby 80b57917f9
Fix _if using named_pin 2019-09-13 16:03:19 -07:00
Connor Rigby 700d875e3a
Fix move_relative and move_abs coordinate math 2019-09-13 16:03:19 -07:00
Connor Rigby b61f1b11be
Fix some misc firmware bugs 2019-09-13 16:03:19 -07:00
Connor Rigby 1b31c32b3d
Implement change_ownership sys_call
This required some changes to NervesHub because of there
being a brief moment after changing ownership where there
are no certs.
2019-09-13 16:03:19 -07:00
Connor Rigby 4f74887990
Implement get_toolslot syscall and fix parameter typos 2019-09-13 16:03:19 -07:00
Connor Rigby 68e15bcce1
Firmware command updates
Fixes `toggle_pin`
Fixes some unknown parameters
2019-09-13 16:03:19 -07:00
Connor Rigby db799f5d64
Update nerves-hub and add check_update sys_call 2019-09-13 16:03:19 -07:00
Connor Rigby f0c8fed4d8
Implement estop and eunlock syscalls 2019-09-13 16:03:19 -07:00
Connor Rigby 2a1db21546
Implement remaining celery nodes 2019-09-13 16:03:19 -07:00
Connor Rigby db54e2117d
Update find_home to not take a speed arg 2019-09-13 16:03:18 -07:00
Connor Rigby 1c7e789eb4
Remove debug log of compiled cellery_script 2019-09-13 16:03:18 -07:00
Rick Carlino b4c05d13e0
Test case: passing default value in a sequence 2019-09-13 16:03:17 -07:00
connor rigby d842c3a123
🎉 Implement Variables in Regimen 2019-09-13 16:03:17 -07:00
Connor Rigby b7f4ebba57
Fixes for bootstrapping firmware.
* TTYDetector no longer causes side effects
* add new RPC for `flash_firmware` that takes a `package` arg
2019-09-13 16:03:17 -07:00
Rick Carlino 3bc84a3d3e
Add `flash_firmware` to Celery compiler. 2019-09-13 16:03:16 -07:00
Rick Carlino cfa86293a4
Refactor fbos_config_worker_test to be less prone to races 2019-09-13 16:03:16 -07:00
connor rigby b204ce3b9e
Add some work for Farmware 2019-09-13 16:03:16 -07:00
Connor Rigby ef3225878d
Change FarmbotCeleryScript Namespace 2019-09-13 16:03:16 -07:00
Connor Rigby 21bb1a6f25
Misc cleanup and bug fixes.
* Fixes botstate changes being dispatched as nested lists
* Fixes jwt failing to decode `nil`. Doesn't fix token actually being`nil` though.
* Adds a syscall for read_pin
* Add/fix calibrate syscall
* tweak tty detector. This is still broken i think
2019-09-13 16:03:16 -07:00
Connor Rigby 4114e26804
Implement new CeleryScript Runtime environment.
This is obviously a rather large change warranting an essay describing
it.

A Brief overview
Basically the old implementation had quite a few down sides preventing
it from really working as intended, especially with the addition of the
variables feature. Here is the shortlist of things that needed
addressing:

* No scoping between sequences.
What this essentially means is that a sequence that executes another
sequence is unable to add data to the calle. This is important for using
Variables.

* Error recovery
certain nodes have a high likelyhood of failing such as anything that
interfaces the firmware. Much focus was spent ensuring that errors would
be recoverable when desired.

* Complexity of control flow asts versus action asts.
Nodes such as `if` will always work in the same way regardless of the
state of the rest of the system meaning there is no reason for it to
have a special implementation per environment. on the other hand
`move_absolute` is bound to a specific part of the system. Seperating
these concerns allows for better testing of each piece independently.

A More In Depth overview
The core of this change resolves around 1 really big change resulting in
many more small changes. This change is the CeleryScript `compiler`. The
TLDR of this system is that now CeleryScript ASTs are deterministicly
compiled to Elixir's AST and executed. Doing this has some big benifits
as described below.

1) CeleryScript "runtime" environment is now much simpiler in favor of a somewhat
complex "compile time" environment. Basically instead of EVERY single
CeleryScript AST having a custom runtime implementation, only a subset
of ASTs that require external services such as the Firmware, Database,
HTTP, etc require having a runtime implementation. This subset of ASTs
are called `SysCalls`. Also the runtime implementations are compiled to
a single function call that can be implemented instead of needing to
have a contextual environment and making decisions at runtime to
evaluate variables and the like.

2) Static analysis is now possible. This means an incorrectly crafted
sequence can be validated at compile time rather than getting half way
through a sequence before finding the error.

3) Having the "external services" separated leads to better plugability.
There is now a behaviour to be implemented for the subset of syscalls
that are system specific.
2019-09-13 16:03:15 -07:00
Connor Rigby 519791e99e
Implement helpful macros for building node implementation 2019-09-13 16:03:15 -07:00
connor rigby a28b6c4141
Variable resolution works? 2019-09-13 16:03:15 -07:00
connor rigby 2b83e76233
Fix type on if block 2019-09-13 16:03:15 -07:00
connor rigby fbb9d294a8
Cleanup+document new ElixirAST compiler 2019-09-13 16:03:15 -07:00
connor rigby cd90a1f18a
Build out compiler for Celery AST to Elixir AST 2019-09-13 16:03:15 -07:00
connor rigby 2d3f81b1e6
Add fixture for a pair of sequences 2019-09-13 16:03:15 -07:00
connor rigby d0edb1e19e
Update to Elixir 1.8!! 2019-09-13 16:03:15 -07:00
Rick Carlino 814b6f1f73
Minor doc updates 2019-09-13 16:03:15 -07:00
connor rigby 236bb1d270
Farmware stuff 2019-09-13 16:03:15 -07:00
Connor Rigby bb7306dabc
Work towards refactoring Farmware
* Add bot state serializer to filesystem
* Add helper to turn ecto types into TSC types
* Add module to extract ecto changes as a list of instructions
* Add new transport for bot_state_ng
* Update CSVM Runtime to poll until `await` can complete
2019-09-13 16:03:15 -07:00
connor rigby 001faaaf2f
Fix CSVM getting stuck when calling process dies 2019-09-13 16:03:15 -07:00
Connor Rigby f7b573e6d5
Update celery_script
Support peripherals in read and write pin
2019-09-13 16:03:15 -07:00
Connor Rigby f2b8abd692
Botstate refactor (#645)
* Refactor PinBindings

* Refactor logging to not require Farmbot.Registry

* Fix AMQP bot_state_transport and log_transport

* Update to use floats everywhere

* Refactor Farmbot.Firmware

* Write Firmware Tests

* dipping into FarmbotOS finally

* Cleanup peripheral_worker error message

* Implement remaining CeleryScript RPCs

* Refactor job progress

* Image Upload status notifications updates

* Fix compiler warnings and things

* Update Firmware submodule

* Fix FarmbotEXT tests
2019-09-13 16:03:14 -07:00
Connor Rigby b3f1369a77
Re-implement image uploading 2019-09-13 16:03:14 -07:00
Connor Rigby 869d1ad1a4
Progress on refactoring data syncing (#643) 2019-09-13 16:03:13 -07:00
connor rigby 10a4379568
Work toward farmware/celeryscript stuff
* Fix logger erros
* Fix migrations
* Fix compile errors
* Fix tzdata child_spec
2019-09-13 16:03:13 -07:00
connor rigby d7ff13422c
Test Migration 2019-09-13 16:03:13 -07:00
connor rigby 32f46b5fd3
Start refactoring Farmware 2019-09-13 16:03:13 -07:00
connor rigby 358a1e209e
Update AMQP workers to reconnect on a 4 second timer
Pull in new csvm implementation

Update circleci config

Implement syncing and write_pin

add migration for syncing

add saftey to write_pin

Implement read_pin

Implement set_servo_angle

Implement more ast nodes

Implement e-stop and e-unlock

Fix a bunch of stuf

Fix missing assets on boot/init

It actually works

Rename csvm -> farmbot_celery_script; fix initial sync/dispatch

Fix a bunch of small bugs

Identify problem

Fix Sqlite bug and increase performance by 10 times!!

Fix sequences inside of sequences
2019-09-13 16:03:13 -07:00