diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3e315c7d..b332ca09 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,2 +1,4 @@ # Contributing To get started, sign the Contributor License Agreement. + +After that, read through the [documentation](docs/index.md) \ No newline at end of file diff --git a/README.md b/README.md index c947514a..7bc42c1f 100644 --- a/README.md +++ b/README.md @@ -49,8 +49,8 @@ _Refer to the [software documentation Configurator page](https://software.farm.b ## Problems? -See the [FAQ](docs/FAQ.md) -If your problem isn't solved there please file an issue on [Github](https://github.com/FarmBot-Labs/farmbot_os/issues/new) +See the [FAQ](docs/target_development/target_faq.md) +If your problem isn't solved there please file an issue on [Github](https://github.com/FarmBot/farmbot_os/issues/new) ## Security Concerns? @@ -58,5 +58,5 @@ We take security seriously and value the input of independent researchers. Pleas ## Want to Help? -[Low Hanging Fruit](https://github.com/FarmBot-Labs/farmbot_os/search?utf8=%E2%9C%93&q=TODO) +[Low Hanging Fruit](https://github.com/FarmBot/farmbot_os/search?utf8=%E2%9C%93&q=TODO) [Development](CONTRIBUTING.md) diff --git a/docs/celery_script/assert expressions.md b/docs/celery_script/assert_expressions.md similarity index 100% rename from docs/celery_script/assert expressions.md rename to docs/celery_script/assert_expressions.md diff --git a/docs/celery_script/celery_script.md b/docs/celery_script/celery_script.md index 541a2d8a..a879547c 100644 --- a/docs/celery_script/celery_script.md +++ b/docs/celery_script/celery_script.md @@ -1,4 +1,5 @@ # CeleryScript + CeleryScript is an AST definition of commands, rpcs, and functions that can all be executed by Farmbot. The basic syntax is as follows: @@ -16,6 +17,7 @@ optional, `body` should be supplied when working with any and all modules in this project. ## kind + `kind` is the identifier for a command. Examples include: * `move_absolute` * `sync` @@ -26,6 +28,7 @@ Each `kind` will have it's own set of rules for execution. These rules will define what is required inside of both `args` and `body`. ## args + `args` is arguments to be passed to `kind`. Each `kind` defines it's own set of optional and required `args`. Args can any of the following types: * `number` @@ -42,6 +45,7 @@ executing the parent AST. Examples of `args` include: * `milliseconds` ## body + `body` is the only way a `list` or `array` type is aloud in CeleryScript. It may only contain _more CeleryScript nodes_. This is useful for enumeration, scripting looping etc. Here's a syntacticly correct example: diff --git a/docs/glossary.md b/docs/glossary.md new file mode 100644 index 00000000..ecad47b1 --- /dev/null +++ b/docs/glossary.md @@ -0,0 +1,48 @@ +# FarmBot OS Source Glossary + +This file contains a basic glossary of commonly used terms + +## FarmBot Specific Terms + +* Asset - REST resource stored in Farmbot's database stored on the SD card +* Arduino Firmware - The code that runs on the Arduino. + * [Source](https://github.com/farmbot/farmbot-arduino-firmware) +* CelleryScript - FarmBot OS's scripting language +* FarmbBot API/Web App - The REST server FarmBot communicates with + +## General Terms + +* Elixir - Programming language FarmBot is developed in + * [More info](https://elixir-lang.org/) + * [Docs](https://hexdocs.pm/elixir/Kernel.html) +* Erlang - Programming language and VM that Elixir compiles down too + * [More info](https://elixir-lang.org/) + * [Even more info](#OTP-Terms) + * [Docs](https://www.erlang.org/docs) +* UART - **U**niversal **A**synchronous **R**eceiver/**T**ransmitter. hardware based transport mechanism +* SSH - **S**ecure **S**hell. +* MQTT/AMQP - network protocols for pub/sub data transport +* HTTP - network protocol for accessing REST resource + +## Nerves Specific Terms + +* Nerves - Framework that allows cross compilation of Elixir code + * [More info](https://nerves-project.org/) + * [Docs](https://hexdocs.pm/nerves/getting-started.html) +* NervesHub - Cloud based firmware management + * [More info](https://www.nerves-hub.org/) + * [Docs](https://github.com/nerves-hub/documentation) +* Firmware - Usually refers to the code that gets deployed onto the Raspberry Pi + +## OTP Terms + +* Beam - Virtual machine that runs compiled Erlang bytecode +* OTP - Open Telecom Platform. Erlang's runtime libraries + * [More info](https://erlang.org/doc/design_principles/des_princ.html) +* Supervisor - OTP `Process` responsible for supervising `Workers` +* Worker - OTP `Process` responsible for doing `work`. Usually `Supervised` +* Process - OTP concept responsible for sending/receiving messages. **everything** is a process in erlang +* Application - OTP concept responsible for containing many `Supervisor`s and `Worker`s +* Distribution - OTP concept of networking multiple Beam instances together +* ETS - **E**rlang **T**erm **S**torage. OTP application for storing data in memory +* DETS - **D**isk **E**rlang **T**erm **S**torage. OTP application for storing data on disk \ No newline at end of file diff --git a/docs/DEVELOPMENT.md b/docs/host_development/host_development.md similarity index 100% rename from docs/DEVELOPMENT.md rename to docs/host_development/host_development.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..50d1c760 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,38 @@ +# FarmBot OS Documentation + +This document will act as an index to available documentation. + +## Glossary + +* [FarmBot Source Code common terms](/docs/glossary.md) + +## Hardware specifics + +Most FarmBot development/testing is done on a standard desktop PC. + +* [Developing on your local PC](/docs/host_development/host_development.md) +* [Deploying on Raspberry Pi](/docs/target_development/building_target_firmware.md) + * [Publishing Firmware (OTAs)](/docs/target_development/releasing_target_firmware.md) + * [Why doesn't my device boot after building firmware](docs/target_development/target_faq.md) + * [Inspecting a running devicve](/docs/target_development/consoles/target_console.md) + +## CeleryScript + +CeleryScript is FarmBot's native scripting language. See the below +documentation for information about it as it relates to FarmBot OS. + +* [CeleryScript intro](/docs/celery_script/celery_script.md) +* [A list of all supported commands](/docs/celery_script/all_nodes.md) +* [Lua (embedded scripting inside CeleryScript)](/docs/celery_script/assert_expressions.md) + +## Project structure + +The FarmBot OS application is broken into several sub applications. + +* [Project structure overview](/docs/project/structure.md) + * [farmbot_celery_script](/docs/project/farmbot_celery_script.md) + * [farmbot_core](/docs/project/farmbot_core.md) + * [farmbot_ext](/docs/project/farmbot_ext.md) + * [farmbot_firmware](/docs/project/farmbot_firmware.md) + * [farmbot_os](/docs/project/farmbot_os.md) + * [farmbot_telemetry](/docs/project/farmbot_telemetry.md) diff --git a/docs/project/farmbot_celery_script.md b/docs/project/farmbot_celery_script.md new file mode 100644 index 00000000..217b8d5a --- /dev/null +++ b/docs/project/farmbot_celery_script.md @@ -0,0 +1,12 @@ +# FarmBot CeleryScript OTP App + +`farmbot_celery_script` is responsible for implementing the +runtime that execute's [CeleryScript](/docs/celery_script/celery_script.md). +It contains a handful of helpers, and several subsystems for working with CeleryScript. +The most important being: +* AST - definition of the AST as it relates to FarmBot OS +* Compiler - Compiles CeleryScript to Elixir AST. + * See the [Elixir Macro Docs](https://hexdocs.pm/elixir/Macro.html) +* StepRunner - Process responsible for actually executing CeleryScript +* Scheduler - Process responsible for scheduling calls to the `StepRunner` +* SysCalls - module responsible for dispatching calls to the configured implementation diff --git a/docs/project/farmbot_core.md b/docs/project/farmbot_core.md new file mode 100644 index 00000000..23787e3e --- /dev/null +++ b/docs/project/farmbot_core.md @@ -0,0 +1,58 @@ +# FarmBot Core OTP App + +`farmbot_core` is responsible for the core funcionality of the FarmBot application. +This contains things such as resource (asset) management, plugin (farmware) management, +central state, and schedule management. FarmBot specific network requests are not made +from the `farmbot_core` app. Below describes the important subsystems + +## Asset storage subsystem + +Sqlite database responsible for storing data needed for FarmBot to operate. +Most device specific REST resources are mirrored here. +* Device +* FarmEvent +* Regimen +* Sequence +* Peripheral + +## Asset Worker subsystem + +All assets that need to have a process assosiated with it will be found +in this subsystem. Examples of this include: +* FarmEvent scheduling +* Regimen scheduling +* PinBinding monitoring +* FbosConfig/FirmwareConfig + +## Farmware subsystem + +Farmbot's external plugin system. See the Farmware documentation for more details. + +## BotState subsystem + +Central in-memory state process/tracker. This process keeps a cache of +all the moving parts of FarmBot. Some examples of what is stored +in this cache: + +* Firmware reporting + * current axis position + * encoder data + * arduino pin data + * currently configured firmware paramaters +* Current configuration + * mirror of `fbos_config` asset +* System info + * version info + * (nerves) firmware info + * memory usage + * disk usage +* Network info + * WiFi signal quality + * private ip address + +## Logging subsystem + +This is where the `Messages` panel gets it's data from. Calls to this subsystem +push data into an sqlite database. This data is collected on a timer and dispatched +over AMQP/MQTT when/if that subsystem is available. See [farmbot_ext](/docs/project/farmbot_ext.md) +for information on how that works. diff --git a/docs/project/farmbot_ext.md b/docs/project/farmbot_ext.md new file mode 100644 index 00000000..d6e32632 --- /dev/null +++ b/docs/project/farmbot_ext.md @@ -0,0 +1,38 @@ +# FarmBot Ext OTP App + +the `farmbot_ext` OTP app contains extended FarmbotCore functionality. This includes +mostly network functionality that isn't possible to do in `farmbot_core`. + +## Bootstrap subsystem + +Subsystem responsible for bootstrapping a connection to the FarmBot network services. +This includes authenticating with the FarmBot API, connecting to AMQP/MQTT and syncing +the bare minimum resources to get up and running. + +## HTTP/Sync subsystem + +This is the subsystem that syncronizes FarmBot with the remote API. It uses HTTP to +download an index of all the data FarmBot cares about, and compares timestamps to +determine who has the most up to date data. The basic flow is whoever has the most +recent `updated_at` field will become the "most truthy". If FarmBot has a more recent +`updated_at` field, FarmBot will do an HTTP PUT of it's data. If the remote resource +does not exist, FarmBot will do an HTTP POST of it's data. If the remote data has a more +recent `updated_at` field, FarmBot will do an HTTP GET and replace it's own data. + +## AMQP/MQTT substem + +FarmBot maintains a connection to the API for real time communication. This real time communication +connection is multiplexed over multiple `channel`s. Below is a description of the channels + +* bot_state - pushes a JSON encoded version of the `bot_state` process (from `farmbot_core`) +* celery_script - receives/sends JSON encoded celery_script. Used for controling FarmBot externally +* log - sends log messages from `farmbot_core`'s logger +* ping/pong - echos everything received. used for detecting active conncetion +* auto_sync - the API dispatches every REST resource change on this channel. Used to speed up HTTP requests +* telemetry - similar to the log channel, but sends consumable events, rather than human readable messages + +## Image uploader subsystem + +This subsystem watches a local directory, and as matching files appear in that directory, +it uploads them using the FarmBot image upload protocol. Basically an HTTP request to fetch +credentials that are used to preform another HTTP request to upload the photo. \ No newline at end of file diff --git a/docs/project/farmbot_firmware.md b/docs/project/farmbot_firmware.md new file mode 100644 index 00000000..5c884a1d --- /dev/null +++ b/docs/project/farmbot_firmware.md @@ -0,0 +1,23 @@ +# FarmBot Firmware OTP App + +The `farmbot_firmware` OTP application is responsible for maintaining a connection to +the arduino-firmware. + +## GCODE encoder/decoder subsystem + +The official Farmbot-Arduino-Firmware communicates over UART using a ASCII based +protocol based on CNC GCODE. This subsystem is responsible for translating +the ASCII data into an intermediate representation that can be `transport`ed +proto agnostically. + +## Transport subsystem + +This subsystem is responsible for abstracting the details of transporting +FarmBot GCODEs to/from the firmware implementation. A `transport` will take +in the intermediate (farmbot specific) representation of a GCODE, and dispatch/handle +it in it's own specific manor. This keeps the usage of the overall application uniform +with or without a firmware plugged in. + +## UART subsystem + +Responsible for the official communication mechanism with the official arduino hardware. \ No newline at end of file diff --git a/docs/project/farmbot_os.md b/docs/project/farmbot_os.md new file mode 100644 index 00000000..c5933ef1 --- /dev/null +++ b/docs/project/farmbot_os.md @@ -0,0 +1,50 @@ +# FarmBot OTP App + +the `farmbot` OTP app is the container Nerves based application. It contains mostly +glue code between all the subsystems in the other applications along with it's own +platform specific subsystems. + +## CeleryScript System Calls + +The "official" implementation of all the CeleryScript syscalls. These calls are mostly +glue to other existing implementations from the other otp apps. + +## Lua Subsystem + +The implementation of the embedded scripting language inside CeleryScript. +Also contains glue code for glueing together the real implementation to the +Lua vm. + +## Configurator Subsystem + +HTTP server responsible for configuring the running FarmBot OS instance. Will +server a web page that allows a user to supply a username, password, network credentials +etc. + +# Platform specific subsystems + +the `farmbot_os` OTP app contains target/hardware specific systems. This code is +located in the `platform` directory. + +## Network subsystem + +Responsible for getting FarmBot connected to the (inter)net. If no network configuration is +available, FarmBot will create a captive portal access point to allow external devices to +configure it. + +## GPIO subsystem + +Responsible for implementing LED and Button support at the hardware level. + +## RTC subsystem + +Responsible for syncronizing network time to an attached hardware clock. + +## Info Worker subsystem + +Responsible for simple workers that handle things like + +* CPU temperature +* CPU usage +* memory usage +* disk usage diff --git a/docs/project/farmbot_telemetry.md b/docs/project/farmbot_telemetry.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/project/structure.md b/docs/project/structure.md new file mode 100644 index 00000000..407c28da --- /dev/null +++ b/docs/project/structure.md @@ -0,0 +1,33 @@ +## FarmBot Source Project structure + +The FarmBot OS application is broken into several sub OTP applications. + +* [farmbot_celery_script](/docs/project/farmbot_celery_script.md) +* [farmbot_core](/docs/project/farmbot_core.md) +* [farmbot_ext](/docs/project/farmbot_ext.md) +* [farmbot_firmware](/docs/project/farmbot_firmware.md) +* [farmbot_os](/docs/project/farmbot_os.md) +* [farmbot_telemetry](/docs/project/farmbot_telemetry.md) + +## Commonality + +All of these folders share a common structure. + + +├── lib/ +│ ├── application.ex +│ └── some_file.ex +| +├── test/ +| └── test_helper.exs +| +├── config/ +| └── config.exs +| +├─── mix.exs +└─── mix.lock + +* The `lib` folder contains Elixir source code +* the `test` folder contains Elixir scripts responsible for testing the `lib` code +* the `config` folder contains Elixir scripts responsible for configuring the **current** OTP app +* `mix.exs` and `mix.lock` files are responsible describing the OTP app, and managing external dependencies \ No newline at end of file diff --git a/docs/BUILDING.md b/docs/target_development/building_target_firmware.md similarity index 100% rename from docs/BUILDING.md rename to docs/target_development/building_target_firmware.md diff --git a/docs/target_development/consoles/erlang_distribution.md b/docs/target_development/consoles/erlang_distribution.md new file mode 100644 index 00000000..090d9fd7 --- /dev/null +++ b/docs/target_development/consoles/erlang_distribution.md @@ -0,0 +1,26 @@ +# Erlang Distribution Console + +Accessing an Erlang Distribution console + +## Setup + +No additional setup is required for setting up erlang distribution on the target. +On your `host` machine, you need to have Erlang Distribution running. This can +usually be accomplished by starting `epmd`: + +```bash +epmd --daemon +``` + +## Connecting + +From your `host` terminal, connecting to a running device can be done by using the +`remsh` feature of elixir's built in console. + +```bash +iex --name console --cookie democookie --remsh farmbot@farmbot-.local +``` + +## Disconnecting + +Issuing a `ctrl+c` to the `host` terminal should disconnect you from the session. \ No newline at end of file diff --git a/docs/SSH.md b/docs/target_development/consoles/ssh.md similarity index 54% rename from docs/SSH.md rename to docs/target_development/consoles/ssh.md index 33ada68f..759a9238 100644 --- a/docs/SSH.md +++ b/docs/target_development/consoles/ssh.md @@ -1,42 +1,25 @@ -# SSH +# SSH console + +Accessing an SSH console. ## Setup + FarmBot can be configured to start an SSH server to aid in debugging and development. During configuration of Network, select `Advanced Settings` and paste your [ssh public key](https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key) into the optional input section labeled: `id_rsa.pub`. FarmBot requires a public key and -will not allow a username + password combination. FarmBot also only allows one -key to be configured for security reasons. +will not allow a username + password combination. If you followed the documentation +described in [building target firmware](/docs/target_development/building_target_firmware.md) +then your SSH key will be automatically added to the device. ## Connecting + From the same machine that owns the `id_rsa.pub` key and assosiated private key you can simply `ssh `. If your machine supports `mdns`, you can also do `ssh farmbot-` where `node_name` can be found in the `Device` panel on the FarmBot web app. -## Usage -The console a user will be presented with is _not_ a Linux console. There are -pretty much no Linux Utilities built-in. This includes but is not limited to: -* `bash` -* `apt-get` -* `make` -* `screen` -* `vi` -* `cp` -* `mkdir` -* `ln` -* `echo` -* etc - -What is available is a console to the FarmBot OS runtime. You will need to be -familiar with the FarmBotOS Source code for this to be helpful. - -If all you are looking for is Logs, you will probably want to do: -```elixir -RingLogger.attach() -``` - -After that command you will see logs come across the screen in real time. +## Disconnecting To exit the SSH session, type `~.`. This is an ssh escape sequence (See the ssh man page for other escape sequences). diff --git a/docs/target_development/consoles/target_console.md b/docs/target_development/consoles/target_console.md new file mode 100644 index 00000000..04bfef3e --- /dev/null +++ b/docs/target_development/consoles/target_console.md @@ -0,0 +1,42 @@ +# FarmBot target console + +The FarmBot OS target console is a repl for interacting with a running +farmbot instance. Depending on your development configuration, there are several +ways to access a console. + +If you are using `host` mode, your console will be presented on stdin of your +terminal. + +If you are on `target` mode (IE: deployed to the raspberry pi), there will +be a console available in 3 locations: +* UART (except on RPI0 since the on-board UART is used for the arduino-firmware) + * [Connect to a UART console](/docs/target_development/consoles/uart.md) +* SSH + * [Connect to an SSH console](/docs/target_development/consoles/ssh.md) +* Erlang Distribution + * [Connect to an Erlang Distribution console](/docs/target_development/consoles/erlang_distribution.md) + +## Console Usage + +The console a user will be presented with is _not_ a Linux console. There are +pretty much no Linux Utilities built-in. This includes but is not limited to: +* `bash` +* `apt-get` +* `make` +* `screen` +* `vi` +* `cp` +* `mkdir` +* `ln` +* `echo` +* etc + +What is available is a console to the FarmBot OS runtime. You will need to be +familiar with the FarmBotOS Source code for this to be helpful. + +If all you are looking for is Logs, you will probably want to do: +```elixir +RingLogger.tail() +``` + +After that command you will see logs come across the screen in real time. diff --git a/docs/target_development/consoles/uart.md b/docs/target_development/consoles/uart.md new file mode 100644 index 00000000..d0fb47b0 --- /dev/null +++ b/docs/target_development/consoles/uart.md @@ -0,0 +1,31 @@ +# UART Console + +Accessing an target UART console + +## Setup + +No additional setup is required for setting up the UART console on the target. + +On your `host` machine, you need to have a console cable, as well as a console client. +The console cable must be 3.3 volts, **not** 5 volts. a 5 volt cable will harm your +Raspberry Pi. Here are some known working cables: +* https://www.amazon.com/Converter-Terminated-Galileo-BeagleBone-Minnowboard/dp/B06ZYPLFNB +* https://www.amazon.com/DSD-TECH-Adapter-FT232RL-Compatible/dp/B07BBPX8B8 +* https://www.amazon.com/JANSANE-PL2303TA-Serial-Console-Raspberry/dp/B07D9R5JFK + +The most common client is probably `screen` on *Nix based systems or `putty` on windows. +See your distribution's package manager for installation and usage instructions. + +## Connecting + +Connecting to a console is dependent on your particular client. Here is an example +`screen` command: + +```bash +screen /dev/ttyUSB0 115200 +``` + +## Disconnecting + +Disconnecting is also dependent on your particular client. To exit screen, +issue a `ctrl+\+y` sequence to escape the console. \ No newline at end of file diff --git a/docs/RELEASE.md b/docs/target_development/releasing_target_firmware.md similarity index 100% rename from docs/RELEASE.md rename to docs/target_development/releasing_target_firmware.md diff --git a/docs/FAQ.md b/docs/target_development/target_faq.md similarity index 100% rename from docs/FAQ.md rename to docs/target_development/target_faq.md