parent
68a37ea874
commit
6a444136f7
|
@ -1,5 +1,6 @@
|
||||||
*.swp
|
*.swp
|
||||||
data/
|
data/
|
||||||
dump*.json
|
|
||||||
download/
|
download/
|
||||||
|
dump*.json
|
||||||
|
preprocess/
|
||||||
tmp/
|
tmp/
|
||||||
|
|
20
README.md
20
README.md
|
@ -28,15 +28,21 @@ Learning/testing, results are inaccurate.
|
||||||
The following scripts are in the repo:
|
The following scripts are in the repo:
|
||||||
|
|
||||||
* `wut` --- Feed it an observation ID and it returns if it is a "good", "bad", or "failed" observation.
|
* `wut` --- Feed it an observation ID and it returns if it is a "good", "bad", or "failed" observation.
|
||||||
|
* `wut-audio-archive` --- Downloads audio files from archive.org.
|
||||||
* `wut-compare` --- Compare an observations' current presumably human vetting with a `wut` vetting.
|
* `wut-compare` --- Compare an observations' current presumably human vetting with a `wut` vetting.
|
||||||
* `wut-compare-all` --- Compare all the observations in `download/` with `wut` vettings.
|
* `wut-compare-all` --- Compare all the observations in `download/` with `wut` vettings.
|
||||||
* `wut-compare-tx` --- Compare all the observations in `download/` with `wut` vettings using selected transmitter UUID.
|
* `wut-compare-tx` --- Compare all the observations in `download/` with `wut` vettings using selected transmitter UUID.
|
||||||
* `wut-compare-txmode` --- Compare all the observations in `download/` with `wut` vettings using selected encoding.
|
* `wut-compare-txmode` --- Compare all the observations in `download/` with `wut` vettings using selected encoding.
|
||||||
|
* `wut-compare-txmode-csv` --- Compare all the observations in `download/` with `wut` vettings using selected encoding, CSV output.
|
||||||
* `wut-dl-sort` --- Populate `data/` dir with waterfalls from `download/`.
|
* `wut-dl-sort` --- Populate `data/` dir with waterfalls from `download/`.
|
||||||
* `wut-dl-sort-tx` --- Populate `data/` dir with waterfalls from `download/` using selected transmitter UUID.
|
* `wut-dl-sort-tx` --- Populate `data/` dir with waterfalls from `download/` using selected transmitter UUID.
|
||||||
* `wut-dl-sort-txmode` --- Populate `data/` dir with waterfalls from `download/` using selected encoding.
|
* `wut-dl-sort-txmode` --- Populate `data/` dir with waterfalls from `download/` using selected encoding.
|
||||||
|
* `wut-files` --- Tells you about what files you have in `downloads/` and `data/`.
|
||||||
* `wut-ml` --- Main machine learning Python script using Tensorflow and Keras.
|
* `wut-ml` --- Main machine learning Python script using Tensorflow and Keras.
|
||||||
|
* `wut-ml-load` --- Machine learning Python script using Tensorflow and Keras, load `data/wut.h5`.
|
||||||
|
* `wut-ml-save` --- Machine learning Python script using Tensorflow and Keras, save `data/wut.h5`.
|
||||||
* `wut-obs` --- Download the JSON for an observation ID.
|
* `wut-obs` --- Download the JSON for an observation ID.
|
||||||
|
* `wut-ogg2wav` --- Convert `.ogg` files in `downloads/` to `.wav` files.
|
||||||
* `wut-review-staging` --- Review all images in `data/staging`.
|
* `wut-review-staging` --- Review all images in `data/staging`.
|
||||||
* `wut-water` --- Download waterfall for an observation ID to `download/[ID]`.
|
* `wut-water` --- Download waterfall for an observation ID to `download/[ID]`.
|
||||||
* `wut-water-range` --- Download waterfalls for a range of observation IDs to `download/[ID]`.
|
* `wut-water-range` --- Download waterfalls for a range of observation IDs to `download/[ID]`.
|
||||||
|
@ -132,15 +138,21 @@ to sync.
|
||||||
## Data Caching Downloads
|
## Data Caching Downloads
|
||||||
The scripts are designed to not download a waterfall or make a JSON request
|
The scripts are designed to not download a waterfall or make a JSON request
|
||||||
for an observation it has already requested. The first time an observation
|
for an observation it has already requested. The first time an observation
|
||||||
is requested, it is downloaded from the SatNOGS network to the `download`
|
is requested, it is downloaded from the SatNOGS network to the `download/`
|
||||||
directory. That `download` directory is the download cache.
|
directory. That `download/` directory is the download cache.
|
||||||
|
|
||||||
|
|
||||||
The `data` directory is just temporary files,mostly linked from the
|
The `data/` directory is just temporary files, mostly linked from the
|
||||||
`downloads` directory. Files in the `data` directory are deleted by many
|
`downloads/` directory. Files in the `data/` directory are deleted by many
|
||||||
scripts, so don't put anything you want to keep in there.
|
scripts, so don't put anything you want to keep in there.
|
||||||
|
|
||||||
|
|
||||||
|
## Preprocessed Files
|
||||||
|
Files in the `preprocess/` directory have been preprocessed to be used
|
||||||
|
further in the pipeline. This contains `.wav` files that have been
|
||||||
|
decoded from `.ogg` files.
|
||||||
|
|
||||||
|
|
||||||
## SatNOGS Observation Data Mirror
|
## SatNOGS Observation Data Mirror
|
||||||
The downloaded waterfalls are available below via `http` and `rsync`.
|
The downloaded waterfalls are available below via `http` and `rsync`.
|
||||||
Use this instead of downloading from SatNOGS to save their bandwidth.
|
Use this instead of downloading from SatNOGS to save their bandwidth.
|
||||||
|
|
|
@ -1,8 +1,10 @@
|
||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# wut-compare-txmode
|
# wut-compare-txmode-csv
|
||||||
#
|
#
|
||||||
# Check the results of a prediction against vetted results
|
# Check the results of a prediction against vetted results
|
||||||
# using a selected encoding.
|
# using a selected encoding.
|
||||||
|
# Outputs CSV (comma separated values).
|
||||||
|
#
|
||||||
# Uses all files in download/ directory.
|
# Uses all files in download/ directory.
|
||||||
# Available encodings:
|
# Available encodings:
|
||||||
# AFSK AFSK1k2 AHRPT APT BPSK BPSK1k2 BPSK9k6 BPSK12k5 BPSK400 CERTO CW DUV
|
# AFSK AFSK1k2 AHRPT APT BPSK BPSK1k2 BPSK9k6 BPSK12k5 BPSK400 CERTO CW DUV
|
||||||
|
@ -11,9 +13,9 @@
|
||||||
# MSK1k2 MSK2k4 MSK4k8 PSK PSK31 SSTV USB WSJT
|
# MSK1k2 MSK2k4 MSK4k8 PSK PSK31 SSTV USB WSJT
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# wut-compare-txmode [Encoding]
|
# wut-compare-txmode-csv [Encoding]
|
||||||
# Example:
|
# Example:
|
||||||
# wut-compare-txmode DUV
|
# wut-compare-txmode-csv DUV
|
||||||
|
|
||||||
MAIN_DIR=`pwd`
|
MAIN_DIR=`pwd`
|
||||||
OBSENC="$1"
|
OBSENC="$1"
|
||||||
|
|
12
wut-ml-load
12
wut-ml-load
|
@ -1,17 +1,21 @@
|
||||||
#!/usr/bin/python3
|
#!/usr/bin/python3
|
||||||
# wut-ml
|
#
|
||||||
|
# XXX This does not work yet XXX
|
||||||
|
#
|
||||||
|
# wut-ml-load
|
||||||
#
|
#
|
||||||
# Vet a SatNOGS image using machine learning (guessing).
|
# Vet a SatNOGS image using machine learning (guessing).
|
||||||
# It will vet the image located at test/unvetted/waterfall.png.
|
# It will vet the image located at test/unvetted/waterfall.png.
|
||||||
|
# Load data/wut.h5 file.
|
||||||
#
|
#
|
||||||
# Note, there is an issue to fix where it will vet everything
|
# Note, there is an issue to fix where it will vet everything
|
||||||
# under the data/test directory, so fix that. For now, just delete
|
# under the data/test directory, so fix that. For now, just delete
|
||||||
# everything else. :)
|
# everything else. :)
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# wut-ml
|
# wut-ml-load
|
||||||
# Example:
|
# Example:
|
||||||
# wut-ml
|
# wut-ml-load
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
@ -26,7 +30,7 @@ from tensorflow.python.keras.models import load_model
|
||||||
from tensorflow.python.keras.preprocessing.image import load_img
|
from tensorflow.python.keras.preprocessing.image import load_img
|
||||||
from tensorflow.python.keras.preprocessing.image import img_to_array
|
from tensorflow.python.keras.preprocessing.image import img_to_array
|
||||||
|
|
||||||
model = load_model('wut.h5')
|
model = load_model('date/wut.h5')
|
||||||
img_width=256
|
img_width=256
|
||||||
img_height=256
|
img_height=256
|
||||||
model = Sequential()
|
model = Sequential()
|
||||||
|
|
|
@ -1,5 +1,8 @@
|
||||||
#!/usr/bin/python3
|
#!/usr/bin/python3
|
||||||
# wut-ml
|
#
|
||||||
|
# XXX This does not work yet XXX
|
||||||
|
#
|
||||||
|
# wut-ml-save
|
||||||
#
|
#
|
||||||
# Vet a SatNOGS image using machine learning (guessing).
|
# Vet a SatNOGS image using machine learning (guessing).
|
||||||
# It will vet the image located at test/unvetted/waterfall.png.
|
# It will vet the image located at test/unvetted/waterfall.png.
|
||||||
|
@ -9,9 +12,9 @@
|
||||||
# everything else. :)
|
# everything else. :)
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# wut-ml
|
# wut-ml-save
|
||||||
# Example:
|
# Example:
|
||||||
# wut-ml
|
# wut-ml-save
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
|
@ -16,7 +16,7 @@ cd download/ || exit
|
||||||
|
|
||||||
# Compile a list of ogg files (NOTE: THIS WILL BREAK AS ARCHIVE GROWS XXX)
|
# Compile a list of ogg files (NOTE: THIS WILL BREAK AS ARCHIVE GROWS XXX)
|
||||||
echo "Total audio files: `ls -1 */satnogs_*.ogg | wc -l`"
|
echo "Total audio files: `ls -1 */satnogs_*.ogg | wc -l`"
|
||||||
for i in 129*/satnogs_*.ogg
|
for i in 13*/satnogs_*.ogg
|
||||||
do OBSID=`dirname $i`
|
do OBSID=`dirname $i`
|
||||||
# Go into directories with audiofiles
|
# Go into directories with audiofiles
|
||||||
echo $OBSID
|
echo $OBSID
|
||||||
|
|
Loading…
Reference in New Issue