# witzit - What In The Zap Is That?
`witzit` --- What In The Zap Is That?
AI categorization of spectra from LIBS/XRF analyzers.


# Status
Alpha software under development.
Need to fix:


* *Cannot Export Spectra*
 https://spacecruft.org/spacecruft/pysalx/issues/1

* *API: 501 Not Implemented*
 https://spacecruft.org/spacecruft/pysalx/issues/2


# Install
## Install Dependencies
Get system dependency and upgrade Python pip.
Perhaps do something like this, or
set up a Python virtual environment.

```
sudo apt update
sudo apt install git python3-pip
pip install --user --upgrade pip
```


## Clone Git Repo
Get source code with `git`.

The default `requirements.txt` installs a Tensorflow without
GPU support. You can edit the `requirements.txt` file to change
which is supported. The "generic" version supports both.


```
git clone https://spacecruft.org/spacecruft/witzit
cd witzit/
pip install --user --upgrade -r requirements.txt
```

# witzit scripts

* `witzit-load.py` --- Load and text display a sample CSV.

Development is most easily done under Jupyter with Tensorboard
for training models. These files are in the `notebooks/` directory.

* `witzit-predict.ipynb` --- witzit Jupyter notebook, prediction application.
* `witzit-train.ipynb` --- witzit Jupyter notebook, training application.

# Data
Note: Files in the `data/` directory may be deleted and/or manipulated
by scripts in this application.

Note well, the `data/` directory is ignored by git, and is a temporary
directory where data to be processed is stored. For example, if you have
a main original archive of 10,000 samples and you want to process just 1,000
of them, they would be copied to the `data/` directory.


Data is also stored here, which can also be deleted/moved by scripts:

`/srv/witzit/`


Each element sample will be stored under here:

`/srv/witzit/data/element/`


Each element model will be stored under here:

`/srv/witzit/data/models/`


Temporary logs during training may be written to the gitignored
`logs/` directory.


# Usage
HOWTO USE. Getting closer...

```
# Example:
debian@workstation:~/spacecruft/witzit$ ./witzit-load.py
Tensorflow Version:  2.7.0
Num GPUs Available:  0
Num CPUs Available:  1
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]
      Pixel Index  wavelength    intensity
0               1  186.551108    13.956667
1               2  186.647845     9.773333
2               3  186.744576    19.723333
3               4  186.841300    17.223333
4               5  186.938018    16.006667
...           ...         ...          ...
8259         8260  960.998400     0.000000
8260         8261  960.998800     0.000000
8261         8262  960.999200     0.000000
8262         8263  960.999600     0.000000
8263         8264  961.000000  1454.250000

[8264 rows x 3 columns]
```

# Hardware

* SciAps LIBS Analyzer

* SciAps XRF Analyzer


# Deep Learning Algorithm
Can use lots from `wut`.

* https://spacecruft.org/spacecruft/satnogs-wut/src/branch/master/wut-worker-mas.py#L67-L79

```
def uncompiled_model():
  model = Sequential([
    Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
    MaxPooling2D(),
    Conv2D(32, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Conv2D(64, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Flatten(),
    Dense(512, activation='relu'),
    Dense(1, activation='sigmoid')
  ])
  return model
```


Amazingly (to me), the paper
*Classification of radioxenon spectra with deep learning algorithm*
(2021) by Azimi, et al. uses nearly the identical CNN `Sequence()` as
`wut` uses, indicating it may be a very good base to start from.

Paper is non-gratis science:

* https://www.sciencedirect.com/science/article/abs/pii/S0265931X21001909


The `Sequence()` diagram is pulled from the Azimi paper, but is the
same as in `wut`, so makes a good reference.


![Deep Learning Sequence](img/sequence.png)


# Articles:

* https://www.sciencedirect.com/science/article/abs/pii/S0265931X21001909
*Classification of radioxenon spectra with deep learning algorithm*

* https://www.sciencedirect.com/science/article/pii/S0030401822000402
*Deep convolutional neural networks as a unified solution for Raman spectroscopy-based classification in biomedical applications*

* https://www.sciencedirect.com/science/article/abs/pii/S058485472030313X
*Automatic preprocessing of laser-induced breakdown spectra using partial least squares regression and feed-forward artificial neural network: Applications to Earth and Mars data*

* https://www.sciencedirect.com/science/article/abs/pii/S0584854719306068 
*Determination of minor metal elements in steel using laser-induced breakdown spectroscopy combined with machine learning algorithms*

* https://www.sciencedirect.com/science/article/abs/pii/S1386142521009380
*Feature selection of infrared spectra analysis with convolutional neural network*


# Misc

Will likely work best with binary categorizations. E.g. like this:
Is it element A or not element A?
Is it element B or not element B?
Is it element C or not element C?
Not "is it element A, B, or C?" 


# See Also

* `pysalx` --- Unofficial scripts for interacting with the SciAps LIBS
  and XRF analyzers.

https://spacecruft.org/spacecruft/pysalx/

* `wut?` --- *What U Think?* SatNOGS Observation AI.

https://spacecruft.org/spacecruft/satnogs-wut/


# Unofficial
Unofficial, unaffiliated with SciAps.


# License
License: GPLv3 or any later version. 

Copyright (C) 2019-2022, Jeff Moe