5.4 KiB

Raw Blame History

witzit - What In The Zap Is That?

witzit --- What In The Zap Is That? AI categorization of spectra from LIBS/XRF analyzers.

Status

Alpha software under development. Need to fix:

Cannot Export Spectra spacecruft/pysalx#1
API: 501 Not Implemented spacecruft/pysalx#2

Install

Install Dependencies

Get system dependency and upgrade Python pip. Perhaps do something like this, or set up a Python virtual environment.

sudo apt update
sudo apt install git python3-pip
pip install --user --upgrade pip

Clone Git Repo

Get source code with git.

The default requirements.txt installs a Tensorflow without GPU support. You can edit the requirements.txt file to change which is supported. The "generic" version supports both.

git clone https://spacecruft.org/spacecruft/witzit
cd witzit/
pip install --user --upgrade -r requirements.txt

witzit scripts

witzit-load.py --- Load and text display a sample CSV.

Development is most easily done under Jupyter with Tensorboard for training models. These files are in the notebooks/ directory.

witzit-predict.ipynb --- witzit Jupyter notebook, prediction application.
witzit-train.ipynb --- witzit Jupyter notebook, training application.

Data

Note: Files in the data/ directory may be deleted and/or manipulated by scripts in this application.

Note well, the data/ directory is ignored by git, and is a temporary directory where data to be processed is stored. For example, if you have a main original archive of 10,000 samples and you want to process just 1,000 of them, they would be copied to the data/ directory.

Data is also stored here, which can also be deleted/moved by scripts:

/srv/witzit/

Each element sample will be stored under here:

/srv/witzit/data/element/

Each element model will be stored under here:

/srv/witzit/data/models/

Temporary logs during training may be written to the gitignored logs/ directory.

Usage

HOWTO USE. Getting closer...

# Example:
debian@workstation:~/spacecruft/witzit$ ./witzit-load.py
Tensorflow Version:  2.7.0
Num GPUs Available:  0
Num CPUs Available:  1
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]
      Pixel Index  wavelength    intensity
0               1  186.551108    13.956667
1               2  186.647845     9.773333
2               3  186.744576    19.723333
3               4  186.841300    17.223333
4               5  186.938018    16.006667
...           ...         ...          ...
8259         8260  960.998400     0.000000
8260         8261  960.998800     0.000000
8261         8262  960.999200     0.000000
8262         8263  960.999600     0.000000
8263         8264  961.000000  1454.250000

[8264 rows x 3 columns]

Hardware

SciAps LIBS Analyzer
SciAps XRF Analyzer

Deep Learning Algorithm

Can use lots from wut.

https://spacecruft.org/spacecruft/satnogs-wut/src/branch/master/wut-worker-mas.py#L67-L79

def uncompiled_model():
  model = Sequential([
    Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
    MaxPooling2D(),
    Conv2D(32, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Conv2D(64, 3, padding='same', activation='relu'),
    MaxPooling2D(),
    Flatten(),
    Dense(512, activation='relu'),
    Dense(1, activation='sigmoid')
  ])
  return model

Amazingly (to me), the paper Classification of radioxenon spectra with deep learning algorithm (2021) by Azimi, et al. uses nearly the identical CNN Sequence() as wut uses, indicating it may be a very good base to start from.

Paper is non-gratis science:

https://www.sciencedirect.com/science/article/abs/pii/S0265931X21001909

The Sequence() diagram is pulled from the Azimi paper, but is the same as in wut, so makes a good reference.

Articles:

https://www.sciencedirect.com/science/article/abs/pii/S0265931X21001909 Classification of radioxenon spectra with deep learning algorithm
https://www.sciencedirect.com/science/article/pii/S0030401822000402 Deep convolutional neural networks as a unified solution for Raman spectroscopy-based classification in biomedical applications
https://www.sciencedirect.com/science/article/abs/pii/S058485472030313X Automatic preprocessing of laser-induced breakdown spectra using partial least squares regression and feed-forward artificial neural network: Applications to Earth and Mars data
https://www.sciencedirect.com/science/article/abs/pii/S0584854719306068 Determination of minor metal elements in steel using laser-induced breakdown spectroscopy combined with machine learning algorithms
https://www.sciencedirect.com/science/article/abs/pii/S1386142521009380 Feature selection of infrared spectra analysis with convolutional neural network

Misc

Will likely work best with binary categorizations. E.g. like this: Is it element A or not element A? Is it element B or not element B? Is it element C or not element C? Not "is it element A, B, or C?"

Unofficial

Unofficial, unaffiliated with SciAps.

License

License: GPLv3 or any later version.

5.4 KiB Raw Blame History