spacecruft
/
satnogs-wut
0
0
Fork 0
Code Issues 6 Releases Activity

Add script to download Internet Archive torrents

master
Jeff Moe 2022-06-10 21:17:08 -06:00
parent 1517670e7c
commit 270178d027
2 changed files with 32 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
README.md
View File

@ -88,6 +88,7 @@ The following scripts are in the repo.
* `wut-files` --- Tells you about what files you have in `downloads/` and `data/`.
* `wut-files-data` --- Tells you about what files you have in `data/`.
* `wut-ia` --- Download SatNOGS data from the Internet Archive at `archive.org`.
* `wut-ia-torrents` --- Download SatNOGS torrents from the Internet Archive at `archive.org`.
* `wut-img-ck.py` --- Validate image files are not corrupt with PIL.
* `wut-ml` --- Main machine learning Python script using Tensorflow and Keras.
* `wut-ml-auto` --- Machine learning Python script using Tensorflow and Keras, auto.
@ -249,6 +250,13 @@ Get an account at archive.org, then run this to set up your account locally:
ia configure
```
To download all the SatNOGS collections `.torrent` files from the
Internet Archive, run:
```
wut-ia-torrents
```
# Caveats
This is the first artificial intelligence script I've done,

24
wut-ia-torrents 100755
View File

@ -0,0 +1,24 @@
#!/usr/bin/env python3
#
# wut-ia-torrents --- Download SatNOGS torrents from the Internet Archive.
#
# https://archive.org/details/satnogs
from internetarchive import get_item
from internetarchive import get_session
from internetarchive import download
from internetarchive import search_items
# Download dir
obs_dl='/srv/dl'
s = get_session()
s.mount_http_adapter()
search_results = s.search_items('satnogs-observations')
for i in search_items('identifier:satnogs-observations-*'):
obs_id=(i['identifier'])
print('Collection', obs_id)
download(obs_id, verbose=True, glob_pattern='*.torrent',
checksum=True, destdir=obs_dl,
retries=4, ignore_errors=True)