Add script to download Internet Archive torrents
parent
1517670e7c
commit
270178d027
|
@ -88,6 +88,7 @@ The following scripts are in the repo.
|
||||||
* `wut-files` --- Tells you about what files you have in `downloads/` and `data/`.
|
* `wut-files` --- Tells you about what files you have in `downloads/` and `data/`.
|
||||||
* `wut-files-data` --- Tells you about what files you have in `data/`.
|
* `wut-files-data` --- Tells you about what files you have in `data/`.
|
||||||
* `wut-ia` --- Download SatNOGS data from the Internet Archive at `archive.org`.
|
* `wut-ia` --- Download SatNOGS data from the Internet Archive at `archive.org`.
|
||||||
|
* `wut-ia-torrents` --- Download SatNOGS torrents from the Internet Archive at `archive.org`.
|
||||||
* `wut-img-ck.py` --- Validate image files are not corrupt with PIL.
|
* `wut-img-ck.py` --- Validate image files are not corrupt with PIL.
|
||||||
* `wut-ml` --- Main machine learning Python script using Tensorflow and Keras.
|
* `wut-ml` --- Main machine learning Python script using Tensorflow and Keras.
|
||||||
* `wut-ml-auto` --- Machine learning Python script using Tensorflow and Keras, auto.
|
* `wut-ml-auto` --- Machine learning Python script using Tensorflow and Keras, auto.
|
||||||
|
@ -249,6 +250,13 @@ Get an account at archive.org, then run this to set up your account locally:
|
||||||
ia configure
|
ia configure
|
||||||
```
|
```
|
||||||
|
|
||||||
|
To download all the SatNOGS collections `.torrent` files from the
|
||||||
|
Internet Archive, run:
|
||||||
|
|
||||||
|
```
|
||||||
|
wut-ia-torrents
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
# Caveats
|
# Caveats
|
||||||
This is the first artificial intelligence script I've done,
|
This is the first artificial intelligence script I've done,
|
||||||
|
|
|
@ -0,0 +1,24 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# wut-ia-torrents --- Download SatNOGS torrents from the Internet Archive.
|
||||||
|
#
|
||||||
|
# https://archive.org/details/satnogs
|
||||||
|
|
||||||
|
from internetarchive import get_item
|
||||||
|
from internetarchive import get_session
|
||||||
|
from internetarchive import download
|
||||||
|
from internetarchive import search_items
|
||||||
|
|
||||||
|
# Download dir
|
||||||
|
obs_dl='/srv/dl'
|
||||||
|
|
||||||
|
s = get_session()
|
||||||
|
s.mount_http_adapter()
|
||||||
|
search_results = s.search_items('satnogs-observations')
|
||||||
|
for i in search_items('identifier:satnogs-observations-*'):
|
||||||
|
obs_id=(i['identifier'])
|
||||||
|
print('Collection', obs_id)
|
||||||
|
download(obs_id, verbose=True, glob_pattern='*.torrent',
|
||||||
|
checksum=True, destdir=obs_dl,
|
||||||
|
retries=4, ignore_errors=True)
|
||||||
|
|
Loading…
Reference in New Issue