sndid

sndid identifies sounds.

At present only birds are identified.

Install

Install thusly.

Using Debian Stable (12/Bookworm).

Install thine dependencies:

sudo apt update
sudo apt install git ffmpeg python3-pip python3-virtualenv \
  netcat-traditional sox alsa-utils

Clone repo:

git clone https://spacecruft.org/deepcrayon/sndid
cd sndid/

If you want some sound samples, pull the submodule:

git submodule init
git submodule update

Set up Python to taste, such as:

virtualenv env
source env/bin/activate
pip install --upgrade pip setuptools wheel

Install Python dependencies. This will download 750 megabytes of files. The install will use an additional 2.1 gigabytes.

pip install -r requirements.txt

Usage

Command line

As such:

./sndid

Help:

$ ./sndid -h
usage: sndid [-h] [-i INPUT] [-t LATITUDE] [-n LONGITUDE] [-y YEAR] [-m MONTH] [-d DAY] [-c CONFIDENCE]

Run sndid

options:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        Input filename to process (default samples/sample.wav)
  -t LATITUDE, --latitude LATITUDE
                        Latitude (default 40.57)
  -n LONGITUDE, --longitude LONGITUDE
                        Longitude (default -105.23)
  -y YEAR, --year YEAR  Year (default 2023)
  -m MONTH, --month MONTH
                        Month (default 9)
  -d DAY, --day DAY     Day (default 19)
  -c CONFIDENCE, --confidence CONFIDENCE
                        Minimum Confidence (default 0.50)

Sample output:

$ ./sndid
Labels loaded.
load model True
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Model loaded.
Labels loaded.
load_species_list_model
Meta model loaded.
read_audio_data
read_audio_data: complete, read  20 chunks.
analyze_recording sample.wav
recording has lon/lat
set_predicted_species_list_from_position
return_predicted_species_list
35
100 species loaded.
[{'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 15.0, 'end_time': 18.0, 'confidence': 0.8371534943580627}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 18.0, 'end_time': 21.0, 'confidence': 0.8111729025840759}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 30.0, 'end_time': 33.0, 'confidence': 0.50068598985672}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 33.0, 'end_time': 36.0, 'confidence': 0.7170186042785645}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 39.0, 'end_time': 42.0, 'confidence': 0.6576249003410339}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 48.0, 'end_time': 51.0, 'confidence': 0.8048814535140991}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 51.0, 'end_time': 54.0, 'confidence': 0.9604988694190979}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 54.0, 'end_time': 57.0, 'confidence': 0.8156641125679016}, {'common_name': 'Hairy Woodpecker', 'scientific_name': 'Dryobates villosus', 'start_time': 57.0, 'end_time': 60.0, 'confidence': 0.8230038285255432}]

Server

Run thusly:

./sndid-server

Help:

$ ./sndid-server -h
usage: sndid-server [-h] [-i IP] [-p PORT] [-t LATITUDE] [-n LONGITUDE] [-y YEAR] [-m MONTH] [-d DAY] [-c CONFIDENCE]

Run sndid-server

options:
  -h, --help            show this help message and exit
  -i IP, --ip IP        Server IP address (default 127.0.0.1)
  -p PORT, --port PORT  Server network port (default 9988)
  -t LATITUDE, --latitude LATITUDE
                        Latitude (default 40.57)
  -n LONGITUDE, --longitude LONGITUDE
                        Longitude (default -105.23)
  -y YEAR, --year YEAR  Year (default 2023)
  -m MONTH, --month MONTH
                        Month (default 9)
  -d DAY, --day DAY     Day (default 19)
  -c CONFIDENCE, --confidence CONFIDENCE

Sample output on startup:

$ ./sndid-server
sndid-server Birdnetlib started
IP: 127.0.0.1
PORT: 9988

After client connects and sends mono wav:

Labels loaded.
load model True
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Model loaded.
Labels loaded.
load_species_list_model
Meta model loaded.
read_audio_data: complete, read  20 chunks.
analyze_recording buffer
recording has lon/lat
set_predicted_species_list_from_position
return_predicted_species_list
35
100 species loaded.
[{'common_name': 'Hairy Woodpecker',
  'confidence': 0.837184488773346,
  'end_time': 18.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 15.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.8105885982513428,
  'end_time': 21.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 18.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.47060683369636536,
  'end_time': 24.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 21.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.5013241171836853,
  'end_time': 33.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 30.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.7146830558776855,
  'end_time': 36.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 33.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.6557420492172241,
  'end_time': 42.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 39.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.3896884322166443,
  'end_time': 45.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 42.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.806126594543457,
  'end_time': 51.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 48.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.9604253768920898,
  'end_time': 54.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 51.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.8147749900817871,
  'end_time': 57.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 54.0},
 {'common_name': 'Hairy Woodpecker',
  'confidence': 0.8241879343986511,
  'end_time': 60.0,
  'scientific_name': 'Dryobates villosus',
  'start_time': 57.0}]

Client

Requires mono wav file.

To convert stereo to mono with sox:

sox -c 2 stereo.wav -c 1 mono.wav

Run client to submit file to server thusly:

./sndid-client

Help:

$ ./sndid-client -h
usage: sndid-client [-h] [-i IP] [-p PORT] [-f FILE]

Run sndid-client

options:
  -h, --help            show this help message and exit
  -i IP, --ip IP        Server IP address (default 127.0.0.1)
  -p PORT, --port PORT  Server network port (default 9988)
  -f FILE, --file FILE  Input filename to process (default samples/mono.wav)

Sample output:

$ ./sndid-client
Sending samples/mono.wav to 127.0.0.1:9988

ALSA Client

Use this script to stream from the microphone to the server, using ALSA.

./sndid-alsa

Help:

$ ./sndid-alsa -h
usage: sndid-alsa [-h] [-i IP] [-p PORT] [-r RATE] [-t TIME]

Run sndid-alsa

options:
  -h, --help            show this help message and exit
  -i IP, --ip IP        Server IP address (default 127.0.0.1)
  -p PORT, --port PORT  Server network port (default 9988)
  -r RATE, --rate RATE  Rate in Hertz (default 48000)
  -t TIME, --time TIME  Length of segments in seconds (default 10)

Sample output:

$ ./sndid-alsa 
Streaming ALSA in to 127.0.0.1:9988
Recording WAVE 'stdin' : Float 32 bit Little Endian, Rate 48000 Hz, Mono

Exit with CTRL-C.

Stream

sndid-stream streams from a URL to the sndid-server. Input URL can be anything ffmpeg can read (everything).

Help:

$ ./sndid-stream -h
usage: sndid-stream [-h] [-i IP] [-p PORT] [-t TIME] -u URL

Run sndid-stream

options:
  -h, --help            show this help message and exit
  -i IP, --ip IP        Server IP address (default 127.0.0.1)
  -p PORT, --port PORT  Server network port (default 9988)
  -t TIME, --time TIME  Length of segments in seconds (default 60)
  -u URL, --url URL     Input url

Exit with CTRL-Z and kill %1 :) por ahora.

List

sndid-list lists sounds available to be identified at a particular time and location.

XXX This script is foobar at the moment.

Use:

./sndid-list

Help:

$ ./sndid-list -h
usage: sndid-list [-h] [-t LATITUDE] [-n LONGITUDE] [-y YEAR] [-m MONTH] [-d DAY] [-g | --geo | --no-geo] [-c | --cal | --no-cal]

Run sndid-list

options:
  -h, --help            show this help message and exit
  -t LATITUDE, --latitude LATITUDE
                        Latitude (default 40.57)
  -n LONGITUDE, --longitude LONGITUDE
                        Longitude (default -105.23)
  -y YEAR, --year YEAR  Year (default today)
  -m MONTH, --month MONTH
                        Month (default today)
  -d DAY, --day DAY     Day (default today)
  -g, --geo, --no-geo   Limit list by geocoordinates (default False)
  -c, --cal, --no-cal   Limit list by calendar date (default False, use with --geo)

Sample output:

$ ./sndid-list -c -g
Labels loaded.
load_species_list_model
Meta model loaded.
35
18 species loaded.
[{'scientific_name': 'Colaptes auratus', 'common_name': 'Northern Flicker', 'threshold': 0.7266561}, {'scientific_name': 'Turdus migratorius', 'common_name': 'American Robin', 'threshold': 0.7251873}, {'scientific_name': 'Cardellina pusilla', 'common_name': "Wilson's Warbler", 'threshold': 0.6379353}, {'scientific_name': 'Setophaga coronata', 'common_name': 'Yellow-rumped Warbler', 'threshold': 0.60027194}, {'scientific_name': 'Pica hudsonia', 'common_name': 'Black-billed Magpie', 'threshold': 0.5446144}, {'scientific_name': 'Streptopelia decaocto', 'common_name': 'Eurasian Collared-Dove', 'threshold': 0.5141223}, {'scientific_name': 'Zenaida macroura', 'common_name': 'Mourning Dove', 'threshold': 0.4939787}, {'scientific_name': 'Sturnus vulgaris', 'common_name': 'European Starling', 'threshold': 0.49090102}, {'scientific_name': 'Sturnella neglecta', 'common_name': 'Western Meadowlark', 'threshold': 0.47289485}, {'scientific_name': 'Buteo jamaicensis', 'common_name': 'Red-tailed Hawk', 'threshold': 0.41278195}, {'scientific_name': 'Zonotrichia leucophrys', 'common_name': 'White-crowned Sparrow', 'threshold': 0.38684908}, {'scientific_name': 'Cathartes aura', 'common_name': 'Turkey Vulture', 'threshold': 0.3806524}, {'scientific_name': 'Corvus corax', 'common_name': 'Common Raven', 'threshold': 0.3722749}, {'scientific_name': 'Anas platyrhynchos', 'common_name': 'Mallard', 'threshold': 0.34827054}, {'scientific_name': 'Poecile gambeli', 'common_name': 'Mountain Chickadee', 'threshold': 0.33535123}, {'scientific_name': 'Spizella passerina', 'common_name': 'Chipping Sparrow', 'threshold': 0.3286236}, {'scientific_name': 'Poecile atricapillus', 'common_name': 'Black-capped Chickadee', 'threshold': 0.3259723}, {'scientific_name': 'Haemorhous mexicanus', 'common_name': 'House Finch', 'threshold': 0.30751282}]

The current count of all species in model:

$ ./sndid-list | grep "species loaded"
6522 species loaded.

Tensorflow may also output lines to stderr such as:

2023-09-22 14:42:37.059356: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-09-22 14:42:37.061652: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-09-22 14:42:37.112102: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-09-22 14:42:37.112659: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-09-22 14:42:37.971311: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

Note, BirdNet tensorflow works fine with just the CPU, no GPU required to use the model.

Development

To "develop", install the requirements:

pip install -r requirements-dev.txt

Then run black on the Python files for nice formatting:

black sndid*

Upstream

Birds

BirdNet

• BirdNet is the main upstream project: https://birdnet.cornell.edu/

• The source code is available here: https://github.com/kahst/BirdNET-Analyzer

Unfortunately the project has a horrible license, CC-NC-BY-SA 4.0, which isn't even a software license. The authors of the license themselves say it shouldn't be used for software. It's not just the data files, such as the models that are NC (which would be ok), but the source code itself is NC. I haven't double checked the Python imports, but it would be interesting to see if they are even compatible with the NC license, since the NC license is incompatible with so many Free Software and Open Source Software licenses. There is a bug open to fix the license, and change it to Apache 2.0, which would be sane. I hope this happens:

https://github.com/kahst/BirdNET-Analyzer/issues/154

birdnetlib

birdnetlib is based on BirdNet, but with a different codebase and author. birdnetlib uses BirdNet's non-libre NC model files. birdnetlib has a dependency on the non-free BirdNet-Analyzer Python code (e.g. via pip install), but the source code to birdnetlib itself is under the Apache 2.0 license. birdnetlib is Free Software / Open Source Software, with non-libre dependency.

https://github.com/joeweiss/birdnetlib

Status

Alpha, initial development. Works, but taped together with duct tape.

Copyright

Unofficial project, not related to BirdNet or Joe Weiss.

Upstream sources under their respective copyrights.

License

Apache 2.0.

Copyright © 2023, Jeff Moe.