dleucas / wmmsdb (public) (License: GPLv3) (since 2018-07-08) (hash sha1)
A collection of scripts to download, transform and normalize the Watkins Marine Mammal Sound Database.

Credit:

“Watkins Marine Mammal Sound Database, Woods Hole Oceanographic Institution.”

http://cis.whoi.edu/science/B/whalesounds/index.cfm
Clone URLs: https://rocketgit.com/user/dleucas/wmmsdb ssh://rocketgit@ssh.rocketgit.com/user/dleucas/wmmsdb git://git.rocketgit.com/user/dleucas/wmmsdb
master species_names
List of commits:
Subject Hash Author Date (UTC)
add recording conditions c5603786d17346690a789c5392137fd9afd78b28 dleucas 2021-12-22 12:18:07
add signal level with comment on missing data af643d86e29e82e0879eaf899e6bfea2a505adab dleucas 2021-12-22 11:20:28
docu updates 0912bbabf50c7c106fc45f623497ca87f7503377 dleucas 2021-12-22 09:54:28
extract storage location b73fc943e55d51fd3df9fe26d46220b67d2efe75 dleucas 2021-12-19 18:32:04
extract and transform author names 0ff3dd29773a1b93c7a893c4971dda5522fa488b dleucas 2021-12-19 17:56:35
extract and transform ship names 12eb822d4ae70fbd2f01b0f03a77eee5e0d9a2b7 dleucas 2021-12-19 14:40:54
extract and transform observation time 595eb911201f8697c9c5f1aa1c40ee502e8009c2 dleucas 2021-12-19 12:04:29
formating f82ed6a3f1c0dccfc50aaa2a0fa3e866f5741acf dleucas 2021-12-18 23:18:45
update readme for GeoJSON command e5c1a6f49ea17a8860cecffdcf280d1d2fdb3ac2 dleucas 2021-12-18 23:13:31
add species names to GeoJSON and World Map 86af1d9597c3cb3c58f71c777902148d2ce333f8 dleucas 2021-12-18 22:51:11
re-worked GeoJSON from transformed JSON. identical output. added license 8ec5e1678966c6b1c441d035a6833f55586325ec dleucas 2021-12-18 22:01:19
remove genus, add species common and scientific names, display names in record details e2025219ade78f9ac2e6dfd81c479d242b9db24b dleucas 2021-12-18 17:18:28
add common and scientific names to schema b4c7dcb4042b9d1cd3da24123f4506113a876f02 dleucas 2021-12-18 17:17:17
remove genus, add species and type to type_of namespace change 2a7757a6c3e6dbb784b41d7a4b29381bc9105803 dleucas 2021-12-18 17:16:04
converted all transforms to functions 9d37303b24abc225f11f9c3d3b622c5167ed43e3 dleucas 2021-12-18 16:00:04
more conversion to functions. match old output for now fc4a8157a6902f4571b54c6ab84174f005adbe0d dleucas 2021-12-18 10:26:23
WIP convert filters to functions 32badc3512dd9094d51ba2cc2ef8112eba2698bf dleucas 2021-12-16 18:33:21
convert html only once. extract species names as json. formating and lint. e755dc7f4fe2d7c9b97826a0f3f2cf5385e90ef9 dleucas 2021-12-16 13:35:31
download once. use wget only. get species names. test for commands. formating 572dbf1eaffe17c43a4a01dc9675737628c5a234 dleucas 2021-12-16 12:14:26
add filter by behavior type, sort by modified date c3f9f9f9d9501e714117af7fff573e7f3fa4052b dleucas 2019-06-14 03:51:19
Commit c5603786d17346690a789c5392137fd9afd78b28 - add recording conditions
Author: dleucas
Author date (UTC): 2021-12-22 12:18
Committer name: dleucas
Committer date (UTC): 2021-12-22 12:18
Parent(s): af643d86e29e82e0879eaf899e6bfea2a505adab
Signing key:
Tree: e5a67d080ae2ae6a26f9ad7123de5d2f379cec57
File Lines added Lines deleted
transform.jq 25 0
File transform.jq changed (mode: 100755) (index 89d26e7..bf40bdc)
... ... def remove_extra_spaces:
30 30 # replace two or more whitespaces with a single one # replace two or more whitespaces with a single one
31 31 sub("\\s{2,}"; " "); sub("\\s{2,}"; " ");
32 32
33 def null_if_empty:
34 if (. == "") then null
35 elif (. | length == 0) then null
36 else . end;
33 37 # #
34 38 # transform functions # transform functions
35 39 # #
 
... ... def as_sound_sample_rate:
485 489 # 10200 # 10200
486 490 if (. | length > 0) then . | sub("[\\.,]"; "") | tonumber else null end; if (. | length > 0) then . | sub("[\\.,]"; "") | tonumber else null end;
487 491
492 def as_recording_conditions:
493 # Conditions, weather, salinity
494 #
495 # Only removing a few species codes. Not useful as filter
496 #
497 # Example source data:
498 #
499 # Cruise ship in area, wind 10 kts. WNW, seas 2-3ft.
500 # Flat, calm, sunny, occassional boats
501 # High generator background RG AX-58 hydrophones
502 # In air CC12F
503 # Magnecorder tape recorder
504 # On ice. Noisy ship background CC12F
505 sub("\\b[A-C][A-Z]\\d{1,2}[A-Z]$"; "") | trim | remove_extra_spaces;
506
507
488 508 # #
489 509 # Assemble the object tree # Assemble the object tree
490 510 # #
 
... ... def as_sound_sample_rate:
505 525 ship: .SH | as_ship, ship: .SH | as_ship,
506 526 author: .AU | split("|") | map(as_author), author: .AU | split("|") | map(as_author),
507 527 storage_location: .LO | split("|") | map(as_storage_location), storage_location: .LO | split("|") | map(as_storage_location),
528 # recording setup
529 recording: {
530 # mostly single values, join the few multi-values
531 "conditions": .RC | split("|") | map(as_recording_conditions) | join(", ") | null_if_empty,
532 },
508 533 # object contains properties of the captured signal # object contains properties of the captured signal
509 534 signal: { signal: {
510 535 position: [ position: [
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/dleucas/wmmsdb

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/dleucas/wmmsdb

Clone this repository using git:
git clone git://git.rocketgit.com/user/dleucas/wmmsdb

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main