References
Sources
The published sources behind the site’s external technical, archival and governance statements; the example recordings and images, with their rights; and the credits for the software, typefaces and music.
References
Primary sources
Each citation supports the statement beside it, scoped as its source scopes it.
California Language Archive / Survey of California and Other Indian Languages, UC Berkeley
“The Collection”
cla.berkeley.edu
Supports archival-scale figures: 150+ linear feet, 2,000+ catalog records, ~130 languages, at least 50 Indigenous languages of California. Tierline has no access to these materials.
California Language Archive / Survey of California and Other Indian Languages, UC Berkeley
“Mission”
cla.berkeley.edu
Supports the statement that 1950s/1960s Survey fieldwork used elicitation protocols and is now part of the permanent collection.
Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, and others
“XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale”
arXiv:2111.09296
Supports pretraining on ~436,000 hours across 128 languages and the 0.3B–2B parameter model family. Tierline uses the 300M configuration.
Meta AI / Hugging Face
Official facebook/wav2vec2-xls-r-300m model card and configuration
huggingface.co
Supports hidden size 1,024, 24 hidden layers, 16 attention heads, conv strides [5,2,2,2,2,2,2]. The 20 ms / 50 fps frame rate on How it works follows from this configuration.
Hervé Bredin and others; pyannote project
“pyannote.audio: neural building blocks for speaker diarization” (arXiv:1911.01255) and the official pyannote/speaker-diarization-community-1 model card
arXiv:1911.01255
Supports the optional diarization adapter and its local-loading behavior. Model assets carry their own license terms (CC-BY-4.0), unaffected by Tierline's own Apache-2.0 license.
Yu A. Malkov and D. A. Yashunin
“Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs”
arXiv:1603.09320
Supports the HNSW method underlying the production candidate index.
Xinjian Li, Siddharth Dalmia, Juncheng Li, Matthew Lee, Patrick Littell, Jiali Yao, and others
“Universal Phone Recognition with a Multilingual Allophone System”, ICASSP 2020
DOI 10.1109/ICASSP40776.2020.9054362 · arXiv:2002.11800
Supports the Allosaurus architecture and its PHOIBLE-based customization to ~2,000 language-dependent recognizers. That paper's reported accuracy on its own eleven evaluation languages is not transferred to Tierline or to any California language.
David R. Mortensen, Patrick Littell, Akash Bharadwaj, Kartik Goyal, Chris Dyer, and Lori Levin
“PanPhon: A Resource for Mapping IPA Segments to Articulatory Feature Vectors”, COLING 2016
ACL Anthology C16-1328
Supports the articulatory feature vectors used in Tierline's feature-weighted edit-distance implementation.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber
“Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks”, ICML 2006
DOI 10.1145/1143844.1143891
Supports the CTC formulation behind the standard per-utterance Viterbi forced alignment that Tierline’s alignment.ctc_adapter implements for a known token sequence.
Stephanie Russo Carroll, Ibrahim Garba, Oscar L. Figueroa-Rodríguez, Jarita Holbrook, Raymond Lovett, Simeon Materechera, and others
“The CARE Principles for Indigenous Data Governance”, Data Science Journal 19 (2020), Article 43
DOI 10.5334/dsj-2020-043
Supports Collective Benefit, Authority to Control, Responsibility, Ethics as a governance reference.
Mukurtu CMS
User Manual, “Understanding Communities and Cultural Protocols”
mukurtu.org
Supports the description of protocol-based access control as an analogue to Tierline's own policy engine.
Local Contexts
“Traditional Knowledge Labels” and “About the Labels”
localcontexts.org
Supports TK Labels as a community-controlled mechanism. Only communities can apply their own Labels.
Jeff Johnson, Matthijs Douze, and Hervé Jégou
“Billion-scale similarity search with GPUs”
arXiv:1702.08734
Supports FAISS as similarity-search infrastructure. Tierline’s deployment target is a CPU.
Nay San, Martijn Bartelds, Mitchell Browne, Lily Clifford, Fiona Gibson, John Mansfield, David Nash, Jane Simpson, Myfany Turpin, Maria Vollmer, Sasha Wilmoth, and Dan Jurafsky
“Leveraging pre-trained representations to improve access to untranscribed speech from endangered languages”
arXiv:2103.14583
Motivates query-by-example retrieval as an approach suited to endangered-language documentation and access to untranscribed speech.
FAISS project
“The index factory” wiki page, including HNSW32_SQ8 and its scalar-quantizer training requirement
github.com/facebookresearch/faiss/wiki
Supports the exact factory-string configuration used for the production index and the scalar-quantizer training requirement. Tierline's builder checks is_trained and trains the index before adding vectors when required.
MLCommons
Multilingual Spoken Words Corpus, release 1.0.0
mlcommons.org/datasets/multilingual-spoken-words
The open dataset Tierline's benchmark builder draws from. CC-BY-4.0. Used to build the engineering stress corpus of the public-data protocol.
Example recordings
Where the voices on this site come from
The example recordings are short excerpts of two public-domain recordings of real people. Each excerpt is listed with the file and the seconds it was cut from, so it can be checked against the original. Transcripts were made for this site from the recordings and checked against each source’s published text.
- Interview with CAPT Ann Bernatitus, NC, USN, (Ret.), World War II nurse. Ann Bernatitus, interviewed by Jan K. Herman, U.S. Navy Bureau of Medicine and Surgery, 1994. Public domain. https://archive.org/details/BERNATITUSANNReleased. Rights: Public Domain Mark 1.0, as published by the U.S. Navy Bureau of Medicine and Surgery history office (archive.org item BERNATITUSANNReleased; licence field http://creativecommons.org/publicdomain/mark/1.0/). A work of the U.S. Government.
- The Diary of Samuel Pepys, 1665: June. Samuel Pepys's diary, June 1665, read by Nicole Lee for LibriVox. Public domain. https://librivox.org/diary-of-samuel-pepys-1665-by-samuel-pepys/. Rights: LibriVox: “all our recordings are public domain (definitely in the USA, and maybe in your country as well)” (https://librivox.org/pages/public-domain/). The 1665 text is in the public domain.
| Excerpt | Title | Source file | Seconds | Processing |
|---|---|---|---|---|
| bernatitus-01 | I would rather go by submarine | Bernatitus 2A.flac | 74.97–82.54 s | Cut from the source at 74.968-82.540 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of +8.20 dB (-20.5 LUFS, -3.9 dBTP) |
| bernatitus-02 | Their own minefield | Bernatitus 2A.flac | 198.81–204.01 s | Cut from the source at 198.814-204.006 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of +6.20 dB (-20.5 LUFS, -5.1 dBTP) |
| bernatitus-03 | Gotten the message | Bernatitus 2A.flac | 205.61–210.64 s | Cut from the source at 205.612-210.638 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of +9.50 dB (-20.5 LUFS, -4.1 dBTP) |
| bernatitus-04 | Taking us so long to get out there | Bernatitus 2A.flac | 216.25–228.40 s | Cut from the source at 216.252-228.404 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of +7.70 dB (-20.5 LUFS, -1.6 dBTP) |
| bernatitus-05 | Tea and chocolate cake | Bernatitus 2A.flac | 403.35–409.44 s | Cut from the source at 403.354-409.440 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of -0.10 dB (-20.5 LUFS, -2.6 dBTP) |
| bernatitus-06 | The slapping of the water | Bernatitus 2A.flac | 251.49–259.83 s | Cut from the source at 251.490-259.830 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of +2.00 dB (-20.5 LUFS, -3.5 dBTP) |
| pepys-01 | Home at almost one in the morning | diaryofsamuelpepys1665_06_pepys.mp3 | 202.83–209.63 s | Cut from the source at 202.834-209.632 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of +2.40 dB (-20.5 LUFS, -2.0 dBTP) |
| pepys-02 | Not very late, to supper and to bed | diaryofsamuelpepys1665_06_pepys.mp3 | 1891.63–1897.67 s | Cut from the source at 1891.630-1897.666 s, mixed to mono, resampled to 16 kHz, 10 ms fades, one linear gain of +2.60 dB (-20.5 LUFS, -4.5 dBTP) |
Photographs and documents
How the images on this site are chosen
- Photographs of the people, places, objects and exhibitions connected to the communities Tierline works with are published with the permission of the photographer or publisher and, where they show people or cultural objects, with the community's approval of that image.
- Other images are in the public domain or under an open licence that allows reuse with attribution, and are credited as the licence asks.
- Captions say only what and where a photograph shows. No sacred or ceremonial sites, no burial places, and nothing a community restricts; an image comes down at the request of the community or the rights holder.
- Images are cropped and resized only: nothing is added, removed, recoloured, generated or enlarged.
Each image opens with its full record: context, credit, rights and treatment.
Credits and licences
Software, typefaces and music
- The film’s score, an original composition for Tierline played with sampled instruments. Samples: VS Chamber Orchestra Community Edition (Versilian Studios, CC0 1.0); University of Iowa Musical Instrument Samples (Lawrence Fritts, free for any use); Salamander Grand Piano V3 by Alexander Holm (CC BY 3.0). Reverberation in part from Voxengo IM Reverbs (Aleksey Vaneev, royalty-free).
- Typefaces: Instrument Serif, Inter Tight and IBM Plex Mono, under the SIL Open Font License 1.1. (licence and file hashes).
- three.js 0.180.0, MIT, which draws the film (licence, provenance).
- The example recordings and the images: the two sections above.
| Component | Used for | Licence |
|---|---|---|
| facebook/wav2vec2-xls-r-300m | Core acoustic representation | Apache-2.0 |
| pyannote/speaker-diarization-community-1 | Optional navigation-only diarization | CC-BY-4.0 |
| FAISS | Exact/approximate candidate retrieval | MIT |
| PyTorch | XLS-R inference | its own upstream terms |
| Transformers / Hugging Face Hub | Local model loading and controlled staging | its own upstream terms |
| ONNX Runtime | Optional optimized inference | MIT |
| NumPy / SciPy / scikit-learn | Numeric processing, degradation tests, PCA fitting | its own upstream terms |
| FFmpeg | Ingest, decoding, resampling | depends on the build |
| Allosaurus | Optional phone-search plugin | GPL-3.0 |
| PanPhon | Optional articulatory-feature distance | its own upstream terms |
| MLCommons Multilingual Spoken Words Corpus 1.0.0 | Open engineering benchmark | CC-BY-4.0 |