Seven stages take a recording from its arrival, with its permissions, to a result a person reviews. You can also watch the same stages on a live search.
From object to record
An archive keeps an object’s knowledge in its record.
A basket carries its maker’s knowledge in its weave. What lets the next generation find it, trust it and learn from it is the record kept with it. Recordings need the same layers, and Tierline keeps each of them for every recording it searches.
Basket with cover by Elizabeth Hickox (Karuk/Wiyot, 1875–1947), about 1920. National Museum of the American Indian, Smithsonian Institution.National Museum of the American Indian, SmithsonianSource ↗
01Metadata
Basket with cover · Elizabeth Hickox, Karuk/Wiyot (1875–1947) · about 1920 · hazel, spruce root, maidenhair fern stem and porcupine quills · 25.5 × 13.5 cm
In Tierline Title, speaker, date, language and notes for every recording, written by the people who keep it. The Workspace →
02Provenance
Its maker, its date, and every hand it passed through on its way to the museum, kept as its collection history
In Tierline Every recording keeps where it came from; the example recordings carry a SHA-256 checksum that your browser checks before it plays them. Explore →
03Collection context
One object among the collections of the National Museum of the American Indian, catalogued with the works of its region and its people
In Tierline Recordings are grouped by the access policy that governs them; a search opens only the collections a person may search. How a search runs →
04Source relationships
Other works by its maker, and the Karuk and Wiyot basketry it belongs to
In Tierline Every result stays bound to its recording and its exact span, and opens there, with the audio around it. Listening room →
05Recorded knowledge
What a maker, a speaker or a teacher said about it, kept as sound
In Tierline Passages found by their sound, in recordings no one has transcribed, for a person to hear and judge. Try a search →
06Community-controlled access
Who may see, hear or reproduce it, decided by the community it belongs to
In Tierline Six permissions on every recording, denied unless granted; a withdrawal takes effect at the next search. Permissions →
One search, start to finish
From a recording to a result, in seven steps.
A passage of a 1994 oral-history interview, searched with Tierline’s reference engine and the XLS-R model. Everything drawn below is what that run recorded. Switch to Inspect for the mechanism, the numbers and the code.
01 consent
Who decides
Before a recording is processed, the people responsible for it decide what may happen to it. Listening, searching, annotating, exporting, training and publishing are six separate permissions, each denied until it is granted. In this example, a language worker may search, play and annotate; export, training and publication stay closed.
The run searched under example-language-worker-policy (examples/policy-language-worker.json, SHA-256 b620f577f5a9…): annotate, play, search granted to role language_worker for language work and teaching; export, publish, train denied. Indexes are partitioned by policy hash, so this decision fixes which partition a search may open. Code: src/tierline/policies/evaluator.py, src/tierline/policies/access.py.
02 audio
Keep the original
The original file is kept exactly as it arrived. Tierline makes one predictable copy to listen to and analyse, and records how it was made, so every result can be traced back to its source. This one is a 1994 oral-history interview with the Navy nurse Ann Bernatitus, in the public domain.
FFmpeg and ffprobe read the stream layout; one 16000 Hz FLAC derivative is made per source channel (no downmix), resampled with soxr. Source and derivative SHA-256, the FFmpeg build and the exact command are recorded, and an IMPORT event is written to the hash-chained audit log. This file: bernatitus-01.wav, 7.57 s, SHA-256 56f1b18bfbc1a29b…. Code: src/tierline/ingest/.
03 encoder
Describe the sound
Choose a few spoken words, here “go by submarine”. A speech model listens and describes every 20 milliseconds as a list of numbers. Passages that sound alike get similar numbers whether or not anyone has transcribed them, and often when different people say them.
facebook/wav2vec2-xls-r-300m at revision 1a640f3, transformer layer 11: 1,024 dimensions per frame, a 20 ms hop (320 samples at 16000 Hz), a 25 ms receptive field, chunked with 2 s of context either side. The span 1.16–2.54 s yields 69 frames. Code: src/tierline/features/xlsr.py.
04 compress
Make it small enough to keep
The description is compressed so a whole archive fits on an ordinary computer: fewer numbers for each moment, and each kept in a single byte.
Incremental PCA 1,024 → 256 (fitted on a calibration split, seed 1729, never on the searched audio), then int8 scalar quantization: 4,096 bytes per frame as float32 become 256. In the storage benchmark on Evaluation, the int8 payload is 75% smaller and the SQ8 index uses 63.4% less memory than float32. Code: src/tierline/features/pca.py, quantize.py.
05 retrieve
Search only what you may search
The collection is cut into overlapping one-second windows. A search opens only the part of the index the person may search, and an approximate index proposes the most promising windows without comparing every one. Here, eight example recordings make 202 windows.
1.0 s windows on a 0.25 s hop; FAISS HNSW32_SQ8 (M 32, efConstruction 200, efSearch 512, L2); each authorized policy partition is searched and the hits are merged and cut to K = 500 candidates in total. This run: 8 recordings, 202 windows, one partition, so every window is a candidate. In the approximate-versus-exact benchmark over the evaluated collections, the index held 3,271,840 windows and kept 99.04% of exact search’s top 500. Code: src/tierline/retrieval/windows.py, faiss_index.py, candidates.py.
06 align
Compare the passages themselves
People rarely say the same words at the same speed. Each candidate is lined up with the query moment by moment, stretching and compressing time where it must, and the passages are ranked by how closely they match once aligned. The line is the alignment the engine chose for its closest other passage.
Subsequence DTW with cosine frame cost over up to 100 candidate regions, duration ratio 0.67–1.5, temporal NMS at IoU 0.5, 50 results kept. The path pairs the query’s 69 frames with 50 frames of bernatitus-03 at 1.65–2.65 s; mean cost 0.538. Code: src/tierline/retrieval/dtw.py, nms.py.
07 review
Hear it in its place
The answer comes back as a passage in its original recording, with its time, its source and the conversation around it. A person listens and decides. Here the closest other passage is the same nurse saying “submarine” again, in another part of the interview: “the submarine had”.
The recorded run returned 8 ranked passages, and a second run returned the same. Rank 1 is the query itself at cost 0. Reviews export to ELAN and Praat. Code: src/tierline/review/service.py, src/tierline/export/.
A recording moves through these stages in this order. Select a stage to read
it: Plain says what happens there; Inspect names the code and
configuration that implement it.
Status
ImplementedSource exists in the repository, with tests.
ConfiguredA fixed value in the architecture configuration.
Run on real recordingsThe reference engine's recorded run on the example recordings went through this stage.
EvaluatedA measurement on Evaluation covers this stage: a system benchmark of storage or approximate search, or the held-out evaluation of the final ranking.
Results are on Evaluation; the recorded run on the
example recordings is published.
Architecture 1.0.0K = 500
The stages are spaced for reading, not by how long each takes.
Configured6 fixed settings in the architecture configuration
Run on real recordingssearched under example-language-worker-policy: search, play and annotate granted; export, train and publish denied, 13 October 2026
Authority and consent
Before anything enters
Start with the people responsible for the recording. Decide whether it should be digitized at all, who may reach it, and for which uses. Deciding to keep something out of the archive is a valid outcome.
Why it works this way
A recording is not a blanket permission. Policy has to travel with the recording, with each time interval inside it, and with every artifact derived from it.
A policy evaluator reads explicit rules per recording interval. Principals are users and roles; conditions are purposes and months; each action is granted separately. Nothing is permitted by omission, and private material additionally requires recorded stewardship consent.
Configuration
Default policy
deny
Separately authorized actions
search, play, annotate, export, train, publish
Condition dimensions
user, role, purpose, month
Private authorization
policy AND explicit stewardship consent
Index partitioning
policy-partitioned
Audit log
SHA-256 hash chained
Implemented in
tierline.policies.evaluator
tierline.policies.inheritance
tierline.policies.access
tierline.policies.audit
Technical permissions support community decisions. They do not create consent, and they are not a substitute for cultural authority.
Configured5 fixed settings in the architecture configuration
Run on real recordingsthe example recordings ingested, source and derivative SHA-256 recorded, 13 October 2026
Prepare the audio
Keep the original
Keep an unchanged copy of the source. Make one predictable audio derivative for each source channel, and record exactly how it was produced.
Why it works this way
Provenance lets a language worker trace any result back to the original recording. Separate channels avoid silently mixing two speakers into one signal.
FFmpeg and ffprobe read the real stream layout and write one derivative per source channel. The source file is copied unmodified. Source and derivative SHA-256, the literal FFmpeg build string and the exact conversion command are stored with the recording.
DERIVATIVEone FLAC per source channel
Configuration
Model input
16,000 Hz s16 FLAC
Resampler
soxr, precision 28
Channel strategy
separate channels; no automatic archive downmix
Denoising / high-pass / loudness
off, off, off
Recorded provenance
source SHA-256, derivative SHA-256, exact command
Implemented in
tierline.ingest.ffmpeg
tierline.ingest.probe
tierline.ingest.provenance
Enhancement and denoising are off by default. This step does not transcribe or translate anything.
Configured7 fixed settings in the architecture configuration
Run on real recordingsXLS-R weights at revision 1a640f32ac3e… ran on CPU, 4 threads, 13 October 2026
Represent the sound
Speech into features
A speech model converts short stretches of audio into numeric features. These are a way to compare acoustic patterns. They are not a transcript, and not a translation.
Why it works this way
Search can then begin from a spoken example, instead of requiring a spelling or a finished transcript that may not exist.
A pinned XLS-R checkpoint runs in float32 PyTorch, loaded strictly from a local staged directory. One hidden layer is read per frame. Long audio is processed in core chunks with context on each side; a frame is kept only when its acoustic centre falls inside the core span, so chunk seams neither duplicate nor drop frames.
Configuration
Encoder
facebook/wav2vec2-xls-r-300m
Configured revision
1a640f3
Layer read
11
Frame representation
1,024-D
Frame hop
20 ms / 320 samples
Receptive field
25 ms / 400 samples
Chunking
20 s core, 2 s left, 2 s right
Implemented in
tierline.features.xlsr
tierline.features.pipeline
tierline.model_registry
The model is staged locally and hash-verified before use.
Configured7 fixed settings in the architecture configuration
Run on real recordingsPCA and int8 quantizer fitted on a separate calibration set, never on the searched audio, 13 October 2026
Evaluatedindex memory, int8 against float32, in a storage benchmark on Evaluation
Make it compact
Preserve a useful signal
PCA reduces the number of features per frame. Quantization stores each value in fewer bits. Together they cut storage substantially.
Why it works this way
A compact representation is what makes a large archive practical on a single local machine.
An incremental PCA basis is fitted once over sampled frames, then reused. Each projected value is stored as a symmetric integer with a per-dimension scale taken from a high absolute percentile of the calibration set. The basis and the quantizer are artifacts with their own recorded provenance.
PROJECTED STORE1,024-D -> 256-D -> int8
Configuration
Projection
sklearn IncrementalPCA, 1,024 -> 256 dimensions
Whitening
off
Calibration frames
100,000
Calibration batch
8,192
Seed
1729
Frame store
int8, symmetric
Calibration percentile
99.9
Implemented in
tierline.features.pca
tierline.features.quantize
tierline.features.store
The PCA basis and quantizer are fitted and validated on real frames before a collection is compressed.
Configured7 fixed settings in the architecture configuration
Run on real recordingsHNSW32_SQ8 and exact Flat indexes built in one permission partition, 13 October 2026
Evaluatedcandidate recall against exact search, in a separate benchmark on Evaluation
Find candidates
A fast first pass
Search only the policy partitions the operator is authorized for. A nearest-neighbour index proposes likely acoustic matches for closer inspection.
Why it works this way
Unauthorized partitions are excluded before candidates are gathered. Material the operator may not search is never loaded, not merely hidden from the result list.
Each authorized policy partition is a separate approximate index over pooled coarse windows. Hits from every authorized partition are merged and truncated globally to K. An exact index is retained per partition so the cost of the approximation can be evaluated later.
Configuration
Candidates (K)
500
Index
HNSW32_SQ8
Graph
M=32, efConstruction=200, efSearch=512
Coarse window
1 s
Coarse hop
0.25 s
Metric
l2
Exact reference index
Flat
Implemented in
tierline.retrieval.windows
tierline.retrieval.faiss_index
tierline.retrieval.search
Approximate search is checked against an exact index of the same windows; Evaluation reports how many of the exact candidates it keeps.
Configured8 fixed settings in the architecture configuration
Run on real recordingsevery search reranked by subsequence DTW, 13 October 2026
Evaluatedthe final ranking, Hit@k on held-out queries, on Evaluation
Align and rerank
Different speeds, compared
Two people rarely say the same phrase at the same speed. Dynamic time warping compares query and candidate frame by frame while allowing timing to stretch, then ranks what is worth a person's attention.
Why it works this way
Rigid timing comparison would miss a match simply because it was spoken more slowly. Ranking still ends with a human listener, not a score.
Coarse hits whose centres fall close together merge into one region; each surviving region expands by a fixed margin, clipped to the authorized interval. Subsequence dynamic time warping consumes the whole query while the matched span may start and end anywhere inside the candidate. Overlapping spans are suppressed per recording.
Configuration
Reranker
subsequence DTW
Frame metric
cosine
Query duration
0.5-2 s
Merge centre distance
1.5 s
Region expansion
-2 s / +2 s, clipped to the authorized interval
Duration ratio
0.67-1.5x
Maximum regions reranked
100
Temporal NMS
IoU 0.5
Implemented in
tierline.retrieval.candidates
tierline.retrieval.dtw
tierline.retrieval.nms
Authorization is rechecked before candidate frames are loaded and again before final spans are returned. Regions are never merged across differently governed intervals.
Configured5 fixed settings in the architecture configuration
Run on real recordingsranked results returned with their sources; audit chain intact, 13 October 2026
Listen and carry forward
A person decides
An authorized reviewer listens, adds context, and judges whether a passage is relevant. Reviewed annotations can be exported into the tools language documentation already uses.
Why it works this way
Acoustic similarity is not meaning. Transcripts, translations and teachings need human authorship and community-defined context.
A ranked result list is paged to a reviewer through a desktop interface or a loopback browser API. Audio is served through an integrity-verified, policy-authorized crop service. Reviewed annotations leave the system only through the documented interchange formats, and every search and export is written to the audit chain.
EXPORTELAN EAF, Praat TextGrid
Configuration
Internal results
50
Review page size
20
Exports
ELAN EAF, Praat TextGrid
Audit log
SHA-256 hash chained
Withdrawal / recovery
external HMAC hash-chained tombstone ledger
Implemented in
tierline.review.service
tierline.audio.access
tierline.export.elan
tierline.export.praat
tierline.web.review_routes
Play, annotate and export are separately authorized. Revoking a policy blocks future access; it cannot recall a copy already delivered.
Architecture version 1.0.0 · source snapshot e3fba160fcf0
Encoder pinned to facebook/wav2vec2-xls-r-300m at revision 1a640f3. Staging resolves it to the full commit and hashes every file: the recorded run of 13 October 2026 used revision 1a640f32ac3e39899438a2931f9924c02f080a54, weights SHA-256 d5e490574712ad0a… (its record).
Status
Built, run, evaluated, in use
How far each part of Tierline has gone. Progress lists what stands at each.
Implemented
Built: source code in the repository.
Run on real recordings
The reference engine run end to end on the example recordings, with its record published: every stage working on real speech.
Evaluated
Measured on a defined test set or in a controlled test: the held-out retrieval evaluation, the system benchmarks, the comparative study and the access-control tests, each with its scope.
In use
Observed in the community pilots: who uses them, what reviewers accepted, how available the hosted pilot was.
Next
Defined and next to run or publish, such as the public-data benchmark protocol.
The reference path
Full configuration matrix
Every value below comes from the architecture configuration, configs/architecture-v1.yaml.
Architecture version
1.0.0
Model input
16,000 Hz mono working derivative per source channel
Encoder
facebook/wav2vec2-xls-r-300m
Transformer layer
11
Frame representation
1024 dimensions every 20 ms; 25 ms receptive field
centers within 1.5 s merged; regions expanded -2/+2 s
Fine reranking
subsequence DTW with cosine frame distance
Fine regions
100
Accepted duration ratio
0.67–1.50× query length
Temporal NMS
IoU 0.50
Ranked depth
50 internal results; 20 per UI page
Query selection length
0.50–2.00 s
00
Ingest & provenance
Implemented (tierline.ingest). FFmpeg/ffprobe read each source file's actual stream layout and create one 16 kHz FLAC derivative per source channel — there is no automatic downmix, so a two-channel field recording with an interviewer on one channel and a speaker on the other yields two independently policy-governed derivatives. Resampling uses FFmpeg's soxr resampler at precision 28.
The source file is copied into the workspace unmodified. Recording and derivative provenance together record the source SHA-256, derivative SHA-256, literal FFmpeg version/build string, and exact conversion command. That makes each transformation auditable and reproducible. An IMPORT event is written to the hash-chained audit log at ingest time.
No denoising, high-pass filtering, or loudness normalization is applied by default. That is a deliberate default (audio.default_denoising: false, etc.): enhancement is optimized for human listening comfort and can attenuate exactly the fine spectral structure some phonemic contrasts depend on.
01
Segmentation — optional, off by default
Two independent optional modules. Neither is enabled by default, and enabling one never excludes material from retrieval — both are navigation and review aids only.
Speaker diarization
Implemented (tierline.segmentation.diarization), wrapping pyannote/speaker-diarization-community-1.[5] Navigation-only; never excludes retrieval candidates. The adapter loads a local, pre-staged model directory only; it is never called against a hosted endpoint.
English-versus-target classification
Implemented (tierline.segmentation.language_filter). Frames are pooled per 1.5-second window (0.5-second hop) as mean-concatenated-with-standard-deviation, giving a 2,048-dimensional vector from 1,024-dimensional XLS-R frames.
Logistic regression
Complete and trainable: StandardScaler + LogisticRegression swept over C ∈ {0.01, 0.1, 1.0, 10.0}, threshold chosen by grid search to maximize English-removal rate while satisfying both the KEEP-recall floor and the configured minimum English-removal gate.
MLP
An alternative head (Linear(2048,256) → GELU → Dropout(0.20) → Linear(256,2)); the logistic path is the one trained end to end.
The KEEP-recall threshold is configured at 0.99 and the English-removal threshold at 0.40. If no threshold satisfies both on the deterministic class-stratified calibration split, training stops with an error rather than returning an ineffective or unsafe classifier.
02
Representation
Implemented (tierline.features.xlsr, tierline.features.pca, tierline.features.quantize). XLS-R runs in float32 PyTorch; weights are loaded strictly from a local staged directory ([3][4]).
Chunked inference: 20s core windows with 2s of context on each side. Frames are kept only if their acoustic center falls inside the core span, so chunk seams neither duplicate nor drop frames.
Projection is sklearn.decomposition.IncrementalPCA, configured to fit once on 100,000 frames sampled in batches of 8,192 (seed 1729), reducing 1024 to 256 dimensions with no whitening. Storage is symmetric int8, with per-dimension scale fixed from the 99.9th absolute percentile of the calibration set.
1,024-D frames project to 256-D via PCA, then quantize to calibrated int8 for storage.
Raw embedding payload only — excludes the HNSW graph, SQLite metadata, and source audio. The public-data protocol’s acceptance limit for it is ≤ 60 MB per hour of audio (Evaluation).
Stored as
Per hour
500 hours
float32, 1,024-dim
737.28 MB
368.64 GB
float16, 1,024-dim
368.64 MB
184.32 GB
float16, PCA to 256
92.16 MB
46.08 GB
int8, PCA to 256 (as shipped)
46.08 MB
23.04 GB
03
Retrieval
Implemented (tierline.retrieval: windows, faiss_index, candidates, dtw, nms, search). Tierline retrieves likely acoustic matches across every recording the requesting operator is authorized to search, for a person to review.
Coarse windowing — why timestamps, not frame indices
1.00s windows at a 0.25s hop. 250ms is not an integer number of 20ms frames, so windows are placed by timestamp rather than a rounded frame count.
Coarse retrieval
Each authorized policy shard is a separate FAISS index. A query searches only the shards the operator's identity is authorized for. Hits from every authorized shard are merged and truncated globally to the top 500, sorted by distance.
The production factory is HNSW32_SQ8 (M=32, efConstruction=200, efSearch=512);[6][13] an exact Flat index is retained per shard for evaluating how much recall the approximation costs.[15]
FAISS training-state handling is explicit:HNSW32_SQ8 includes a scalar-quantizer component that can require training before vectors are added. The production builder checks index.is_trained and calls index.train() when required.
Candidate consolidation
Coarse hits within 1.5s merge into one region; each surviving region expands 2s/2s, clipped to policy bounds. At most 100 regions proceed to reranking.
Fine reranking
Subsequence DTW: the query is fully consumed; the matched span may start and end anywhere in the candidate.
Subsequence dynamic time warping is implemented directly in NumPy. Frame cost is cosine distance; there is no Sakoe-Chiba global band. A match is accepted only if the matched span's duration is 0.67–1.50× the query's frame count.
Temporal non-maximum suppression at IoU >= 0.50 removes redundant overlapping results per recording.
Results are sorted by DTW cost, then coarse distance, then a deterministic tie-break, truncated to 50 internal results (20 per UI page). A selectable query passage must be 0.50–2.00 seconds long and must lie entirely inside one policy interval; using a stored recording as a query requires both search and play permission on that interval.
Every search is persisted with a hash of the full configuration that produced it and its wall-clock latency, and logged to the audit chain with its result count.
04
Phone & spelling — optional, off by default
Phone-level indexing
Implemented (tierline.phones.allosaurus_adapter), a subprocess adapter around Allosaurus[7] kept deliberately separable because Allosaurus is GPL-3.0-licensed code, distinct from Tierline's own Apache-2.0 source. It requests the uni2005 model with the top 5 phone candidates per position and timestamps. Nothing here is displayed to a user as a transcript.
Articulatory-feature-weighted distance
Implemented (tierline.phones.panphon_distance), using PanPhon's mapping of IPA segments to articulatory features.[8] Substitution cost is the mean absolute difference across PanPhon's ternary feature vector, normalized to [0,1]; insertion and deletion each default to cost 1.0. A uniform-edit-distance baseline is implemented alongside it for comparison.
Spelling query
Implemented (tierline.phones.spelling). Given a partner-supplied grapheme-to-phone rule set (NFC-normalized, longest-match, explicit exceptions), it compiles written queries into phone sequences with a pronunciation beam of 32.
05
Alignment & offline deployment
Alignment — optional
Implemented (tierline.alignment.ctc_adapter): a generic Viterbi forced-alignment algorithm over CTC-style log-probabilities and a known token sequence. It is model- and tokenizer-agnostic by construction and is not used by, or required for, query-by-example.
Export
Implemented and round-trip-tested (tierline.export.elan, tierline.export.praat): ELAN .eaf and Praat TextGrid. Every export writes a companion provenance manifest and an audit event.
Offline deployment
Build stage
Model weights may be downloaded during a controlled staging step, subject to each model's own license and access terms.
Deployment package
Implemented (scripts/build_offline_bundle.py): assembles application code, a local wheelhouse, an install script, and a checksum manifest.
Process-level socket denial
Executed: scripts/run_offline_suite.py denies outbound sockets inside the test process, and its self-test confirms the guard blocks an external destination. It works at the process level; kernel-level isolation is the next row.
Kernel namespace + strace E2E
Implemented in source: scripts/run_offline_e2e_linux.sh uses Linux unshare -n with strace and fails on attempted network syscalls during a staged end-to-end run. Next to run.
Optional ONNX deployment
Implemented (tierline.deployment.onnx): float and dynamic-int8 ONNX export with a validation hook. PyTorch float32 remains the reference encoder until a converted model meets the same retrieval-quality thresholds.
Scope
What stays with people
Meaning
A result is a passage that sounds similar. Whether it says the same thing, and what it means, is decided by the person who reviews it.
Transcription
Search works on sound. Alignment uses a transcript you already have, and phone-level output is never labelled as a transcript.
The decision
The system narrows what a person has to listen to. Their review, not the ranking, is what gets exported.
Training
The policy engine's train action is off unless it is granted separately, per policy.
