A systematic review of the analytical techniques used across shark, ray, and chimaera research.
Elasmobranch science (sharks, rays, and chimaeras) spans dozens of methods, from age-and-growth histology to acoustic telemetry to environmental DNA. No one has systematically mapped which techniques are used, where, by whom, and how that has changed over 75 years. This project does exactly that.
We assembled a near-complete corpus of elasmobranch literature (~31,650 papers, 1950–present), acquired the full-text PDFs, and ran an automated pipeline that reads each paper and records what analytical approaches, ecosystems, pressures, gear, and study locations it involves. The result is a queryable database of the field's methodological history, plus author, geography, and citation-impact enrichment layered on top.
The work began as a panel session at the European Elasmobranch Association (EEA) Conference 2025 in Rotterdam, and is intended to become a living database, updated annually.
New here? Start with the results index — live links to the interactive Author Atlas, the collaboration network, key figures, and the validation report.
Helping with downloads? The publisher & journal index lists every source with papers still missing, each linking to a pre-filtered view of the download helper.
Want the tour? The slide deck with speaker notes walks through the whole project in 17 slides, each with the notes spoken alongside it, so the argument reads end to end without the talk.
| Metric | Value |
|---|---|
| Papers catalogued | ~31,770 |
| PDF files on disk | ~21,250 |
| Extraction columns | ~1,740 total; 127 binary across the 6 core schemas |
| Evidence rows (audit trail) | ~271,400 across ~18,600 papers |
| Unique authors (OpenAlex) | ~30,300 |
| Species columns | ~1,300 |
| Techniques in taxonomy | 215 |
| Conference abstracts (grey literature) | ~23,400, of which ~7,800 chondrichthyan |
Rounded, and growing with each monthly sync. Exact current figures are generated by python3 scripts/report_corpus_stats.py into outputs/corpus_stats.json; the full as-built pipeline and the canonical table live in the pipeline overview.
The project is an as-built data pipeline. Each stage has a dedicated script; the arrows show the flow from a bare citation to an answerable question.
The pipeline at a glance (PDF, SVG). The stages below expand each step.
1. Acquire. scripts/acquire_cascade.py is the unified acquisition entry point. Given a paper, it tries each source in turn (open-access resolvers, DOI recovery, the Biodiversity Heritage Library, Unpaywall) until it finds the PDF. finalize_acquisitions() then chains ingest → verified-delete → incremental extraction so a newly-found paper flows straight into the database. Monthly top-ups come from scripts/sync_shark_references.py (an anacron job that crawls Shark-References, downloads new papers, and re-extracts).
This unifies what used to be dozens of per-source download scripts. If you find older
download_*/ manual-download / Sci-Hub / Tor guides underdocs/database/, they're superseded by the cascade — see the design spec atdocs/superpowers/specs/2026-07-07-cascade-finalize-ingest-extract-design.md.
2. OCR. Scanned and historical PDFs are made text-searchable so the extractor can read them: screen (find_non_extractable_pdfs.py) → OCR (ocr_library.py, per-file language) → verify. See the OCR pipeline guide.
3. Extract. scripts/extract_schema_columns.py reads each PDF's text and scores it against 127 binary columns across the 6 core schemas (discipline, ecosystem, pressure, gear, impact, ocean basin), plus 43 sub-basin, 1,308 species, and 215 analytical-technique columns, using keyword-plus-context matching with section-weighted scoring. Every classification is backed by a quote in the evidence table. See extraction logic.
4. Enrich. Authors (OpenAlex + NamSor for gender/origin), citation impact (Altmetric), journal quality (SCImago), and geography are joined on. See the enrichment proposals under docs/schema_proposals/.
5. Analyse. The scripts/viz_*.R suite produces the temporal, spatial, and cross-cutting figures; the Author Atlas maps the collaboration network.
6. Ask. A local retrieval-augmented-generation (RAG) prototype (BGE-small embeddings + FAISS + Ollama, all CPU-local and private) lets you query the corpus in natural language. See RAG prototype status.
- Biology, Life History, & Health (BIO) — age/growth, reproduction, physiology, anatomy, disease
- Behaviour & Sensory Ecology (BEH) — behavioural observation, social structure, sensory biology
- Trophic & Community Ecology (TRO) — diet analysis, trophic position, food webs
- Genetics, Genomics, & eDNA (GEN) — population genetics, phylogenetics, genomics, environmental DNA
- Movement, Space Use, & Habitat Modelling (MOV) — telemetry, movement models, species distribution models
- Fisheries, Stock Assessment, & Management (FISH) — stock assessment, CPUE standardisation, bycatch
- Conservation Policy & Human Dimensions (CON) — IUCN assessments, policy, human-wildlife conflict
- Data Science & Integrative Methods (DATA) — statistical frameworks, machine learning, reproducibility
Six schema categories extract binary classifications from each PDF. Fully implemented and run across the corpus. Column counts below are as built in the parquet, not as originally proposed.
| Category | Prefix | Columns | Proposal |
|---|---|---|---|
| Discipline | d_ |
19 | discipline_proposal.md |
| Ecosystem | eco_ |
23 | ecosystem_component_proposal.md |
| Pressure / Threat | pr_ |
26 | pressure_proposal.md |
| Fishing Gear | gear_ |
29 | gear_proposal.md |
| Impact / Response | imp_ |
25 (21 binary + 4 metadata) | impact_proposal.md |
| Ocean Basin | b_ |
9 | ocean_basin_proposal.md |
Beyond these six: sb_ sub-basin (43), sp_ species (1,308), a_ analytical technique (215), geo_ geography (19), ob_ ocean basin from the geographic pipeline (9), depth_ (3).
Pipeline internals: extraction_logic.md. Known false-positive patterns: extraction_quality_issues.md. Column-design discussion: Issue #2.
docs/ Documentation (see docs/readme.md for the index)
core/ Master plan & current status
schema_proposals/ Extraction column design + logic (the analytical core)
database/ Schema, extraction, acquisition, OCR
superpowers/ Current design specs (cascade, RAG, validation loop)
integrations/ Sharkipedia, MegaMove, Altmetric
network_atlas/ Deployed interactive Author Atlas
scripts/ Acquisition, OCR, extraction, enrichment, viz, RAG
database/ SQLite databases (technique taxonomy + tracking)
outputs/ Generated data: enriched parquet, analysis CSVs, figures, validation XLSX
data/ Input & integration data (Sharkipedia, SCImago)
Phase: validation & analysis (2026). Acquisition, OCR, schema extraction, and author/citation/geography enrichment are complete and running on a monthly cycle. Current focus:
- Validation — an automated loop scores the rule-based extractor against human-reviewed and LLM-oracle labels; the first rule-improvement round is complete (report). The community validation UI is live: search for your papers and review the extraction.
- Analysis — temporal trends, discipline × pressure heatmaps, geographic and parachute-science dashboards, species × discipline gaps.
- Database query interface — natural-language querying of the corpus, running over the full index with schema pre-filters. Its retrieval quality has not yet been measured; that work is scoped in the retrieval evaluation design and should land before the interface is opened to external users.
- Next — methods manuscript and public database release.
Known open gaps are listed in the pipeline overview.
Contributions from the elasmobranch research community are welcome:
- Validate extraction results — search for your papers on the validation site and check false positives/negatives; each correction opens a pull request for the schema lead to review.
- Review schema columns — suggest rule changes for missed or over-called techniques.
- Share missing papers — contribute PDFs via Shark-References.
See CONTRIBUTING.md.
Dedman, S., Tiktak, G., et al. (2025). Elasmobranch Analytical Methods Review:
A systematic extraction of analytical techniques from the elasmobranch
literature (1950–2025). European Elasmobranch Association Conference 2025,
Rotterdam, Netherlands. https://github.com/SimonDedman/elasmo_analyses
Leads: Dr. Simon Dedman (simondedman@gmail.com, GitHub) & Dr. Guuske Tiktak.
Collaborators: David Ruiz Garcia (Mediterranean fisheries; schema proposals), David Schiffman (citation/social analysis), Elena Fernández Corredor (Mediterranean trophic review), Jürgen Pollerspöck & Nico Straube (Shark-References).
Licensed under CC BY 4.0.
Builds on the work of thousands of researchers worldwide, the Shark-References literature repository, Sharkipedia trait data, and the EEA and AES conference communities.
Last updated: 2026-07-21