Compare commits

...
3 changed files with 240 additions and 13 deletions
@@ -2,9 +2,9 @@
title: "Adverse Event Detection - Tool Parameter Reference"
task: ""
lineage_type: import
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/e2520a96/skills/tooluniverse-adverse-event-detection/TOOL_REFERENCE.md
upstream_sha: e2520a96
imported_at: 2026-06-26
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/4d14233e/skills/tooluniverse-adverse-event-detection/TOOL_REFERENCE.md
upstream_sha: 4d14233e
imported_at: 2026-08-18
prompt_class: unknown
upstream_changes: accepted
author: upstream
@@ -29,7 +29,7 @@ Verified parameter names, response formats, and fallback chains for all tools us
| `FAERS_count_reportercountry_by_drug_event` | `medicinalproduct` (REQUIRED), `patientsex`, `patientagegroup`, `serious` | Returns [{term: "US"/"GB"/..., count}] |
| `FAERS_search_adverse_event_reports` | `medicinalproduct`, `limit` (max 100), `skip` | Returns individual case reports with patient/drug/reaction data |
| `FAERS_search_reports_by_drug_and_reaction` | `medicinalproduct` (REQUIRED), `reactionmeddrapt` (REQUIRED), `limit`, `skip`, `patientsex`, `serious` | Returns individual reports filtered by specific reaction |
| `FAERS_search_serious_reports_by_drug` | `medicinalproduct` (REQUIRED), `seriousnessdeath`, `seriousnesshospitalization`, `seriousnesslifethreatening`, `seriousnessdisabling`, `limit` | Returns serious event reports |
| `FAERS_search_serious_reports_by_drug` | `medicinalproduct` (REQUIRED), `serious`, `seriousnessdeath`, `seriousnesshospitalization`, `seriousnesslifethreatening`, `seriousnessdisabling`, `limit` | Case reports. Despite the name it returns serious AND non-serious reports unless you pass `serious='Yes'` or one of the `seriousness*` criteria |
## FAERS Analytics Tools (operation-based)
@@ -1,8 +1,8 @@
---
lineage_type: import
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/cfd26718/skills/tooluniverse-biomedical-fact-lookup/SKILL.md
upstream_sha: cfd26718
imported_at: 2026-08-08
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/4d14233e/skills/tooluniverse-biomedical-fact-lookup/SKILL.md
upstream_sha: 4d14233e
imported_at: 2026-08-18
prompt_class: unknown
upstream_changes: accepted
name: tooluniverse-biomedical-fact-lookup
@@ -14,6 +14,36 @@ when_to_use: "A factual biomedical question has a single database-checkable answ
Factual biomedical questions — "which gene is in set X", "which gene is associated with disease Y according to DisGeNet", "which gene has a TF binding site per GTRD" — have an authoritative answer in a public database. Guessing from memory is unreliable (≈chance on niche annotations); the matching ToolUniverse tool returns the ground truth.
## Six traps that produce a confidently wrong answer
Each was observed producing a wrong answer on a real question. Check them before
answering; the detail for each is further down.
1. **"Highest p-value" in GWAS means most significant** — the *smallest* number.
Read literally it picks the study's weakest hit (`rs2476491` at 1e-06 instead
of `rs7775055-G` at 3e-174).
2. **A window of "N bp upstream plus M bp downstream" spans N+M+1 bases** — the
anchor counts. 100 either side of a TSS is 201 nt, not 200. Check the length
you got against the length you asked for.
3. **HPA subcellular locations pool every cell line the antibody was tested in.**
Report both `main_locations` and `additional_locations`, but when the question
names a line, treat them as candidates and drop annotations belonging to
another line — reciting all five is as wrong as reciting one.
4. **Allen Brain: answer the leaf structure, not its parent.** The atlas colours
the specific structure and gives the parent a different colour, so
"Hypothalamus" is wrong where `Lateral preoptic area` (#F2483B) is right.
`AllenBrain_search_structures` returns `color_hex_triplet`.
5. **SCREEN's `is_proximal` is unreliable; filter on `element_type`**`PLS`
and `pELS` are TSS-proximal, `dELS` distal.
6. **Derived scores are release-pinned.** gnomAD pLI for APOC2 is 0.047 in r4
and 0.402 in r2.1 — an 8.5x difference for the same gene. Set the release the
question names and say which you used.
## RULE ZERO: Look it up, never guess
If a question names a database, a gene set, or any annotation that lives in a database, you MUST query the tool before answering. Answering a "according to <database>" question from memory is a failure mode — these annotations (predicted miRNA targets, ChIP-seq binding, curated gene sets, disease associations) are exactly what models hallucinate. A tool-verified answer beats any recalled fact.
@@ -43,9 +73,59 @@ Most of these questions are MCQ with an "Insufficient information to answer the
| **drug / compound** target, MoA, approval | `ChEMBL_*`, `OpenFDA_*`, `GtoPdb_*`, `PubChem_*` | resolve drug, query the relation |
| **which drug for this patient** (clinical vignette naming a modifier) | `FDA_*_by_drug_name` — pick the section by modifier | see "Drug choice for a described patient" below |
| **protein** function / domain / sequence | `UniProt_*` | resolve accession, read annotation |
| **protein localization / expression** "according to the Human Protein Atlas" | `HPA_get_subcellular_location`, `HPA_get_rna_expression_by_source`, `HPA_get_comprehensive_gene_details_by_ensembl_id` | pass the gene symbol — an **antibody ID such as `HPA073143` also works** and resolves to its target gene. **Report main *and* additional locations** — see below |
| **brain region** in the Allen Mouse/Human Brain Atlas | `AllenBrain_search_structures` (`name` or `acronym`), `AllenBrain_get_structure` | reference-atlas regions are **colour-coded**: the result carries `color_hex_triplet`, so "the region shown in red" is answerable — see below |
| **regulatory element / cCRE** near a gene (ENCODE SCREEN) | `SCREEN_search_cCREs_by_region` | filter on `element_type` (**PLS** and **pELS** are TSS-proximal, **dELS** distal) and read `dnase_zscore` |
| **which variant is at / overlaps** a genomic region (ClinVar) | `ClinVar_search_by_region` | **not** `ClinVar_search_variants` — Entrez matches a variant's START, so a narrow window misses a CNV that spans the region but begins megabases upstream. Returns true overlaps, smallest span first |
| **how many peaks / which datasets** for a TF experiment (ReMap) | `ReMap_list_datasets_for_target` | one GEO series can hold several datasets (GSE23852/FOXA1 = 2, with 60,158 and 67,736 peaks) — report them separately unless a total is asked for; `count_peaks: true` to get counts |
| **protein interaction partners** (STRING) | `STRING_get_protein_interactions` | read the **`partner`** field, not `preferredName_B`: edges are ordered A/B by internal ID, so the queried protein sits in column A on about half of them |
When unsure which tool wraps a database, search the catalog by the *relation* (e.g. "gene disease association", "gene set members"), not the brand name — ToolUniverse usually already has it.
### Allen Brain Atlas — answer with the specific structure, not its parent
The reference atlas colours every structure, and `AllenBrain_search_structures`
returns `color_hex_triplet`. A question naming a colour ("which region is
annotated in red at coronal position 181") is asking which **leaf structure**
carries that colour, e.g. `Lateral preoptic area` = `#F2483B`.
Answering with the enclosing region ("Hypothalamus") is wrong even though it
contains the right area: the atlas colours the specific structure, and the
parent has its own different colour. Search by name or acronym, compare
`color_hex_triplet`, and give the structure whose colour matches. Note the same
acronym can return several rows (hemisphere-specific and ontology-version
entries) with different colours — prefer the row whose `name` matches the
question's wording.
### Human Protein Atlas — report both location fields
`HPA_get_subcellular_location` splits its answer in two, and the split is not
significance ranking:
```
main_locations : ['Nucleoplasm']
additional_locations : ['Primary cilium', ..., 'Cytosol']
```
A question asking "what localization does this antibody show" wants the
locations HPA reports, which is **both lists** — answering from `main_locations`
alone drops real localizations and is a common way to be half-right (e.g.
answering "Nucleoplasm" where HPA reports "Nucleoplasm, Cytosol"). Use
`location_summary`, which already joins them, or read both fields.
Two further cautions:
- **Locations aggregate over cell lines.** HPA pools immunofluorescence across
every line an antibody was tested in. If the question names one line (HEK293,
U-2 OS), treat the list as the candidate set and say which line you are
reporting for, rather than implying the aggregate is line-specific.
- **Per-cell-type RNA values are only published for enriched cell types.** HPA's
machine-readable fields give specificity plus nTPM/nCPM for the cell types a
gene is enriched in; a value for an arbitrary cell type is not exposed. If a
question asks for one that is absent, say so instead of substituting the
nearest available number — those differ by an order of magnitude.
## MSigDB set-name conventions (the most common LAB-Bench pattern)
ToolUniverse's `MSigDB_*` tools cover several collections that LAB-Bench questions are built from. Get the set name right:
@@ -127,6 +207,42 @@ Try the `MP_<PHENOTYPE>` MSigDB set first (above): it answers in one call per op
## Computational procedures (when the answer is COMPUTED, not looked up)
### GWAS "highest p-value" means most significant
In GWAS writing, "the highest p-value", "the top hit" and "the strongest
association" all mean the **most significant** result — the *smallest* numeric
p-value. Read literally, "highest" picks the weakest association in the study
and is almost never what was meant.
For GCST005528 the literal reading gives `rs2476491-?` at p = 1e-06; the
intended answer is `rs7775055-G` at p = 3e-174.
Sort ascending by p-value and report that hit. If the phrasing genuinely could
go either way, give the most significant one and say in a clause that the
numerically largest p-value is a different SNP — do not silently pick the
literal reading.
### Genomic windows — count the anchor base
A window described as "N bp upstream plus M bp downstream of X" spans
**N + M + 1** bases, because the anchor base X is itself included. Asking for
100 up and 100 down around a TSS is 201 nt, not 200. Off-by-one here is the
single most common way a sequence answer is wrong while looking right.
The same care applies to the coordinate convention of whichever tool you call:
| convention | span of `start`..`end` | used by |
|---|---|---|
| 1-based inclusive | `end - start + 1` | Ensembl `region`, UCSC browser text, IGV, samtools |
| 0-based half-open | `end - start` | UCSC REST API, BED |
`UCSC_get_sequence` takes a written locus via `region` (1-based inclusive) or
explicit `chrom`/`start`/`end` with `coordinate_system`; it echoes
`region_1based` and `requested_length` so the span is checkable. **Always check
the returned length against what the question asked for** before answering — a
sequence of the wrong length is wrong even when every base you kept is right.
Any question with a **single deterministic numeric/combinatorial answer** must be obtained by **RUNNING code**, never by estimating or doing it in your head. This covers sequence questions (ORF counts, restriction fragments/sizes, GC content, translation) **and** any other exactly-computable question — e.g. **genetics segregation / Mendelian or polyploid gamete ratios, combinatorial probabilities, stoichiometry, dosage/PK arithmetic, counting problems**. Mental arithmetic on these is the #1 avoidable error: the model reliably mis-counts or mis-multiplies. If a question reduces to "enumerate the cases / multiply the probabilities / count the objects", **write a short Python snippet, execute it, and report exactly what it returns** — even when the topic looks like a biology "reasoning" question, if the answer is a definite number, compute it rather than reason it out. Match the question's wording for conventions (which strand; linear vs circular; which cross/segregation model) and **state the convention you used** so the answer is auditable.
**Final-answer discipline (avoid "computed right, answered wrong").** After the code returns the value, map it back to the option letters **carefully and explicitly**: quote the computed value, then find the option that matches it exactly (for a set of fragment sizes, match the whole multiset; for a count, match the integer). A surprising number of misses are cases where the computation was correct but the wrong letter was selected — do not let this happen; re-read each option against the computed result before emitting `[ANSWER]`.
@@ -219,5 +335,19 @@ Interpretation: report the **exact value the code returns** (ORF count; fragment
## Limitations (honest)
- **Key-gated sources**: `DisGeNET_*` and OMIM tools need `DISGENET_API_KEY` / OMIM key. Without a key, fall back to `OpenTargets_*` / `MyDisease_*` (keyless) and state the source used. If no keyless source can answer and the question is database-specific, this is a genuine "Insufficient information" case — say so.
- **Release mismatch**: a tool's snapshot of a database may differ slightly from the exact release a question cites; report the source and version when it matters.
- **Release mismatch**: a tool's snapshot of a database may differ from the exact
release a question cites — and for some quantities the difference is not
slight. Derived scores get recomputed between releases, so the *same gene* can
differ by an order of magnitude. gnomAD pLI, via `gnomad_get_constraint`:
| gene | gnomAD r4 | gnomAD r2.1 |
|---|---|---|
| APOC2 | 0.046875 | 0.401638 |
| APOC1 | 0.086323 | 0.216848 |
Where a tool exposes a `dataset`/release parameter, set it to the release the
question names and **say which release you used**. If the question names one
the tool cannot serve, report the release you did use rather than presenting
the number as if it were release-independent — a bare pLI value is ambiguous
by a factor of eight here.
- This skill grounds *factual* lookups. For computing over user data files, use the data-analysis router skills instead.
@@ -1,8 +1,8 @@
---
lineage_type: import
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/cfd26718/skills/tooluniverse-phylogenetics/SKILL.md
upstream_sha: cfd26718
imported_at: 2026-08-08
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/4d14233e/skills/tooluniverse-phylogenetics/SKILL.md
upstream_sha: 4d14233e
imported_at: 2026-08-18
prompt_class: catalogue
upstream_changes: accepted
name: tooluniverse-phylogenetics
@@ -12,6 +12,45 @@ disable-model-invocation: true
# Phylogenetics and Sequence Analysis
## Four traps that produce a confidently wrong number
Each of these was observed producing a wrong answer *while the correct guidance
was already present further down this file*. Check them before you answer.
1. **PhyKIT prints more than one column, and for `saturation` the two
conventions disagree — state which you used.** `phykit saturation` prints
`saturation <TAB> |saturation-1|`. Its own `--help` is explicit: *"The first
value is the saturation value and the second column is the absolute value of
saturation minus 1."* But several published analyses (and some reference
answers derived from them) report the **second** column as "the saturation
value". The two always sum to 1.0000, which is the tell that you may be
looking at the wrong one — on the fungal scogs set the medians are 0.39
(col 1) and 0.61 (col 2).
So: **follow phykit and use column 1** unless the question or source defines
saturation the other way, and say in your answer which column you read. Do
not silently pick the one that looks closer to an expected number.
`treeness_over_rcv` has no such ambiguity: it gives
`ratio <TAB> treeness <TAB> RCV` and the ratio is first.
2. **"Gap percentage" means the fraction of alignment COLUMNS containing at
least one gap**, not the fraction of residues that are gaps. On the fungal
scogs set the residue definition maxes out at 0.556, so a ">70% gaps" filter
selects **nothing** and the question looks unanswerable; by columns, three
orthologs qualify (max 0.783).
3. **`treeness_over_rcv` and `rcv` take the UNTRIMMED `.faa.mafft`**, while
`saturation` takes the trimmed `.clipkit`. RCV measures variability across
columns, so trimming changes it: median 0.2683 untrimmed against 0.3050
trimmed, and among >70%-gap genes the maximum is 0.2572 untrimmed against
0.4174 trimmed.
4. **Never loop PhyKIT per file.** `phykit_batch_analysis` is parallel and does
~250 trees in about 35 seconds; a shell loop takes ~9 minutes and runs out of
turns mid-way, producing no answer at all. It also selects the right column
for every function, which removes trap 1 entirely.
## RULE ZERO — Check for pre-computed results FIRST
Before following any instruction below, scan the data folder for:
@@ -182,8 +221,9 @@ set (249 orthologs, canonical shipped files):
```
median treeness/RCV untrimmed .faa.mafft = 0.2683 trimmed .clipkit = 0.3050
max treeness/RCV (>70% gap genes)
untrimmed .faa.mafft = 0.1866 trimmed .clipkit = 0.4205
max treeness/RCV (over the 3 genes with >70% gapped columns:
1260807at2759 0.0861, 1567796at2759 0.1866, 939345at2759 0.2572)
untrimmed .faa.mafft = 0.2572 trimmed .clipkit = 0.4174
```
Plain `treeness` needs no alignment and is unaffected — it reproduces exactly
@@ -351,6 +391,63 @@ tu run phykit_batch_analysis '{"operation":"gap_percentage","directory":"./align
```
Do NOT run phykit manually in a loop — the tool handles all files and returns correct summary statistics.
**The batch tool is parallel: ~250 trees finish in about 35 seconds.** A per-tree
shell loop takes ~9 minutes for the same work and is the single most common way
these questions end with no answer at all — the run hits its turn or time budget
mid-loop and reports "I'll report when it finishes" instead of a number. If you
find yourself writing `for f in *.treefile`, stop and call the batch tool.
Supported `function` values include `treeness`, `saturation`, `dvmc`,
`long_branch_score`, `total_tree_length`, `parsimony_informative`,
`treeness_over_rcv` (alias `toverr`). `dvmc` and `long_branch_score` are
covered — you do not need to loop for those.
**Two-group comparisons (Mann-Whitney U, differences of medians).** Questions
comparing fungi against animals need one batch call per group, then the test on
the two value lists — not a per-tree loop over both groups:
```bash
tu run phykit_batch_analysis '{"operation":"batch","function":"dvmc","directory":"<fungi>","extension":".treefile"}'
tu run phykit_batch_analysis '{"operation":"batch","function":"dvmc","directory":"<animals>","extension":".treefile"}'
# then scipy.stats.mannwhitneyu(fungi_values, animal_values)
```
Ask for `values` in the result when you need the full list for a test; the batch
tool returns them for sets up to 50 and summary statistics always. For larger
sets, compute the statistic from the per-group summaries the tool returns rather
than re-deriving every value by hand.
### PhyKIT column conventions — take the right one
Several PhyKIT subcommands print more than one number per file, and the value
the question wants is usually not the first:
| subcommand | prints | the value asked for |
|---|---|---|
| `saturation` | `saturation <TAB> \|saturation-1\|` | **column 1** per phykit's docs; some sources report col 2 — say which you used |
| `treeness_over_rcv` | `treeness/RCV <TAB> treeness <TAB> RCV` | **column 1**, the ratio |
| `parsimony_informative_sites` | `n_pi <TAB> n_total <TAB> %PIS` | column 3 for a percentage |
Taking `saturation`'s first column gives exactly `1 - answer`: a fungal set
whose saturation is 0.6146 reports 0.3854 instead, and the two sum to 1.0000,
which is the tell. `phykit_batch_analysis` already selects the right column for
each function — another reason to call it rather than run the CLI yourself.
### Commit the value you computed
Two failures in this benchmark came from computing the right number and then
answering a different one:
- a tree-length ratio computed as **2.1775**, then answered as 1.9 after
re-reading "paired orthologs";
- an average treeness that listed **19** among the alternatives, then committed 10.
When a question is ambiguous, compute the reading you judge most literal, state
the alternative in one clause, and **answer with the value you actually
computed**. Do not replace a computed result with a re-derived one at the last
step — if two readings are both defensible, give the computed number first and
name the other, rather than silently switching.
### PhyKIT column-position cheat sheet (parse output carefully)
When parsing PhyKIT stdout for batch metrics, the **column you want** depends on the metric: