Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0b7324d180 | ||
|
|
ca8242906e | ||
|
|
dcb71b5f48 |
+4
-4
@@ -2,9 +2,9 @@
|
||||
title: "Adverse Event Detection - Tool Parameter Reference"
|
||||
task: ""
|
||||
lineage_type: import
|
||||
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/e2520a96/skills/tooluniverse-adverse-event-detection/TOOL_REFERENCE.md
|
||||
upstream_sha: e2520a96
|
||||
imported_at: 2026-06-26
|
||||
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/4d14233e/skills/tooluniverse-adverse-event-detection/TOOL_REFERENCE.md
|
||||
upstream_sha: 4d14233e
|
||||
imported_at: 2026-08-18
|
||||
prompt_class: unknown
|
||||
upstream_changes: accepted
|
||||
author: upstream
|
||||
@@ -29,7 +29,7 @@ Verified parameter names, response formats, and fallback chains for all tools us
|
||||
| `FAERS_count_reportercountry_by_drug_event` | `medicinalproduct` (REQUIRED), `patientsex`, `patientagegroup`, `serious` | Returns [{term: "US"/"GB"/..., count}] |
|
||||
| `FAERS_search_adverse_event_reports` | `medicinalproduct`, `limit` (max 100), `skip` | Returns individual case reports with patient/drug/reaction data |
|
||||
| `FAERS_search_reports_by_drug_and_reaction` | `medicinalproduct` (REQUIRED), `reactionmeddrapt` (REQUIRED), `limit`, `skip`, `patientsex`, `serious` | Returns individual reports filtered by specific reaction |
|
||||
| `FAERS_search_serious_reports_by_drug` | `medicinalproduct` (REQUIRED), `seriousnessdeath`, `seriousnesshospitalization`, `seriousnesslifethreatening`, `seriousnessdisabling`, `limit` | Returns serious event reports |
|
||||
| `FAERS_search_serious_reports_by_drug` | `medicinalproduct` (REQUIRED), `serious`, `seriousnessdeath`, `seriousnesshospitalization`, `seriousnesslifethreatening`, `seriousnessdisabling`, `limit` | Case reports. Despite the name it returns serious AND non-serious reports unless you pass `serious='Yes'` or one of the `seriousness*` criteria |
|
||||
|
||||
## FAERS Analytics Tools (operation-based)
|
||||
|
||||
|
||||
+134
-4
@@ -1,8 +1,8 @@
|
||||
---
|
||||
lineage_type: import
|
||||
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/cfd26718/skills/tooluniverse-biomedical-fact-lookup/SKILL.md
|
||||
upstream_sha: cfd26718
|
||||
imported_at: 2026-08-08
|
||||
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/4d14233e/skills/tooluniverse-biomedical-fact-lookup/SKILL.md
|
||||
upstream_sha: 4d14233e
|
||||
imported_at: 2026-08-18
|
||||
prompt_class: unknown
|
||||
upstream_changes: accepted
|
||||
name: tooluniverse-biomedical-fact-lookup
|
||||
@@ -14,6 +14,36 @@ when_to_use: "A factual biomedical question has a single database-checkable answ
|
||||
|
||||
Factual biomedical questions — "which gene is in set X", "which gene is associated with disease Y according to DisGeNet", "which gene has a TF binding site per GTRD" — have an authoritative answer in a public database. Guessing from memory is unreliable (≈chance on niche annotations); the matching ToolUniverse tool returns the ground truth.
|
||||
|
||||
## Six traps that produce a confidently wrong answer
|
||||
|
||||
Each was observed producing a wrong answer on a real question. Check them before
|
||||
answering; the detail for each is further down.
|
||||
|
||||
1. **"Highest p-value" in GWAS means most significant** — the *smallest* number.
|
||||
Read literally it picks the study's weakest hit (`rs2476491` at 1e-06 instead
|
||||
of `rs7775055-G` at 3e-174).
|
||||
|
||||
2. **A window of "N bp upstream plus M bp downstream" spans N+M+1 bases** — the
|
||||
anchor counts. 100 either side of a TSS is 201 nt, not 200. Check the length
|
||||
you got against the length you asked for.
|
||||
|
||||
3. **HPA subcellular locations pool every cell line the antibody was tested in.**
|
||||
Report both `main_locations` and `additional_locations`, but when the question
|
||||
names a line, treat them as candidates and drop annotations belonging to
|
||||
another line — reciting all five is as wrong as reciting one.
|
||||
|
||||
4. **Allen Brain: answer the leaf structure, not its parent.** The atlas colours
|
||||
the specific structure and gives the parent a different colour, so
|
||||
"Hypothalamus" is wrong where `Lateral preoptic area` (#F2483B) is right.
|
||||
`AllenBrain_search_structures` returns `color_hex_triplet`.
|
||||
|
||||
5. **SCREEN's `is_proximal` is unreliable; filter on `element_type`** — `PLS`
|
||||
and `pELS` are TSS-proximal, `dELS` distal.
|
||||
|
||||
6. **Derived scores are release-pinned.** gnomAD pLI for APOC2 is 0.047 in r4
|
||||
and 0.402 in r2.1 — an 8.5x difference for the same gene. Set the release the
|
||||
question names and say which you used.
|
||||
|
||||
## RULE ZERO: Look it up, never guess
|
||||
|
||||
If a question names a database, a gene set, or any annotation that lives in a database, you MUST query the tool before answering. Answering a "according to <database>" question from memory is a failure mode — these annotations (predicted miRNA targets, ChIP-seq binding, curated gene sets, disease associations) are exactly what models hallucinate. A tool-verified answer beats any recalled fact.
|
||||
@@ -43,9 +73,59 @@ Most of these questions are MCQ with an "Insufficient information to answer the
|
||||
| **drug / compound** target, MoA, approval | `ChEMBL_*`, `OpenFDA_*`, `GtoPdb_*`, `PubChem_*` | resolve drug, query the relation |
|
||||
| **which drug for this patient** (clinical vignette naming a modifier) | `FDA_*_by_drug_name` — pick the section by modifier | see "Drug choice for a described patient" below |
|
||||
| **protein** function / domain / sequence | `UniProt_*` | resolve accession, read annotation |
|
||||
| **protein localization / expression** "according to the Human Protein Atlas" | `HPA_get_subcellular_location`, `HPA_get_rna_expression_by_source`, `HPA_get_comprehensive_gene_details_by_ensembl_id` | pass the gene symbol — an **antibody ID such as `HPA073143` also works** and resolves to its target gene. **Report main *and* additional locations** — see below |
|
||||
|
||||
| **brain region** in the Allen Mouse/Human Brain Atlas | `AllenBrain_search_structures` (`name` or `acronym`), `AllenBrain_get_structure` | reference-atlas regions are **colour-coded**: the result carries `color_hex_triplet`, so "the region shown in red" is answerable — see below |
|
||||
| **regulatory element / cCRE** near a gene (ENCODE SCREEN) | `SCREEN_search_cCREs_by_region` | filter on `element_type` (**PLS** and **pELS** are TSS-proximal, **dELS** distal) and read `dnase_zscore` |
|
||||
| **which variant is at / overlaps** a genomic region (ClinVar) | `ClinVar_search_by_region` | **not** `ClinVar_search_variants` — Entrez matches a variant's START, so a narrow window misses a CNV that spans the region but begins megabases upstream. Returns true overlaps, smallest span first |
|
||||
| **how many peaks / which datasets** for a TF experiment (ReMap) | `ReMap_list_datasets_for_target` | one GEO series can hold several datasets (GSE23852/FOXA1 = 2, with 60,158 and 67,736 peaks) — report them separately unless a total is asked for; `count_peaks: true` to get counts |
|
||||
| **protein interaction partners** (STRING) | `STRING_get_protein_interactions` | read the **`partner`** field, not `preferredName_B`: edges are ordered A/B by internal ID, so the queried protein sits in column A on about half of them |
|
||||
|
||||
When unsure which tool wraps a database, search the catalog by the *relation* (e.g. "gene disease association", "gene set members"), not the brand name — ToolUniverse usually already has it.
|
||||
|
||||
### Allen Brain Atlas — answer with the specific structure, not its parent
|
||||
|
||||
The reference atlas colours every structure, and `AllenBrain_search_structures`
|
||||
returns `color_hex_triplet`. A question naming a colour ("which region is
|
||||
annotated in red at coronal position 181") is asking which **leaf structure**
|
||||
carries that colour, e.g. `Lateral preoptic area` = `#F2483B`.
|
||||
|
||||
Answering with the enclosing region ("Hypothalamus") is wrong even though it
|
||||
contains the right area: the atlas colours the specific structure, and the
|
||||
parent has its own different colour. Search by name or acronym, compare
|
||||
`color_hex_triplet`, and give the structure whose colour matches. Note the same
|
||||
acronym can return several rows (hemisphere-specific and ontology-version
|
||||
entries) with different colours — prefer the row whose `name` matches the
|
||||
question's wording.
|
||||
|
||||
### Human Protein Atlas — report both location fields
|
||||
|
||||
`HPA_get_subcellular_location` splits its answer in two, and the split is not
|
||||
significance ranking:
|
||||
|
||||
```
|
||||
main_locations : ['Nucleoplasm']
|
||||
additional_locations : ['Primary cilium', ..., 'Cytosol']
|
||||
```
|
||||
|
||||
A question asking "what localization does this antibody show" wants the
|
||||
locations HPA reports, which is **both lists** — answering from `main_locations`
|
||||
alone drops real localizations and is a common way to be half-right (e.g.
|
||||
answering "Nucleoplasm" where HPA reports "Nucleoplasm, Cytosol"). Use
|
||||
`location_summary`, which already joins them, or read both fields.
|
||||
|
||||
Two further cautions:
|
||||
|
||||
- **Locations aggregate over cell lines.** HPA pools immunofluorescence across
|
||||
every line an antibody was tested in. If the question names one line (HEK293,
|
||||
U-2 OS), treat the list as the candidate set and say which line you are
|
||||
reporting for, rather than implying the aggregate is line-specific.
|
||||
- **Per-cell-type RNA values are only published for enriched cell types.** HPA's
|
||||
machine-readable fields give specificity plus nTPM/nCPM for the cell types a
|
||||
gene is enriched in; a value for an arbitrary cell type is not exposed. If a
|
||||
question asks for one that is absent, say so instead of substituting the
|
||||
nearest available number — those differ by an order of magnitude.
|
||||
|
||||
## MSigDB set-name conventions (the most common LAB-Bench pattern)
|
||||
|
||||
ToolUniverse's `MSigDB_*` tools cover several collections that LAB-Bench questions are built from. Get the set name right:
|
||||
@@ -127,6 +207,42 @@ Try the `MP_<PHENOTYPE>` MSigDB set first (above): it answers in one call per op
|
||||
|
||||
## Computational procedures (when the answer is COMPUTED, not looked up)
|
||||
|
||||
### GWAS "highest p-value" means most significant
|
||||
|
||||
In GWAS writing, "the highest p-value", "the top hit" and "the strongest
|
||||
association" all mean the **most significant** result — the *smallest* numeric
|
||||
p-value. Read literally, "highest" picks the weakest association in the study
|
||||
and is almost never what was meant.
|
||||
|
||||
For GCST005528 the literal reading gives `rs2476491-?` at p = 1e-06; the
|
||||
intended answer is `rs7775055-G` at p = 3e-174.
|
||||
|
||||
Sort ascending by p-value and report that hit. If the phrasing genuinely could
|
||||
go either way, give the most significant one and say in a clause that the
|
||||
numerically largest p-value is a different SNP — do not silently pick the
|
||||
literal reading.
|
||||
|
||||
### Genomic windows — count the anchor base
|
||||
|
||||
A window described as "N bp upstream plus M bp downstream of X" spans
|
||||
**N + M + 1** bases, because the anchor base X is itself included. Asking for
|
||||
100 up and 100 down around a TSS is 201 nt, not 200. Off-by-one here is the
|
||||
single most common way a sequence answer is wrong while looking right.
|
||||
|
||||
The same care applies to the coordinate convention of whichever tool you call:
|
||||
|
||||
| convention | span of `start`..`end` | used by |
|
||||
|---|---|---|
|
||||
| 1-based inclusive | `end - start + 1` | Ensembl `region`, UCSC browser text, IGV, samtools |
|
||||
| 0-based half-open | `end - start` | UCSC REST API, BED |
|
||||
|
||||
`UCSC_get_sequence` takes a written locus via `region` (1-based inclusive) or
|
||||
explicit `chrom`/`start`/`end` with `coordinate_system`; it echoes
|
||||
`region_1based` and `requested_length` so the span is checkable. **Always check
|
||||
the returned length against what the question asked for** before answering — a
|
||||
sequence of the wrong length is wrong even when every base you kept is right.
|
||||
|
||||
|
||||
Any question with a **single deterministic numeric/combinatorial answer** must be obtained by **RUNNING code**, never by estimating or doing it in your head. This covers sequence questions (ORF counts, restriction fragments/sizes, GC content, translation) **and** any other exactly-computable question — e.g. **genetics segregation / Mendelian or polyploid gamete ratios, combinatorial probabilities, stoichiometry, dosage/PK arithmetic, counting problems**. Mental arithmetic on these is the #1 avoidable error: the model reliably mis-counts or mis-multiplies. If a question reduces to "enumerate the cases / multiply the probabilities / count the objects", **write a short Python snippet, execute it, and report exactly what it returns** — even when the topic looks like a biology "reasoning" question, if the answer is a definite number, compute it rather than reason it out. Match the question's wording for conventions (which strand; linear vs circular; which cross/segregation model) and **state the convention you used** so the answer is auditable.
|
||||
|
||||
**Final-answer discipline (avoid "computed right, answered wrong").** After the code returns the value, map it back to the option letters **carefully and explicitly**: quote the computed value, then find the option that matches it exactly (for a set of fragment sizes, match the whole multiset; for a count, match the integer). A surprising number of misses are cases where the computation was correct but the wrong letter was selected — do not let this happen; re-read each option against the computed result before emitting `[ANSWER]`.
|
||||
@@ -219,5 +335,19 @@ Interpretation: report the **exact value the code returns** (ORF count; fragment
|
||||
## Limitations (honest)
|
||||
|
||||
- **Key-gated sources**: `DisGeNET_*` and OMIM tools need `DISGENET_API_KEY` / OMIM key. Without a key, fall back to `OpenTargets_*` / `MyDisease_*` (keyless) and state the source used. If no keyless source can answer and the question is database-specific, this is a genuine "Insufficient information" case — say so.
|
||||
- **Release mismatch**: a tool's snapshot of a database may differ slightly from the exact release a question cites; report the source and version when it matters.
|
||||
- **Release mismatch**: a tool's snapshot of a database may differ from the exact
|
||||
release a question cites — and for some quantities the difference is not
|
||||
slight. Derived scores get recomputed between releases, so the *same gene* can
|
||||
differ by an order of magnitude. gnomAD pLI, via `gnomad_get_constraint`:
|
||||
|
||||
| gene | gnomAD r4 | gnomAD r2.1 |
|
||||
|---|---|---|
|
||||
| APOC2 | 0.046875 | 0.401638 |
|
||||
| APOC1 | 0.086323 | 0.216848 |
|
||||
|
||||
Where a tool exposes a `dataset`/release parameter, set it to the release the
|
||||
question names and **say which release you used**. If the question names one
|
||||
the tool cannot serve, report the release you did use rather than presenting
|
||||
the number as if it were release-independent — a bare pLI value is ambiguous
|
||||
by a factor of eight here.
|
||||
- This skill grounds *factual* lookups. For computing over user data files, use the data-analysis router skills instead.
|
||||
|
||||
+102
-5
@@ -1,8 +1,8 @@
|
||||
---
|
||||
lineage_type: import
|
||||
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/cfd26718/skills/tooluniverse-phylogenetics/SKILL.md
|
||||
upstream_sha: cfd26718
|
||||
imported_at: 2026-08-08
|
||||
upstream_source: https://github.com/mims-harvard/ToolUniverse/blob/4d14233e/skills/tooluniverse-phylogenetics/SKILL.md
|
||||
upstream_sha: 4d14233e
|
||||
imported_at: 2026-08-18
|
||||
prompt_class: catalogue
|
||||
upstream_changes: accepted
|
||||
name: tooluniverse-phylogenetics
|
||||
@@ -12,6 +12,45 @@ disable-model-invocation: true
|
||||
|
||||
# Phylogenetics and Sequence Analysis
|
||||
|
||||
## Four traps that produce a confidently wrong number
|
||||
|
||||
Each of these was observed producing a wrong answer *while the correct guidance
|
||||
was already present further down this file*. Check them before you answer.
|
||||
|
||||
1. **PhyKIT prints more than one column, and for `saturation` the two
|
||||
conventions disagree — state which you used.** `phykit saturation` prints
|
||||
`saturation <TAB> |saturation-1|`. Its own `--help` is explicit: *"The first
|
||||
value is the saturation value and the second column is the absolute value of
|
||||
saturation minus 1."* But several published analyses (and some reference
|
||||
answers derived from them) report the **second** column as "the saturation
|
||||
value". The two always sum to 1.0000, which is the tell that you may be
|
||||
looking at the wrong one — on the fungal scogs set the medians are 0.39
|
||||
(col 1) and 0.61 (col 2).
|
||||
|
||||
So: **follow phykit and use column 1** unless the question or source defines
|
||||
saturation the other way, and say in your answer which column you read. Do
|
||||
not silently pick the one that looks closer to an expected number.
|
||||
|
||||
`treeness_over_rcv` has no such ambiguity: it gives
|
||||
`ratio <TAB> treeness <TAB> RCV` and the ratio is first.
|
||||
|
||||
2. **"Gap percentage" means the fraction of alignment COLUMNS containing at
|
||||
least one gap**, not the fraction of residues that are gaps. On the fungal
|
||||
scogs set the residue definition maxes out at 0.556, so a ">70% gaps" filter
|
||||
selects **nothing** and the question looks unanswerable; by columns, three
|
||||
orthologs qualify (max 0.783).
|
||||
|
||||
3. **`treeness_over_rcv` and `rcv` take the UNTRIMMED `.faa.mafft`**, while
|
||||
`saturation` takes the trimmed `.clipkit`. RCV measures variability across
|
||||
columns, so trimming changes it: median 0.2683 untrimmed against 0.3050
|
||||
trimmed, and among >70%-gap genes the maximum is 0.2572 untrimmed against
|
||||
0.4174 trimmed.
|
||||
|
||||
4. **Never loop PhyKIT per file.** `phykit_batch_analysis` is parallel and does
|
||||
~250 trees in about 35 seconds; a shell loop takes ~9 minutes and runs out of
|
||||
turns mid-way, producing no answer at all. It also selects the right column
|
||||
for every function, which removes trap 1 entirely.
|
||||
|
||||
## RULE ZERO — Check for pre-computed results FIRST
|
||||
|
||||
Before following any instruction below, scan the data folder for:
|
||||
@@ -182,8 +221,9 @@ set (249 orthologs, canonical shipped files):
|
||||
|
||||
```
|
||||
median treeness/RCV untrimmed .faa.mafft = 0.2683 trimmed .clipkit = 0.3050
|
||||
max treeness/RCV (>70% gap genes)
|
||||
untrimmed .faa.mafft = 0.1866 trimmed .clipkit = 0.4205
|
||||
max treeness/RCV (over the 3 genes with >70% gapped columns:
|
||||
1260807at2759 0.0861, 1567796at2759 0.1866, 939345at2759 0.2572)
|
||||
untrimmed .faa.mafft = 0.2572 trimmed .clipkit = 0.4174
|
||||
```
|
||||
|
||||
Plain `treeness` needs no alignment and is unaffected — it reproduces exactly
|
||||
@@ -351,6 +391,63 @@ tu run phykit_batch_analysis '{"operation":"gap_percentage","directory":"./align
|
||||
```
|
||||
Do NOT run phykit manually in a loop — the tool handles all files and returns correct summary statistics.
|
||||
|
||||
**The batch tool is parallel: ~250 trees finish in about 35 seconds.** A per-tree
|
||||
shell loop takes ~9 minutes for the same work and is the single most common way
|
||||
these questions end with no answer at all — the run hits its turn or time budget
|
||||
mid-loop and reports "I'll report when it finishes" instead of a number. If you
|
||||
find yourself writing `for f in *.treefile`, stop and call the batch tool.
|
||||
|
||||
Supported `function` values include `treeness`, `saturation`, `dvmc`,
|
||||
`long_branch_score`, `total_tree_length`, `parsimony_informative`,
|
||||
`treeness_over_rcv` (alias `toverr`). `dvmc` and `long_branch_score` are
|
||||
covered — you do not need to loop for those.
|
||||
|
||||
**Two-group comparisons (Mann-Whitney U, differences of medians).** Questions
|
||||
comparing fungi against animals need one batch call per group, then the test on
|
||||
the two value lists — not a per-tree loop over both groups:
|
||||
|
||||
```bash
|
||||
tu run phykit_batch_analysis '{"operation":"batch","function":"dvmc","directory":"<fungi>","extension":".treefile"}'
|
||||
tu run phykit_batch_analysis '{"operation":"batch","function":"dvmc","directory":"<animals>","extension":".treefile"}'
|
||||
# then scipy.stats.mannwhitneyu(fungi_values, animal_values)
|
||||
```
|
||||
|
||||
Ask for `values` in the result when you need the full list for a test; the batch
|
||||
tool returns them for sets up to 50 and summary statistics always. For larger
|
||||
sets, compute the statistic from the per-group summaries the tool returns rather
|
||||
than re-deriving every value by hand.
|
||||
|
||||
### PhyKIT column conventions — take the right one
|
||||
|
||||
Several PhyKIT subcommands print more than one number per file, and the value
|
||||
the question wants is usually not the first:
|
||||
|
||||
| subcommand | prints | the value asked for |
|
||||
|---|---|---|
|
||||
| `saturation` | `saturation <TAB> \|saturation-1\|` | **column 1** per phykit's docs; some sources report col 2 — say which you used |
|
||||
| `treeness_over_rcv` | `treeness/RCV <TAB> treeness <TAB> RCV` | **column 1**, the ratio |
|
||||
| `parsimony_informative_sites` | `n_pi <TAB> n_total <TAB> %PIS` | column 3 for a percentage |
|
||||
|
||||
Taking `saturation`'s first column gives exactly `1 - answer`: a fungal set
|
||||
whose saturation is 0.6146 reports 0.3854 instead, and the two sum to 1.0000,
|
||||
which is the tell. `phykit_batch_analysis` already selects the right column for
|
||||
each function — another reason to call it rather than run the CLI yourself.
|
||||
|
||||
### Commit the value you computed
|
||||
|
||||
Two failures in this benchmark came from computing the right number and then
|
||||
answering a different one:
|
||||
|
||||
- a tree-length ratio computed as **2.1775**, then answered as 1.9 after
|
||||
re-reading "paired orthologs";
|
||||
- an average treeness that listed **19** among the alternatives, then committed 10.
|
||||
|
||||
When a question is ambiguous, compute the reading you judge most literal, state
|
||||
the alternative in one clause, and **answer with the value you actually
|
||||
computed**. Do not replace a computed result with a re-derived one at the last
|
||||
step — if two readings are both defensible, give the computed number first and
|
||||
name the other, rather than silently switching.
|
||||
|
||||
### PhyKIT column-position cheat sheet (parse output carefully)
|
||||
|
||||
When parsing PhyKIT stdout for batch metrics, the **column you want** depends on the metric:
|
||||
|
||||
Reference in New Issue
Block a user