[Upstream sync] K-Dense-AI/scientific-agent-skills (github) — 45 added, 56 modified #33

Merged
promptadmin merged 101 commits from upstream-sync/scientific-agent-skills-20260808-991bd9-sesb into main 2026-08-09 18:31:46 +00:00
Showing only changes of commit 2b8271c0b2 - Show all commits
@@ -1,8 +1,8 @@
---
lineage_type: import
upstream_source: https://github.com/K-Dense-AI/scientific-agent-skills/blob/9c9bd2e9/skills/datamol/SKILL.md
upstream_sha: 9c9bd2e9
imported_at: 2026-06-26
upstream_source: https://github.com/K-Dense-AI/scientific-agent-skills/blob/991bd993/skills/datamol/SKILL.md
upstream_sha: 991bd993
imported_at: 2026-08-08
prompt_class: prompt
upstream_changes: accepted
name: datamol
@@ -10,7 +10,9 @@ description: Pythonic wrapper around RDKit with simplified interface and sensibl
license: Apache-2.0 license
allowed-tools: Read Write Edit Bash
compatibility: Requires Python 3.8+ and datamol (uv pip install). RDKit is installed automatically as a datamol dependency (since 0.12.2). Optional s3fs/gcsfs for cloud I/O via fsspec.
metadata: {"version": "1.0", "skill-author": "K-Dense Inc."}
metadata:
version: "1.1"
skill-author: K-Dense Inc.
---
# Datamol Cheminformatics Skill
@@ -55,448 +57,24 @@ import datamol as dm
## Core Workflows
### 1. Basic Molecule Handling
**Creating molecules from SMILES**:
```python
import datamol as dm
# Single molecule
mol = dm.to_mol("CCO") # Ethanol
# From list of SMILES
smiles_list = ["CCO", "c1ccccc1", "CC(=O)O"]
mols = [dm.to_mol(smi) for smi in smiles_list]
# Error handling
mol = dm.to_mol("invalid_smiles") # Returns None
if mol is None:
print("Failed to parse SMILES")
```
**Converting molecules to SMILES**:
```python
# Canonical SMILES
smiles = dm.to_smiles(mol)
# Isomeric SMILES (includes stereochemistry)
smiles = dm.to_smiles(mol, isomeric=True)
# Other formats
inchi = dm.to_inchi(mol)
inchikey = dm.to_inchikey(mol)
selfies = dm.to_selfies(mol)
```
**Standardization and sanitization** (always recommend for user-provided molecules):
```python
# Sanitize molecule
mol = dm.sanitize_mol(mol)
# Full standardization (recommended for datasets)
mol = dm.standardize_mol(
mol,
disconnect_metals=True,
normalize=True,
reionize=True
)
# For SMILES strings directly
clean_smiles = dm.standardize_smiles(smiles)
```
### 2. Reading and Writing Molecular Files
Refer to `references/io_module.md` for comprehensive I/O documentation.
**Reading files**:
```python
# SDF files (most common in chemistry)
df = dm.read_sdf("compounds.sdf", mol_column='mol')
# SMILES files
df = dm.read_smi("molecules.smi", smiles_column='smiles', mol_column='mol')
# CSV with SMILES column
df = dm.read_csv("data.csv", smiles_column="SMILES", mol_column="mol")
# Excel files
df = dm.read_excel("compounds.xlsx", sheet_name=0, mol_column="mol")
# Universal reader/writer (auto-detects format; supports compression)
df = dm.open_df("file.sdf") # .sdf, .csv, .xlsx, .parquet, .json, .gz, etc.
dm.save_df(df, "output.parquet")
```
**Writing files**:
```python
# Save as SDF
dm.to_sdf(mols, "output.sdf")
# Or from DataFrame
dm.to_sdf(df, "output.sdf", mol_column="mol")
# Save as SMILES file
dm.to_smi(mols, "output.smi")
# Excel with rendered molecule images
dm.to_xlsx(df, "output.xlsx", mol_columns=["mol"])
```
**Remote file support** (S3, GCS, HTTP via fsspec):
Only use cloud paths when the user explicitly requests them. Confirm the destination before writing.
```python
# Read from cloud storage or HTTPS (user-provided URLs only)
df = dm.read_sdf("s3://bucket/compounds.sdf")
df = dm.read_csv("https://example.com/data.csv")
# Write to cloud storage — confirm path with user first
dm.to_sdf(mols, "s3://bucket/output.sdf")
```
Cloud backends read credentials from the standard provider environment (for example `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`, or `GOOGLE_APPLICATION_CREDENTIALS`). Datamol passes these to fsspec locally; it does not collect or transmit environment variables to third-party endpoints. Scope credential access to the named provider variables only.
### 3. Molecular Descriptors and Properties
Refer to `references/descriptors_viz.md` for detailed descriptor documentation.
**Computing descriptors for a single molecule**:
```python
# Get standard descriptor set
descriptors = dm.descriptors.compute_many_descriptors(mol)
# Returns: {'mw': 46.07, 'logp': -0.03, 'hbd': 1, 'hba': 1,
# 'tpsa': 20.23, 'n_aromatic_atoms': 0, ...}
```
**Batch descriptor computation** (recommended for datasets):
```python
# Compute for all molecules in parallel
desc_df = dm.descriptors.batch_compute_many_descriptors(
mols,
n_jobs=-1, # Use all CPU cores
progress=True # Show progress bar
)
```
**Specific descriptors**:
```python
# Aromaticity
n_aromatic = dm.descriptors.n_aromatic_atoms(mol)
aromatic_ratio = dm.descriptors.n_aromatic_atoms_proportion(mol)
# Stereochemistry
n_stereo = dm.descriptors.n_stereo_centers(mol)
n_unspec = dm.descriptors.n_stereo_centers_unspecified(mol)
# Flexibility
n_rigid = dm.descriptors.n_rigid_bonds(mol)
```
**Drug-likeness filtering (Lipinski's Rule of Five)**:
```python
# Filter compounds
def is_druglike(mol):
desc = dm.descriptors.compute_many_descriptors(mol)
return (
desc['mw'] <= 500 and
desc['logp'] <= 5 and
desc['hbd'] <= 5 and
desc['hba'] <= 10
)
druglike_mols = [mol for mol in mols if is_druglike(mol)]
```
### 4. Molecular Fingerprints and Similarity
**Generating fingerprints**:
Datamol defaults to ECFP6 (`radius=3`, `n_bits=2048`). Pass `radius=2` explicitly for ECFP4.
```python
# ECFP4 (common in similarity screening)
fp = dm.to_fp(mol, fp_type='ecfp', radius=2, n_bits=2048)
# Other fingerprint types
fp_maccs = dm.to_fp(mol, fp_type='maccs')
fp_topological = dm.to_fp(mol, fp_type='topological')
fp_atompair = dm.to_fp(mol, fp_type='atompair')
fp_rdkit = dm.to_fp(mol, fp_type='rdkit')
```
**Similarity calculations**:
```python
# Pairwise distances within a set
distance_matrix = dm.pdist(mols, n_jobs=-1)
# Distances between two sets
distances = dm.cdist(query_mols, library_mols, n_jobs=-1)
# Find most similar molecules (scipy is a PyPI package, not a file in this skill)
from scipy.spatial.distance import squareform # third-party library
dist_matrix = squareform(dm.pdist(mols))
# Lower distance = higher similarity (Tanimoto distance = 1 - Tanimoto similarity)
```
### 5. Clustering and Diversity Selection
Refer to `references/core_api.md` for clustering details.
**Butina clustering**:
```python
# Cluster molecules by structural similarity
clusters = dm.cluster_mols(
mols,
cutoff=0.2, # Tanimoto distance threshold (0=identical, 1=completely different)
n_jobs=-1 # Parallel processing
)
# Each cluster is a list of molecule indices
for i, cluster in enumerate(clusters):
print(f"Cluster {i}: {len(cluster)} molecules")
cluster_mols = [mols[idx] for idx in cluster]
```
**Important**: Butina clustering builds a full distance matrix - suitable for ~1000 molecules, not for 10,000+.
**Diversity selection**:
```python
# Pick diverse subset
diverse_mols = dm.pick_diverse(
mols,
npick=100 # Select 100 diverse molecules
)
# Pick cluster centroids
centroids = dm.pick_centroids(
mols,
npick=50 # Select 50 representative molecules
)
```
### 6. Scaffold Analysis
Refer to `references/fragments_scaffolds.md` for complete scaffold documentation.
**Extracting Murcko scaffolds**:
```python
# Get Bemis-Murcko scaffold (core structure)
scaffold = dm.to_scaffold_murcko(mol)
scaffold_smiles = dm.to_smiles(scaffold)
```
**Scaffold-based analysis**:
```python
# Group compounds by scaffold
from collections import Counter
scaffolds = [dm.to_scaffold_murcko(mol) for mol in mols]
scaffold_smiles = [dm.to_smiles(s) for s in scaffolds]
# Count scaffold frequency
scaffold_counts = Counter(scaffold_smiles)
most_common = scaffold_counts.most_common(10)
# Create scaffold-to-molecules mapping
scaffold_groups = {}
for mol, scaf_smi in zip(mols, scaffold_smiles):
if scaf_smi not in scaffold_groups:
scaffold_groups[scaf_smi] = []
scaffold_groups[scaf_smi].append(mol)
```
**Scaffold-based train/test splitting** (for ML):
```python
# Ensure train and test sets have different scaffolds
scaffold_to_mols = {}
for mol, scaf in zip(mols, scaffold_smiles):
if scaf not in scaffold_to_mols:
scaffold_to_mols[scaf] = []
scaffold_to_mols[scaf].append(mol)
# Split scaffolds into train/test
import random
scaffolds = list(scaffold_to_mols.keys())
random.shuffle(scaffolds)
split_idx = int(0.8 * len(scaffolds))
train_scaffolds = scaffolds[:split_idx]
test_scaffolds = scaffolds[split_idx:]
# Get molecules for each split
train_mols = [mol for scaf in train_scaffolds for mol in scaffold_to_mols[scaf]]
test_mols = [mol for scaf in test_scaffolds for mol in scaffold_to_mols[scaf]]
```
### 7. Molecular Fragmentation
Refer to `references/fragments_scaffolds.md` for fragmentation details.
**BRICS fragmentation** (16 bond types):
```python
# Fragment molecule
fragments = dm.fragment.brics(mol)
# Returns: set of fragment SMILES with attachment points like '[1*]CCN'
```
**RECAP fragmentation** (11 bond types):
```python
fragments = dm.fragment.recap(mol)
```
**Fragment analysis**:
```python
# Find common fragments across compound library
from collections import Counter
all_fragments = []
for mol in mols:
frags = dm.fragment.brics(mol)
all_fragments.extend(frags)
fragment_counts = Counter(all_fragments)
common_frags = fragment_counts.most_common(20)
# Fragment-based scoring
def fragment_score(mol, reference_fragments):
mol_frags = dm.fragment.brics(mol)
overlap = mol_frags.intersection(reference_fragments)
return len(overlap) / len(mol_frags) if mol_frags else 0
```
### 8. 3D Conformer Generation
Refer to `references/conformers_module.md` for detailed conformer documentation.
**Generating conformers**:
```python
# Generate 3D conformers
mol_3d = dm.conformers.generate(
mol,
n_confs=50, # Number to generate (auto if None)
rms_cutoff=0.5, # Filter similar conformers (Ångströms)
minimize_energy=True, # Minimize with UFF force field
method='ETKDGv3' # Embedding method (recommended)
)
# Access conformers
n_conformers = mol_3d.GetNumConformers()
conf = mol_3d.GetConformer(0) # Get first conformer
positions = conf.GetPositions() # Nx3 array of atom coordinates
```
**Conformer clustering**:
```python
# Cluster conformers by RMSD
clusters = dm.conformers.cluster(
mol_3d,
rms_cutoff=1.0,
centroids=False
)
# Get representative conformers
centroids = dm.conformers.return_centroids(mol_3d, clusters)
```
**SASA calculation**:
```python
# Calculate solvent accessible surface area
sasa_values = dm.conformers.sasa(mol_3d, n_jobs=-1)
# Access SASA from conformer properties
conf = mol_3d.GetConformer(0)
sasa = conf.GetDoubleProp('rdkit_free_sasa')
```
### 9. Visualization
Refer to `references/descriptors_viz.md` for visualization documentation.
**Basic molecule grid**:
```python
# Visualize molecules
dm.viz.to_image(
mols[:20],
legends=[dm.to_smiles(m) for m in mols[:20]],
n_cols=5,
mol_size=(300, 300)
)
# Save to file
dm.viz.to_image(mols, outfile="molecules.png")
# SVG for publications
dm.viz.to_image(mols, outfile="molecules.svg", use_svg=True)
```
**Aligned visualization** (for SAR analysis):
```python
# Align molecules by common substructure
dm.viz.to_image(
similar_mols,
align=True, # Enable MCS alignment
legends=activity_labels,
n_cols=4
)
```
**Highlighting substructures**:
```python
# Highlight specific atoms and bonds
dm.viz.to_image(
mol,
highlight_atom=[0, 1, 2, 3], # Atom indices
highlight_bond=[0, 1, 2] # Bond indices
)
```
**Conformer visualization**:
```python
# Display multiple conformers
dm.viz.conformers(
mol_3d,
n_confs=10,
align_conf=True,
n_cols=3
)
```
### 10. Chemical Reactions
Refer to `references/reactions_data.md` for reactions documentation.
**Applying reactions**:
```python
from rdkit.Chem import rdChemReactions
# Define reaction from SMARTS
rxn_smarts = '[C:1](=[O:2])[OH:3]>>[C:1](=[O:2])[Cl:3]'
rxn = rdChemReactions.ReactionFromSmarts(rxn_smarts)
# Apply to molecule
reactant = dm.to_mol("CC(=O)O") # Acetic acid
product = dm.reactions.apply_reaction(
rxn,
(reactant,),
sanitize=True
)
# Convert to SMILES
product_smiles = dm.to_smiles(product)
```
**Batch reaction application**:
```python
# Apply reaction to library
products = []
for mol in reactant_mols:
try:
prod = dm.reactions.apply_reaction(rxn, (mol,))
if prod is not None:
products.append(prod)
except Exception as e:
print(f"Reaction failed: {e}")
```
Ten workflow areas, each with worked code, are documented in
[references/core_workflows.md](references/core_workflows.md):
| # | Area | Covers |
| --- | --- | --- |
| 1 | Basic molecule handling | `to_mol`, batch conversion, error handling, canonical and isomeric SMILES, sanitization and full standardization |
| 2 | Reading and writing files | SDF, SMILES, CSV, Excel with rendered structures, the universal reader/writer, and cloud or HTTPS paths |
| 3 | Descriptors and properties | the standard descriptor set, parallel computation, aromaticity, stereochemistry, flexibility, and filtering |
| 4 | Fingerprints and similarity | ECFP4 and other types, pairwise and cross-set distances, nearest-neighbour lookup (Tanimoto distance = 1 similarity) |
| 5 | Clustering and diversity | similarity clustering, diverse subset picking, and cluster centroids |
| 6 | Scaffold analysis | Bemis-Murcko scaffolds, grouping and counting, and scaffold-disjoint train/test splits |
| 7 | Fragmentation | fragmenting molecules, finding common fragments across a library, and fragment-based scoring |
| 8 | 3D conformers | generation, access, RMSD clustering, representative selection, and SASA |
| 9 | Visualization | grids, files, publication SVG, substructure alignment, atom and bond highlighting, conformer display |
| 10 | Chemical reactions | reaction SMARTS, applying to a molecule or a whole library |
Three end-to-end pipelines — load/filter/analyze, SAR by scaffold series, and virtual
screening — are in [references/workflow_patterns.md](references/workflow_patterns.md).
## Parallelization
@@ -514,106 +92,6 @@ Datamol includes built-in parallelization for many operations. Use `n_jobs` para
**Progress bars**: Many batch operations support `progress=True` parameter.
## Common Workflows and Patterns
### Complete Pipeline: Data Loading → Filtering → Analysis
```python
import datamol as dm
import pandas as pd
# 1. Load molecules
df = dm.read_sdf("compounds.sdf")
# 2. Standardize
df['mol'] = df['mol'].apply(lambda m: dm.standardize_mol(m) if m else None)
df = df[df['mol'].notna()] # Remove failed molecules
# 3. Compute descriptors
desc_df = dm.descriptors.batch_compute_many_descriptors(
df['mol'].tolist(),
n_jobs=-1,
progress=True
)
# 4. Filter by drug-likeness
druglike = (
(desc_df['mw'] <= 500) &
(desc_df['logp'] <= 5) &
(desc_df['hbd'] <= 5) &
(desc_df['hba'] <= 10)
)
filtered_df = df[druglike]
# 5. Cluster and select diverse subset
diverse_mols = dm.pick_diverse(
filtered_df['mol'].tolist(),
npick=100
)
# 6. Visualize results
dm.viz.to_image(
diverse_mols,
legends=[dm.to_smiles(m) for m in diverse_mols],
outfile="diverse_compounds.png",
n_cols=10
)
```
### Structure-Activity Relationship (SAR) Analysis
```python
# Group by scaffold
scaffolds = [dm.to_scaffold_murcko(mol) for mol in mols]
scaffold_smiles = [dm.to_smiles(s) for s in scaffolds]
# Create DataFrame with activities
sar_df = pd.DataFrame({
'mol': mols,
'scaffold': scaffold_smiles,
'activity': activities # User-provided activity data
})
# Analyze each scaffold series
for scaffold, group in sar_df.groupby('scaffold'):
if len(group) >= 3: # Need multiple examples
print(f"\nScaffold: {scaffold}")
print(f"Count: {len(group)}")
print(f"Activity range: {group['activity'].min():.2f} - {group['activity'].max():.2f}")
# Visualize with activities as legends
dm.viz.to_image(
group['mol'].tolist(),
legends=[f"Activity: {act:.2f}" for act in group['activity']],
align=True # Align by common substructure
)
```
### Virtual Screening Pipeline
```python
import numpy as np
# 1. Calculate Tanimoto distances between query actives and library
distances = dm.cdist(query_actives, library_mols, n_jobs=-1)
# 3. Find closest matches (min distance to any query)
min_distances = distances.min(axis=0)
similarities = 1 - min_distances # Convert distance to similarity
# 4. Rank and select top hits
top_indices = np.argsort(similarities)[::-1][:100] # Top 100
top_hits = [library_mols[i] for i in top_indices]
top_scores = [similarities[i] for i in top_indices]
# 5. Visualize hits
dm.viz.to_image(
top_hits[:20],
legends=[f"Sim: {score:.3f}" for score in top_scores[:20]],
outfile="screening_hits.png"
)
```
## Reference Documentation
For detailed API documentation, consult these reference files:
@@ -726,4 +204,3 @@ predictions = model.predict(X_test)
- **Datamol Documentation**: https://docs.datamol.io/
- **RDKit Documentation**: https://www.rdkit.org/docs/
- **GitHub Repository**: https://github.com/datamol-io/datamol