Import AITURK IDE 1.0.0-beta.1 from Hermes 63279301; preserve MIT license
This commit is contained in:
@@ -0,0 +1,237 @@
|
||||
---
|
||||
name: bioinformatics
|
||||
description: Gateway to 400+ genomics and computational biology skills.
|
||||
version: 1.0.0
|
||||
author: Teknium (teknium1), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [bioinformatics, genomics, sequencing, biology, research, science]
|
||||
category: research
|
||||
---
|
||||
|
||||
# Bioinformatics Skills Gateway
|
||||
|
||||
Use when asked about bioinformatics, genomics, sequencing, variant calling, gene expression, single-cell analysis, protein structure, pharmacogenomics, metagenomics, phylogenetics, or any computational biology task.
|
||||
|
||||
This skill is a gateway to two open-source bioinformatics skill libraries. Instead of bundling hundreds of domain-specific skills, it indexes them and fetches what you need on demand.
|
||||
|
||||
## Sources
|
||||
|
||||
◆ **bioSkills** — 385 reference skills (code patterns, parameter guides, decision trees)
|
||||
Repo: https://github.com/GPTomics/bioSkills
|
||||
Format: SKILL.md per topic with code examples. Python/R/CLI.
|
||||
|
||||
◆ **ClawBio** — 33 runnable pipeline skills (executable scripts, reproducibility bundles)
|
||||
Repo: https://github.com/ClawBio/ClawBio
|
||||
Format: Python scripts with demos. Each analysis exports report.md + commands.sh + environment.yml.
|
||||
|
||||
## How to fetch and use a skill
|
||||
|
||||
1. Identify the domain and skill name from the index below.
|
||||
2. Clone the relevant repo (shallow clone to save time):
|
||||
```bash
|
||||
# bioSkills (reference material)
|
||||
git clone --depth 1 https://github.com/GPTomics/bioSkills.git /tmp/bioSkills
|
||||
|
||||
# ClawBio (runnable pipelines)
|
||||
git clone --depth 1 https://github.com/ClawBio/ClawBio.git /tmp/ClawBio
|
||||
```
|
||||
3. Read the specific skill:
|
||||
```bash
|
||||
# bioSkills — each skill is at: <category>/<skill-name>/SKILL.md
|
||||
cat /tmp/bioSkills/variant-calling/gatk-variant-calling/SKILL.md
|
||||
|
||||
# ClawBio — each skill is at: skills/<skill-name>/
|
||||
cat /tmp/ClawBio/skills/pharmgx-reporter/README.md
|
||||
```
|
||||
4. Follow the fetched skill as reference material. These are NOT Hermes-format skills — treat them as expert domain guides. They contain correct parameters, proper tool flags, and validated pipelines.
|
||||
|
||||
## Skill Index by Domain
|
||||
|
||||
### Sequence Fundamentals
|
||||
bioSkills:
|
||||
sequence-io/ — read-sequences, write-sequences, format-conversion, batch-processing, compressed-files, fastq-quality, filter-sequences, paired-end-fastq, sequence-statistics
|
||||
sequence-manipulation/ — seq-objects, reverse-complement, transcription-translation, motif-search, codon-usage, sequence-properties, sequence-slicing
|
||||
ClawBio:
|
||||
seq-wrangler — Sequence QC, alignment, and BAM processing (wraps FastQC, BWA, SAMtools)
|
||||
|
||||
### Read QC & Alignment
|
||||
bioSkills:
|
||||
read-qc/ — quality-reports, fastp-workflow, adapter-trimming, quality-filtering, umi-processing, contamination-screening, rnaseq-qc
|
||||
read-alignment/ — bwa-alignment, star-alignment, hisat2-alignment, bowtie2-alignment
|
||||
alignment-files/ — sam-bam-basics, alignment-sorting, alignment-filtering, bam-statistics, duplicate-handling, pileup-generation
|
||||
|
||||
### Variant Calling & Annotation
|
||||
bioSkills:
|
||||
variant-calling/ — gatk-variant-calling, deepvariant, variant-calling (bcftools), joint-calling, structural-variant-calling, filtering-best-practices, variant-annotation, variant-normalization, vcf-basics, vcf-manipulation, vcf-statistics, consensus-sequences, clinical-interpretation
|
||||
ClawBio:
|
||||
vcf-annotator — VEP + ClinVar + gnomAD annotation with ancestry-aware context
|
||||
variant-annotation — Variant annotation pipeline
|
||||
|
||||
### Differential Expression (Bulk RNA-seq)
|
||||
bioSkills:
|
||||
differential-expression/ — deseq2-basics, edger-basics, batch-correction, de-results, de-visualization, timeseries-de
|
||||
rna-quantification/ — alignment-free-quant (Salmon/kallisto), featurecounts-counting, tximport-workflow, count-matrix-qc
|
||||
expression-matrix/ — counts-ingest, gene-id-mapping, metadata-joins, sparse-handling
|
||||
ClawBio:
|
||||
rnaseq-de — Full DE pipeline with QC, normalization, and visualization
|
||||
diff-visualizer — Rich visualization and reporting for DE results
|
||||
|
||||
### Single-Cell RNA-seq
|
||||
bioSkills:
|
||||
single-cell/ — preprocessing, clustering, batch-integration, cell-annotation, cell-communication, doublet-detection, markers-annotation, trajectory-inference, multimodal-integration, perturb-seq, scatac-analysis, lineage-tracing, metabolite-communication, data-io
|
||||
ClawBio:
|
||||
scrna-orchestrator — Full Scanpy pipeline (QC, clustering, markers, annotation)
|
||||
scrna-embedding — scVI-based latent embedding and batch integration
|
||||
|
||||
### Spatial Transcriptomics
|
||||
bioSkills:
|
||||
spatial-transcriptomics/ — spatial-data-io, spatial-preprocessing, spatial-domains, spatial-deconvolution, spatial-communication, spatial-neighbors, spatial-statistics, spatial-visualization, spatial-multiomics, spatial-proteomics, image-analysis
|
||||
|
||||
### Epigenomics
|
||||
bioSkills:
|
||||
chip-seq/ — peak-calling, differential-binding, motif-analysis, peak-annotation, chipseq-qc, chipseq-visualization, super-enhancers
|
||||
atac-seq/ — atac-peak-calling, atac-qc, differential-accessibility, footprinting, motif-deviation, nucleosome-positioning
|
||||
methylation-analysis/ — bismark-alignment, methylation-calling, dmr-detection, methylkit-analysis
|
||||
hi-c-analysis/ — hic-data-io, tad-detection, loop-calling, compartment-analysis, contact-pairs, matrix-operations, hic-visualization, hic-differential
|
||||
ClawBio:
|
||||
methylation-clock — Epigenetic age estimation
|
||||
|
||||
### Pharmacogenomics & Clinical
|
||||
bioSkills:
|
||||
clinical-databases/ — clinvar-lookup, gnomad-frequencies, dbsnp-queries, pharmacogenomics, polygenic-risk, hla-typing, variant-prioritization, somatic-signatures, tumor-mutational-burden, myvariant-queries
|
||||
ClawBio:
|
||||
pharmgx-reporter — PGx report from 23andMe/AncestryDNA (12 genes, 31 SNPs, 51 drugs)
|
||||
drug-photo — Photo of medication → personalized PGx dosage card (via vision)
|
||||
clinpgx — ClinPGx API for gene-drug data and CPIC guidelines
|
||||
gwas-lookup — Federated variant lookup across 9 genomic databases
|
||||
gwas-prs — Polygenic risk scores from consumer genetic data
|
||||
nutrigx_advisor — Personalized nutrition from consumer genetic data
|
||||
|
||||
### Population Genetics & GWAS
|
||||
bioSkills:
|
||||
population-genetics/ — association-testing (PLINK GWAS), plink-basics, population-structure, linkage-disequilibrium, scikit-allel-analysis, selection-statistics
|
||||
causal-genomics/ — mendelian-randomization, fine-mapping, colocalization-analysis, mediation-analysis, pleiotropy-detection
|
||||
phasing-imputation/ — haplotype-phasing, genotype-imputation, imputation-qc, reference-panels
|
||||
ClawBio:
|
||||
claw-ancestry-pca — Ancestry PCA against SGDP reference panel
|
||||
|
||||
### Metagenomics & Microbiome
|
||||
bioSkills:
|
||||
metagenomics/ — kraken-classification, metaphlan-profiling, abundance-estimation, functional-profiling, amr-detection, strain-tracking, metagenome-visualization
|
||||
microbiome/ — amplicon-processing, diversity-analysis, differential-abundance, taxonomy-assignment, functional-prediction, qiime2-workflow
|
||||
ClawBio:
|
||||
claw-metagenomics — Shotgun metagenomics profiling (taxonomy, resistome, functional pathways)
|
||||
|
||||
### Genome Assembly & Annotation
|
||||
bioSkills:
|
||||
genome-assembly/ — hifi-assembly, long-read-assembly, short-read-assembly, metagenome-assembly, assembly-polishing, assembly-qc, scaffolding, contamination-detection
|
||||
genome-annotation/ — eukaryotic-gene-prediction, prokaryotic-annotation, functional-annotation, ncrna-annotation, repeat-annotation, annotation-transfer
|
||||
long-read-sequencing/ — basecalling, long-read-alignment, long-read-qc, clair3-variants, structural-variants, medaka-polishing, nanopore-methylation, isoseq-analysis
|
||||
|
||||
### Structural Biology & Chemoinformatics
|
||||
bioSkills:
|
||||
structural-biology/ — alphafold-predictions, modern-structure-prediction, structure-io, structure-navigation, structure-modification, geometric-analysis
|
||||
chemoinformatics/ — molecular-io, molecular-descriptors, similarity-searching, substructure-search, virtual-screening, admet-prediction, reaction-enumeration
|
||||
ClawBio:
|
||||
struct-predictor — Local AlphaFold/Boltz/Chai structure prediction with comparison
|
||||
|
||||
### Proteomics
|
||||
bioSkills:
|
||||
proteomics/ — data-import, peptide-identification, protein-inference, quantification, differential-abundance, dia-analysis, ptm-analysis, proteomics-qc, spectral-libraries
|
||||
ClawBio:
|
||||
proteomics-de — Proteomics differential expression
|
||||
|
||||
### Pathway Analysis & Gene Networks
|
||||
bioSkills:
|
||||
pathway-analysis/ — go-enrichment, gsea, kegg-pathways, reactome-pathways, wikipathways, enrichment-visualization
|
||||
gene-regulatory-networks/ — scenic-regulons, coexpression-networks, differential-networks, multiomics-grn, perturbation-simulation
|
||||
|
||||
### Immunoinformatics
|
||||
bioSkills:
|
||||
immunoinformatics/ — mhc-binding-prediction, epitope-prediction, neoantigen-prediction, immunogenicity-scoring, tcr-epitope-binding
|
||||
tcr-bcr-analysis/ — mixcr-analysis, scirpy-analysis, immcantation-analysis, repertoire-visualization, vdjtools-analysis
|
||||
|
||||
### CRISPR & Genome Engineering
|
||||
bioSkills:
|
||||
crispr-screens/ — mageck-analysis, jacks-analysis, hit-calling, screen-qc, library-design, crispresso-editing, base-editing-analysis, batch-correction
|
||||
genome-engineering/ — grna-design, off-target-prediction, hdr-template-design, base-editing-design, prime-editing-design
|
||||
|
||||
### Workflow Management
|
||||
bioSkills:
|
||||
workflow-management/ — snakemake-workflows, nextflow-pipelines, cwl-workflows, wdl-workflows
|
||||
ClawBio:
|
||||
repro-enforcer — Export any analysis as reproducibility bundle (Conda env + Singularity + checksums)
|
||||
galaxy-bridge — Access 8,000+ Galaxy tools from usegalaxy.org
|
||||
|
||||
### Specialized Domains
|
||||
bioSkills:
|
||||
alternative-splicing/ — splicing-quantification, differential-splicing, isoform-switching, sashimi-plots, single-cell-splicing, splicing-qc
|
||||
ecological-genomics/ — edna-metabarcoding, landscape-genomics, conservation-genetics, biodiversity-metrics, community-ecology, species-delimitation
|
||||
epidemiological-genomics/ — pathogen-typing, variant-surveillance, phylodynamics, transmission-inference, amr-surveillance
|
||||
liquid-biopsy/ — cfdna-preprocessing, ctdna-mutation-detection, fragment-analysis, tumor-fraction-estimation, methylation-based-detection, longitudinal-monitoring
|
||||
epitranscriptomics/ — m6a-peak-calling, m6a-differential, m6anet-analysis, merip-preprocessing, modification-visualization
|
||||
metabolomics/ — xcms-preprocessing, metabolite-annotation, normalization-qc, statistical-analysis, pathway-mapping, lipidomics, targeted-analysis, msdial-preprocessing
|
||||
flow-cytometry/ — fcs-handling, gating-analysis, compensation-transformation, clustering-phenotyping, differential-analysis, cytometry-qc, doublet-detection, bead-normalization
|
||||
systems-biology/ — flux-balance-analysis, metabolic-reconstruction, gene-essentiality, context-specific-models, model-curation
|
||||
rna-structure/ — secondary-structure-prediction, ncrna-search, structure-probing
|
||||
|
||||
### Data Visualization & Reporting
|
||||
bioSkills:
|
||||
data-visualization/ — ggplot2-fundamentals, heatmaps-clustering, volcano-customization, circos-plots, genome-browser-tracks, interactive-visualization, multipanel-figures, network-visualization, upset-plots, color-palettes, specialized-omics-plots, genome-tracks
|
||||
reporting/ — rmarkdown-reports, quarto-reports, jupyter-reports, automated-qc-reports, figure-export
|
||||
ClawBio:
|
||||
profile-report — Analysis profile reporting
|
||||
data-extractor — Extract numerical data from scientific figure images (via vision)
|
||||
lit-synthesizer — PubMed/bioRxiv search, summarization, citation graphs
|
||||
pubmed-summariser — Gene/disease PubMed search with structured briefing
|
||||
|
||||
### Database Access
|
||||
bioSkills:
|
||||
database-access/ — entrez-search, entrez-fetch, entrez-link, blast-searches, local-blast, sra-data, geo-data, uniprot-access, batch-downloads, interaction-databases, sequence-similarity
|
||||
ClawBio:
|
||||
ukb-navigator — Semantic search across 12,000+ UK Biobank fields
|
||||
clinical-trial-finder — Clinical trial discovery
|
||||
|
||||
### Experimental Design
|
||||
bioSkills:
|
||||
experimental-design/ — power-analysis, sample-size, batch-design, multiple-testing
|
||||
|
||||
### Machine Learning for Omics
|
||||
bioSkills:
|
||||
machine-learning/ — omics-classifiers, biomarker-discovery, survival-analysis, model-validation, prediction-explanation, atlas-mapping
|
||||
ClawBio:
|
||||
claw-semantic-sim — Semantic similarity index for disease literature (PubMedBERT)
|
||||
omics-target-evidence-mapper — Aggregate target-level evidence across omics sources
|
||||
|
||||
## Environment Setup
|
||||
|
||||
These skills assume a bioinformatics workstation. Common dependencies:
|
||||
|
||||
```bash
|
||||
# Python
|
||||
pip install biopython pysam cyvcf2 pybedtools pyBigWig scikit-allel anndata scanpy mygene
|
||||
|
||||
# R/Bioconductor
|
||||
Rscript -e 'BiocManager::install(c("DESeq2","edgeR","Seurat","clusterProfiler","methylKit"))'
|
||||
|
||||
# CLI tools (Ubuntu/Debian)
|
||||
sudo apt install samtools bcftools ncbi-blast+ minimap2 bedtools
|
||||
|
||||
# CLI tools (macOS)
|
||||
brew install samtools bcftools blast minimap2 bedtools
|
||||
|
||||
# Or via Conda (recommended for reproducibility)
|
||||
conda install -c bioconda samtools bcftools blast minimap2 bedtools fastp kraken2
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- The fetched skills are NOT in Hermes SKILL.md format. They use their own structure (bioSkills: code pattern cookbooks; ClawBio: README + Python scripts). Read them as expert reference material.
|
||||
- bioSkills are reference guides — they show correct parameters and code patterns but aren't executable pipelines.
|
||||
- ClawBio skills are executable — many have `--demo` flags and can be run directly.
|
||||
- Both repos assume bioinformatics tools are installed. Check prerequisites before running pipelines.
|
||||
- For ClawBio, run `pip install -r requirements.txt` in the cloned repo first.
|
||||
- Genomic data files can be very large. Be mindful of disk space when downloading reference genomes, SRA datasets, or building indices.
|
||||
@@ -0,0 +1,146 @@
|
||||
---
|
||||
name: blogwatcher
|
||||
description: "Monitor blogs and RSS/Atom feeds via blogwatcher-cli tool."
|
||||
version: 2.0.0
|
||||
author: JulienTant (fork of Hyaxia/blogwatcher)
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [RSS, Blogs, Feed-Reader, Monitoring]
|
||||
homepage: https://github.com/JulienTant/blogwatcher-cli
|
||||
prerequisites:
|
||||
commands: [blogwatcher-cli]
|
||||
---
|
||||
|
||||
# Blogwatcher
|
||||
|
||||
Track blog and RSS/Atom feed updates with the `blogwatcher-cli` tool. Supports automatic feed discovery, HTML scraping fallback, OPML import, and read/unread article management.
|
||||
|
||||
## Working with Hermes tools (read this first)
|
||||
|
||||
`blogwatcher-cli` is the feed database; Hermes tools do the automation around it:
|
||||
|
||||
- **Recurring watch — use the cronjob tool's `monitor` field, not a bare schedule.** `monitor` runs a script each tick and only wakes the agent when output changes: set it to a script that runs `blogwatcher-cli scan >/dev/null 2>&1 && blogwatcher-cli articles` (deterministic output; new articles = changed output = agent wakes with the diff injected). Unchanged ticks cost zero LLM calls. Set `deliver` to route digests to a chat/channel; add `continuity: true` so consecutive digests can dedupe.
|
||||
- **Reading an article the user asks about**: `web_extract([url])` on the article URL from `blogwatcher-cli articles` — do not re-scrape by hand.
|
||||
- **One-off "watch this page for changes" without feed semantics**: skip this skill; the cronjob tool's `monitor` field accepts an http(s) URL directly.
|
||||
- **Company/competitor tracking with analysis and citations**: prefer the `competitor-news-monitor` skill; blogwatcher is the lighter raw-feed layer it can sit on.
|
||||
|
||||
## Installation
|
||||
|
||||
Pick one method:
|
||||
|
||||
- **Go:** `go install github.com/JulienTant/blogwatcher-cli/cmd/blogwatcher-cli@latest`
|
||||
- **Docker:** `docker run --rm -v blogwatcher-cli:/data ghcr.io/julientant/blogwatcher-cli`
|
||||
- **Binary (Linux amd64):** `curl -sL https://github.com/JulienTant/blogwatcher-cli/releases/latest/download/blogwatcher-cli_linux_amd64.tar.gz | tar xz -C /usr/local/bin blogwatcher-cli`
|
||||
- **Binary (Linux arm64):** `curl -sL https://github.com/JulienTant/blogwatcher-cli/releases/latest/download/blogwatcher-cli_linux_arm64.tar.gz | tar xz -C /usr/local/bin blogwatcher-cli`
|
||||
- **Binary (macOS Apple Silicon):** `curl -sL https://github.com/JulienTant/blogwatcher-cli/releases/latest/download/blogwatcher-cli_darwin_arm64.tar.gz | tar xz -C /usr/local/bin blogwatcher-cli`
|
||||
- **Binary (macOS Intel):** `curl -sL https://github.com/JulienTant/blogwatcher-cli/releases/latest/download/blogwatcher-cli_darwin_amd64.tar.gz | tar xz -C /usr/local/bin blogwatcher-cli`
|
||||
|
||||
All releases: https://github.com/JulienTant/blogwatcher-cli/releases
|
||||
|
||||
### Docker with persistent storage
|
||||
|
||||
By default the database lives at `~/.blogwatcher-cli/blogwatcher-cli.db`. In Docker this is lost on container restart. Use `BLOGWATCHER_DB` or a volume mount to persist it:
|
||||
|
||||
```bash
|
||||
# Named volume (simplest)
|
||||
docker run --rm -v blogwatcher-cli:/data -e BLOGWATCHER_DB=/data/blogwatcher-cli.db ghcr.io/julientant/blogwatcher-cli scan
|
||||
|
||||
# Host bind mount
|
||||
docker run --rm -v /path/on/host:/data -e BLOGWATCHER_DB=/data/blogwatcher-cli.db ghcr.io/julientant/blogwatcher-cli scan
|
||||
```
|
||||
|
||||
### Migrating from the original blogwatcher
|
||||
|
||||
If upgrading from `Hyaxia/blogwatcher`, move your database:
|
||||
|
||||
```bash
|
||||
mv ~/.blogwatcher/blogwatcher.db ~/.blogwatcher-cli/blogwatcher-cli.db
|
||||
```
|
||||
|
||||
The binary name changed from `blogwatcher` to `blogwatcher-cli`.
|
||||
|
||||
## Common Commands
|
||||
|
||||
### Managing blogs
|
||||
|
||||
- Add a blog: `blogwatcher-cli add "My Blog" https://example.com`
|
||||
- Add with explicit feed: `blogwatcher-cli add "My Blog" https://example.com --feed-url https://example.com/feed.xml`
|
||||
- Add with HTML scraping: `blogwatcher-cli add "My Blog" https://example.com --scrape-selector "article h2 a"`
|
||||
- List tracked blogs: `blogwatcher-cli blogs`
|
||||
- Remove a blog: `blogwatcher-cli remove "My Blog" --yes`
|
||||
- Import from OPML: `blogwatcher-cli import subscriptions.opml`
|
||||
|
||||
### Scanning and reading
|
||||
|
||||
- Scan all blogs: `blogwatcher-cli scan`
|
||||
- Scan one blog: `blogwatcher-cli scan "My Blog"`
|
||||
- List unread articles: `blogwatcher-cli articles`
|
||||
- List all articles: `blogwatcher-cli articles --all`
|
||||
- Filter by blog: `blogwatcher-cli articles --blog "My Blog"`
|
||||
- Filter by category: `blogwatcher-cli articles --category "Engineering"`
|
||||
- Mark article read: `blogwatcher-cli read 1`
|
||||
- Mark article unread: `blogwatcher-cli unread 1`
|
||||
- Mark all read: `blogwatcher-cli read-all`
|
||||
- Mark all read for a blog: `blogwatcher-cli read-all --blog "My Blog" --yes`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All flags can be set via environment variables with the `BLOGWATCHER_` prefix:
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `BLOGWATCHER_DB` | Path to SQLite database file |
|
||||
| `BLOGWATCHER_WORKERS` | Number of concurrent scan workers (default: 8) |
|
||||
| `BLOGWATCHER_SILENT` | Only output "scan done" when scanning |
|
||||
| `BLOGWATCHER_YES` | Skip confirmation prompts |
|
||||
| `BLOGWATCHER_CATEGORY` | Default filter for articles by category |
|
||||
|
||||
## Example Output
|
||||
|
||||
```
|
||||
$ blogwatcher-cli blogs
|
||||
Tracked blogs (1):
|
||||
|
||||
xkcd
|
||||
URL: https://xkcd.com
|
||||
Feed: https://xkcd.com/atom.xml
|
||||
Last scanned: 2026-04-03 10:30
|
||||
```
|
||||
|
||||
```
|
||||
$ blogwatcher-cli scan
|
||||
Scanning 1 blog(s)...
|
||||
|
||||
xkcd
|
||||
Source: RSS | Found: 4 | New: 4
|
||||
|
||||
Found 4 new article(s) total!
|
||||
```
|
||||
|
||||
```
|
||||
$ blogwatcher-cli articles
|
||||
Unread articles (2):
|
||||
|
||||
[1] [new] Barrel - Part 13
|
||||
Blog: xkcd
|
||||
URL: https://xkcd.com/3095/
|
||||
Published: 2026-04-02
|
||||
Categories: Comics, Science
|
||||
|
||||
[2] [new] Volcano Fact
|
||||
Blog: xkcd
|
||||
URL: https://xkcd.com/3094/
|
||||
Published: 2026-04-01
|
||||
Categories: Comics
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Auto-discovers RSS/Atom feeds from blog homepages when no `--feed-url` is provided.
|
||||
- Falls back to HTML scraping if RSS fails and `--scrape-selector` is configured.
|
||||
- Categories from RSS/Atom feeds are stored and can be used to filter articles.
|
||||
- Import blogs in bulk from OPML files exported by Feedly, Inoreader, NewsBlur, etc.
|
||||
- Database stored at `~/.blogwatcher-cli/blogwatcher-cli.db` by default (override with `--db` or `BLOGWATCHER_DB`).
|
||||
- Use `blogwatcher-cli <command> --help` to discover all flags and options.
|
||||
@@ -0,0 +1,199 @@
|
||||
---
|
||||
name: darwinian-evolver
|
||||
description: Evolve prompts/regex/SQL/code with Imbue's evolution loop.
|
||||
version: 0.1.0
|
||||
author: Bihruze (Asahi0x), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [evolution, optimization, prompt-engineering, research]
|
||||
related_skills: [arxiv, jupyter-notebook]
|
||||
---
|
||||
|
||||
# Darwinian Evolver
|
||||
|
||||
Run Imbue's [darwinian_evolver](https://github.com/imbue-ai/darwinian_evolver) — an
|
||||
LLM-driven evolutionary search loop — to optimize a **prompt, regex, SQL query,
|
||||
or small code snippet** against a fitness function.
|
||||
|
||||
Status: thin wrapper around the upstream tool. The skill installs it, walks the
|
||||
agent through writing a `Problem` definition (organism + evaluator + mutator),
|
||||
and drives the loop via the upstream CLI or a small custom Python driver.
|
||||
|
||||
**License:** the upstream tool is **AGPL-3.0**. The skill ONLY ever invokes it
|
||||
via the upstream CLI or a `subprocess`/`uv run` call (mere aggregation). Do NOT
|
||||
import upstream classes into Hermes itself.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User says "optimize this prompt", "evolve a regex for X", "auto-improve this
|
||||
code/SQL", "search for a better instruction".
|
||||
- You have a scorer (exact match, regex pass-rate, unit test, LLM-judge, runtime
|
||||
metric) AND a starting candidate (organism). If you don't have a scorer, stop
|
||||
and define one first — that's the hard part.
|
||||
- Cost is OK: a typical run is 50–500 LLM calls. On gpt-4o-mini that's pennies;
|
||||
on Claude Sonnet it can be a few dollars.
|
||||
|
||||
Do **not** use this when:
|
||||
- The optimization target is differentiable (use gradient descent / DSPy).
|
||||
- You only need to try 2–3 variants — just write them by hand.
|
||||
- The fitness signal is purely subjective with no measurable criterion.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python ≥3.11
|
||||
- `git`, `uv` (or `pip`)
|
||||
- One of: `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`, or `OPENAI_API_KEY`
|
||||
|
||||
The skill ships a small `parrot_openrouter.py` driver that uses `OPENROUTER_API_KEY`
|
||||
via the OpenAI SDK, so any model on OpenRouter works. The upstream CLI itself
|
||||
hardcodes Anthropic and needs `ANTHROPIC_API_KEY`.
|
||||
|
||||
## Install (One-Time)
|
||||
|
||||
Run via the `terminal` tool:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.hermes/cache/darwinian-evolver && cd ~/.hermes/cache/darwinian-evolver
|
||||
[ -d darwinian_evolver ] || git clone --depth 1 https://github.com/imbue-ai/darwinian_evolver.git
|
||||
cd darwinian_evolver && uv sync
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
cd ~/.hermes/cache/darwinian-evolver/darwinian_evolver \
|
||||
&& uv run darwinian_evolver --help | head -5
|
||||
```
|
||||
|
||||
## Quick Start — The Built-In Parrot Example
|
||||
|
||||
Tiny smoke test (requires `ANTHROPIC_API_KEY`):
|
||||
|
||||
```bash
|
||||
cd ~/.hermes/cache/darwinian-evolver/darwinian_evolver
|
||||
uv run darwinian_evolver parrot \
|
||||
--num_iterations 2 \
|
||||
--num_parents_per_iteration 2 \
|
||||
--mutator_concurrency 2 --evaluator_concurrency 2 \
|
||||
--output_dir /tmp/parrot_demo
|
||||
```
|
||||
|
||||
Outputs:
|
||||
- `/tmp/parrot_demo/snapshots/iteration_N.pkl` — pickled population per iteration
|
||||
- `/tmp/parrot_demo/<jsonl>` — per-iteration JSON log (path printed at end)
|
||||
|
||||
Open `~/.hermes/cache/darwinian-evolver/darwinian_evolver/darwinian_evolver/lineage_visualizer.html`
|
||||
in a browser and load the JSON log to see the evolutionary tree.
|
||||
|
||||
## Quick Start — OpenRouter Driver (No Anthropic Key)
|
||||
|
||||
The skill ships `scripts/parrot_openrouter.py` — same parrot problem, but the
|
||||
LLM call goes through OpenRouter so any provider works.
|
||||
|
||||
```bash
|
||||
# From wherever the skill is installed:
|
||||
SKILL_DIR=~/.hermes/skills/research/darwinian-evolver
|
||||
DE_DIR=~/.hermes/cache/darwinian-evolver/darwinian_evolver
|
||||
|
||||
cd "$DE_DIR" && \
|
||||
EVOLVER_MODEL='openai/gpt-4o-mini' \
|
||||
uv run --with openai python "$SKILL_DIR/scripts/parrot_openrouter.py" \
|
||||
--num_iterations 3 --num_parents_per_iteration 2 \
|
||||
--output_dir /tmp/parrot_or
|
||||
```
|
||||
|
||||
Inspect the result with `scripts/show_snapshot.py`:
|
||||
|
||||
```bash
|
||||
uv run --with openai python "$SKILL_DIR/scripts/show_snapshot.py" \
|
||||
/tmp/parrot_or/snapshots/iteration_3.pkl
|
||||
```
|
||||
|
||||
Expected output: 7 evolved prompt templates ranked by score, with the best
|
||||
landing around 0.6–0.8 (the seed `Say {{ phrase }}` scored 0.000).
|
||||
|
||||
## Defining a Custom Problem
|
||||
|
||||
The skill ships `templates/custom_problem_template.py` — copy, edit, run.
|
||||
Three things you must define:
|
||||
|
||||
1. **`Organism`** — a Pydantic `BaseModel` subclass holding the artifact being
|
||||
evolved (`prompt_template: str`, `regex_pattern: str`, `sql_query: str`,
|
||||
`code_block: str`, etc.). Add a `run(*args)` method that exercises it.
|
||||
|
||||
2. **`Evaluator`** — `.evaluate(organism) -> EvaluationResult(score=..., trainable_failure_cases=[...], holdout_failure_cases=[...], is_viable=True)`.
|
||||
- **`score`** is in `[0, 1]`. Higher is better.
|
||||
- **`trainable_failure_cases`** — what the mutator sees. Include enough
|
||||
context (input, expected, actual) for the LLM to diagnose.
|
||||
- **`holdout_failure_cases`** — kept out of the mutator's view. Use these
|
||||
to detect overfitting.
|
||||
- **`is_viable=True`** unless the organism is completely broken (raises,
|
||||
returns None, etc.). A 0-score viable organism is fine — it just gets
|
||||
down-weighted in parent selection.
|
||||
|
||||
3. **`Mutator`** — `.mutate(organism, failure_cases, learning_log_entries) -> list[Organism]`.
|
||||
Typically: build an LLM prompt that includes the current organism + a
|
||||
failure case + an ask to propose a fix; parse the LLM's response; return
|
||||
a new `Organism`. Return `[]` on parse failure — the loop handles it.
|
||||
|
||||
Then write a driver script that wires `Problem(initial_organism, evaluator, [mutators])`
|
||||
into `EvolveProblemLoop` and iterates over `loop.run(num_iterations=N)` — the
|
||||
shipped `scripts/parrot_openrouter.py` is the reference.
|
||||
|
||||
## Hyperparameters That Actually Matter
|
||||
|
||||
| flag | default | when to change |
|
||||
|---|---|---|
|
||||
| `--num_iterations` | 5 | bump to 10–20 once you trust the evaluator |
|
||||
| `--num_parents_per_iteration` | 4 | drop to 2 for cheap exploration |
|
||||
| `--mutator_concurrency` | 10 | drop to 2–4 to avoid rate limits |
|
||||
| `--evaluator_concurrency` | 10 | same; evaluator hits the LLM too |
|
||||
| `--batch_size` | 1 | raise to 3–5 once your mutator handles multiple failures |
|
||||
| `--verify_mutations` | off | turn on once mutator is wasteful (>10× cost saving on later runs per Imbue) |
|
||||
| `--midpoint_score` | `p75` | leave alone unless scores cluster |
|
||||
| `--sharpness` | 10 | leave alone |
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **`Initial organism must be viable`** — set `is_viable=True` in your
|
||||
`EvaluationResult` even on a 0-score seed. The loop refuses non-viable
|
||||
organisms because they imply the loop has nothing to evolve from.
|
||||
2. **Provider content filters kill runs.** Azure-backed OpenRouter models
|
||||
reject phrases like "ignore previous instructions" with HTTP 400. Wrap
|
||||
the LLM call in `try/except` and return `f"<LLM_ERROR: {e}>"` — the
|
||||
evolver will just score that organism 0 and move on.
|
||||
3. **`loop.run()` is a generator** — calling it doesn't run anything until
|
||||
you iterate. Use `for snap in loop.run(num_iterations=N):`.
|
||||
4. **Snapshots are nested pickles.** `iteration_N.pkl` contains a dict with
|
||||
`population_snapshot` (more pickled bytes). To unpickle you must have the
|
||||
`Organism` class importable under the same dotted path it was pickled at.
|
||||
5. **Concurrency defaults are aggressive.** 10/10 will hit rate limits on
|
||||
most providers. Start with 2/2.
|
||||
6. **CLI is hardcoded to Anthropic.** `uv run darwinian_evolver <problem>`
|
||||
reaches for `ANTHROPIC_API_KEY` and uses Claude Sonnet. To use any other
|
||||
provider, write a driver like `parrot_openrouter.py`.
|
||||
7. **AGPL.** Never `from darwinian_evolver import ...` inside Hermes core.
|
||||
Custom driver scripts under `~/.hermes/skills/...` are user-side and fine.
|
||||
8. **No PyPI package.** `pip install darwinian-evolver` will pull the wrong
|
||||
thing. Always install from the GitHub repo.
|
||||
|
||||
## Verification
|
||||
|
||||
After install + a parrot run, exit code 0 from this is sufficient:
|
||||
|
||||
```bash
|
||||
DE_DIR=~/.hermes/cache/darwinian-evolver/darwinian_evolver
|
||||
ls "$DE_DIR/darwinian_evolver/lineage_visualizer.html" >/dev/null && \
|
||||
cd "$DE_DIR" && uv run darwinian_evolver --help >/dev/null && \
|
||||
echo "darwinian-evolver: OK"
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- [Imbue research post](https://imbue.com/research/2026-02-27-darwinian-evolver/)
|
||||
- [ARC-AGI-2 results](https://imbue.com/research/2026-02-27-arc-agi-2-evolution/)
|
||||
- [imbue-ai/darwinian_evolver](https://github.com/imbue-ai/darwinian_evolver) (AGPL-3.0)
|
||||
- [Darwin Gödel Machines](https://arxiv.org/abs/2505.22954)
|
||||
- [PromptBreeder](https://arxiv.org/abs/2309.16797)
|
||||
@@ -0,0 +1,218 @@
|
||||
"""
|
||||
parrot_openrouter: same as the upstream `parrot` example but the LLM call goes
|
||||
through OpenRouter (OpenAI SDK) instead of Anthropic native. Lets us run an
|
||||
end-to-end evolution with whatever model the user already has paid access to.
|
||||
|
||||
Run with:
|
||||
uv --project darwinian_evolver run python parrot_openrouter.py \
|
||||
--num_iterations 3 --output_dir /tmp/parrot_out
|
||||
|
||||
Reads `OPENROUTER_API_KEY` from the environment.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import jinja2
|
||||
from openai import OpenAI
|
||||
|
||||
# Vendored problem types from upstream (AGPL — only run via subprocess in production)
|
||||
from darwinian_evolver.cli_common import build_hyperparameter_config_from_args
|
||||
from darwinian_evolver.cli_common import register_hyperparameter_args
|
||||
from darwinian_evolver.cli_common import parse_learning_log_view_type
|
||||
from darwinian_evolver.evolve_problem_loop import EvolveProblemLoop
|
||||
from darwinian_evolver.learning_log import LearningLogEntry
|
||||
from darwinian_evolver.problem import EvaluationFailureCase
|
||||
from darwinian_evolver.problem import EvaluationResult
|
||||
from darwinian_evolver.problem import Evaluator
|
||||
from darwinian_evolver.problem import Mutator
|
||||
from darwinian_evolver.problem import Organism
|
||||
from darwinian_evolver.problem import Problem
|
||||
|
||||
DEFAULT_MODEL = os.environ.get("EVOLVER_MODEL", "openai/gpt-4o-mini")
|
||||
|
||||
|
||||
def _client() -> OpenAI:
|
||||
key = os.environ.get("OPENROUTER_API_KEY")
|
||||
if not key:
|
||||
sys.exit("OPENROUTER_API_KEY is not set")
|
||||
return OpenAI(api_key=key, base_url="https://openrouter.ai/api/v1")
|
||||
|
||||
|
||||
def _prompt_llm(prompt: str) -> str:
|
||||
try:
|
||||
r = _client().chat.completions.create(
|
||||
model=DEFAULT_MODEL,
|
||||
max_tokens=1024,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
return r.choices[0].message.content or ""
|
||||
except Exception as e:
|
||||
# Treat any provider error (rate limit, content filter, schema reject)
|
||||
# as a failed response. The evolver will simply see this as a low score
|
||||
# on this organism and move on — much friendlier than killing the run.
|
||||
return f"<LLM_ERROR: {type(e).__name__}: {e}>"
|
||||
|
||||
|
||||
class ParrotOrganism(Organism):
|
||||
prompt_template: str
|
||||
|
||||
def run(self, phrase: str) -> str:
|
||||
try:
|
||||
prompt = jinja2.Template(self.prompt_template).render(phrase=phrase)
|
||||
except jinja2.exceptions.TemplateError as e:
|
||||
return f"Error rendering prompt: {e}"
|
||||
if not prompt:
|
||||
return ""
|
||||
return _prompt_llm(prompt)
|
||||
|
||||
|
||||
class ParrotEvaluationFailureCase(EvaluationFailureCase):
|
||||
phrase: str
|
||||
response: str
|
||||
|
||||
|
||||
class ImproveParrotMutator(Mutator[ParrotOrganism, ParrotEvaluationFailureCase]):
|
||||
IMPROVEMENT_PROMPT_TEMPLATE = """
|
||||
We want to build a prompt that causes an LLM to repeat back a given phrase verbatim.
|
||||
|
||||
The current prompt template is:
|
||||
```
|
||||
{{ organism.prompt_template }}
|
||||
```
|
||||
|
||||
Unfortunately, on this phrase:
|
||||
```
|
||||
{{ failure_case.phrase }}
|
||||
```
|
||||
the LLM responded with:
|
||||
```
|
||||
{{ failure_case.response }}
|
||||
```
|
||||
|
||||
Diagnose what went wrong, then propose an improved prompt template. Put the new
|
||||
template in the LAST triple-backtick block of your response.
|
||||
""".strip()
|
||||
|
||||
def mutate(
|
||||
self,
|
||||
organism: ParrotOrganism,
|
||||
failure_cases: list[ParrotEvaluationFailureCase],
|
||||
learning_log_entries: list[LearningLogEntry],
|
||||
) -> list[ParrotOrganism]:
|
||||
fc = failure_cases[0]
|
||||
prompt = jinja2.Template(self.IMPROVEMENT_PROMPT_TEMPLATE).render(
|
||||
organism=organism, failure_case=fc
|
||||
)
|
||||
try:
|
||||
resp = _prompt_llm(prompt)
|
||||
parts = resp.split("```")
|
||||
if len(parts) < 3:
|
||||
return []
|
||||
new_tpl = parts[-2].strip()
|
||||
return [ParrotOrganism(prompt_template=new_tpl)]
|
||||
except Exception as e:
|
||||
print(f"mutate error: {e}", file=sys.stderr)
|
||||
return []
|
||||
|
||||
|
||||
class ParrotEvaluator(Evaluator[ParrotOrganism, EvaluationResult, ParrotEvaluationFailureCase]):
|
||||
TRAINABLE_PHRASES = [
|
||||
"Hello world.",
|
||||
"bla",
|
||||
"Bla",
|
||||
"bla.",
|
||||
'"bla bla".',
|
||||
"Just say 'foo' once with no extra words.",
|
||||
]
|
||||
HOLDOUT_PHRASES = [
|
||||
"bla, but only once.",
|
||||
"'bla'",
|
||||
]
|
||||
|
||||
def evaluate(self, organism: ParrotOrganism) -> EvaluationResult:
|
||||
train_fails: list[ParrotEvaluationFailureCase] = []
|
||||
hold_fails: list[ParrotEvaluationFailureCase] = []
|
||||
for i, p in enumerate(self.TRAINABLE_PHRASES):
|
||||
r = organism.run(p)
|
||||
if r != p:
|
||||
train_fails.append(ParrotEvaluationFailureCase(
|
||||
phrase=p, response=r, data_point_id=f"trainable_{i}"))
|
||||
for i, p in enumerate(self.HOLDOUT_PHRASES):
|
||||
r = organism.run(p)
|
||||
if r != p:
|
||||
hold_fails.append(ParrotEvaluationFailureCase(
|
||||
phrase=p, response=r, data_point_id=f"holdout_{i}"))
|
||||
n_total = len(self.TRAINABLE_PHRASES) + len(self.HOLDOUT_PHRASES)
|
||||
n_ok = n_total - len(train_fails) - len(hold_fails)
|
||||
return EvaluationResult(
|
||||
score=n_ok / n_total,
|
||||
trainable_failure_cases=train_fails,
|
||||
holdout_failure_cases=hold_fails,
|
||||
# Always viable. Even a 0-score seed is a valid starting point; the
|
||||
# mutator should still get a chance to fix it.
|
||||
is_viable=True,
|
||||
)
|
||||
|
||||
|
||||
def make_problem() -> Problem:
|
||||
return Problem[ParrotOrganism, EvaluationResult, ParrotEvaluationFailureCase](
|
||||
evaluator=ParrotEvaluator(),
|
||||
mutators=[ImproveParrotMutator()],
|
||||
initial_organism=ParrotOrganism(prompt_template="Say {{ phrase }}"),
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
register_hyperparameter_args(ap.add_argument_group("hyperparameters"))
|
||||
ap.add_argument("--num_iterations", type=int, default=3)
|
||||
ap.add_argument("--mutator_concurrency", type=int, default=4)
|
||||
ap.add_argument("--evaluator_concurrency", type=int, default=4)
|
||||
ap.add_argument("--output_dir", type=str, required=True)
|
||||
args = ap.parse_args()
|
||||
|
||||
out = Path(args.output_dir)
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
hp = build_hyperparameter_config_from_args(args)
|
||||
loop = EvolveProblemLoop(
|
||||
problem=make_problem(),
|
||||
learning_log_view_type=parse_learning_log_view_type(hp.learning_log_view_type),
|
||||
num_parents_per_iteration=hp.num_parents_per_iteration,
|
||||
mutator_concurrency=args.mutator_concurrency,
|
||||
evaluator_concurrency=args.evaluator_concurrency,
|
||||
fixed_midpoint_score=hp.fixed_midpoint_score,
|
||||
midpoint_score_percentile=hp.midpoint_score_percentile,
|
||||
sharpness=hp.sharpness,
|
||||
novelty_weight=hp.novelty_weight,
|
||||
batch_size=hp.batch_size,
|
||||
should_verify_mutations=hp.verify_mutations,
|
||||
)
|
||||
|
||||
import json
|
||||
log_path = out / "results.jsonl"
|
||||
snap_dir = out / "snapshots"
|
||||
snap_dir.mkdir(exist_ok=True)
|
||||
print("Evaluating initial organism...")
|
||||
for snap in loop.run(num_iterations=args.num_iterations):
|
||||
(snap_dir / f"iteration_{snap.iteration}.pkl").write_bytes(snap.snapshot)
|
||||
_, best_eval = snap.best_organism_result
|
||||
print(f"iter={snap.iteration} pop={snap.population_size} "
|
||||
f"best_score={best_eval.score:.3f}")
|
||||
with log_path.open("a") as f:
|
||||
f.write(json.dumps({
|
||||
"iteration": snap.iteration,
|
||||
"best_score": best_eval.score,
|
||||
"pop_size": snap.population_size,
|
||||
"score_percentiles": {str(k): v for k, v in snap.score_percentiles.items()},
|
||||
}) + "\n")
|
||||
print(f"\nDone. Results in: {out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,92 @@
|
||||
"""
|
||||
show_snapshot.py — Dump the population from a darwinian-evolver snapshot pickle.
|
||||
|
||||
Usage:
|
||||
python show_snapshot.py PATH/TO/iteration_N.pkl [--field prompt_template]
|
||||
|
||||
The script is intentionally Organism-agnostic: it walks `org.__dict__` and prints
|
||||
all str fields. By default it shows `prompt_template` if present; pass --field to
|
||||
target a different attribute (e.g. `regex_pattern`, `sql_query`, `code_block`).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import pickle
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("snapshot", type=Path)
|
||||
ap.add_argument(
|
||||
"--field",
|
||||
default=None,
|
||||
help="Organism attribute to display. Defaults to the first str field found.",
|
||||
)
|
||||
ap.add_argument("--top", type=int, default=None, help="Show only top N by score.")
|
||||
ap.add_argument(
|
||||
"--i-trust-this-file",
|
||||
action="store_true",
|
||||
help=(
|
||||
"Required acknowledgement that the snapshot is from a trusted source. "
|
||||
"pickle.loads executes arbitrary code embedded in the file (RCE) and "
|
||||
"must NEVER be run on snapshots received from untrusted parties."
|
||||
),
|
||||
)
|
||||
args = ap.parse_args()
|
||||
|
||||
if not args.snapshot.exists():
|
||||
sys.exit(f"snapshot not found: {args.snapshot}")
|
||||
|
||||
if not args.i_trust_this_file:
|
||||
sys.exit(
|
||||
"refusing to unpickle: pickle.loads is equivalent to executing arbitrary "
|
||||
"code from the snapshot file. Only proceed if you created/control this "
|
||||
"file, then re-run with --i-trust-this-file.\n"
|
||||
f" file: {args.snapshot}"
|
||||
)
|
||||
|
||||
print(
|
||||
f"WARNING: unpickling {args.snapshot} — this executes code embedded in the "
|
||||
"file. Only safe for snapshots you produced yourself.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
# The outer pickle wraps a dict; the inner pickle contains the actual organism
|
||||
# objects, which must be importable under their original dotted path. If you
|
||||
# ran a custom driver, make sure its module is on sys.path before calling this.
|
||||
outer = pickle.loads(args.snapshot.read_bytes()) # noqa: S301 — gated by --i-trust-this-file
|
||||
if not isinstance(outer, dict) or "population_snapshot" not in outer:
|
||||
sys.exit("not a darwinian-evolver snapshot (no population_snapshot key)")
|
||||
inner = pickle.loads(outer["population_snapshot"]) # noqa: S301 — gated by --i-trust-this-file
|
||||
pairs = inner["organisms"] # list of (Organism, EvaluationResult)
|
||||
|
||||
print(f"# organisms: {len(pairs)}\n")
|
||||
ranked = sorted(pairs, key=lambda p: getattr(p[1], "score", 0) or 0, reverse=True)
|
||||
if args.top:
|
||||
ranked = ranked[: args.top]
|
||||
|
||||
for i, (org, res) in enumerate(ranked):
|
||||
score = getattr(res, "score", float("nan"))
|
||||
print(f"=== rank {i} score={score:.3f} ===")
|
||||
# pick field
|
||||
field = args.field
|
||||
if field is None:
|
||||
for k, v in vars(org).items():
|
||||
if isinstance(v, str) and not k.startswith("_") and k not in {"id",}:
|
||||
field = k
|
||||
break
|
||||
val = getattr(org, field, None) if field else None
|
||||
if val is None:
|
||||
print(f" (no string field; org fields: {list(vars(org).keys())})")
|
||||
else:
|
||||
print(f" {field} ({len(val)} chars):")
|
||||
for ln in val.splitlines()[:30]:
|
||||
print(f" {ln}")
|
||||
print()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,240 @@
|
||||
"""
|
||||
Template: a custom darwinian-evolver problem.
|
||||
|
||||
Copy this file, fill in the THREE marked spots (Organism, Evaluator, Mutator),
|
||||
then run it as a driver script. The skeleton handles all the wiring so you only
|
||||
write the domain-specific logic.
|
||||
|
||||
To run:
|
||||
cd ~/.hermes/cache/darwinian-evolver/darwinian_evolver
|
||||
OPENROUTER_API_KEY=... uv run --with openai python /path/to/this_file.py \
|
||||
--num_iterations 3 --num_parents_per_iteration 2 \
|
||||
--output_dir /tmp/my_problem
|
||||
|
||||
The pattern mirrors `scripts/parrot_openrouter.py` (the working reference).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
# Upstream types (AGPL — invoked via subprocess in production; importing here
|
||||
# is fine for skill-side driver scripts the user owns).
|
||||
from darwinian_evolver.cli_common import (
|
||||
build_hyperparameter_config_from_args,
|
||||
parse_learning_log_view_type,
|
||||
register_hyperparameter_args,
|
||||
)
|
||||
from darwinian_evolver.evolve_problem_loop import EvolveProblemLoop
|
||||
from darwinian_evolver.learning_log import LearningLogEntry
|
||||
from darwinian_evolver.problem import (
|
||||
EvaluationFailureCase,
|
||||
EvaluationResult,
|
||||
Evaluator,
|
||||
Mutator,
|
||||
Organism,
|
||||
Problem,
|
||||
)
|
||||
|
||||
DEFAULT_MODEL = os.environ.get("EVOLVER_MODEL", "openai/gpt-4o-mini")
|
||||
|
||||
|
||||
def _client() -> OpenAI:
|
||||
key = os.environ.get("OPENROUTER_API_KEY")
|
||||
if not key:
|
||||
sys.exit("OPENROUTER_API_KEY is not set")
|
||||
return OpenAI(api_key=key, base_url="https://openrouter.ai/api/v1")
|
||||
|
||||
|
||||
def _prompt_llm(prompt: str, max_tokens: int = 1024) -> str:
|
||||
try:
|
||||
r = _client().chat.completions.create(
|
||||
model=DEFAULT_MODEL,
|
||||
max_tokens=max_tokens,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
return r.choices[0].message.content or ""
|
||||
except Exception as e:
|
||||
# Never let one bad LLM response kill the run.
|
||||
return f"<LLM_ERROR: {type(e).__name__}: {e}>"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. ORGANISM — what you are evolving.
|
||||
# ---------------------------------------------------------------------------
|
||||
class MyOrganism(Organism):
|
||||
# TODO: replace with your artifact field. Common shapes:
|
||||
# prompt_template: str
|
||||
# regex_pattern: str
|
||||
# sql_query: str
|
||||
# code_block: str
|
||||
artifact: str
|
||||
|
||||
def run(self, *inputs) -> str:
|
||||
"""Exercise the organism on a test input. Return whatever your
|
||||
evaluator wants to score."""
|
||||
# TODO: implement. For prompt evolution this typically calls _prompt_llm
|
||||
# with the artifact rendered against the input. For regex/SQL it would
|
||||
# call `re.findall(self.artifact, input)` / execute SQL / etc.
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. EVALUATOR — score organisms and surface failures the mutator can learn from.
|
||||
# ---------------------------------------------------------------------------
|
||||
class MyFailureCase(EvaluationFailureCase):
|
||||
# TODO: include enough context for the LLM to diagnose the failure.
|
||||
input: str
|
||||
expected: str
|
||||
actual: str
|
||||
|
||||
|
||||
class MyEvaluator(Evaluator[MyOrganism, EvaluationResult, MyFailureCase]):
|
||||
# Split your dataset. Mutator only sees trainable; holdout detects overfitting.
|
||||
TRAINABLE = [
|
||||
# TODO: list of (input, expected) tuples
|
||||
# ("input1", "expected1"),
|
||||
]
|
||||
HOLDOUT = [
|
||||
# TODO: separate set the mutator never sees
|
||||
]
|
||||
|
||||
def evaluate(self, organism: MyOrganism) -> EvaluationResult:
|
||||
train_fails: list[MyFailureCase] = []
|
||||
hold_fails: list[MyFailureCase] = []
|
||||
for i, (inp, expected) in enumerate(self.TRAINABLE):
|
||||
actual = organism.run(inp)
|
||||
if actual != expected:
|
||||
train_fails.append(MyFailureCase(
|
||||
input=inp, expected=expected, actual=actual,
|
||||
data_point_id=f"trainable_{i}",
|
||||
))
|
||||
for i, (inp, expected) in enumerate(self.HOLDOUT):
|
||||
actual = organism.run(inp)
|
||||
if actual != expected:
|
||||
hold_fails.append(MyFailureCase(
|
||||
input=inp, expected=expected, actual=actual,
|
||||
data_point_id=f"holdout_{i}",
|
||||
))
|
||||
n_total = len(self.TRAINABLE) + len(self.HOLDOUT)
|
||||
n_ok = n_total - len(train_fails) - len(hold_fails)
|
||||
return EvaluationResult(
|
||||
score=n_ok / n_total if n_total else 0.0,
|
||||
trainable_failure_cases=train_fails,
|
||||
holdout_failure_cases=hold_fails,
|
||||
# Always-viable. The evolver only blocks completely-broken organisms;
|
||||
# a 0-score organism is fine and will simply be sampled less often.
|
||||
is_viable=True,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. MUTATOR — LLM proposes an improved organism from a failure case.
|
||||
# ---------------------------------------------------------------------------
|
||||
class MyMutator(Mutator[MyOrganism, MyFailureCase]):
|
||||
PROMPT = """
|
||||
The current artifact is:
|
||||
```
|
||||
{artifact}
|
||||
```
|
||||
|
||||
On this input:
|
||||
```
|
||||
{input}
|
||||
```
|
||||
it produced:
|
||||
```
|
||||
{actual}
|
||||
```
|
||||
but we wanted:
|
||||
```
|
||||
{expected}
|
||||
```
|
||||
|
||||
Diagnose what went wrong, then propose an improved version of the artifact.
|
||||
Put the new version in the LAST triple-backtick block of your response.
|
||||
""".strip()
|
||||
|
||||
def mutate(
|
||||
self,
|
||||
organism: MyOrganism,
|
||||
failure_cases: list[MyFailureCase],
|
||||
learning_log_entries: list[LearningLogEntry],
|
||||
) -> list[MyOrganism]:
|
||||
fc = failure_cases[0]
|
||||
prompt = self.PROMPT.format(
|
||||
artifact=organism.artifact,
|
||||
input=fc.input,
|
||||
actual=fc.actual,
|
||||
expected=fc.expected,
|
||||
)
|
||||
resp = _prompt_llm(prompt)
|
||||
parts = resp.split("```")
|
||||
if len(parts) < 3:
|
||||
return []
|
||||
new_artifact = parts[-2].strip()
|
||||
# Strip an opening language tag like "python\n" or "sql\n"
|
||||
if "\n" in new_artifact:
|
||||
first_line, rest = new_artifact.split("\n", 1)
|
||||
if first_line and not first_line.startswith(" ") and len(first_line) < 20:
|
||||
new_artifact = rest
|
||||
return [MyOrganism(artifact=new_artifact)]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Driver — fills in the EvolveProblemLoop boilerplate. You shouldn't need to
|
||||
# touch anything below this line for a typical run.
|
||||
# ---------------------------------------------------------------------------
|
||||
def make_problem() -> Problem:
|
||||
initial = MyOrganism(artifact="TODO: starting artifact here") # TODO
|
||||
return Problem[MyOrganism, EvaluationResult, MyFailureCase](
|
||||
evaluator=MyEvaluator(),
|
||||
mutators=[MyMutator()],
|
||||
initial_organism=initial,
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
register_hyperparameter_args(ap.add_argument_group("hyperparameters"))
|
||||
ap.add_argument("--num_iterations", type=int, default=3)
|
||||
ap.add_argument("--mutator_concurrency", type=int, default=2)
|
||||
ap.add_argument("--evaluator_concurrency", type=int, default=2)
|
||||
ap.add_argument("--output_dir", type=str, required=True)
|
||||
args = ap.parse_args()
|
||||
|
||||
out = Path(args.output_dir)
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
(out / "snapshots").mkdir(exist_ok=True)
|
||||
|
||||
hp = build_hyperparameter_config_from_args(args)
|
||||
loop = EvolveProblemLoop(
|
||||
problem=make_problem(),
|
||||
learning_log_view_type=parse_learning_log_view_type(hp.learning_log_view_type),
|
||||
num_parents_per_iteration=hp.num_parents_per_iteration,
|
||||
mutator_concurrency=args.mutator_concurrency,
|
||||
evaluator_concurrency=args.evaluator_concurrency,
|
||||
fixed_midpoint_score=hp.fixed_midpoint_score,
|
||||
midpoint_score_percentile=hp.midpoint_score_percentile,
|
||||
sharpness=hp.sharpness,
|
||||
novelty_weight=hp.novelty_weight,
|
||||
batch_size=hp.batch_size,
|
||||
should_verify_mutations=hp.verify_mutations,
|
||||
)
|
||||
|
||||
print("Evaluating initial organism...")
|
||||
for snap in loop.run(num_iterations=args.num_iterations):
|
||||
(out / "snapshots" / f"iteration_{snap.iteration}.pkl").write_bytes(snap.snapshot)
|
||||
_, best = snap.best_organism_result
|
||||
print(f"iter={snap.iteration} pop={snap.population_size} best_score={best.score:.3f}")
|
||||
|
||||
print(f"\nDone. Results in: {out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,104 @@
|
||||
---
|
||||
name: domain-intel
|
||||
description: Passive recon of subdomains, SSL certs, WHOIS, and DNS.
|
||||
version: 1.0.0
|
||||
author: FurkanL0, Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Domains, OSINT, DNS, Research]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# Domain Intelligence — Passive OSINT
|
||||
|
||||
Passive domain reconnaissance using only Python stdlib.
|
||||
**Zero dependencies. Zero API keys. Works on Linux, macOS, and Windows.**
|
||||
|
||||
## Helper script
|
||||
|
||||
This skill includes `scripts/domain_intel.py` — a complete CLI tool for all domain intelligence operations.
|
||||
|
||||
```bash
|
||||
# Subdomain discovery via Certificate Transparency logs
|
||||
python SKILL_DIR/scripts/domain_intel.py subdomains example.com
|
||||
|
||||
# SSL certificate inspection (expiry, cipher, SANs, issuer)
|
||||
python SKILL_DIR/scripts/domain_intel.py ssl example.com
|
||||
|
||||
# WHOIS lookup (registrar, dates, name servers — 100+ TLDs)
|
||||
python SKILL_DIR/scripts/domain_intel.py whois example.com
|
||||
|
||||
# DNS records (A, AAAA, MX, NS, TXT, CNAME)
|
||||
python SKILL_DIR/scripts/domain_intel.py dns example.com
|
||||
|
||||
# Domain availability check (passive: DNS + WHOIS + SSL signals)
|
||||
python SKILL_DIR/scripts/domain_intel.py available coolstartup.io
|
||||
|
||||
# Bulk analysis — multiple domains, multiple checks in parallel
|
||||
python SKILL_DIR/scripts/domain_intel.py bulk example.com github.com google.com
|
||||
python SKILL_DIR/scripts/domain_intel.py bulk example.com github.com --checks ssl,dns
|
||||
```
|
||||
|
||||
`SKILL_DIR` is the directory containing this SKILL.md file. All output is structured JSON.
|
||||
|
||||
## Available commands
|
||||
|
||||
| Command | What it does | Data source |
|
||||
|---------|-------------|-------------|
|
||||
| `subdomains` | Find subdomains from certificate logs | crt.sh (HTTPS) |
|
||||
| `ssl` | Inspect TLS certificate details | Direct TCP:443 to target |
|
||||
| `whois` | Registration info, registrar, dates | WHOIS servers (TCP:43) |
|
||||
| `dns` | A, AAAA, MX, NS, TXT, CNAME records | System DNS + Google DoH |
|
||||
| `available` | Check if domain is registered | DNS + WHOIS + SSL signals |
|
||||
| `bulk` | Run multiple checks on multiple domains | All of the above |
|
||||
|
||||
## When to use this vs built-in tools
|
||||
|
||||
- **Use this skill** for infrastructure questions: subdomains, SSL certs, WHOIS, DNS records, availability
|
||||
- **Use `web_search`** for general research about what a domain/company does
|
||||
- **Use `web_extract`** to get the actual content of a webpage
|
||||
- **Use `terminal` with `curl -I`** for a simple "is this URL reachable" check
|
||||
|
||||
| Task | Better tool | Why |
|
||||
|------|-------------|-----|
|
||||
| "What does example.com do?" | `web_extract` | Gets page content, not DNS/WHOIS data |
|
||||
| "Find info about a company" | `web_search` | General research, not domain-specific |
|
||||
| "Is this website safe?" | `web_search` | Reputation checks need web context |
|
||||
| "Check if a URL is reachable" | `terminal` with `curl -I` | Simple HTTP check |
|
||||
| "Find subdomains of X" | **This skill** | Only passive source for this |
|
||||
| "When does the SSL cert expire?" | **This skill** | Built-in tools can't inspect TLS |
|
||||
| "Who registered this domain?" | **This skill** | WHOIS data not in web search |
|
||||
| "Is coolstartup.io available?" | **This skill** | Passive availability via DNS+WHOIS+SSL |
|
||||
|
||||
## Platform compatibility
|
||||
|
||||
Pure Python stdlib (`socket`, `ssl`, `urllib`, `json`, `concurrent.futures`).
|
||||
Works identically on Linux, macOS, and Windows with no dependencies.
|
||||
|
||||
- **crt.sh queries** use HTTPS (port 443) — works behind most firewalls
|
||||
- **WHOIS queries** use TCP port 43 — may be blocked on restrictive networks
|
||||
- **DNS queries** use Google DoH (HTTPS) for MX/NS/TXT — firewall-friendly
|
||||
- **SSL checks** connect to the target on port 443 — the only "active" operation
|
||||
|
||||
## Data sources
|
||||
|
||||
All queries are **passive** — no port scanning, no vulnerability testing:
|
||||
|
||||
- **crt.sh** — Certificate Transparency logs (subdomain discovery, HTTPS only)
|
||||
- **WHOIS servers** — Direct TCP to 100+ authoritative TLD registrars
|
||||
- **Google DNS-over-HTTPS** — MX, NS, TXT, CNAME resolution (firewall-friendly)
|
||||
- **System DNS** — A/AAAA record resolution
|
||||
- **SSL check** is the only "active" operation (TCP connection to target:443)
|
||||
|
||||
## Notes
|
||||
|
||||
- WHOIS queries use TCP port 43 — may be blocked on restrictive networks
|
||||
- Some WHOIS servers redact registrant info (GDPR) — mention this to the user
|
||||
- crt.sh can be slow for very popular domains (thousands of certs) — set reasonable expectations
|
||||
- The availability check is heuristic-based (3 passive signals) — not authoritative like a registrar API
|
||||
|
||||
---
|
||||
|
||||
*Contributed by [@FurkanL0](https://github.com/FurkanL0)*
|
||||
@@ -0,0 +1,397 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Domain Intelligence — Passive OSINT via Python stdlib.
|
||||
|
||||
Usage:
|
||||
python domain_intel.py subdomains example.com
|
||||
python domain_intel.py ssl example.com
|
||||
python domain_intel.py whois example.com
|
||||
python domain_intel.py dns example.com
|
||||
python domain_intel.py available example.com
|
||||
python domain_intel.py bulk example.com github.com google.com --checks ssl,dns
|
||||
|
||||
All output is structured JSON. No dependencies beyond Python stdlib.
|
||||
Works on Linux, macOS, and Windows.
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import socket
|
||||
import ssl
|
||||
import sys
|
||||
import urllib.request
|
||||
import urllib.parse
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
# ─── Subdomain Discovery (crt.sh) ──────────────────────────────────────────
|
||||
|
||||
def subdomains(domain, include_expired=False, limit=200):
|
||||
"""Find subdomains via Certificate Transparency logs."""
|
||||
url = f"https://crt.sh/?q=%25.{urllib.parse.quote(domain)}&output=json"
|
||||
req = urllib.request.Request(url, headers={
|
||||
"User-Agent": "domain-intel-skill/1.0", "Accept": "application/json",
|
||||
})
|
||||
with urllib.request.urlopen(req, timeout=15) as r:
|
||||
entries = json.loads(r.read().decode())
|
||||
|
||||
seen, results = set(), []
|
||||
now = datetime.now(timezone.utc)
|
||||
for e in entries:
|
||||
not_after = e.get("not_after", "")
|
||||
if not include_expired and not_after:
|
||||
try:
|
||||
dt = datetime.strptime(not_after[:19], "%Y-%m-%dT%H:%M:%S").replace(tzinfo=timezone.utc)
|
||||
if dt <= now:
|
||||
continue
|
||||
except ValueError:
|
||||
pass
|
||||
for name in e.get("name_value", "").splitlines():
|
||||
name = name.strip().lower()
|
||||
if name and name not in seen:
|
||||
seen.add(name)
|
||||
results.append({
|
||||
"subdomain": name,
|
||||
"issuer": e.get("issuer_name", ""),
|
||||
"not_after": not_after,
|
||||
})
|
||||
|
||||
results.sort(key=lambda r: (r["subdomain"].startswith("*"), r["subdomain"]))
|
||||
return {"domain": domain, "count": min(len(results), limit), "subdomains": results[:limit]}
|
||||
|
||||
|
||||
# ─── SSL Certificate Inspection ────────────────────────────────────────────
|
||||
|
||||
def check_ssl(host, port=443, timeout=10):
|
||||
"""Inspect the TLS certificate of a host."""
|
||||
def flat(rdns):
|
||||
r = {}
|
||||
for rdn in rdns:
|
||||
for item in rdn:
|
||||
if isinstance(item, (list, tuple)) and len(item) == 2:
|
||||
r[item[0]] = item[1]
|
||||
return r
|
||||
|
||||
def parse_date(s):
|
||||
for fmt in ("%b %d %H:%M:%S %Y %Z", "%b %d %H:%M:%S %Y %Z"):
|
||||
try:
|
||||
return datetime.strptime(s, fmt).replace(tzinfo=timezone.utc)
|
||||
except ValueError:
|
||||
pass
|
||||
return None
|
||||
|
||||
warning = None
|
||||
try:
|
||||
ctx = ssl.create_default_context()
|
||||
with socket.create_connection((host, port), timeout=timeout) as sock:
|
||||
with ctx.wrap_socket(sock, server_hostname=host) as s:
|
||||
cert, cipher, proto = s.getpeercert(), s.cipher(), s.version()
|
||||
except ssl.SSLCertVerificationError as e:
|
||||
warning = str(e)
|
||||
ctx = ssl.create_default_context()
|
||||
ctx.check_hostname = False
|
||||
ctx.verify_mode = ssl.CERT_NONE
|
||||
with socket.create_connection((host, port), timeout=timeout) as sock:
|
||||
with ctx.wrap_socket(sock, server_hostname=host) as s:
|
||||
cert, cipher, proto = s.getpeercert(), s.cipher(), s.version()
|
||||
|
||||
not_after = parse_date(cert.get("notAfter", ""))
|
||||
now = datetime.now(timezone.utc)
|
||||
days = (not_after - now).days if not_after else None
|
||||
is_expired = days is not None and days < 0
|
||||
|
||||
if is_expired:
|
||||
status = f"EXPIRED ({abs(days)} days ago)"
|
||||
elif days is not None and days <= 14:
|
||||
status = f"CRITICAL — {days} day(s) left"
|
||||
elif days is not None and days <= 30:
|
||||
status = f"WARNING — {days} day(s) left"
|
||||
else:
|
||||
status = f"OK — {days} day(s) remaining" if days is not None else "unknown"
|
||||
|
||||
return {
|
||||
"host": host, "port": port,
|
||||
"subject": flat(cert.get("subject", [])),
|
||||
"issuer": flat(cert.get("issuer", [])),
|
||||
"subject_alt_names": [f"{t}:{v}" for t, v in cert.get("subjectAltName", [])],
|
||||
"not_before": parse_date(cert.get("notBefore", "")).isoformat() if parse_date(cert.get("notBefore", "")) else "",
|
||||
"not_after": not_after.isoformat() if not_after else "",
|
||||
"days_remaining": days, "is_expired": is_expired, "expiry_status": status,
|
||||
"tls_version": proto,
|
||||
"cipher_suite": cipher[0] if cipher else None,
|
||||
"serial_number": cert.get("serialNumber", ""),
|
||||
"verification_warning": warning,
|
||||
}
|
||||
|
||||
|
||||
# ─── WHOIS Lookup ──────────────────────────────────────────────────────────
|
||||
|
||||
WHOIS_SERVERS = {
|
||||
"com": "whois.verisign-grs.com", "net": "whois.verisign-grs.com",
|
||||
"org": "whois.pir.org", "io": "whois.nic.io", "co": "whois.nic.co",
|
||||
"ai": "whois.nic.ai", "dev": "whois.nic.google", "app": "whois.nic.google",
|
||||
"tech": "whois.nic.tech", "shop": "whois.nic.shop", "store": "whois.nic.store",
|
||||
"online": "whois.nic.online", "site": "whois.nic.site", "cloud": "whois.nic.cloud",
|
||||
"digital": "whois.nic.digital", "media": "whois.nic.media", "blog": "whois.nic.blog",
|
||||
"info": "whois.afilias.net", "biz": "whois.biz", "me": "whois.nic.me",
|
||||
"tv": "whois.nic.tv", "cc": "whois.nic.cc", "ws": "whois.website.ws",
|
||||
"uk": "whois.nic.uk", "co.uk": "whois.nic.uk", "de": "whois.denic.de",
|
||||
"nl": "whois.domain-registry.nl", "fr": "whois.nic.fr", "it": "whois.nic.it",
|
||||
"es": "whois.nic.es", "pl": "whois.dns.pl", "ru": "whois.tcinet.ru",
|
||||
"se": "whois.iis.se", "no": "whois.norid.no", "fi": "whois.fi",
|
||||
"ch": "whois.nic.ch", "at": "whois.nic.at", "be": "whois.dns.be",
|
||||
"cz": "whois.nic.cz", "br": "whois.registro.br", "ca": "whois.cira.ca",
|
||||
"mx": "whois.mx", "au": "whois.auda.org.au", "jp": "whois.jprs.jp",
|
||||
"cn": "whois.cnnic.cn", "in": "whois.inregistry.net", "kr": "whois.kr",
|
||||
"sg": "whois.sgnic.sg", "hk": "whois.hkirc.hk", "tr": "whois.nic.tr",
|
||||
"ae": "whois.aeda.net.ae", "za": "whois.registry.net.za",
|
||||
"space": "whois.nic.space", "zone": "whois.nic.zone", "ninja": "whois.nic.ninja",
|
||||
"guru": "whois.nic.guru", "rocks": "whois.nic.rocks", "live": "whois.nic.live",
|
||||
"game": "whois.nic.game", "games": "whois.nic.games",
|
||||
}
|
||||
|
||||
|
||||
def whois_lookup(domain):
|
||||
"""Query WHOIS servers for domain registration info."""
|
||||
parts = domain.split(".")
|
||||
server = WHOIS_SERVERS.get(".".join(parts[-2:])) or WHOIS_SERVERS.get(parts[-1])
|
||||
if not server:
|
||||
return {"error": f"No WHOIS server for .{parts[-1]}"}
|
||||
|
||||
try:
|
||||
with socket.create_connection((server, 43), timeout=10) as s:
|
||||
s.sendall((domain + "\r\n").encode())
|
||||
chunks = []
|
||||
while True:
|
||||
c = s.recv(4096)
|
||||
if not c:
|
||||
break
|
||||
chunks.append(c)
|
||||
raw = b"".join(chunks).decode("utf-8", errors="replace")
|
||||
except Exception as e:
|
||||
return {"error": str(e)}
|
||||
|
||||
patterns = {
|
||||
"registrar": r"(?:Registrar|registrar):\s*(.+)",
|
||||
"creation_date": r"(?:Creation Date|Created|created):\s*(.+)",
|
||||
"expiration_date": r"(?:Registry Expiry Date|Expiration Date|Expiry Date):\s*(.+)",
|
||||
"updated_date": r"(?:Updated Date|Last Modified):\s*(.+)",
|
||||
"name_servers": r"(?:Name Server|nserver):\s*(.+)",
|
||||
"status": r"(?:Domain Status|status):\s*(.+)",
|
||||
"dnssec": r"DNSSEC:\s*(.+)",
|
||||
}
|
||||
result = {"domain": domain, "whois_server": server}
|
||||
for key, pat in patterns.items():
|
||||
matches = re.findall(pat, raw, re.IGNORECASE)
|
||||
if matches:
|
||||
if key in {"name_servers", "status"}:
|
||||
result[key] = list(dict.fromkeys(m.strip().lower() for m in matches))
|
||||
else:
|
||||
result[key] = matches[0].strip()
|
||||
|
||||
for field in ("creation_date", "expiration_date", "updated_date"):
|
||||
if field in result:
|
||||
for fmt in ("%Y-%m-%dT%H:%M:%S", "%Y-%m-%dT%H:%M:%SZ", "%Y-%m-%d %H:%M:%S", "%Y-%m-%d"):
|
||||
try:
|
||||
dt = datetime.strptime(result[field][:19], fmt).replace(tzinfo=timezone.utc)
|
||||
result[field] = dt.isoformat()
|
||||
if field == "expiration_date":
|
||||
days = (dt - datetime.now(timezone.utc)).days
|
||||
result["expiration_days_remaining"] = days
|
||||
result["is_expired"] = days < 0
|
||||
break
|
||||
except ValueError:
|
||||
pass
|
||||
return result
|
||||
|
||||
|
||||
# ─── DNS Records ───────────────────────────────────────────────────────────
|
||||
|
||||
def dns_records(domain, types=None):
|
||||
"""Resolve DNS records using system DNS + Google DoH."""
|
||||
if not types:
|
||||
types = ["A", "AAAA", "MX", "NS", "TXT", "CNAME"]
|
||||
records = {}
|
||||
|
||||
for qtype in types:
|
||||
if qtype == "A":
|
||||
try:
|
||||
records["A"] = list(dict.fromkeys(
|
||||
i[4][0] for i in socket.getaddrinfo(domain, None, socket.AF_INET)
|
||||
))
|
||||
except Exception:
|
||||
records["A"] = []
|
||||
elif qtype == "AAAA":
|
||||
try:
|
||||
records["AAAA"] = list(dict.fromkeys(
|
||||
i[4][0] for i in socket.getaddrinfo(domain, None, socket.AF_INET6)
|
||||
))
|
||||
except Exception:
|
||||
records["AAAA"] = []
|
||||
else:
|
||||
url = f"https://dns.google/resolve?name={urllib.parse.quote(domain)}&type={qtype}"
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "domain-intel-skill/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=10) as r:
|
||||
data = json.loads(r.read())
|
||||
records[qtype] = [
|
||||
a.get("data", "").strip().rstrip(".")
|
||||
for a in data.get("Answer", []) if a.get("data")
|
||||
]
|
||||
except Exception:
|
||||
records[qtype] = []
|
||||
|
||||
return {"domain": domain, "records": records}
|
||||
|
||||
|
||||
# ─── Domain Availability Check ─────────────────────────────────────────────
|
||||
|
||||
def check_available(domain):
|
||||
"""Check domain availability using passive signals (DNS + WHOIS + SSL)."""
|
||||
signals = {}
|
||||
|
||||
# DNS
|
||||
try:
|
||||
a = [i[4][0] for i in socket.getaddrinfo(domain, None, socket.AF_INET)]
|
||||
except Exception:
|
||||
a = []
|
||||
|
||||
try:
|
||||
ns_url = f"https://dns.google/resolve?name={urllib.parse.quote(domain)}&type=NS"
|
||||
req = urllib.request.Request(ns_url, headers={"User-Agent": "domain-intel-skill/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=10) as r:
|
||||
ns = [x.get("data", "") for x in json.loads(r.read()).get("Answer", [])]
|
||||
except Exception:
|
||||
ns = []
|
||||
|
||||
signals["dns_a"] = a
|
||||
signals["dns_ns"] = ns
|
||||
dns_exists = bool(a or ns)
|
||||
|
||||
# SSL
|
||||
ssl_up = False
|
||||
try:
|
||||
ctx = ssl.create_default_context()
|
||||
ctx.check_hostname = False
|
||||
ctx.verify_mode = ssl.CERT_NONE
|
||||
with socket.create_connection((domain, 443), timeout=3) as s:
|
||||
with ctx.wrap_socket(s, server_hostname=domain):
|
||||
ssl_up = True
|
||||
except Exception:
|
||||
pass
|
||||
signals["ssl_reachable"] = ssl_up
|
||||
|
||||
# WHOIS (quick check)
|
||||
tld = domain.rsplit(".", 1)[-1]
|
||||
server = WHOIS_SERVERS.get(tld)
|
||||
whois_avail = None
|
||||
whois_note = ""
|
||||
if server:
|
||||
try:
|
||||
with socket.create_connection((server, 43), timeout=10) as s:
|
||||
s.sendall((domain + "\r\n").encode())
|
||||
raw = b""
|
||||
while True:
|
||||
c = s.recv(4096)
|
||||
if not c:
|
||||
break
|
||||
raw += c
|
||||
raw = raw.decode("utf-8", errors="replace").lower()
|
||||
if any(p in raw for p in ["no match", "not found", "no data found", "status: free"]):
|
||||
whois_avail = True
|
||||
whois_note = "WHOIS: not found"
|
||||
elif "registrar:" in raw or "creation date:" in raw:
|
||||
whois_avail = False
|
||||
whois_note = "WHOIS: registered"
|
||||
else:
|
||||
whois_note = "WHOIS: inconclusive"
|
||||
except Exception as e:
|
||||
whois_note = f"WHOIS error: {e}"
|
||||
|
||||
signals["whois_available"] = whois_avail
|
||||
signals["whois_note"] = whois_note
|
||||
|
||||
if not dns_exists and whois_avail is True:
|
||||
verdict, conf = "LIKELY AVAILABLE", "high"
|
||||
elif dns_exists or whois_avail is False or ssl_up:
|
||||
verdict, conf = "REGISTERED / IN USE", "high"
|
||||
elif not dns_exists and whois_avail is None:
|
||||
verdict, conf = "POSSIBLY AVAILABLE", "medium"
|
||||
else:
|
||||
verdict, conf = "UNCERTAIN", "low"
|
||||
|
||||
return {"domain": domain, "verdict": verdict, "confidence": conf, "signals": signals}
|
||||
|
||||
|
||||
# ─── Bulk Analysis ─────────────────────────────────────────────────────────
|
||||
|
||||
COMMAND_MAP = {
|
||||
"subdomains": subdomains,
|
||||
"ssl": check_ssl,
|
||||
"whois": whois_lookup,
|
||||
"dns": dns_records,
|
||||
"available": check_available,
|
||||
}
|
||||
|
||||
|
||||
def bulk_check(domains, checks=None, max_workers=5):
|
||||
"""Run multiple checks across multiple domains in parallel."""
|
||||
if not checks:
|
||||
checks = ["ssl", "whois", "dns"]
|
||||
|
||||
def run_one(d):
|
||||
entry = {"domain": d}
|
||||
for check in checks:
|
||||
fn = COMMAND_MAP.get(check)
|
||||
if fn:
|
||||
try:
|
||||
entry[check] = fn(d)
|
||||
except Exception as e:
|
||||
entry[check] = {"error": str(e)}
|
||||
return entry
|
||||
|
||||
results = []
|
||||
with ThreadPoolExecutor(max_workers=min(max_workers, 10)) as ex:
|
||||
futures = {ex.submit(run_one, d): d for d in domains[:20]}
|
||||
for f in as_completed(futures):
|
||||
results.append(f.result())
|
||||
|
||||
return {"total": len(results), "checks": checks, "results": results}
|
||||
|
||||
|
||||
# ─── CLI Entry Point ───────────────────────────────────────────────────────
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 3:
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
|
||||
command = sys.argv[1].lower()
|
||||
args = sys.argv[2:]
|
||||
|
||||
if command == "bulk":
|
||||
# Parse --checks flag
|
||||
checks = None
|
||||
domains = []
|
||||
i = 0
|
||||
while i < len(args):
|
||||
if args[i] == "--checks" and i + 1 < len(args):
|
||||
checks = [c.strip() for c in args[i + 1].split(",")]
|
||||
i += 2
|
||||
else:
|
||||
domains.append(args[i])
|
||||
i += 1
|
||||
result = bulk_check(domains, checks)
|
||||
elif command in COMMAND_MAP:
|
||||
result = COMMAND_MAP[command](args[0])
|
||||
else:
|
||||
print(f"Unknown command: {command}")
|
||||
print(f"Available: {', '.join(COMMAND_MAP.keys())}, bulk")
|
||||
sys.exit(1)
|
||||
|
||||
print(json.dumps(result, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,221 @@
|
||||
---
|
||||
name: drug-discovery
|
||||
description: "Drug discovery: ChEMBL search, drug-likeness, interactions."
|
||||
platforms: [linux, macos, windows]
|
||||
version: 1.0.0
|
||||
author: bennytimz
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [science, chemistry, pharmacology, research, health]
|
||||
prerequisites:
|
||||
commands: [curl, python]
|
||||
---
|
||||
|
||||
# Drug Discovery & Pharmaceutical Research
|
||||
|
||||
You are an expert pharmaceutical scientist and medicinal chemist with deep
|
||||
knowledge of drug discovery, cheminformatics, and clinical pharmacology.
|
||||
Use this skill for all pharma/chemistry research tasks.
|
||||
|
||||
## Core Workflows
|
||||
|
||||
### 1 — Bioactive Compound Search (ChEMBL)
|
||||
|
||||
Search ChEMBL (the world's largest open bioactivity database) for compounds
|
||||
by target, activity, or molecule name. No API key required.
|
||||
|
||||
```bash
|
||||
# Search compounds by target name (e.g. "EGFR", "COX-2", "ACE")
|
||||
TARGET="$1"
|
||||
ENCODED=$(python -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$TARGET")
|
||||
curl -s "https://www.ebi.ac.uk/chembl/api/data/target/search?q=${ENCODED}&format=json" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
targets=data.get('targets',[])[:5]
|
||||
for t in targets:
|
||||
print(f\"ChEMBL ID : {t.get('target_chembl_id')}\")
|
||||
print(f\"Name : {t.get('pref_name')}\")
|
||||
print(f\"Type : {t.get('target_type')}\")
|
||||
print()
|
||||
"
|
||||
```
|
||||
|
||||
```bash
|
||||
# Get bioactivity data for a ChEMBL target ID
|
||||
TARGET_ID="$1" # e.g. CHEMBL203
|
||||
curl -s "https://www.ebi.ac.uk/chembl/api/data/activity?target_chembl_id=${TARGET_ID}&pchembl_value__gte=6&limit=10&format=json" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
acts=data.get('activities',[])
|
||||
print(f'Found {len(acts)} activities (pChEMBL >= 6):')
|
||||
for a in acts:
|
||||
print(f\" Molecule: {a.get('molecule_chembl_id')} | {a.get('standard_type')}: {a.get('standard_value')} {a.get('standard_units')} | pChEMBL: {a.get('pchembl_value')}\")
|
||||
"
|
||||
```
|
||||
|
||||
```bash
|
||||
# Look up a specific molecule by ChEMBL ID
|
||||
MOL_ID="$1" # e.g. CHEMBL25 (aspirin)
|
||||
curl -s "https://www.ebi.ac.uk/chembl/api/data/molecule/${MOL_ID}?format=json" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
m=json.load(sys.stdin)
|
||||
props=m.get('molecule_properties',{}) or {}
|
||||
print(f\"Name : {m.get('pref_name','N/A')}\")
|
||||
print(f\"SMILES : {m.get('molecule_structures',{}).get('canonical_smiles','N/A') if m.get('molecule_structures') else 'N/A'}\")
|
||||
print(f\"MW : {props.get('full_mwt','N/A')} Da\")
|
||||
print(f\"LogP : {props.get('alogp','N/A')}\")
|
||||
print(f\"HBD : {props.get('hbd','N/A')}\")
|
||||
print(f\"HBA : {props.get('hba','N/A')}\")
|
||||
print(f\"TPSA : {props.get('psa','N/A')} Ų\")
|
||||
print(f\"Ro5 violations: {props.get('num_ro5_violations','N/A')}\")
|
||||
print(f\"QED : {props.get('qed_weighted','N/A')}\")
|
||||
"
|
||||
```
|
||||
|
||||
### 2 — Drug-Likeness Calculation (Lipinski Ro5 + Veber)
|
||||
|
||||
Assess any molecule against established oral bioavailability rules using
|
||||
PubChem's free property API — no RDKit install needed.
|
||||
|
||||
```bash
|
||||
COMPOUND="$1"
|
||||
ENCODED=$(python -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$COMPOUND")
|
||||
curl -s "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/${ENCODED}/property/MolecularWeight,XLogP,HBondDonorCount,HBondAcceptorCount,RotatableBondCount,TPSA,InChIKey/JSON" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
props=data['PropertyTable']['Properties'][0]
|
||||
mw = float(props.get('MolecularWeight', 0))
|
||||
logp = float(props.get('XLogP', 0))
|
||||
hbd = int(props.get('HBondDonorCount', 0))
|
||||
hba = int(props.get('HBondAcceptorCount', 0))
|
||||
rot = int(props.get('RotatableBondCount', 0))
|
||||
tpsa = float(props.get('TPSA', 0))
|
||||
print('=== Lipinski Rule of Five (Ro5) ===')
|
||||
print(f' MW {mw:.1f} Da {\"✓\" if mw<=500 else \"✗ VIOLATION (>500)\"}')
|
||||
print(f' LogP {logp:.2f} {\"✓\" if logp<=5 else \"✗ VIOLATION (>5)\"}')
|
||||
print(f' HBD {hbd} {\"✓\" if hbd<=5 else \"✗ VIOLATION (>5)\"}')
|
||||
print(f' HBA {hba} {\"✓\" if hba<=10 else \"✗ VIOLATION (>10)\"}')
|
||||
viol = sum([mw>500, logp>5, hbd>5, hba>10])
|
||||
print(f' Violations: {viol}/4 {\"→ Likely orally bioavailable\" if viol<=1 else \"→ Poor oral bioavailability predicted\"}')
|
||||
print()
|
||||
print('=== Veber Oral Bioavailability Rules ===')
|
||||
print(f' TPSA {tpsa:.1f} Ų {\"✓\" if tpsa<=140 else \"✗ VIOLATION (>140)\"}')
|
||||
print(f' Rot. bonds {rot} {\"✓\" if rot<=10 else \"✗ VIOLATION (>10)\"}')
|
||||
print(f' Both rules met: {\"Yes → good oral absorption predicted\" if tpsa<=140 and rot<=10 else \"No → reduced oral absorption\"}')
|
||||
"
|
||||
```
|
||||
|
||||
### 3 — Drug Interaction & Safety Lookup (OpenFDA)
|
||||
|
||||
```bash
|
||||
DRUG="$1"
|
||||
ENCODED=$(python -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$DRUG")
|
||||
curl -s "https://api.fda.gov/drug/label.json?search=drug_interactions:\"${ENCODED}\"&limit=3" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
results=data.get('results',[])
|
||||
if not results:
|
||||
print('No interaction data found in FDA labels.')
|
||||
sys.exit()
|
||||
for r in results[:2]:
|
||||
brand=r.get('openfda',{}).get('brand_name',['Unknown'])[0]
|
||||
generic=r.get('openfda',{}).get('generic_name',['Unknown'])[0]
|
||||
interactions=r.get('drug_interactions',['N/A'])[0]
|
||||
print(f'--- {brand} ({generic}) ---')
|
||||
print(interactions[:800])
|
||||
print()
|
||||
"
|
||||
```
|
||||
|
||||
```bash
|
||||
DRUG="$1"
|
||||
ENCODED=$(python -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$DRUG")
|
||||
curl -s "https://api.fda.gov/drug/event.json?search=patient.drug.medicinalproduct:\"${ENCODED}\"&count=patient.reaction.reactionmeddrapt.exact&limit=10" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
results=data.get('results',[])
|
||||
if not results:
|
||||
print('No adverse event data found.')
|
||||
sys.exit()
|
||||
print(f'Top adverse events reported:')
|
||||
for r in results[:10]:
|
||||
print(f\" {r['count']:>5}x {r['term']}\")
|
||||
"
|
||||
```
|
||||
|
||||
### 4 — PubChem Compound Search
|
||||
|
||||
```bash
|
||||
COMPOUND="$1"
|
||||
ENCODED=$(python -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$COMPOUND")
|
||||
CID=$(curl -s "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/${ENCODED}/cids/TXT" | head -1 | tr -d '[:space:]')
|
||||
echo "PubChem CID: $CID"
|
||||
curl -s "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/cid/${CID}/property/IsomericSMILES,InChIKey,IUPACName/JSON" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
p=json.load(sys.stdin)['PropertyTable']['Properties'][0]
|
||||
print(f\"IUPAC Name : {p.get('IUPACName','N/A')}\")
|
||||
print(f\"SMILES : {p.get('IsomericSMILES','N/A')}\")
|
||||
print(f\"InChIKey : {p.get('InChIKey','N/A')}\")
|
||||
"
|
||||
```
|
||||
|
||||
### 5 — Target & Disease Literature (OpenTargets)
|
||||
|
||||
```bash
|
||||
GENE="$1"
|
||||
curl -s -X POST "https://api.platform.opentargets.org/api/v4/graphql" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{\"query\":\"{ search(queryString: \\\"${GENE}\\\", entityNames: [\\\"target\\\"], page: {index: 0, size: 1}) { hits { id score object { ... on Target { id approvedSymbol approvedName associatedDiseases(page: {index: 0, size: 5}) { count rows { score disease { id name } } } } } } } }\"}" \
|
||||
| python -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
hits=data.get('data',{}).get('search',{}).get('hits',[])
|
||||
if not hits:
|
||||
print('Target not found.')
|
||||
sys.exit()
|
||||
obj=hits[0]['object']
|
||||
print(f\"Target: {obj.get('approvedSymbol')} — {obj.get('approvedName')}\")
|
||||
assoc=obj.get('associatedDiseases',{})
|
||||
print(f\"Associated with {assoc.get('count',0)} diseases. Top associations:\")
|
||||
for row in assoc.get('rows',[]):
|
||||
print(f\" Score {row['score']:.3f} | {row['disease']['name']}\")
|
||||
"
|
||||
```
|
||||
|
||||
## Reasoning Guidelines
|
||||
|
||||
When analysing drug-likeness or molecular properties, always:
|
||||
|
||||
1. **State raw values first** — MW, LogP, HBD, HBA, TPSA, RotBonds
|
||||
2. **Apply rule sets** — Ro5 (Lipinski), Veber, Ghose filter where relevant
|
||||
3. **Flag liabilities** — metabolic hotspots, hERG risk, high TPSA for CNS penetration
|
||||
4. **Suggest optimizations** — bioisosteric replacements, prodrug strategies, ring truncation
|
||||
5. **Cite the source API** — ChEMBL, PubChem, OpenFDA, or OpenTargets
|
||||
|
||||
For ADMET questions, reason through Absorption, Distribution, Metabolism, Excretion, Toxicity systematically. See references/ADMET_REFERENCE.md for detailed guidance.
|
||||
|
||||
## Important Notes
|
||||
|
||||
- All APIs are free, public, require no authentication
|
||||
- ChEMBL rate limits: add sleep 1 between batch requests
|
||||
- FDA data reflects reported adverse events, not necessarily causation
|
||||
- Always recommend consulting a licensed pharmacist or physician for clinical decisions
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | API | Endpoint |
|
||||
|------|-----|----------|
|
||||
| Find target | ChEMBL | `/api/data/target/search?q=` |
|
||||
| Get bioactivity | ChEMBL | `/api/data/activity?target_chembl_id=` |
|
||||
| Molecule properties | PubChem | `/rest/pug/compound/name/{name}/property/` |
|
||||
| Drug interactions | OpenFDA | `/drug/label.json?search=drug_interactions:` |
|
||||
| Adverse events | OpenFDA | `/drug/event.json?search=...&count=reaction` |
|
||||
| Gene-disease | OpenTargets | GraphQL POST `/api/v4/graphql` |
|
||||
@@ -0,0 +1,66 @@
|
||||
# ADMET Reference Guide
|
||||
|
||||
Comprehensive reference for Absorption, Distribution, Metabolism, Excretion, and Toxicity (ADMET) analysis in drug discovery.
|
||||
|
||||
## Drug-Likeness Rule Sets
|
||||
|
||||
### Lipinski's Rule of Five (Ro5)
|
||||
|
||||
| Property | Threshold |
|
||||
|----------|-----------|
|
||||
| Molecular Weight (MW) | ≤ 500 Da |
|
||||
| Lipophilicity (LogP) | ≤ 5 |
|
||||
| H-Bond Donors (HBD) | ≤ 5 |
|
||||
| H-Bond Acceptors (HBA) | ≤ 10 |
|
||||
|
||||
Reference: Lipinski et al., Adv. Drug Deliv. Rev. 23, 3–25 (1997).
|
||||
|
||||
### Veber's Oral Bioavailability Rules
|
||||
|
||||
| Property | Threshold |
|
||||
|----------|-----------|
|
||||
| TPSA | ≤ 140 Ų |
|
||||
| Rotatable Bonds | ≤ 10 |
|
||||
|
||||
Reference: Veber et al., J. Med. Chem. 45, 2615–2623 (2002).
|
||||
|
||||
### CNS Penetration (BBB)
|
||||
|
||||
| Property | CNS-Optimal |
|
||||
|----------|-------------|
|
||||
| MW | ≤ 400 Da |
|
||||
| LogP | 1–3 |
|
||||
| TPSA | < 90 Ų |
|
||||
| HBD | ≤ 3 |
|
||||
|
||||
## CYP450 Metabolism
|
||||
|
||||
| Isoform | % Drugs | Notable inhibitors |
|
||||
|---------|---------|-------------------|
|
||||
| CYP3A4 | ~50% | Grapefruit, ketoconazole |
|
||||
| CYP2D6 | ~25% | Fluoxetine, paroxetine |
|
||||
| CYP2C9 | ~15% | Fluconazole, amiodarone |
|
||||
| CYP2C19 | ~10% | Omeprazole, fluoxetine |
|
||||
| CYP1A2 | ~5% | Fluvoxamine, ciprofloxacin |
|
||||
|
||||
## hERG Cardiac Toxicity Risk
|
||||
|
||||
Structural alerts: basic nitrogen (pKa 7–9) + aromatic ring + hydrophobic moiety, LogP > 3.5 + basic amine.
|
||||
|
||||
Mitigation: reduce basicity, introduce polar groups, break planarity.
|
||||
|
||||
## Common Bioisosteric Replacements
|
||||
|
||||
| Original | Bioisostere | Purpose |
|
||||
|----------|-------------|---------|
|
||||
| -COOH | -tetrazole, -SO₂NH₂ | Improve permeability |
|
||||
| -OH (phenol) | -F, -CN | Reduce glucuronidation |
|
||||
| Phenyl | Pyridine, thiophene | Reduce LogP |
|
||||
| Ester | -CONHR | Reduce hydrolysis |
|
||||
|
||||
## Key APIs
|
||||
|
||||
- ChEMBL: https://www.ebi.ac.uk/chembl/api/data/
|
||||
- PubChem: https://pubchem.ncbi.nlm.nih.gov/rest/pug/
|
||||
- OpenFDA: https://api.fda.gov/drug/
|
||||
- OpenTargets GraphQL: https://api.platform.opentargets.org/api/v4/graphql
|
||||
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
chembl_target.py — Search ChEMBL for a target and retrieve top active compounds.
|
||||
Usage: python3 chembl_target.py "EGFR" --min-pchembl 7 --limit 20
|
||||
No external dependencies.
|
||||
"""
|
||||
import sys, json, time, argparse
|
||||
import urllib.request, urllib.parse
|
||||
|
||||
BASE = "https://www.ebi.ac.uk/chembl/api/data"
|
||||
|
||||
def get(endpoint):
|
||||
try:
|
||||
req = urllib.request.Request(f"{BASE}{endpoint}", headers={"Accept":"application/json"})
|
||||
with urllib.request.urlopen(req, timeout=15) as r:
|
||||
return json.loads(r.read())
|
||||
except Exception as e:
|
||||
print(f"API error: {e}", file=sys.stderr); return None
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="ChEMBL target → active compounds")
|
||||
parser.add_argument("target")
|
||||
parser.add_argument("--min-pchembl", type=float, default=6.0)
|
||||
parser.add_argument("--limit", type=int, default=10)
|
||||
args = parser.parse_args()
|
||||
|
||||
enc = urllib.parse.quote(args.target)
|
||||
data = get(f"/target/search?q={enc}&limit=5&format=json")
|
||||
if not data or not data.get("targets"):
|
||||
print("No targets found."); sys.exit(1)
|
||||
|
||||
t = data["targets"][0]
|
||||
tid = t.get("target_chembl_id","")
|
||||
print(f"\nTarget: {t.get('pref_name')} ({tid})")
|
||||
print(f"Type: {t.get('target_type')} | Organism: {t.get('organism','N/A')}")
|
||||
print(f"\nFetching compounds with pChEMBL ≥ {args.min_pchembl}...\n")
|
||||
|
||||
acts = get(f"/activity?target_chembl_id={tid}&pchembl_value__gte={args.min_pchembl}&assay_type=B&limit={args.limit}&order_by=-pchembl_value&format=json")
|
||||
if not acts or not acts.get("activities"):
|
||||
print("No activities found."); sys.exit(0)
|
||||
|
||||
print(f"{'Molecule':<18} {'pChEMBL':>8} {'Type':<12} {'Value':<10} {'Units'}")
|
||||
print("-"*65)
|
||||
seen = set()
|
||||
for a in acts["activities"]:
|
||||
mid = a.get("molecule_chembl_id","N/A")
|
||||
if mid in seen: continue
|
||||
seen.add(mid)
|
||||
print(f"{mid:<18} {str(a.get('pchembl_value','N/A')):>8} {str(a.get('standard_type','N/A')):<12} {str(a.get('standard_value','N/A')):<10} {a.get('standard_units','N/A')}")
|
||||
time.sleep(0.1)
|
||||
print(f"\nTotal: {len(seen)} unique molecules")
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
ro5_screen.py — Batch Lipinski Ro5 + Veber screening via PubChem API.
|
||||
Usage: python3 ro5_screen.py aspirin ibuprofen paracetamol
|
||||
No external dependencies beyond stdlib.
|
||||
"""
|
||||
import sys, json, time
|
||||
import urllib.request, urllib.parse
|
||||
|
||||
BASE = "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name"
|
||||
PROPS = "MolecularWeight,XLogP,HBondDonorCount,HBondAcceptorCount,RotatableBondCount,TPSA"
|
||||
|
||||
def fetch(name):
|
||||
url = f"{BASE}/{urllib.parse.quote(name)}/property/{PROPS}/JSON"
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=10) as r:
|
||||
return json.loads(r.read())["PropertyTable"]["Properties"][0]
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def check(p):
|
||||
mw,logp,hbd,hba,rot,tpsa = float(p.get("MolecularWeight",0)),float(p.get("XLogP",0)),int(p.get("HBondDonorCount",0)),int(p.get("HBondAcceptorCount",0)),int(p.get("RotatableBondCount",0)),float(p.get("TPSA",0))
|
||||
v = sum([mw>500,logp>5,hbd>5,hba>10])
|
||||
return dict(mw=mw,logp=logp,hbd=hbd,hba=hba,rot=rot,tpsa=tpsa,violations=v,ro5=v<=1,veber=tpsa<=140 and rot<=10,ok=v<=1 and tpsa<=140 and rot<=10)
|
||||
|
||||
def report(name, r):
|
||||
if not r: print(f"✗ {name:30s} — not found"); return
|
||||
s = "✓ PASS" if r["ok"] else "✗ FAIL"
|
||||
flags = (f" [Ro5 violations:{r['violations']}]" if not r["ro5"] else "") + (" [Veber fail]" if not r["veber"] else "")
|
||||
print(f"{s} {name:28s} MW={r['mw']:.0f} LogP={r['logp']:.2f} HBD={r['hbd']} HBA={r['hba']} TPSA={r['tpsa']:.0f} RotB={r['rot']}{flags}")
|
||||
|
||||
def main():
|
||||
compounds = sys.stdin.read().splitlines() if len(sys.argv)<2 or sys.argv[1]=="-" else sys.argv[1:]
|
||||
print(f"\n{'Status':<8} {'Compound':<30} Properties\n" + "-"*85)
|
||||
passed = 0
|
||||
for name in compounds:
|
||||
props = fetch(name.strip())
|
||||
result = check(props) if props else None
|
||||
report(name.strip(), result)
|
||||
if result and result["ok"]: passed += 1
|
||||
time.sleep(0.3)
|
||||
print(f"\nSummary: {passed}/{len(compounds)} passed Ro5 + Veber.\n")
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
@@ -0,0 +1,238 @@
|
||||
---
|
||||
name: duckduckgo-search
|
||||
description: Free keyless web, news, and image search via ddgs.
|
||||
version: 1.3.0
|
||||
author: gamedevCloudy
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [search, duckduckgo, web-search, free, fallback]
|
||||
related_skills: [arxiv]
|
||||
fallback_for_toolsets: [web]
|
||||
---
|
||||
|
||||
# DuckDuckGo Search
|
||||
|
||||
Free web search using DuckDuckGo. **No API key required.**
|
||||
|
||||
Preferred when `web_search` is unavailable or unsuitable (for example when `FIRECRAWL_API_KEY` is not set). Can also be used as a standalone search path when DuckDuckGo results are specifically desired.
|
||||
|
||||
## Detection Flow
|
||||
|
||||
Check what is actually available before choosing an approach:
|
||||
|
||||
```bash
|
||||
# Check CLI availability
|
||||
command -v ddgs >/dev/null && echo "DDGS_CLI=installed" || echo "DDGS_CLI=missing"
|
||||
```
|
||||
|
||||
Decision tree:
|
||||
1. If `ddgs` CLI is installed, prefer `terminal` + `ddgs`
|
||||
2. If `ddgs` CLI is missing, do not assume `execute_code` can import `ddgs`
|
||||
3. If the user wants DuckDuckGo specifically, install `ddgs` first in the relevant environment
|
||||
4. Otherwise fall back to built-in web/browser tools
|
||||
|
||||
Important runtime note:
|
||||
- Terminal and `execute_code` are separate runtimes
|
||||
- A successful shell install does not guarantee `execute_code` can import `ddgs`
|
||||
- Never assume third-party Python packages are preinstalled inside `execute_code`
|
||||
|
||||
## Installation
|
||||
|
||||
Install `ddgs` only when DuckDuckGo search is specifically needed and the runtime does not already provide it.
|
||||
|
||||
```bash
|
||||
# Python package + CLI entrypoint
|
||||
pip install ddgs
|
||||
|
||||
# Verify CLI
|
||||
ddgs --help
|
||||
```
|
||||
|
||||
If a workflow depends on Python imports, verify that same runtime can import `ddgs` before using `from ddgs import DDGS`.
|
||||
|
||||
## Method 1: CLI Search (Preferred)
|
||||
|
||||
Use the `ddgs` command via `terminal` when it exists. This is the preferred path because it avoids assuming the `execute_code` sandbox has the `ddgs` Python package installed.
|
||||
|
||||
```bash
|
||||
# Text search
|
||||
ddgs text -q "python async programming" -m 5
|
||||
|
||||
# News search
|
||||
ddgs news -q "artificial intelligence" -m 5
|
||||
|
||||
# Image search
|
||||
ddgs images -q "landscape photography" -m 10
|
||||
|
||||
# Video search
|
||||
ddgs videos -q "python tutorial" -m 5
|
||||
|
||||
# With region filter
|
||||
ddgs text -q "best restaurants" -m 5 -r us-en
|
||||
|
||||
# Recent results only (d=day, w=week, m=month, y=year)
|
||||
ddgs text -q "latest AI news" -m 5 -t w
|
||||
|
||||
# JSON output for parsing
|
||||
ddgs text -q "fastapi tutorial" -m 5 -o json
|
||||
```
|
||||
|
||||
### CLI Flags
|
||||
|
||||
| Flag | Description | Example |
|
||||
|------|-------------|---------|
|
||||
| `-q` | Query — **required** | `-q "search terms"` |
|
||||
| `-m` | Max results | `-m 5` |
|
||||
| `-r` | Region | `-r us-en` |
|
||||
| `-t` | Time limit | `-t w` (week) |
|
||||
| `-s` | Safe search | `-s off` |
|
||||
| `-o` | Output format | `-o json` |
|
||||
|
||||
## Method 2: Python API (Only After Verification)
|
||||
|
||||
Use the `DDGS` class in `execute_code` or another Python runtime only after verifying that `ddgs` is installed there. Do not assume `execute_code` includes third-party packages by default.
|
||||
|
||||
Safe wording:
|
||||
- "Use `execute_code` with `ddgs` after installing or verifying the package if needed"
|
||||
|
||||
Avoid saying:
|
||||
- "`execute_code` includes `ddgs`"
|
||||
- "DuckDuckGo search works by default in `execute_code`"
|
||||
|
||||
**Important:** `max_results` must always be passed as a **keyword argument** — positional usage raises an error on all methods.
|
||||
|
||||
### Text Search
|
||||
|
||||
Best for: general research, companies, documentation.
|
||||
|
||||
```python
|
||||
from ddgs import DDGS
|
||||
|
||||
with DDGS() as ddgs:
|
||||
for r in ddgs.text("python async programming", max_results=5):
|
||||
print(r["title"])
|
||||
print(r["href"])
|
||||
print(r.get("body", "")[:200])
|
||||
print()
|
||||
```
|
||||
|
||||
Returns: `title`, `href`, `body`
|
||||
|
||||
### News Search
|
||||
|
||||
Best for: current events, breaking news, latest updates.
|
||||
|
||||
```python
|
||||
from ddgs import DDGS
|
||||
|
||||
with DDGS() as ddgs:
|
||||
for r in ddgs.news("AI regulation 2026", max_results=5):
|
||||
print(r["date"], "-", r["title"])
|
||||
print(r.get("source", ""), "|", r["url"])
|
||||
print(r.get("body", "")[:200])
|
||||
print()
|
||||
```
|
||||
|
||||
Returns: `date`, `title`, `body`, `url`, `image`, `source`
|
||||
|
||||
### Image Search
|
||||
|
||||
Best for: visual references, product images, diagrams.
|
||||
|
||||
```python
|
||||
from ddgs import DDGS
|
||||
|
||||
with DDGS() as ddgs:
|
||||
for r in ddgs.images("semiconductor chip", max_results=5):
|
||||
print(r["title"])
|
||||
print(r["image"])
|
||||
print(r.get("thumbnail", ""))
|
||||
print(r.get("source", ""))
|
||||
print()
|
||||
```
|
||||
|
||||
Returns: `title`, `image`, `thumbnail`, `url`, `height`, `width`, `source`
|
||||
|
||||
### Video Search
|
||||
|
||||
Best for: tutorials, demos, explainers.
|
||||
|
||||
```python
|
||||
from ddgs import DDGS
|
||||
|
||||
with DDGS() as ddgs:
|
||||
for r in ddgs.videos("FastAPI tutorial", max_results=5):
|
||||
print(r["title"])
|
||||
print(r.get("content", ""))
|
||||
print(r.get("duration", ""))
|
||||
print(r.get("provider", ""))
|
||||
print(r.get("published", ""))
|
||||
print()
|
||||
```
|
||||
|
||||
Returns: `title`, `content`, `description`, `duration`, `provider`, `published`, `statistics`, `uploader`
|
||||
|
||||
### Quick Reference
|
||||
|
||||
| Method | Use When | Key Fields |
|
||||
|--------|----------|------------|
|
||||
| `text()` | General research, companies | title, href, body |
|
||||
| `news()` | Current events, updates | date, title, source, body, url |
|
||||
| `images()` | Visuals, diagrams | title, image, thumbnail, url |
|
||||
| `videos()` | Tutorials, demos | title, content, duration, provider |
|
||||
|
||||
## Workflow: Search then Extract
|
||||
|
||||
DuckDuckGo returns titles, URLs, and snippets — not full page content. To get full page content, search first and then extract the most relevant URL with `web_extract`, browser tools, or curl.
|
||||
|
||||
CLI example:
|
||||
|
||||
```bash
|
||||
ddgs text -q "fastapi deployment guide" -m 3 -o json
|
||||
```
|
||||
|
||||
Python example, only after verifying `ddgs` is installed in that runtime:
|
||||
|
||||
```python
|
||||
from ddgs import DDGS
|
||||
|
||||
with DDGS() as ddgs:
|
||||
results = list(ddgs.text("fastapi deployment guide", max_results=3))
|
||||
for r in results:
|
||||
print(r["title"], "->", r["href"])
|
||||
```
|
||||
|
||||
Then extract the best URL with `web_extract` or another content-retrieval tool.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Rate limiting**: DuckDuckGo may throttle after many rapid requests. Add a short delay between searches if needed.
|
||||
- **No content extraction**: `ddgs` returns snippets, not full page content. Use `web_extract`, browser tools, or curl for the full article/page.
|
||||
- **Results quality**: Generally good but less configurable than Firecrawl's search.
|
||||
- **Availability**: DuckDuckGo may block requests from some cloud IPs. If searches return empty, try different keywords or wait a few seconds.
|
||||
- **Field variability**: Return fields may vary between results or `ddgs` versions. Use `.get()` for optional fields to avoid `KeyError`.
|
||||
- **Separate runtimes**: A successful `ddgs` install in terminal does not automatically mean `execute_code` can import it.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Likely Cause | What To Do |
|
||||
|---------|--------------|------------|
|
||||
| `ddgs: command not found` | CLI not installed in the shell environment | Install `ddgs`, or use built-in web/browser tools instead |
|
||||
| `ModuleNotFoundError: No module named 'ddgs'` | Python runtime does not have the package installed | Do not use Python DDGS there until that runtime is prepared |
|
||||
| Search returns nothing | Temporary rate limiting or poor query | Wait a few seconds, retry, or adjust the query |
|
||||
| CLI works but `execute_code` import fails | Terminal and `execute_code` are different runtimes | Keep using CLI, or separately prepare the Python runtime |
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **`max_results` is keyword-only**: `ddgs.text("query", 5)` raises an error. Use `ddgs.text("query", max_results=5)`.
|
||||
- **Do not assume the CLI exists**: Check `command -v ddgs` before using it.
|
||||
- **Do not assume `execute_code` can import `ddgs`**: `from ddgs import DDGS` may fail with `ModuleNotFoundError` unless that runtime was prepared separately.
|
||||
- **Package name**: The package is `ddgs` (previously `duckduckgo-search`). Install with `pip install ddgs`.
|
||||
- **Don't confuse `-q` and `-m`** (CLI): `-q` is for the query, `-m` is for max results count.
|
||||
- **Empty results**: If `ddgs` returns nothing, it may be rate-limited. Wait a few seconds and retry.
|
||||
|
||||
## Validated With
|
||||
|
||||
Validated examples against `ddgs==9.11.2` semantics. Skill guidance now treats CLI availability and Python import availability as separate concerns so the documented workflow matches actual runtime behavior.
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/bin/bash
|
||||
# DuckDuckGo Search Helper Script
|
||||
# Wrapper around ddgs CLI with sensible defaults
|
||||
# Usage: ./duckduckgo.sh <query> [max_results]
|
||||
|
||||
set -e
|
||||
|
||||
QUERY="$1"
|
||||
MAX_RESULTS="${2:-5}"
|
||||
|
||||
if [ -z "$QUERY" ]; then
|
||||
echo "Usage: $0 <query> [max_results]"
|
||||
echo ""
|
||||
echo "Examples:"
|
||||
echo " $0 'python async programming' 5"
|
||||
echo " $0 'latest AI news' 10"
|
||||
echo ""
|
||||
echo "Requires: pip install ddgs"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if ddgs is available
|
||||
if ! command -v ddgs &> /dev/null; then
|
||||
echo "Error: ddgs not found. Install with: pip install ddgs"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ddgs text -q "$QUERY" -m "$MAX_RESULTS"
|
||||
@@ -0,0 +1,214 @@
|
||||
---
|
||||
name: gitnexus-explorer
|
||||
description: Serve an interactive codebase knowledge graph web UI.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent + Teknium
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [gitnexus, code-intelligence, knowledge-graph, visualization]
|
||||
related_skills: [hermes-agent, codebase-inspection]
|
||||
---
|
||||
|
||||
# GitNexus Explorer
|
||||
|
||||
Index any codebase into a knowledge graph and serve an interactive web UI for exploring
|
||||
symbols, call chains, clusters, and execution flows. Tunneled via Cloudflare for remote access.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User wants to visually explore a codebase's architecture
|
||||
- User asks for a knowledge graph / dependency graph of a repo
|
||||
- User wants to share an interactive codebase explorer with someone
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node.js** (v18+) — required for GitNexus and the proxy
|
||||
- **git** — repo must have a `.git` directory
|
||||
- **cloudflared** — for tunneling (auto-installed to ~/.local/bin if missing)
|
||||
|
||||
## Size Warning
|
||||
|
||||
The web UI renders all nodes in the browser. Repos under ~5,000 files work well. Large
|
||||
repos (30k+ nodes) will be sluggish or crash the browser tab. The CLI/MCP tools work
|
||||
at any scale — only the web visualization has this limit.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Clone and Build GitNexus (one-time setup)
|
||||
|
||||
```bash
|
||||
GITNEXUS_DIR="${GITNEXUS_DIR:-$HOME/.local/share/gitnexus}"
|
||||
|
||||
if [ ! -d "$GITNEXUS_DIR/gitnexus-web/dist" ]; then
|
||||
git clone https://github.com/abhigyanpatwari/GitNexus.git "$GITNEXUS_DIR"
|
||||
cd "$GITNEXUS_DIR/gitnexus-shared" && npm install && npm run build
|
||||
cd "$GITNEXUS_DIR/gitnexus-web" && npm install
|
||||
fi
|
||||
```
|
||||
|
||||
### 2. Patch the Web UI for Remote Access
|
||||
|
||||
The web UI defaults to `localhost:4747` for API calls. Patch it to use same-origin
|
||||
so it works through a tunnel/proxy:
|
||||
|
||||
**File: `$GITNEXUS_DIR/gitnexus-web/src/config/ui-constants.ts`**
|
||||
Change:
|
||||
```typescript
|
||||
export const DEFAULT_BACKEND_URL = 'http://localhost:4747';
|
||||
```
|
||||
To:
|
||||
```typescript
|
||||
export const DEFAULT_BACKEND_URL = typeof window !== 'undefined' && window.location.hostname !== 'localhost' ? window.location.origin : 'http://localhost:4747';
|
||||
```
|
||||
|
||||
**File: `$GITNEXUS_DIR/gitnexus-web/vite.config.ts`**
|
||||
Add `allowedHosts: true` inside the `server: { }` block (only needed if running dev
|
||||
mode instead of production build):
|
||||
```typescript
|
||||
server: {
|
||||
allowedHosts: true,
|
||||
// ... existing config
|
||||
},
|
||||
```
|
||||
|
||||
Then build the production bundle:
|
||||
```bash
|
||||
cd "$GITNEXUS_DIR/gitnexus-web" && npx vite build
|
||||
```
|
||||
|
||||
### 3. Index the Target Repo
|
||||
|
||||
```bash
|
||||
cd /path/to/target-repo
|
||||
npx gitnexus analyze --skip-agents-md
|
||||
rm -rf .claude/ # remove Claude Code-specific artifacts
|
||||
```
|
||||
|
||||
Add `--embeddings` for semantic search (slower — minutes instead of seconds).
|
||||
|
||||
The index lives in `.gitnexus/` inside the repo (auto-gitignored).
|
||||
|
||||
### 4. Create the Proxy Script
|
||||
|
||||
Write this to a file (e.g., `$GITNEXUS_DIR/proxy.mjs`). It serves the production
|
||||
web UI and proxies `/api/*` to the GitNexus backend — same origin, no CORS issues,
|
||||
no sudo, no nginx.
|
||||
|
||||
```javascript
|
||||
import http from 'node:http';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
const API_PORT = parseInt(process.env.API_PORT || '4747');
|
||||
const DIST_DIR = process.argv[2] || './dist';
|
||||
const PORT = parseInt(process.argv[3] || '8888');
|
||||
|
||||
const MIME = {
|
||||
'.html': 'text/html', '.js': 'application/javascript', '.css': 'text/css',
|
||||
'.json': 'application/json', '.png': 'image/png', '.svg': 'image/svg+xml',
|
||||
'.ico': 'image/x-icon', '.woff2': 'font/woff2', '.woff': 'font/woff',
|
||||
'.wasm': 'application/wasm',
|
||||
};
|
||||
|
||||
function proxyToApi(req, res) {
|
||||
const opts = {
|
||||
hostname: '127.0.0.1', port: API_PORT,
|
||||
path: req.url, method: req.method, headers: req.headers,
|
||||
};
|
||||
const proxy = http.request(opts, (upstream) => {
|
||||
res.writeHead(upstream.statusCode, upstream.headers);
|
||||
upstream.pipe(res, { end: true });
|
||||
});
|
||||
proxy.on('error', () => { res.writeHead(502); res.end('Backend unavailable'); });
|
||||
req.pipe(proxy, { end: true });
|
||||
}
|
||||
|
||||
function serveStatic(req, res) {
|
||||
let filePath = path.join(DIST_DIR, req.url === '/' ? 'index.html' : req.url.split('?')[0]);
|
||||
if (!fs.existsSync(filePath)) filePath = path.join(DIST_DIR, 'index.html');
|
||||
const ext = path.extname(filePath);
|
||||
const mime = MIME[ext] || 'application/octet-stream';
|
||||
try {
|
||||
const data = fs.readFileSync(filePath);
|
||||
res.writeHead(200, { 'Content-Type': mime, 'Cache-Control': 'public, max-age=3600' });
|
||||
res.end(data);
|
||||
} catch { res.writeHead(404); res.end('Not found'); }
|
||||
}
|
||||
|
||||
http.createServer((req, res) => {
|
||||
if (req.url.startsWith('/api')) proxyToApi(req, res);
|
||||
else serveStatic(req, res);
|
||||
}).listen(PORT, () => console.log(`GitNexus proxy on http://localhost:${PORT}`));
|
||||
```
|
||||
|
||||
### 5. Start the Services
|
||||
|
||||
```bash
|
||||
# Terminal 1: GitNexus backend API
|
||||
npx gitnexus serve &
|
||||
|
||||
# Terminal 2: Proxy (web UI + API on one port)
|
||||
node "$GITNEXUS_DIR/proxy.mjs" "$GITNEXUS_DIR/gitnexus-web/dist" 8888 &
|
||||
```
|
||||
|
||||
Verify: `curl -s http://localhost:8888/api/repos` should return the indexed repo(s).
|
||||
|
||||
### 6. Tunnel with Cloudflare (optional — for remote access)
|
||||
|
||||
```bash
|
||||
# Install cloudflared if needed (no sudo)
|
||||
if ! command -v cloudflared &>/dev/null; then
|
||||
mkdir -p ~/.local/bin
|
||||
curl -sL https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64 \
|
||||
-o ~/.local/bin/cloudflared
|
||||
chmod +x ~/.local/bin/cloudflared
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
fi
|
||||
|
||||
# Start tunnel (--config /dev/null avoids conflicts with existing named tunnels)
|
||||
cloudflared tunnel --config /dev/null --url http://localhost:8888 --no-autoupdate --protocol http2
|
||||
```
|
||||
|
||||
The tunnel URL (e.g., `https://random-words.trycloudflare.com`) is printed to stderr.
|
||||
Share it — anyone with the link can explore the graph.
|
||||
|
||||
### 7. Cleanup
|
||||
|
||||
```bash
|
||||
# Stop services
|
||||
pkill -f "gitnexus serve"
|
||||
pkill -f "proxy.mjs"
|
||||
pkill -f cloudflared
|
||||
|
||||
# Remove index from the target repo
|
||||
cd /path/to/target-repo
|
||||
npx gitnexus clean
|
||||
rm -rf .claude/
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **`--config /dev/null` is required for cloudflared** if the user has an existing
|
||||
named tunnel config at `~/.cloudflared/config.yml`. Without it, the catch-all
|
||||
ingress rule in the config returns 404 for all quick tunnel requests.
|
||||
|
||||
- **Production build is mandatory for tunneling.** The Vite dev server blocks
|
||||
non-localhost hosts by default (`allowedHosts`). The production build + Node
|
||||
proxy avoids this entirely.
|
||||
|
||||
- **The web UI does NOT create `.claude/` or `CLAUDE.md`.** Those are created by
|
||||
`npx gitnexus analyze`. Use `--skip-agents-md` to suppress the markdown files,
|
||||
then `rm -rf .claude/` for the rest. These are Claude Code integrations that
|
||||
hermes-agent users don't need.
|
||||
|
||||
- **Browser memory limit.** The web UI loads the entire graph into browser memory.
|
||||
Repos with 5k+ files may be sluggish. 30k+ files will likely crash the tab.
|
||||
|
||||
- **Embeddings are optional.** `--embeddings` enables semantic search but takes
|
||||
minutes on large repos. Skip it for quick exploration; add it if you want
|
||||
natural language queries via the AI chat panel.
|
||||
|
||||
- **Multiple repos.** `gitnexus serve` serves ALL indexed repos. Index several
|
||||
repos, start serve once, and the web UI lets you switch between them.
|
||||
@@ -0,0 +1,92 @@
|
||||
/**
|
||||
* GitNexus reverse proxy — serves production web UI + proxies /api/* to backend.
|
||||
* Zero dependencies, Node.js built-ins only.
|
||||
*
|
||||
* Usage: node proxy.mjs <dist-dir> [port]
|
||||
* dist-dir: path to gitnexus-web/dist (production build)
|
||||
* port: listen port (default: 8888)
|
||||
*
|
||||
* Environment:
|
||||
* API_PORT: GitNexus serve backend port (default: 4747)
|
||||
*/
|
||||
import http from 'node:http';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
const API_PORT = parseInt(process.env.API_PORT || '4747');
|
||||
const DIST_DIR = process.argv[2] || './dist';
|
||||
const PORT = parseInt(process.argv[3] || '8888');
|
||||
|
||||
const MIME = {
|
||||
'.html': 'text/html',
|
||||
'.js': 'application/javascript',
|
||||
'.css': 'text/css',
|
||||
'.json': 'application/json',
|
||||
'.png': 'image/png',
|
||||
'.svg': 'image/svg+xml',
|
||||
'.ico': 'image/x-icon',
|
||||
'.woff2': 'font/woff2',
|
||||
'.woff': 'font/woff',
|
||||
'.wasm': 'application/wasm',
|
||||
'.ttf': 'font/ttf',
|
||||
'.map': 'application/json',
|
||||
};
|
||||
|
||||
function proxyToApi(req, res) {
|
||||
const opts = {
|
||||
hostname: '127.0.0.1',
|
||||
port: API_PORT,
|
||||
path: req.url,
|
||||
method: req.method,
|
||||
headers: { ...req.headers, host: `127.0.0.1:${API_PORT}` },
|
||||
};
|
||||
const proxy = http.request(opts, (upstream) => {
|
||||
res.writeHead(upstream.statusCode, upstream.headers);
|
||||
upstream.pipe(res, { end: true });
|
||||
});
|
||||
proxy.on('error', () => {
|
||||
res.writeHead(502, { 'Content-Type': 'text/plain' });
|
||||
res.end('GitNexus backend unavailable — is `npx gitnexus serve` running?');
|
||||
});
|
||||
req.pipe(proxy, { end: true });
|
||||
}
|
||||
|
||||
function serveStatic(req, res) {
|
||||
const urlPath = req.url.split('?')[0];
|
||||
let filePath = path.join(DIST_DIR, urlPath === '/' ? 'index.html' : urlPath);
|
||||
|
||||
// SPA fallback: if file doesn't exist and isn't a static asset, serve index.html
|
||||
if (!fs.existsSync(filePath) && !path.extname(filePath)) {
|
||||
filePath = path.join(DIST_DIR, 'index.html');
|
||||
}
|
||||
|
||||
const ext = path.extname(filePath);
|
||||
const mime = MIME[ext] || 'application/octet-stream';
|
||||
|
||||
try {
|
||||
const data = fs.readFileSync(filePath);
|
||||
res.writeHead(200, {
|
||||
'Content-Type': mime,
|
||||
'Cache-Control': ext === '.html' ? 'no-cache' : 'public, max-age=86400',
|
||||
});
|
||||
res.end(data);
|
||||
} catch {
|
||||
res.writeHead(404, { 'Content-Type': 'text/plain' });
|
||||
res.end('Not found');
|
||||
}
|
||||
}
|
||||
|
||||
const server = http.createServer((req, res) => {
|
||||
if (req.url.startsWith('/api')) {
|
||||
proxyToApi(req, res);
|
||||
} else {
|
||||
serveStatic(req, res);
|
||||
}
|
||||
});
|
||||
|
||||
server.listen(PORT, () => {
|
||||
console.log(`GitNexus proxy listening on http://localhost:${PORT}`);
|
||||
console.log(` Web UI: http://localhost:${PORT}/`);
|
||||
console.log(` API: http://localhost:${PORT}/api/repos`);
|
||||
console.log(` Backend: http://127.0.0.1:${API_PORT}`);
|
||||
});
|
||||
@@ -0,0 +1,278 @@
|
||||
---
|
||||
name: osint-investigation
|
||||
description: Follow the money via public records and sanctions data.
|
||||
version: 0.1.0
|
||||
platforms: [linux, macos, windows]
|
||||
author: Hermes Agent (adapted from ShinMegamiBoson/OpenPlanter, MIT)
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [osint, investigation, public-records, sec, sanctions, corporate-registry, property, courts, due-diligence, journalism]
|
||||
category: research
|
||||
related_skills: [domain-intel, arxiv]
|
||||
---
|
||||
|
||||
# OSINT Investigation — Public Records Cross-Reference
|
||||
|
||||
Investigative framework for public-records OSINT: government contracts,
|
||||
corporate filings, lobbying, sanctions, offshore leaks, property records,
|
||||
court records, web archives, knowledge bases, and global news. Resolve
|
||||
entities across heterogeneous sources, build cross-links with explicit
|
||||
confidence, run statistical timing tests, and produce structured evidence
|
||||
chains.
|
||||
|
||||
**Python stdlib only.** Zero install. Works on Linux, macOS, Windows. Most
|
||||
sources work with no API key (OpenCorporates has an optional free token
|
||||
that raises rate limits).
|
||||
|
||||
Adapted from the MIT-licensed ShinMegamiBoson/OpenPlanter project; expanded
|
||||
to cover identity / property / litigation / archives / news sources that
|
||||
the original didn't address.
|
||||
|
||||
## When to use this skill
|
||||
|
||||
Use when the user asks for:
|
||||
|
||||
- "follow the money" — government contracts, lobbying → legislation, sanctions
|
||||
- corporate due diligence — who controls company X, where are they
|
||||
incorporated, who serves on their boards, what filings have they made
|
||||
- sanctions screening — is entity X on OFAC SDN, ICIJ offshore leaks
|
||||
- pay-to-play investigation — contractors with offshore ties, lobbying
|
||||
clients winning awards
|
||||
- property ownership — find recorded deeds/mortgages by name or address
|
||||
(NYC; for other counties point users at the relevant recorder)
|
||||
- litigation history — find federal + state court opinions and PACER dockets
|
||||
- multi-source entity resolution where naming varies (LLC suffixes, abbreviations)
|
||||
- evidence-chain construction with explicit confidence levels
|
||||
- "what's been said about X" — international news (GDELT) + Wikipedia
|
||||
narrative + Wayback Machine to recover dead URLs
|
||||
|
||||
Do NOT use this skill for:
|
||||
|
||||
- general web research → `web_search` / `web_extract`
|
||||
- domain/infrastructure OSINT → `domain-intel` skill
|
||||
- academic literature → `arxiv` skill
|
||||
- social-media profile discovery → `sherlock` skill (optional)
|
||||
- US **federal** campaign finance — FEC is intentionally NOT covered here
|
||||
(the API is unreliable for ad-hoc contributor-name queries on the free
|
||||
DEMO_KEY tier). For federal donations, point users at
|
||||
https://www.fec.gov/data/ directly.
|
||||
|
||||
## Workflow
|
||||
|
||||
The agent runs scripts via the `terminal` tool. `SKILL_DIR` is the directory
|
||||
holding this SKILL.md.
|
||||
|
||||
### 1. Identify which sources apply
|
||||
|
||||
Read the data-source wiki entries to plan the investigation:
|
||||
|
||||
```
|
||||
ls SKILL_DIR/references/sources/
|
||||
|
||||
# Federal financial / regulatory
|
||||
cat SKILL_DIR/references/sources/sec-edgar.md # corporate filings
|
||||
cat SKILL_DIR/references/sources/usaspending.md # federal contracts
|
||||
cat SKILL_DIR/references/sources/senate-ld.md # lobbying
|
||||
cat SKILL_DIR/references/sources/ofac-sdn.md # sanctions
|
||||
cat SKILL_DIR/references/sources/icij-offshore.md # offshore leaks
|
||||
|
||||
# Identity / property / litigation / archives / news
|
||||
cat SKILL_DIR/references/sources/nyc-acris.md # NYC property records
|
||||
cat SKILL_DIR/references/sources/opencorporates.md # global corporate registry
|
||||
cat SKILL_DIR/references/sources/courtlistener.md # court records (federal + state)
|
||||
cat SKILL_DIR/references/sources/wayback.md # Wayback Machine archives
|
||||
cat SKILL_DIR/references/sources/wikipedia.md # Wikipedia + Wikidata
|
||||
cat SKILL_DIR/references/sources/gdelt.md # global news monitoring
|
||||
```
|
||||
|
||||
Each entry follows a 9-section template: summary, access, schema, coverage,
|
||||
cross-reference keys, data quality, acquisition, legal, references.
|
||||
|
||||
The **cross-reference potential** section maps join keys between sources — read
|
||||
those first to pick the right pair.
|
||||
|
||||
### 2. Acquire data
|
||||
|
||||
Each source has a stdlib-only fetch script in `SKILL_DIR/scripts/`:
|
||||
|
||||
**Federal financial / regulatory**
|
||||
|
||||
```bash
|
||||
# SEC EDGAR filings (corporate disclosures)
|
||||
python SKILL_DIR/scripts/fetch_sec_edgar.py --cik 0000320193 \
|
||||
--types 10-K,10-Q --out data/edgar_filings.csv
|
||||
|
||||
# USAspending federal contracts
|
||||
python SKILL_DIR/scripts/fetch_usaspending.py --recipient "EXAMPLE CORP" \
|
||||
--fy 2024 --out data/contracts.csv
|
||||
|
||||
# Senate LD-1 / LD-2 lobbying disclosures
|
||||
python SKILL_DIR/scripts/fetch_senate_ld.py --client "EXAMPLE CORP" \
|
||||
--year 2024 --out data/lobbying.csv
|
||||
|
||||
# OFAC SDN sanctions list (full snapshot)
|
||||
python SKILL_DIR/scripts/fetch_ofac_sdn.py --out data/ofac_sdn.csv
|
||||
|
||||
# ICIJ Offshore Leaks — downloads ~70 MB bulk CSV on first use,
|
||||
# then searches it locally. Cached for 30 days under
|
||||
# $HERMES_OSINT_CACHE/icij/ (default: ~/.cache/hermes-osint/icij/).
|
||||
python SKILL_DIR/scripts/fetch_icij_offshore.py --entity "EXAMPLE CORP" \
|
||||
--out data/icij.csv
|
||||
```
|
||||
|
||||
**Identity / property / litigation / archives / news**
|
||||
|
||||
```bash
|
||||
# NYC property records (deeds, mortgages, liens) — ACRIS via Socrata
|
||||
python SKILL_DIR/scripts/fetch_nyc_acris.py --name "SMITH, JOHN" \
|
||||
--out data/acris.csv
|
||||
python SKILL_DIR/scripts/fetch_nyc_acris.py --address "571 HUDSON" \
|
||||
--out data/acris_addr.csv
|
||||
|
||||
# OpenCorporates — 130+ jurisdiction corporate registry
|
||||
# (free token required; set OPENCORPORATES_API_TOKEN or pass --token)
|
||||
python SKILL_DIR/scripts/fetch_opencorporates.py --query "Example Corp" \
|
||||
--jurisdiction us_ny --out data/opencorporates.csv
|
||||
|
||||
# CourtListener — federal + state court opinions, PACER dockets
|
||||
python SKILL_DIR/scripts/fetch_courtlistener.py --query "Smith v. Example Corp" \
|
||||
--type opinions --out data/courts.csv
|
||||
|
||||
# Wayback Machine — historical web captures
|
||||
python SKILL_DIR/scripts/fetch_wayback.py --url "example.com" \
|
||||
--match host --collapse digest --out data/wayback.csv
|
||||
|
||||
# Wikipedia + Wikidata — narrative bio + structured facts
|
||||
# Set HERMES_OSINT_UA=your-app/1.0 (your@email) to identify yourself
|
||||
python SKILL_DIR/scripts/fetch_wikipedia.py --query "Bill Gates" \
|
||||
--out data/wp.csv
|
||||
|
||||
# GDELT — global news in 100+ languages, ~2015→present
|
||||
python SKILL_DIR/scripts/fetch_gdelt.py --query '"Example Corp"' \
|
||||
--timespan 1y --out data/gdelt.csv
|
||||
```
|
||||
|
||||
All outputs are normalized CSV with a header row. Re-run scripts idempotently.
|
||||
|
||||
When a private individual won't be in a source (e.g. SEC EDGAR for a non-public-
|
||||
company person, USAspending for someone who isn't a federal contractor, Senate
|
||||
LDA for someone who isn't a lobbying client), the script returns 0 rows with a
|
||||
clear warning rather than silently writing an empty CSV. EDGAR specifically
|
||||
flags when the company-name resolver matched an individual Form 3/4/5 filer
|
||||
rather than a corporate registrant.
|
||||
|
||||
Rate-limit notes are in each source's wiki entry. Default fetchers sleep
|
||||
politely between paginated requests. **API keys raise rate limits** for
|
||||
sources that support them (`SEC_USER_AGENT`, `SENATE_LDA_TOKEN`,
|
||||
`OPENCORPORATES_API_TOKEN`, `COURTLISTENER_TOKEN`). All scripts surface
|
||||
429 responses immediately with the upstream's quota message so the user
|
||||
knows to slow down or supply a key.
|
||||
|
||||
### 3. Resolve entities across sources
|
||||
|
||||
Normalize names and find matches between two CSV files:
|
||||
|
||||
```bash
|
||||
# Match lobbying clients (Senate LDA) against contract recipients (USAspending)
|
||||
python SKILL_DIR/scripts/entity_resolution.py \
|
||||
--left data/lobbying.csv --left-name-col client_name \
|
||||
--right data/contracts.csv --right-name-col recipient_name \
|
||||
--out data/cross_links.csv
|
||||
```
|
||||
|
||||
Three matching tiers with explicit confidence:
|
||||
|
||||
| Tier | Method | Confidence |
|
||||
|------|--------|------------|
|
||||
| `exact` | Normalized strings equal after suffix/punctuation strip | high |
|
||||
| `fuzzy` | Sorted-token equality (word-bag match) | medium |
|
||||
| `token_overlap` | ≥60% token overlap, ≥2 shared tokens, tokens ≥4 chars | low |
|
||||
|
||||
Output `cross_links.csv` columns: `match_type, confidence, left_name,
|
||||
right_name, left_normalized, right_normalized, left_row, right_row`.
|
||||
|
||||
### 4. Statistical timing correlation (optional)
|
||||
|
||||
Test whether two time series cluster suspiciously close together — e.g.
|
||||
lobbying filings near contract awards — using a permutation test:
|
||||
|
||||
```bash
|
||||
python SKILL_DIR/scripts/timing_analysis.py \
|
||||
--donations data/lobbying.csv --donation-date-col filing_date \
|
||||
--donation-amount-col income --donation-donor-col client_name \
|
||||
--donation-recipient-col registrant_name \
|
||||
--contracts data/contracts.csv --contract-date-col award_date \
|
||||
--contract-vendor-col recipient_name \
|
||||
--cross-links data/cross_links.csv \
|
||||
--permutations 1000 \
|
||||
--out data/timing.json
|
||||
```
|
||||
|
||||
The script's column flags are intentionally generic — the original tool was
|
||||
written for donations vs awards, but it works for any (event, payee) time
|
||||
series joined through cross-links. Null hypothesis: event timing is
|
||||
independent of award dates. One-tailed p-value = fraction of permutations
|
||||
with mean nearest-award distance ≤ observed. Minimum 3 events per (payer,
|
||||
vendor) pair to run the test.
|
||||
|
||||
### 5. Build the findings JSON (evidence chain)
|
||||
|
||||
```bash
|
||||
python SKILL_DIR/scripts/build_findings.py \
|
||||
--cross-links data/cross_links.csv \
|
||||
--timing data/timing.json \
|
||||
--out data/findings.json
|
||||
```
|
||||
|
||||
Every finding has `id, title, severity, confidence, summary, evidence[], sources[]`.
|
||||
Each evidence item points back to a specific row in a source CSV. The user (or a
|
||||
follow-up agent) can verify every claim against its source.
|
||||
|
||||
## Confidence and evidence discipline
|
||||
|
||||
This is the load-bearing rule of the skill. Tell the user:
|
||||
|
||||
- Every claim must trace to a record. No naked assertions.
|
||||
- Confidence tier travels with the claim. `match_type=fuzzy` is "probable",
|
||||
not "confirmed."
|
||||
- Entity resolution produces candidates, NOT conclusions. A `fuzzy` match
|
||||
between "ACME LLC" and "Acme Holdings Group" is a lead, not a fact.
|
||||
- Statistical significance ≠ wrongdoing. p < 0.05 means the timing pattern
|
||||
is unlikely under the null. It does not establish corruption.
|
||||
- All data sources here are public records. They may still contain
|
||||
inaccuracies, stale info, or redactions (GDPR, sealed records).
|
||||
|
||||
## Adding a new data source
|
||||
|
||||
Use the template:
|
||||
|
||||
```bash
|
||||
cp SKILL_DIR/templates/source-template.md \
|
||||
SKILL_DIR/references/sources/<your-source>.md
|
||||
```
|
||||
|
||||
Fill in all 9 sections. Write a `fetch_<source>.py` script in `scripts/` that
|
||||
uses stdlib only and writes a normalized CSV. Update the source list in the
|
||||
"When to use" section above.
|
||||
|
||||
## Tools and their limits
|
||||
|
||||
- `entity_resolution.py` does NOT use external fuzzy libraries (no rapidfuzz,
|
||||
no jellyfish). Token-bag matching is the upper bound here. If you need
|
||||
Levenshtein, transliteration, or phonetic matching, pip-install separately.
|
||||
- `timing_analysis.py` uses Python's `random` for permutations. For
|
||||
reproducibility, pass `--seed N`.
|
||||
- `fetch_*.py` scripts use `urllib.request` and respect `Retry-After`. Heavy
|
||||
bulk usage may still violate ToS — read each source's legal section first.
|
||||
|
||||
## Legal note
|
||||
|
||||
All Phase-1 sources are public records. Bulk acquisition is permitted under
|
||||
their respective access terms (FOIA, public records law, ICIJ explicit
|
||||
publication, OFAC public data). However:
|
||||
|
||||
- Some sources rate-limit aggressively. Respect their headers.
|
||||
- Some redact registrant info (GDPR on WHOIS, sealed filings).
|
||||
- Cross-referencing public records to identify private individuals can have
|
||||
ethical implications. The skill produces evidence chains, not accusations.
|
||||
@@ -0,0 +1,98 @@
|
||||
# CourtListener — Free Law Project
|
||||
|
||||
## 1. Summary
|
||||
|
||||
CourtListener (Free Law Project) aggregates court opinions, dockets, oral
|
||||
arguments, and judge data. Covers ~10M federal and state court opinions
|
||||
back to colonial America, plus PACER docket data from RECAP submissions.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **REST API v4:** `https://www.courtlistener.com/api/rest/v4/`
|
||||
- **Auth:** Anonymous reads allowed on most endpoints; token raises rate
|
||||
limits and unlocks bulk export
|
||||
- **Rate limit:** ~5,000 req/hour unauthenticated for search; higher with token
|
||||
|
||||
Set `COURTLISTENER_TOKEN` env var. Get a free token at
|
||||
https://www.courtlistener.com/sign-in/ then create an API key.
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_courtlistener.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `case_name` | str | Case name |
|
||||
| `court` | str | Court name |
|
||||
| `court_id` | str | Court ID (e.g. `nysd`, `scotus`, `ca9`) |
|
||||
| `date_filed` | str | YYYY-MM-DD |
|
||||
| `docket_number` | str | Court docket number |
|
||||
| `judge` | str | Judge name(s) |
|
||||
| `citation` | str | Reporter citation(s) |
|
||||
| `result_type` | str | opinions / dockets / oral / people |
|
||||
| `snippet` | str | Search-match snippet (up to 500 chars) |
|
||||
| `absolute_url` | str | Direct CourtListener URL |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- Federal: all circuit and district courts, SCOTUS
|
||||
- State: all 50 state supreme/appellate courts, many trial courts
|
||||
- Opinions: ~10M back to 1600s (colonial), full coverage 1950 → present
|
||||
- Dockets via RECAP: ~3M+ from user-submitted PACER PDFs
|
||||
- Updated continuously
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **OpenCorporates** ↔ `case_name` (corporate litigation)
|
||||
- **SEC EDGAR** ↔ `case_name` (securities class actions)
|
||||
- **OFAC SDN** ↔ `case_name` (sanctions-related civil/criminal cases)
|
||||
|
||||
Join key: party name from `case_name`. Note: `case_name` often abbreviates
|
||||
("Smith v. Jones" rather than full party names) — use the full case URL
|
||||
to get all parties.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Older opinions (pre-1990) often lack docket numbers and judges
|
||||
- State coverage is more uneven than federal
|
||||
- PACER docket coverage depends on RECAP user submissions — not exhaustive
|
||||
- Sealed documents are excluded
|
||||
- Party names in case captions don't always match filing names exactly
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_courtlistener.py`
|
||||
|
||||
```bash
|
||||
# Search opinions for a party / keyword
|
||||
python3 SKILL_DIR/scripts/fetch_courtlistener.py --query "Example Corp" \
|
||||
--out data/cl.csv
|
||||
|
||||
# PACER dockets (best for recent litigation)
|
||||
python3 SKILL_DIR/scripts/fetch_courtlistener.py --query "Example Corp" \
|
||||
--type dockets --out data/cl_dockets.csv
|
||||
|
||||
# Restrict to a court
|
||||
python3 SKILL_DIR/scripts/fetch_courtlistener.py --query "Microsoft" \
|
||||
--court ca9 --out data/cl_9th.csv
|
||||
|
||||
# Date range
|
||||
python3 SKILL_DIR/scripts/fetch_courtlistener.py --query "Example Corp" \
|
||||
--date-from 2020-01-01 --date-to 2024-12-31 --out data/cl.csv
|
||||
```
|
||||
|
||||
Pass `--token` or set `COURTLISTENER_TOKEN`.
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Court opinions are public domain
|
||||
- Free Law Project provides the data under CC0 / public domain dedication
|
||||
- No commercial use restrictions on opinion text or metadata
|
||||
- Some PACER PDFs have copyright on layout (not text) — fair use applies
|
||||
|
||||
## 9. References
|
||||
|
||||
- API docs: https://www.courtlistener.com/help/api/rest/
|
||||
- Court IDs: https://www.courtlistener.com/api/jurisdictions/
|
||||
- RECAP archive: https://www.courtlistener.com/recap/
|
||||
- Bulk data: https://www.courtlistener.com/help/api/bulk-data/
|
||||
@@ -0,0 +1,104 @@
|
||||
# GDELT — Global News Monitoring
|
||||
|
||||
## 1. Summary
|
||||
|
||||
GDELT (Global Database of Events, Language, and Tone) monitors world news
|
||||
in 100+ languages with full-text indexing. Updated every 15 minutes.
|
||||
~2015 → present, ~1B+ articles indexed. Free anonymous access.
|
||||
|
||||
GDELT is wider than Google News (more international, more long-tail
|
||||
sources) and indexed by tone/sentiment, themes (CAMEO codes), people, and
|
||||
organizations.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **DOC 2.0 API:** `https://api.gdeltproject.org/api/v2/doc/doc`
|
||||
- **Events / GKG 2.0:** `https://api.gdeltproject.org/api/v2/events/events`
|
||||
- **Auth:** None
|
||||
- **Rate limit:** **1 request per 5 seconds** for the DOC API — strict
|
||||
|
||||
The fetch script automatically retries after a 6-second sleep when a
|
||||
429 is received.
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_gdelt.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `title` | str | Article title |
|
||||
| `url` | str | Article URL |
|
||||
| `seen_date` | str | When GDELT first saw the article (UTC) |
|
||||
| `domain` | str | Publisher domain |
|
||||
| `language` | str | Source language |
|
||||
| `source_country` | str | 2-letter country code |
|
||||
| `tone` | str | GDELT-computed tone score (negative = negative coverage) |
|
||||
| `social_image` | str | Open Graph image URL when available |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- Worldwide news in 100+ languages
|
||||
- ~2015 → present (Events back to 1979 via a separate stream)
|
||||
- Update frequency: 15 minutes
|
||||
- Bias: heavily Anglophone in volume but very wide source list overall
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **All sources** ↔ `title` / `url` (news context for any subject)
|
||||
- **Wikipedia** ↔ event timeline for notable entities
|
||||
- **Wayback Machine** ↔ recover articles whose URLs have died
|
||||
- **OFAC SDN** ↔ news context for sanctions designations
|
||||
- **SEC EDGAR** ↔ news context for 8-K material events
|
||||
|
||||
Join key: entity name appearing in article title or full-text. GDELT also
|
||||
extracts named entities into a separate stream (GKG) not exposed by this
|
||||
fetcher — query GDELT directly for entity-level filtering.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Title extraction is automated and can be wrong (sometimes captures the
|
||||
site name + delimiter + article title; sometimes a generic page title)
|
||||
- Sentiment / tone is computed by GDELT, not source-supplied
|
||||
- Some domains are oversampled (newswires, aggregators)
|
||||
- Source country is inferred from domain registration / TLD — can be
|
||||
wrong for international news sites with country-neutral domains
|
||||
- Article URLs can rot — pair with Wayback Machine to preserve content
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_gdelt.py`
|
||||
|
||||
```bash
|
||||
# Recent news mentioning an entity
|
||||
python3 SKILL_DIR/scripts/fetch_gdelt.py --query "Nous Research" \
|
||||
--timespan 6m --out data/gdelt.csv
|
||||
|
||||
# Phrase-exact (use double quotes inside single quotes for the shell)
|
||||
python3 SKILL_DIR/scripts/fetch_gdelt.py --query '"Dillon Rolnick"' \
|
||||
--timespan 1y --out data/gdelt.csv
|
||||
|
||||
# Filter to a country / language
|
||||
python3 SKILL_DIR/scripts/fetch_gdelt.py --query "Microsoft" \
|
||||
--source-country US --source-lang English --out data/gdelt.csv
|
||||
|
||||
# Date range
|
||||
python3 SKILL_DIR/scripts/fetch_gdelt.py --query "Microsoft" \
|
||||
--start 2024-01-01 --end 2024-12-31 --out data/gdelt.csv
|
||||
```
|
||||
|
||||
GDELT supports its own query operators: phrase quoting, AND/OR/NOT,
|
||||
`sourcecountry:US`, `theme:ECON_BANKRUPTCY`, `tone<-5`, etc.
|
||||
See https://blog.gdeltproject.org/gdelt-doc-2-0-api-debuts/ for syntax.
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- GDELT data is provided free for academic and journalistic use
|
||||
- Article URLs link out to original publishers — copyright remains with
|
||||
the publisher
|
||||
- GDELT is NOT a content archive; it's a metadata index
|
||||
|
||||
## 9. References
|
||||
|
||||
- DOC 2.0 API: https://blog.gdeltproject.org/gdelt-doc-2-0-api-debuts/
|
||||
- Themes & query syntax: https://blog.gdeltproject.org/gkg-2-0-our-global-knowledge-graph-2-0-amazing-data-at-your-fingertips/
|
||||
- Project home: https://www.gdeltproject.org/
|
||||
@@ -0,0 +1,104 @@
|
||||
# ICIJ Offshore Leaks Database
|
||||
|
||||
## 1. Summary
|
||||
|
||||
The International Consortium of Investigative Journalists (ICIJ) publishes a
|
||||
combined database of offshore entities from the Panama Papers, Paradise Papers,
|
||||
Pandora Papers, Bahamas Leaks, and Offshore Leaks. ~800,000+ offshore entities
|
||||
with their officers, intermediaries, and addresses.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **Bulk download (primary):** `https://offshoreleaks-data.icij.org/offshoreleaks/csv/full-oldb.LATEST.zip` (~70 MB ZIP, refreshed periodically)
|
||||
- **Search UI (human):** `https://offshoreleaks.icij.org/`
|
||||
- **Auth:** None
|
||||
- **Note:** The previous Open Refine reconciliation endpoint at
|
||||
`/reconcile` now returns 404. ICIJ has removed it. The bulk ZIP is the
|
||||
remaining stable access path. The skill's `fetch_icij_offshore.py` caches
|
||||
the ZIP locally (default `~/.cache/hermes-osint/icij/`, refreshes after
|
||||
30 days) and searches it offline.
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_icij_offshore.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `node_id` | int | ICIJ canonical node ID |
|
||||
| `name` | str | Entity / officer / intermediary name |
|
||||
| `node_type` | str | entity / officer / intermediary / address |
|
||||
| `country_codes` | str | Semicolon-separated ISO codes |
|
||||
| `countries` | str | Country names |
|
||||
| `jurisdiction` | str | Offshore jurisdiction (BVI, Panama, etc.) |
|
||||
| `incorporation_date` | str | YYYY-MM-DD |
|
||||
| `inactivation_date` | str | YYYY-MM-DD (if struck) |
|
||||
| `source` | str | Panama Papers / Paradise Papers / Pandora Papers / etc. |
|
||||
| `entity_url` | str | Link to ICIJ page |
|
||||
| `connections` | str | Semicolon-separated node IDs of related entities |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- Worldwide offshore entity records
|
||||
- Earliest records: 1970s (Bahamas Leaks). Most data 1990–2018.
|
||||
- NOT updated in real-time — new leaks added when ICIJ publishes them
|
||||
- ~810,000 offshore entities + ~750,000 officers + ~150,000 intermediaries
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **SEC EDGAR** ↔ `name` (public companies with offshore arms)
|
||||
- **USAspending** ↔ `name` (federal contractors with offshore structure)
|
||||
- **OFAC SDN** ↔ `name` (sanctioned entities using offshore vehicles)
|
||||
|
||||
Join key: normalized entity/officer name. `node_id` is canonical for cross-
|
||||
referencing within ICIJ. Connections graph traversal is in-script (BFS over
|
||||
`connections`).
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Offshore entity names sometimes appear in multiple leaks with slight variations
|
||||
- Officers may be nominees (front persons), not beneficial owners
|
||||
- Some entries have minimal info (just a name + jurisdiction)
|
||||
- The connections graph is incomplete — some relationships are documented in
|
||||
source materials but not in the structured database
|
||||
- Inactive/struck-off entities are still included with `inactivation_date`
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_icij_offshore.py`
|
||||
|
||||
```bash
|
||||
# Search by entity name (case-insensitive substring across the bulk DB)
|
||||
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --entity "EXAMPLE CORP" \
|
||||
--out data/icij.csv
|
||||
|
||||
# Search by officer (individual person)
|
||||
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --officer "SMITH JOHN" \
|
||||
--out data/icij.csv
|
||||
|
||||
# Search by jurisdiction (filter on cached results)
|
||||
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --officer "SMITH" \
|
||||
--jurisdiction "BRITISH VIRGIN ISLANDS" --out data/icij_bvi.csv
|
||||
|
||||
# Force a fresh download (default refresh window is 30 days)
|
||||
python3 SKILL_DIR/scripts/fetch_icij_offshore.py --entity "EXAMPLE CORP" \
|
||||
--force-refresh --out data/icij.csv
|
||||
```
|
||||
|
||||
First call downloads the ~70 MB ZIP under `~/.cache/hermes-osint/icij/`
|
||||
(or `$HERMES_OSINT_CACHE/icij/`). Subsequent calls reuse the cache for 30 days.
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Public record as published by ICIJ under explicit publication
|
||||
- No copyright on the underlying facts (entity names, jurisdictions)
|
||||
- ICIJ asks for attribution if used in derivative reporting
|
||||
- **Ethical note**: Presence in this database does NOT imply wrongdoing. Many
|
||||
offshore structures are legal. The database is a research tool, not a list of
|
||||
criminals.
|
||||
|
||||
## 9. References
|
||||
|
||||
- Database: https://offshoreleaks.icij.org/
|
||||
- About the data: https://offshoreleaks.icij.org/pages/about
|
||||
- Methodology: https://www.icij.org/investigations/panama-papers/
|
||||
- API hints: Open Refine reconciliation endpoint at `https://offshoreleaks.icij.org/reconcile`
|
||||
@@ -0,0 +1,90 @@
|
||||
# NYC ACRIS — NYC Real Property Records
|
||||
|
||||
## 1. Summary
|
||||
|
||||
The Automated City Register Information System (ACRIS) is NYC's index of
|
||||
recorded property documents: deeds, mortgages, satisfactions, liens, UCC
|
||||
filings. Covers Manhattan, Bronx, Brooklyn, Queens, Staten Island.
|
||||
Published as 4 linked Socrata datasets on the NYC Open Data portal.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **Socrata API:** `https://data.cityofnewyork.us/resource/636b-3b5g.json` (Parties)
|
||||
- **Other datasets:** `bnx9-e6tj` (Master), `8h5j-fqxa` (Legal), `uqqa-hym2` (References)
|
||||
- **Auth:** None for read access (Socrata `$app_token` raises rate limits if needed)
|
||||
- **Rate limit:** Generous (~1000 req/hour unauthenticated)
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_nyc_acris.py` (Parties joined to Master):
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `document_id` | str | ACRIS document ID |
|
||||
| `name` | str | Party name as recorded (often "LAST, FIRST" but varies) |
|
||||
| `party_type` | str | 1=grantor, 2=grantee, 3=other |
|
||||
| `party_role` | str | Human-readable role label |
|
||||
| `address_1` | str | Property or party address line 1 |
|
||||
| `city`, `state`, `zip`, `country` | str | Address parts |
|
||||
| `doc_type` | str | DEED, MTGE (mortgage), SAT (satisfaction), AGMT, etc. |
|
||||
| `doc_date`, `recorded_date` | str | YYYY-MM-DD |
|
||||
| `borough` | str | Manhattan / Bronx / Brooklyn / Queens / Staten Island |
|
||||
| `amount` | str | Document amount (USD, when applicable) |
|
||||
| `filing_url` | str | Direct ACRIS DocumentImageView link |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- NYC 5 boroughs only — other counties have their own recorders
|
||||
- 1966 → present (older filings exist on microfilm at the County Clerk)
|
||||
- Updated nightly
|
||||
- ~70M+ party records cumulative
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **SEC EDGAR** ↔ `name` (insider filers with NYC property)
|
||||
- **USAspending** ↔ `name` (federal contractors with NYC property)
|
||||
- **Senate LDA** ↔ `name` (lobbyists / clients with NYC property)
|
||||
- **ICIJ Offshore** ↔ `name` (NYC properties owned via offshore vehicles)
|
||||
|
||||
Join key: normalized party name. NYC property records typically store names
|
||||
as "LAST, FIRST" or full LLC names — use `entity_resolution.py`.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Same person appears with multiple name formats over time
|
||||
- LLC and trust ownership obscures beneficial owners
|
||||
- Recording lag can be 2-4 weeks after closing
|
||||
- Older documents have spottier address data
|
||||
- Sealed records (e.g. domestic violence shelters) are excluded by law
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_nyc_acris.py`
|
||||
|
||||
```bash
|
||||
# By party name
|
||||
python3 SKILL_DIR/scripts/fetch_nyc_acris.py --name "ROLNICK" --out data/acris.csv
|
||||
|
||||
# By address (useful when you know the property but not the names)
|
||||
python3 SKILL_DIR/scripts/fetch_nyc_acris.py --address "571 HUDSON" --out data/acris.csv
|
||||
|
||||
# Restrict to grantees (buyers / mortgagees)
|
||||
python3 SKILL_DIR/scripts/fetch_nyc_acris.py --name "ROLNICK" --party-type 2 \
|
||||
--out data/acris_buyers.csv
|
||||
```
|
||||
|
||||
The script joins Parties → Master to populate doc_type, dates, borough, and
|
||||
amount. Pass `--no-enrich` to skip the join (faster, fewer columns).
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Public record under NYS Real Property Law and NYC Charter
|
||||
- No commercial use restrictions on the data
|
||||
- All ACRIS data is public information by statute
|
||||
|
||||
## 9. References
|
||||
|
||||
- ACRIS portal: https://a836-acris.nyc.gov/CP/
|
||||
- NYC Open Data: https://data.cityofnewyork.us/
|
||||
- Parties dataset: https://data.cityofnewyork.us/City-Government/ACRIS-Real-Property-Parties/636b-3b5g
|
||||
- Document type codes: https://www1.nyc.gov/site/finance/taxes/acris.page
|
||||
@@ -0,0 +1,92 @@
|
||||
# OFAC SDN — Specially Designated Nationals List
|
||||
|
||||
## 1. Summary
|
||||
|
||||
The Office of Foreign Assets Control (OFAC) publishes the Specially Designated
|
||||
Nationals and Blocked Persons List (SDN). US persons are generally prohibited
|
||||
from dealing with individuals and entities on this list. Also published:
|
||||
non-SDN consolidated lists (BIS Denied Persons, FSE, etc.).
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **Full XML:** `https://www.treasury.gov/ofac/downloads/sdn.xml`
|
||||
- **Delimited:** `https://www.treasury.gov/ofac/downloads/sdn.csv`
|
||||
- **Consolidated:** `https://www.treasury.gov/ofac/downloads/consolidated/consolidated.xml`
|
||||
- **Auth:** None
|
||||
- **Rate limit:** None (static file downloads). Updated continuously.
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_ofac_sdn.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `entity_id` | int | OFAC unique ID |
|
||||
| `name` | str | Primary name |
|
||||
| `entity_type` | str | individual / entity / vessel / aircraft |
|
||||
| `program_list` | str | Semicolon-separated sanctions programs (e.g. SDGT;IRAN) |
|
||||
| `title` | str | For individuals: title/role |
|
||||
| `nationalities` | str | Semicolon-separated country codes |
|
||||
| `aka_list` | str | Semicolon-separated "also known as" names |
|
||||
| `addresses` | str | Semicolon-separated known addresses |
|
||||
| `dob` | str | Date of birth (individuals) |
|
||||
| `pob` | str | Place of birth (individuals) |
|
||||
| `remarks` | str | OFAC's free-text remarks |
|
||||
| `last_updated` | str | YYYY-MM-DD (publication date) |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- Worldwide — all entities sanctioned by US Treasury
|
||||
- ~10,000 entries on SDN, ~15,000 on consolidated lists
|
||||
- Updated continuously (sometimes daily during active enforcement)
|
||||
- Includes AKAs (very common, can be 10+ per entity)
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **SEC EDGAR** ↔ `name` (public companies sanctioned)
|
||||
- **USAspending** ↔ `name` (sanctioned entity as federal contractor — should
|
||||
be impossible but verify)
|
||||
- **ICIJ Offshore** ↔ `name` (offshore entities also sanctioned)
|
||||
|
||||
Join key: normalized name. **CRITICAL**: must match against `aka_list` too.
|
||||
Many sanctioned entities are caught only via aliases.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Names are transliterated from many scripts — multiple romanizations possible
|
||||
- AKAs often differ wildly from primary name
|
||||
- Some entries have minimal info (no DOB, no address) for individuals
|
||||
- Free-text `remarks` contain critical context — read them
|
||||
- "Specially Designated Global Terrorists" (SDGT) and "Cyber-related" (CYBER2)
|
||||
programs add and remove entries frequently
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_ofac_sdn.py`
|
||||
|
||||
```bash
|
||||
# Full snapshot
|
||||
python3 SKILL_DIR/scripts/fetch_ofac_sdn.py --out data/ofac_sdn.csv
|
||||
|
||||
# Filter to specific program
|
||||
python3 SKILL_DIR/scripts/fetch_ofac_sdn.py --program SDGT --out data/sdn_sdgt.csv
|
||||
|
||||
# Entities only (skip individuals, vessels, aircraft)
|
||||
python3 SKILL_DIR/scripts/fetch_ofac_sdn.py --entity-type entity --out data/sdn_entities.csv
|
||||
```
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Public record under Executive Order authority and statutory sanctions programs
|
||||
- US persons MUST screen against this list — it is enforced
|
||||
- No restrictions on the data itself; restrictions are on transactions with
|
||||
the listed entities
|
||||
- ZERO penalty for "over-matching" — false positives must be cleared but are not
|
||||
prohibited
|
||||
|
||||
## 9. References
|
||||
|
||||
- OFAC home: https://ofac.treasury.gov/
|
||||
- SDN list: https://ofac.treasury.gov/specially-designated-nationals-and-blocked-persons-list-sdn-human-readable-lists
|
||||
- Data formats: https://ofac.treasury.gov/sdn-list/sanctions-list-search-tool
|
||||
- Compliance guidance: https://ofac.treasury.gov/recent-actions
|
||||
@@ -0,0 +1,103 @@
|
||||
# OpenCorporates — Global Corporate Registry
|
||||
|
||||
## 1. Summary
|
||||
|
||||
OpenCorporates aggregates corporate registry data from 130+ jurisdictions
|
||||
worldwide (~200M companies). Covers US state-level filings (NY DOS, Delaware
|
||||
DOC, California SOS, etc.), UK Companies House, EU registries, and most
|
||||
common-law jurisdictions.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **REST API:** `https://api.opencorporates.com/v0.4/`
|
||||
- **HTML fallback:** `https://opencorporates.com/companies?q=...`
|
||||
- **Auth:** API token required (free tier 500 calls/month, paid plans available)
|
||||
- **Rate limit:** Token-bound; un-tokened requests return 401
|
||||
|
||||
Set `OPENCORPORATES_API_TOKEN` env var. Get a free token at
|
||||
https://opencorporates.com/api_accounts/new.
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_opencorporates.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `name` | str | Company legal name |
|
||||
| `company_number` | str | Registry-assigned number |
|
||||
| `jurisdiction_code` | str | e.g. `us_ny`, `us_de`, `gb` |
|
||||
| `jurisdiction_name` | str | Human-readable jurisdiction |
|
||||
| `incorporation_date` | str | YYYY-MM-DD |
|
||||
| `dissolution_date` | str | YYYY-MM-DD (empty if active) |
|
||||
| `company_type` | str | Domestic LLC / Foreign Corp / etc. |
|
||||
| `status` | str | Active / Inactive / Dissolved |
|
||||
| `registered_address` | str | Registered office address |
|
||||
| `opencorporates_url` | str | Link to OpenCorporates entity page |
|
||||
| `officers_count` | str | Total officers on record |
|
||||
| `source` | str | `api`, `html`, or `html-fallback` |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- US: all 50 states + DC at state level (LLCs, corps, LPs)
|
||||
- International: UK, EU, Canada, Australia, NZ, many APAC + LATAM jurisdictions
|
||||
- ~200M company records cumulative
|
||||
- Update frequency varies by jurisdiction (UK CH is near-realtime; some
|
||||
state registries lag months)
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **NYC ACRIS** ↔ `name` (LLC/corp owners of NYC property)
|
||||
- **USAspending** ↔ `name` (corporate federal contractors)
|
||||
- **SEC EDGAR** ↔ `name` (public companies + their subsidiaries)
|
||||
- **ICIJ Offshore** ↔ `name` (international corporate structures)
|
||||
|
||||
Join key: normalized company name. Some entries have `previous_names` arrays
|
||||
which are not currently exported by the fetch script — query OC directly
|
||||
for that.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Company-name spellings vary across re-incorporations and renames
|
||||
- Officer records are spottier than company records (many jurisdictions
|
||||
don't require officer disclosure)
|
||||
- Beneficial-ownership data is generally NOT here — most jurisdictions
|
||||
don't require it. UK Companies House has PSC (people with significant
|
||||
control) but that's not universal.
|
||||
- Cross-jurisdictional links (parent / subsidiary) are based on registry
|
||||
filings only; corporate trees are often incomplete
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_opencorporates.py`
|
||||
|
||||
```bash
|
||||
# Search globally by name
|
||||
python3 SKILL_DIR/scripts/fetch_opencorporates.py --query "Example Corp" \
|
||||
--out data/oc.csv
|
||||
|
||||
# Restrict to a jurisdiction
|
||||
python3 SKILL_DIR/scripts/fetch_opencorporates.py --query "Example Corp" \
|
||||
--jurisdiction us_ny --out data/oc_ny.csv
|
||||
|
||||
# Set token via env or flag
|
||||
OPENCORPORATES_API_TOKEN=xxx python3 SKILL_DIR/scripts/fetch_opencorporates.py \
|
||||
--query "Microsoft" --out data/oc.csv
|
||||
```
|
||||
|
||||
Without a token the script falls back to scraping the HTML search page.
|
||||
The fallback is brittle and only fills in `name`, `jurisdiction_code`,
|
||||
`opencorporates_url` — set the token for serious work.
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- OpenCorporates aggregates public records — the underlying facts are
|
||||
public domain
|
||||
- OpenCorporates own database is licensed CC-BY-SA-4.0; attribution required
|
||||
- API ToS prohibits redistributing the full dataset; per-record reference
|
||||
is fine
|
||||
|
||||
## 9. References
|
||||
|
||||
- API docs: https://api.opencorporates.com/documentation/API-Reference
|
||||
- Jurisdiction codes: https://api.opencorporates.com/v0.4/jurisdictions.json
|
||||
- Schema: https://opencorporates.com/info/our_data
|
||||
@@ -0,0 +1,83 @@
|
||||
# SEC EDGAR — Corporate Filings
|
||||
|
||||
## 1. Summary
|
||||
|
||||
EDGAR (Electronic Data Gathering, Analysis, and Retrieval) is the SEC's system
|
||||
for corporate disclosure filings: 10-K (annual), 10-Q (quarterly), 8-K (current
|
||||
events), DEF 14A (proxy), Form 4 (insider trading), 13F (institutional holdings).
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **API:** `https://data.sec.gov/submissions/CIK<10-digit-padded>.json` (no auth)
|
||||
- **Filing index:** `https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=...`
|
||||
- **Full-text search:** `https://efts.sec.gov/LATEST/search-index?q=...`
|
||||
- **Auth:** None — requires `User-Agent` header with contact info per SEC policy
|
||||
- **Rate limit:** 10 requests/second per IP (enforced)
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_sec_edgar.py` (filings index):
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `cik` | str | Central Index Key (10-digit padded) |
|
||||
| `company_name` | str | Registrant name |
|
||||
| `form_type` | str | 10-K, 10-Q, 8-K, etc. |
|
||||
| `filing_date` | str | YYYY-MM-DD |
|
||||
| `accession_number` | str | Filing accession (e.g. 0000320193-24-000123) |
|
||||
| `primary_document` | str | Filename of main document |
|
||||
| `filing_url` | str | Direct URL to filing index |
|
||||
| `reporting_period` | str | Period of report (where applicable) |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- All public US registrants from 1993 → present
|
||||
- 1993-2000 has spotty coverage of older filings (paper-to-electronic migration)
|
||||
- ~12M filings cumulative
|
||||
- Updated within minutes of filing acceptance
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **USAspending** ↔ `company_name` (public companies as federal contractors)
|
||||
- **Senate LD** ↔ `company_name` (public companies hire lobbyists)
|
||||
- **OFAC SDN** ↔ `company_name` (sanctions screening of public registrants)
|
||||
|
||||
Join key: company name OR CIK if you have it. CIK is canonical and stable.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Subsidiaries often filed under parent CIK — be careful with name matches
|
||||
- Name changes over time (rebrands, acquisitions) — CIK remains constant
|
||||
- 10-K Item 1A Risk Factors are free-form text — useful for `web_extract`-style
|
||||
parsing, not structured queries
|
||||
- Foreign private issuers file 20-F instead of 10-K
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_sec_edgar.py`
|
||||
|
||||
```bash
|
||||
# By CIK
|
||||
python3 SKILL_DIR/scripts/fetch_sec_edgar.py --cik 0000320193 \
|
||||
--types 10-K,10-Q --out data/edgar_filings.csv
|
||||
|
||||
# By company name (resolves to CIK first via name search)
|
||||
python3 SKILL_DIR/scripts/fetch_sec_edgar.py --company "APPLE INC" \
|
||||
--types 8-K --since 2024-01-01 --out data/edgar_filings.csv
|
||||
```
|
||||
|
||||
Set `SEC_USER_AGENT` env var with your contact email (SEC requirement).
|
||||
Example: `SEC_USER_AGENT="Research example@example.com"`.
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Public record under SEC Rule 24b-2 / 17 CFR § 230.401
|
||||
- No commercial use restrictions on filing content
|
||||
- SEC asks all bulk users to include a `User-Agent` with contact info and to
|
||||
respect 10 req/s — failure to do so can result in IP blocking
|
||||
|
||||
## 9. References
|
||||
|
||||
- Developer docs: https://www.sec.gov/edgar/sec-api-documentation
|
||||
- EDGAR full-text search: https://efts.sec.gov/LATEST/search-index
|
||||
- Fair access policy: https://www.sec.gov/os/accessing-edgar-data
|
||||
@@ -0,0 +1,89 @@
|
||||
# Senate LD — Lobbying Disclosure (LD-1 / LD-2)
|
||||
|
||||
## 1. Summary
|
||||
|
||||
The Senate Office of Public Records publishes lobbying disclosures under the
|
||||
Lobbying Disclosure Act of 1995 (LDA, as amended by HLOGA 2007). LD-1 is
|
||||
registration of a new client-lobbyist relationship; LD-2 is the quarterly
|
||||
activity report.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **API:** `https://lda.senate.gov/api/v1/` (no auth required for read-only)
|
||||
- **Bulk download:** `https://lda.senate.gov/api/v1/filings/?format=csv` (paginated)
|
||||
- **Auth:** Token required for >120 req/hour — register at https://lda.senate.gov/api/auth/register/
|
||||
- **Rate limit:** 120 req/hour unauthenticated, 1,200 req/hour authenticated
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_senate_ld.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `filing_uuid` | str | Unique filing ID |
|
||||
| `filing_type` | str | LD-1, LD-2, LD-203, etc. |
|
||||
| `filing_year` | int | Year |
|
||||
| `filing_period` | str | Q1/Q2/Q3/Q4 or annual |
|
||||
| `registrant_name` | str | Lobbying firm or organization |
|
||||
| `registrant_id` | str | Senate-assigned registrant ID |
|
||||
| `client_name` | str | Client being represented |
|
||||
| `client_id` | str | Senate-assigned client ID |
|
||||
| `client_general_description` | str | Client industry / business |
|
||||
| `income` | float | LD-2 income from client this quarter (USD) |
|
||||
| `expenses` | float | LD-2 expenses (in-house lobbying) |
|
||||
| `lobbyists` | str | Semicolon-separated lobbyist names |
|
||||
| `issues` | str | Semicolon-separated issue areas |
|
||||
| `government_entities` | str | Agencies/chambers contacted |
|
||||
| `filing_date` | str | YYYY-MM-DD |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- US federal lobbying only (state lobbying handled by individual state ethics offices)
|
||||
- 1999 → present (full electronic coverage from 2008)
|
||||
- Quarterly reporting cycle (LD-2)
|
||||
- ~1M+ filings cumulative
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **USAspending** ↔ `client_name` (clients lobbying for contracts)
|
||||
- **SEC EDGAR** ↔ `client_name` (public companies as lobbying clients)
|
||||
- **OFAC SDN** ↔ `client_name` (sanctions screening of lobbying clients)
|
||||
|
||||
Join key: normalized client_name. registrant_id and client_id are canonical
|
||||
when joining Senate-internal records.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Many lobbyist names appear in multiple registrants over time (job changes)
|
||||
- `issues` and `government_entities` are free-text — Inconsistent capitalization
|
||||
- Foreign agents register under FARA (Department of Justice), NOT here
|
||||
- Income/expenses are reported in $10,000 brackets in some older filings
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_senate_ld.py`
|
||||
|
||||
```bash
|
||||
# By client
|
||||
python3 SKILL_DIR/scripts/fetch_senate_ld.py --client "EXAMPLE CORP" \
|
||||
--year 2024 --out data/lobbying.csv
|
||||
|
||||
# By registrant (lobbying firm)
|
||||
python3 SKILL_DIR/scripts/fetch_senate_ld.py --registrant "BIG K STREET LLP" \
|
||||
--year 2024 --out data/lobbying.csv
|
||||
```
|
||||
|
||||
Set `SENATE_LDA_TOKEN` env var if you have one (or pass `--token`).
|
||||
Defaults to anonymous (120 req/hour).
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Public record under 2 U.S.C. § 1604 (LDA)
|
||||
- No commercial use restrictions
|
||||
- Reuse is unconditional — see Senate Public Records Office disclaimer
|
||||
|
||||
## 9. References
|
||||
|
||||
- API docs: https://lda.senate.gov/api/redoc/v1/
|
||||
- LDA guidance: https://lobbyingdisclosure.house.gov/ld_guidance.pdf
|
||||
- Senate Public Records: https://lda.senate.gov/
|
||||
@@ -0,0 +1,97 @@
|
||||
# USAspending — Federal Government Contracts and Grants
|
||||
|
||||
## 1. Summary
|
||||
|
||||
USAspending.gov is the official source of federal spending data. Coverage:
|
||||
contracts, grants, loans, direct payments, sub-awards. Required by the DATA Act
|
||||
of 2014 — all federal agencies must report to a single schema.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **API v2:** `https://api.usaspending.gov/api/v2/` (no auth, no key)
|
||||
- **Bulk:** `https://files.usaspending.gov/` (CSV / Parquet by award type)
|
||||
- **Auth:** None
|
||||
- **Rate limit:** Not strictly enforced, but be polite — keep to <10 req/s
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_usaspending.py` (prime awards):
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `award_id` | str | Federal award ID (PIID for contracts, FAIN for grants) |
|
||||
| `recipient_name` | str | Awardee legal name |
|
||||
| `recipient_uei` | str | Unique Entity Identifier (replaced DUNS in 2022) |
|
||||
| `recipient_duns` | str | Legacy DUNS number (historical only) |
|
||||
| `recipient_parent_name` | str | Ultimate parent organization |
|
||||
| `recipient_state` | str | Recipient state |
|
||||
| `awarding_agency` | str | Department / agency name |
|
||||
| `awarding_sub_agency` | str | Sub-tier (e.g. DoD → Army) |
|
||||
| `award_type` | str | Contract / Grant / Loan / Direct Payment |
|
||||
| `award_amount` | float | Current total obligation in USD |
|
||||
| `award_date` | str | Action / signed date YYYY-MM-DD |
|
||||
| `period_of_performance_start` | str | YYYY-MM-DD |
|
||||
| `period_of_performance_end` | str | YYYY-MM-DD |
|
||||
| `naics_code` | str | Industry classification |
|
||||
| `psc_code` | str | Product / Service Code |
|
||||
| `competition_extent` | str | Full / limited / sole-source |
|
||||
| `description` | str | Award description (free-text) |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- US federal awards only (state/local not included)
|
||||
- FY 2008 → present (full coverage from FY 2017)
|
||||
- Updated bi-weekly from agency reporting
|
||||
- ~100M+ transaction records cumulative
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **SEC EDGAR** ↔ `recipient_name` (public companies as contractors)
|
||||
- **Senate LD** ↔ `recipient_name` (lobbying clients winning contracts)
|
||||
- **OFAC SDN** ↔ `recipient_name` (sanctions screening of contractors — must be
|
||||
filtered out by SAM.gov but verify)
|
||||
- **ICIJ Offshore** ↔ `recipient_name` (offshore-linked contractors)
|
||||
|
||||
Join key: normalized recipient name. UEI is canonical when present.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- DUNS → UEI transition (April 2022) — old records have DUNS, new records have UEI
|
||||
- Some sub-awards aren't reported (FFATA threshold is $30k)
|
||||
- Award amount changes over time (mod actions) — fetch script reports current total
|
||||
- `competition_extent` field is free-text in older records — `fetch_usaspending.py`
|
||||
normalizes to canonical values
|
||||
- Recipient name variations are extensive — "ACME LLC", "Acme L.L.C.", "ACME, INC"
|
||||
all appear. Use `entity_resolution.py`.
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_usaspending.py`
|
||||
|
||||
```bash
|
||||
# By recipient name
|
||||
python3 SKILL_DIR/scripts/fetch_usaspending.py --recipient "EXAMPLE CORP" \
|
||||
--fy 2024 --out data/contracts.csv
|
||||
|
||||
# By awarding agency
|
||||
python3 SKILL_DIR/scripts/fetch_usaspending.py --agency "Department of Defense" \
|
||||
--fy 2024 --out data/contracts.csv
|
||||
|
||||
# Filter to sole-source only
|
||||
python3 SKILL_DIR/scripts/fetch_usaspending.py --recipient "EXAMPLE CORP" \
|
||||
--fy 2024 --sole-source-only --out data/contracts.csv
|
||||
```
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Public record under the Federal Funding Accountability and Transparency Act
|
||||
(FFATA, 2006) and DATA Act (2014)
|
||||
- No commercial use restrictions on the data
|
||||
- Personal information of award recipients (e.g. small business owners' addresses
|
||||
in some grants) should be handled per the source agency's privacy notice
|
||||
|
||||
## 9. References
|
||||
|
||||
- API docs: https://api.usaspending.gov/
|
||||
- Data dictionary: https://www.usaspending.gov/data-dictionary
|
||||
- Award schema: https://files.usaspending.gov/docs/Data_Dictionary_Crosswalk.xlsx
|
||||
@@ -0,0 +1,93 @@
|
||||
# Wayback Machine — Internet Archive CDX
|
||||
|
||||
## 1. Summary
|
||||
|
||||
The Internet Archive's Wayback Machine has captured ~900B+ web pages since
|
||||
1996. The CDX server API indexes those captures by URL, timestamp, and
|
||||
content hash. Free, anonymous, no auth.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **CDX server:** `https://web.archive.org/cdx/search/cdx`
|
||||
- **Wayback URL:** `https://web.archive.org/web/<timestamp>/<url>`
|
||||
- **Save Page Now (write):** `https://web.archive.org/save/<url>` (different API)
|
||||
- **Auth:** None
|
||||
- **Rate limit:** Generous; be polite (~1 req/s)
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_wayback.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `url` | str | Original URL captured |
|
||||
| `timestamp` | str | YYYYMMDDHHMMSS (CDX format) |
|
||||
| `wayback_url` | str | Direct replay URL |
|
||||
| `mimetype` | str | Content-type at capture |
|
||||
| `status` | str | HTTP status (typically 200) |
|
||||
| `digest` | str | SHA1 of capture content (collapse-friendly) |
|
||||
| `length` | str | Byte length of capture |
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- 1996 → present
|
||||
- ~900B+ captures across ~700M domains
|
||||
- Updated continuously by automated crawls + manual saves
|
||||
- Some domains have aggressive coverage (news), others sparse (private)
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **Wikipedia** ↔ Reverse-lookup pages cited as references that have since
|
||||
disappeared
|
||||
- **News URLs** ↔ Original article content when present-day URLs 404
|
||||
- **Corporate websites** ↔ Historical "About" pages, executive bios that
|
||||
have been scrubbed
|
||||
|
||||
The Wayback CDX is most useful as a **content-recovery** layer when other
|
||||
sources point to URLs that no longer exist.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- robots.txt-blocked domains may have spotty or no coverage
|
||||
- Captures vary in completeness (HTML may be saved without CSS/JS)
|
||||
- Some content is excluded by domain owner request (DMCA, etc.)
|
||||
- Coverage of "deep links" (URLs with query strings) is uneven
|
||||
- Time resolution is per-capture, not continuous — gaps are common
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_wayback.py`
|
||||
|
||||
```bash
|
||||
# All captures of a specific URL
|
||||
python3 SKILL_DIR/scripts/fetch_wayback.py --url "https://example.com/page" \
|
||||
--out data/wb.csv
|
||||
|
||||
# All captures of a host
|
||||
python3 SKILL_DIR/scripts/fetch_wayback.py --url "example.com" \
|
||||
--match host --out data/wb.csv
|
||||
|
||||
# All captures of a domain + subdomains
|
||||
python3 SKILL_DIR/scripts/fetch_wayback.py --url "example.com" \
|
||||
--match domain --out data/wb.csv
|
||||
|
||||
# Only unique-content captures within a date window
|
||||
python3 SKILL_DIR/scripts/fetch_wayback.py --url "example.com" \
|
||||
--match host --collapse digest \
|
||||
--from-date 2020-01-01 --to-date 2023-12-31 \
|
||||
--out data/wb.csv
|
||||
```
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Internet Archive captures are made under fair-use research provisions
|
||||
- Replay URLs are stable references — citing them is encouraged
|
||||
- Internet Archive non-profit terms of use govern content
|
||||
- Some content is rights-restricted; replay may be blocked even if the
|
||||
CDX entry shows it as captured
|
||||
|
||||
## 9. References
|
||||
|
||||
- CDX server docs: https://github.com/internetarchive/wayback/blob/master/wayback-cdx-server/README.md
|
||||
- Wayback API: https://archive.org/help/wayback_api.php
|
||||
- Internet Archive: https://archive.org/
|
||||
@@ -0,0 +1,107 @@
|
||||
# Wikipedia + Wikidata
|
||||
|
||||
## 1. Summary
|
||||
|
||||
Wikipedia is the canonical narrative-bio source for notable people, places,
|
||||
and organizations. Wikidata is its structured-data counterpart: ~110M
|
||||
items, each with claims, dates, identifiers, and cross-references to
|
||||
external authorities (VIAF, ISNI, ORCID, GRID, etc.).
|
||||
|
||||
Together they're a high-precision entity-resolution layer — the bar for
|
||||
inclusion is real, but anything past that bar is well-cross-referenced.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- **Wikipedia OpenSearch:** `https://en.wikipedia.org/w/api.php?action=opensearch`
|
||||
- **Wikipedia REST summary:** `https://en.wikipedia.org/api/rest_v1/page/summary/<title>`
|
||||
- **Wikidata Action API:** `https://www.wikidata.org/w/api.php?action=wbgetentities`
|
||||
- **Wikidata SPARQL:** `https://query.wikidata.org/sparql` (more powerful but aggressively rate-limited)
|
||||
- **Auth:** None, but **a meaningful User-Agent is required**
|
||||
|
||||
Set `HERMES_OSINT_UA` to something identifying (e.g. `your-app/1.0 (you@example.com)`).
|
||||
Wikimedia returns HTTP 429 to generic UAs.
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields emitted by `fetch_wikipedia.py`:
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `source` | str | `wikipedia` or `wikipedia+wikidata` |
|
||||
| `label` | str | Wikipedia article title |
|
||||
| `description` | str | Short Wikidata description |
|
||||
| `qid` | str | Wikidata QID (e.g. Q2283 for Microsoft) |
|
||||
| `wikipedia_title`, `wikipedia_url` | str | Article identifier + URL |
|
||||
| `wikidata_url` | str | Wikidata entity URL |
|
||||
| `instance_of` | str | What kind of thing it is (P31) |
|
||||
| `country` | str | Country (P17 for orgs/places, P27 for people) |
|
||||
| `occupation` | str | P106 |
|
||||
| `employer` | str | P108 |
|
||||
| `date_of_birth` | str | P569, YYYY-MM-DD |
|
||||
| `place_of_birth` | str | P19 |
|
||||
| `summary` | str | Wikipedia REST extract (~1000 chars) |
|
||||
|
||||
The fetch script uses Wikidata's Action API (NOT SPARQL) for structured
|
||||
facts — far more lenient on rate limits.
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- Wikipedia EN: ~7M articles
|
||||
- Wikidata: ~110M items, ~1.5B statements
|
||||
- Updated continuously; abuse filters and bots run constantly
|
||||
- High notability bar — most private individuals are not in Wikipedia
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
- **All sources** ↔ `label` (entity identity resolution)
|
||||
- **SEC EDGAR** ↔ `label` (public companies)
|
||||
- **CourtListener** ↔ `label` (parties to notable litigation)
|
||||
- **Wikidata external identifiers** (not currently in this fetcher's output)
|
||||
link to VIAF, ISNI, ORCID, GRID, GitHub, Twitter, IMDb, ...
|
||||
|
||||
Join key: Wikidata QID is canonical. Wikipedia titles are stable for
|
||||
most articles but can be renamed.
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
- Notability filter — only notable entities (criteria vary by topic)
|
||||
- Recency lag — current events take days to weeks to be reflected
|
||||
- POV / vandalism — moderated, but edits between sweeps can be bad
|
||||
- Living-persons biographies have stricter sourcing requirements
|
||||
- Wikidata claims have qualifiers and references — the fetch script
|
||||
doesn't currently export them
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_wikipedia.py`
|
||||
|
||||
```bash
|
||||
# Look up a notable entity
|
||||
python3 SKILL_DIR/scripts/fetch_wikipedia.py --query "Microsoft" --out data/wp.csv
|
||||
|
||||
# A specific person
|
||||
python3 SKILL_DIR/scripts/fetch_wikipedia.py --query "Bill Gates" --out data/wp_bg.csv
|
||||
|
||||
# Skip the Wikidata enrichment for speed
|
||||
python3 SKILL_DIR/scripts/fetch_wikipedia.py --query "Microsoft" --no-wikidata \
|
||||
--limit 5 --out data/wp.csv
|
||||
```
|
||||
|
||||
The OpenSearch is fuzzy — `--limit 5` returns the top 5 Wikipedia article
|
||||
matches. Each is enriched with the QID + structured facts unless
|
||||
`--no-wikidata` is passed.
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Wikipedia text: CC-BY-SA-3.0 / GFDL
|
||||
- Wikidata claims: CC0 (public domain)
|
||||
- API ToS: respect rate limits, identify your agent
|
||||
- Commercial use allowed with attribution
|
||||
|
||||
## 9. References
|
||||
|
||||
- Wikipedia OpenSearch: https://www.mediawiki.org/wiki/API:Opensearch
|
||||
- Wikipedia REST: https://en.wikipedia.org/api/rest_v1/
|
||||
- Wikidata Action API: https://www.wikidata.org/wiki/Wikidata:Data_access
|
||||
- Wikidata SPARQL: https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service
|
||||
- User-Agent policy: https://meta.wikimedia.org/wiki/User-Agent_policy
|
||||
@@ -0,0 +1,82 @@
|
||||
"""Tiny stdlib HTTP helper used by fetch_*.py scripts.
|
||||
|
||||
Provides polite retry + JSON convenience + User-Agent enforcement.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
DEFAULT_UA = (
|
||||
"hermes-osint-investigation/0.2 "
|
||||
"(+https://github.com/NousResearch/hermes-agent; "
|
||||
"set HERMES_OSINT_UA env var to identify yourself per "
|
||||
"Wikimedia / SEC fair-use guidance)"
|
||||
)
|
||||
|
||||
|
||||
def get(
|
||||
url: str,
|
||||
*,
|
||||
params: dict | None = None,
|
||||
headers: dict | None = None,
|
||||
user_agent: str | None = None,
|
||||
max_retries: int = 3,
|
||||
backoff: float = 1.5,
|
||||
timeout: float = 30.0,
|
||||
) -> bytes:
|
||||
"""GET with retry on 5xx and Retry-After honoring.
|
||||
|
||||
429 (rate-limit) is raised IMMEDIATELY with a clear message — retrying
|
||||
when the upstream says "you're over quota" just wastes time. The caller
|
||||
should slow down or supply real credentials.
|
||||
"""
|
||||
if params:
|
||||
sep = "&" if "?" in url else "?"
|
||||
url = f"{url}{sep}{urllib.parse.urlencode(params)}"
|
||||
h = {"User-Agent": user_agent or os.environ.get("HERMES_OSINT_UA", DEFAULT_UA)}
|
||||
if headers:
|
||||
h.update(headers)
|
||||
|
||||
last_err: Exception | None = None
|
||||
for attempt in range(max_retries + 1):
|
||||
req = urllib.request.Request(url, headers=h)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 429:
|
||||
# Surface immediately. Read the body so the caller sees the
|
||||
# provider's actual message ("OVER_RATE_LIMIT" etc.).
|
||||
try:
|
||||
body = e.read(2048).decode("utf-8", errors="replace")
|
||||
except Exception: # noqa: BLE001
|
||||
body = ""
|
||||
raise RuntimeError(
|
||||
f"HTTP 429 rate-limited by {urllib.parse.urlsplit(url).netloc}. "
|
||||
f"Slow down or supply a real API key. Body: {body[:300]}"
|
||||
) from e
|
||||
if e.code in {500, 502, 503, 504} and attempt < max_retries:
|
||||
retry_after = e.headers.get("Retry-After") if e.headers else None
|
||||
wait = float(retry_after) if (retry_after and retry_after.isdigit()) else backoff ** (attempt + 1)
|
||||
time.sleep(wait)
|
||||
last_err = e
|
||||
continue
|
||||
raise
|
||||
except urllib.error.URLError as e:
|
||||
if attempt < max_retries:
|
||||
time.sleep(backoff ** (attempt + 1))
|
||||
last_err = e
|
||||
continue
|
||||
raise
|
||||
if last_err:
|
||||
raise last_err
|
||||
raise RuntimeError("unreachable")
|
||||
|
||||
|
||||
def get_json(url: str, **kwargs) -> dict | list:
|
||||
return json.loads(get(url, **kwargs).decode("utf-8"))
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Shared entity-name normalization helpers (stdlib-only).
|
||||
|
||||
Used by entity_resolution.py and timing_analysis.py.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
# Legal suffixes / corporate boilerplate to strip during normalization.
|
||||
_SUFFIX_TOKENS = {
|
||||
"INC", "INCORPORATED", "LLC", "LLP", "LP", "LTD", "LIMITED",
|
||||
"CORP", "CORPORATION", "CO", "COMPANY",
|
||||
"GROUP", "GRP", "HOLDINGS", "HOLDING",
|
||||
"PARTNERS", "ASSOCIATES",
|
||||
"INTERNATIONAL", "INTL",
|
||||
"ENTERPRISES", "ENTERPRISE",
|
||||
"SERVICES", "SERVICE", "SVCS",
|
||||
"SOLUTIONS", "MANAGEMENT", "MGMT", "CONSULTING",
|
||||
"TECHNOLOGY", "TECHNOLOGIES", "TECH",
|
||||
"INDUSTRIES", "INDUSTRY",
|
||||
"AMERICA", "AMERICAN",
|
||||
"USA", "US",
|
||||
"PLLC", "PC",
|
||||
"TRUST", "FOUNDATION",
|
||||
}
|
||||
|
||||
_PUNCT_RE = re.compile(r"[^\w\s]")
|
||||
_WS_RE = re.compile(r"\s+")
|
||||
|
||||
|
||||
def normalize_name(name: str | None) -> str:
|
||||
"""Standard normalization: uppercase, strip suffixes, drop punctuation."""
|
||||
if not name:
|
||||
return ""
|
||||
s = _PUNCT_RE.sub(" ", name.upper())
|
||||
s = _WS_RE.sub(" ", s).strip()
|
||||
tokens = [t for t in s.split() if t and t not in _SUFFIX_TOKENS]
|
||||
return " ".join(tokens)
|
||||
|
||||
|
||||
def normalize_aggressive(name: str | None) -> str:
|
||||
"""Aggressive normalization: sorted unique tokens (word-bag)."""
|
||||
base = normalize_name(name)
|
||||
if not base:
|
||||
return ""
|
||||
return " ".join(sorted(set(base.split())))
|
||||
|
||||
|
||||
def name_tokens(name: str | None, min_len: int = 4) -> set[str]:
|
||||
"""Token set used for overlap matching."""
|
||||
base = normalize_name(name)
|
||||
if not base:
|
||||
return set()
|
||||
return {t for t in base.split() if len(t) >= min_len}
|
||||
|
||||
|
||||
def token_overlap_ratio(left: str | None, right: str | None) -> tuple[float, int]:
|
||||
"""Return (jaccard-like ratio, shared token count) over min-len tokens."""
|
||||
a = name_tokens(left)
|
||||
b = name_tokens(right)
|
||||
if not a or not b:
|
||||
return 0.0, 0
|
||||
shared = a & b
|
||||
if not shared:
|
||||
return 0.0, 0
|
||||
union = a | b
|
||||
return len(shared) / len(union), len(shared)
|
||||
@@ -0,0 +1,221 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build a structured findings.json with evidence chains (stdlib-only).
|
||||
|
||||
Aggregates cross_links.csv (entity_resolution output) and an optional
|
||||
timing.json (timing_analysis output) into a single evidence-chain document.
|
||||
|
||||
Output structure:
|
||||
{
|
||||
"metadata": {...},
|
||||
"findings": [
|
||||
{
|
||||
"id": "F0001",
|
||||
"title": "...",
|
||||
"severity": "HIGH|MEDIUM|LOW",
|
||||
"confidence": "high|medium|low",
|
||||
"summary": "...",
|
||||
"evidence": [
|
||||
{"source": "cross_links.csv", "row": 12, "fields": {...}},
|
||||
...
|
||||
],
|
||||
"sources": ["cross_links.csv", "timing.json"]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Every finding traces to specific source rows. No naked claims.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
CONFIDENCE_ORDER = {"high": 0, "medium": 1, "low": 2}
|
||||
SEVERITY_ORDER = {"HIGH": 0, "MEDIUM": 1, "LOW": 2}
|
||||
|
||||
|
||||
def _read_cross_links(path: str) -> list[dict[str, str]]:
|
||||
with open(path, newline="", encoding="utf-8") as fh:
|
||||
return list(csv.DictReader(fh))
|
||||
|
||||
|
||||
def build_findings(
|
||||
cross_links_path: str,
|
||||
timing_path: str | None = None,
|
||||
out_path: str = "findings.json",
|
||||
bundled_threshold: int = 3,
|
||||
) -> dict:
|
||||
findings: list[dict] = []
|
||||
next_id = 1
|
||||
|
||||
# 1. Match-based findings, grouped by (left_normalized, right_normalized).
|
||||
matches = _read_cross_links(cross_links_path)
|
||||
grouped: dict[tuple[str, str], list[dict[str, str]]] = defaultdict(list)
|
||||
for i, row in enumerate(matches):
|
||||
row["__row__"] = str(i)
|
||||
grouped[(row.get("left_normalized", ""), row.get("right_normalized", ""))].append(row)
|
||||
|
||||
for (left_norm, right_norm), rows in grouped.items():
|
||||
if not left_norm or not right_norm:
|
||||
continue
|
||||
# Use the highest-confidence match for the finding's overall confidence.
|
||||
best = min(rows, key=lambda r: CONFIDENCE_ORDER.get(r.get("confidence", "low"), 2))
|
||||
finding_id = f"F{next_id:04d}"
|
||||
next_id += 1
|
||||
evidence = [
|
||||
{
|
||||
"source": "cross_links.csv",
|
||||
"row": int(r["__row__"]),
|
||||
"fields": {
|
||||
"match_type": r.get("match_type", ""),
|
||||
"confidence": r.get("confidence", ""),
|
||||
"left_name": r.get("left_name", ""),
|
||||
"right_name": r.get("right_name", ""),
|
||||
"overlap_ratio": r.get("overlap_ratio", ""),
|
||||
"shared_tokens": r.get("shared_tokens", ""),
|
||||
},
|
||||
}
|
||||
for r in rows
|
||||
]
|
||||
findings.append(
|
||||
{
|
||||
"id": finding_id,
|
||||
"title": f"Entity match: {best.get('left_name', '')} ↔ {best.get('right_name', '')}",
|
||||
"severity": "MEDIUM" if best.get("confidence") == "high" else "LOW",
|
||||
"confidence": best.get("confidence", "low"),
|
||||
"summary": (
|
||||
f"{len(rows)} cross-link record(s) tie "
|
||||
f"'{best.get('left_name', '')}' to "
|
||||
f"'{best.get('right_name', '')}' "
|
||||
f"(best tier: {best.get('match_type', '')})."
|
||||
),
|
||||
"evidence": evidence,
|
||||
"sources": ["cross_links.csv"],
|
||||
}
|
||||
)
|
||||
|
||||
# 2. Bundled-donations findings (if cross_links carries donor↔candidate pattern).
|
||||
# Heuristic: many distinct left names sharing the same right name.
|
||||
by_right: dict[str, set[str]] = defaultdict(set)
|
||||
by_right_rows: dict[str, list[dict[str, str]]] = defaultdict(list)
|
||||
for r in matches:
|
||||
right = r.get("right_normalized", "")
|
||||
left_raw = r.get("left_name", "").strip()
|
||||
if right and left_raw:
|
||||
by_right[right].add(left_raw)
|
||||
by_right_rows[right].append(r)
|
||||
for right_norm, lefts in by_right.items():
|
||||
if len(lefts) < bundled_threshold:
|
||||
continue
|
||||
rows = by_right_rows[right_norm]
|
||||
right_raw = rows[0].get("right_name", "")
|
||||
findings.append(
|
||||
{
|
||||
"id": f"F{next_id:04d}",
|
||||
"title": f"Bundled cross-links: {len(lefts)} distinct left entities ↔ '{right_raw}'",
|
||||
"severity": "HIGH",
|
||||
"confidence": "medium",
|
||||
"summary": (
|
||||
f"{len(lefts)} distinct left-side entities link to "
|
||||
f"'{right_raw}'. Pattern suggests coordinated relationship "
|
||||
f"(e.g. bundled donations, multi-vendor employer)."
|
||||
),
|
||||
"evidence": [
|
||||
{
|
||||
"source": "cross_links.csv",
|
||||
"row": int(r.get("__row__", "0")),
|
||||
"fields": {
|
||||
"left_name": r.get("left_name", ""),
|
||||
"match_type": r.get("match_type", ""),
|
||||
},
|
||||
}
|
||||
for r in rows
|
||||
],
|
||||
"sources": ["cross_links.csv"],
|
||||
}
|
||||
)
|
||||
next_id += 1
|
||||
|
||||
# 3. Timing-based findings.
|
||||
if timing_path and Path(timing_path).exists():
|
||||
timing = json.loads(Path(timing_path).read_text(encoding="utf-8"))
|
||||
for r in timing.get("results", []):
|
||||
if not r.get("significant"):
|
||||
continue
|
||||
findings.append(
|
||||
{
|
||||
"id": f"F{next_id:04d}",
|
||||
"title": (
|
||||
f"Donation timing significantly clusters near awards: "
|
||||
f"{r['donor']} ↔ {r['recipient']}"
|
||||
),
|
||||
"severity": "HIGH" if r["p_value"] < 0.01 else "MEDIUM",
|
||||
"confidence": "medium",
|
||||
"summary": (
|
||||
f"Mean nearest-award distance {r['observed_mean_days']} days "
|
||||
f"(null {r['null_mean_days']} days). p={r['p_value']}, "
|
||||
f"effect size {r['effect_size_sd']} SD. "
|
||||
f"{r['n_donations']} donations, {r['n_award_dates']} awards."
|
||||
),
|
||||
"evidence": [
|
||||
{
|
||||
"source": "timing.json",
|
||||
"row": None,
|
||||
"fields": r,
|
||||
}
|
||||
],
|
||||
"sources": ["timing.json"],
|
||||
}
|
||||
)
|
||||
next_id += 1
|
||||
|
||||
# Sort: severity → confidence → id.
|
||||
findings.sort(
|
||||
key=lambda f: (
|
||||
SEVERITY_ORDER.get(f["severity"], 3),
|
||||
CONFIDENCE_ORDER.get(f["confidence"], 3),
|
||||
f["id"],
|
||||
)
|
||||
)
|
||||
|
||||
payload = {
|
||||
"metadata": {
|
||||
"n_findings": len(findings),
|
||||
"cross_links_path": cross_links_path,
|
||||
"timing_path": timing_path,
|
||||
"bundled_threshold": bundled_threshold,
|
||||
},
|
||||
"findings": findings,
|
||||
}
|
||||
Path(out_path).write_text(json.dumps(payload, indent=2), encoding="utf-8")
|
||||
return payload
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--cross-links", required=True)
|
||||
p.add_argument("--timing", help="Optional timing.json from timing_analysis.py")
|
||||
p.add_argument("--out", default="findings.json")
|
||||
p.add_argument(
|
||||
"--bundled-threshold",
|
||||
type=int,
|
||||
default=3,
|
||||
help="Minimum distinct left entities to flag as bundled (default 3)",
|
||||
)
|
||||
a = p.parse_args()
|
||||
|
||||
payload = build_findings(
|
||||
cross_links_path=a.cross_links,
|
||||
timing_path=a.timing,
|
||||
out_path=a.out,
|
||||
bundled_threshold=a.bundled_threshold,
|
||||
)
|
||||
print(f"Wrote {payload['metadata']['n_findings']} findings to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,228 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Cross-source entity resolution (stdlib-only).
|
||||
|
||||
Given two CSV files with name columns, find candidate matches using three
|
||||
tiers of normalization:
|
||||
|
||||
1. exact — normalized strings equal
|
||||
2. fuzzy — sorted-token (word-bag) match
|
||||
3. token_overlap — >=60% Jaccard overlap on >=4-char tokens, >=2 shared
|
||||
|
||||
Adapted from ShinMegamiBoson/OpenPlanter (MIT) but generalized: no Boston-
|
||||
specific record types, no contribution-code filters, no fixed schemas.
|
||||
|
||||
Output CSV columns:
|
||||
match_type, confidence, left_name, right_name,
|
||||
left_normalized, right_normalized, left_row, right_row,
|
||||
overlap_ratio, shared_tokens
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Allow running directly or as a module.
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _normalize import ( # noqa: E402
|
||||
normalize_name,
|
||||
normalize_aggressive,
|
||||
token_overlap_ratio,
|
||||
)
|
||||
|
||||
CONFIDENCE = {
|
||||
"exact": "high",
|
||||
"fuzzy": "medium",
|
||||
"token_overlap": "low",
|
||||
}
|
||||
|
||||
|
||||
def _read_csv(path: str, name_col: str) -> list[dict[str, str]]:
|
||||
rows = []
|
||||
with open(path, newline="", encoding="utf-8") as fh:
|
||||
reader = csv.DictReader(fh)
|
||||
if name_col not in (reader.fieldnames or []):
|
||||
raise SystemExit(
|
||||
f"Column {name_col!r} not in {path}. "
|
||||
f"Available: {reader.fieldnames}"
|
||||
)
|
||||
for i, row in enumerate(reader):
|
||||
row["__row__"] = str(i)
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
|
||||
def _build_index(rows: list[dict[str, str]], name_col: str):
|
||||
"""Index by exact-normalized and aggressive (sorted-token) form."""
|
||||
exact: dict[str, list[dict[str, str]]] = {}
|
||||
aggressive: dict[str, list[dict[str, str]]] = {}
|
||||
for row in rows:
|
||||
raw = row.get(name_col, "")
|
||||
n = normalize_name(raw)
|
||||
if n:
|
||||
exact.setdefault(n, []).append(row)
|
||||
a = normalize_aggressive(raw)
|
||||
if a:
|
||||
aggressive.setdefault(a, []).append(row)
|
||||
return exact, aggressive
|
||||
|
||||
|
||||
def _emit(
|
||||
out_rows: list[dict[str, str]],
|
||||
seen: set[tuple],
|
||||
match_type: str,
|
||||
left_row: dict[str, str],
|
||||
right_row: dict[str, str],
|
||||
left_col: str,
|
||||
right_col: str,
|
||||
ratio: float = 0.0,
|
||||
shared: int = 0,
|
||||
):
|
||||
left_raw = left_row.get(left_col, "")
|
||||
right_raw = right_row.get(right_col, "")
|
||||
key = (
|
||||
left_row["__row__"],
|
||||
right_row["__row__"],
|
||||
match_type,
|
||||
)
|
||||
if key in seen:
|
||||
return
|
||||
seen.add(key)
|
||||
out_rows.append(
|
||||
{
|
||||
"match_type": match_type,
|
||||
"confidence": CONFIDENCE[match_type],
|
||||
"left_name": left_raw,
|
||||
"right_name": right_raw,
|
||||
"left_normalized": normalize_name(left_raw),
|
||||
"right_normalized": normalize_name(right_raw),
|
||||
"left_row": left_row["__row__"],
|
||||
"right_row": right_row["__row__"],
|
||||
"overlap_ratio": f"{ratio:.3f}" if ratio else "",
|
||||
"shared_tokens": str(shared) if shared else "",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def resolve(
|
||||
left_path: str,
|
||||
left_col: str,
|
||||
right_path: str,
|
||||
right_col: str,
|
||||
out_path: str,
|
||||
overlap_threshold: float = 0.60,
|
||||
min_shared: int = 2,
|
||||
skip_overlap: bool = False,
|
||||
) -> int:
|
||||
left_rows = _read_csv(left_path, left_col)
|
||||
right_rows = _read_csv(right_path, right_col)
|
||||
|
||||
right_exact, right_aggressive = _build_index(right_rows, right_col)
|
||||
|
||||
out_rows: list[dict[str, str]] = []
|
||||
seen: set[tuple] = set()
|
||||
|
||||
# Pass 1+2: exact / fuzzy via index lookup.
|
||||
for lrow in left_rows:
|
||||
raw = lrow.get(left_col, "")
|
||||
n = normalize_name(raw)
|
||||
if not n:
|
||||
continue
|
||||
for rrow in right_exact.get(n, []):
|
||||
_emit(out_rows, seen, "exact", lrow, rrow, left_col, right_col)
|
||||
a = normalize_aggressive(raw)
|
||||
if a:
|
||||
for rrow in right_aggressive.get(a, []):
|
||||
_emit(out_rows, seen, "fuzzy", lrow, rrow, left_col, right_col)
|
||||
|
||||
if not skip_overlap:
|
||||
# Pass 3: token overlap (O(N*M) — expensive; allow opt-out).
|
||||
for lrow in left_rows:
|
||||
l_raw = lrow.get(left_col, "")
|
||||
if not normalize_name(l_raw):
|
||||
continue
|
||||
for rrow in right_rows:
|
||||
ratio, shared = token_overlap_ratio(
|
||||
l_raw, rrow.get(right_col, "")
|
||||
)
|
||||
if ratio >= overlap_threshold and shared >= min_shared:
|
||||
_emit(
|
||||
out_rows,
|
||||
seen,
|
||||
"token_overlap",
|
||||
lrow,
|
||||
rrow,
|
||||
left_col,
|
||||
right_col,
|
||||
ratio=ratio,
|
||||
shared=shared,
|
||||
)
|
||||
|
||||
fieldnames = [
|
||||
"match_type",
|
||||
"confidence",
|
||||
"left_name",
|
||||
"right_name",
|
||||
"left_normalized",
|
||||
"right_normalized",
|
||||
"left_row",
|
||||
"right_row",
|
||||
"overlap_ratio",
|
||||
"shared_tokens",
|
||||
]
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
writer = csv.DictWriter(fh, fieldnames=fieldnames)
|
||||
writer.writeheader()
|
||||
writer.writerows(out_rows)
|
||||
return len(out_rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--left", required=True, help="Left CSV path")
|
||||
p.add_argument(
|
||||
"--left-name-col", required=True, help="Name column in left CSV"
|
||||
)
|
||||
p.add_argument("--right", required=True, help="Right CSV path")
|
||||
p.add_argument(
|
||||
"--right-name-col",
|
||||
required=True,
|
||||
help="Name column in right CSV",
|
||||
)
|
||||
p.add_argument("--out", required=True, help="Output CSV path")
|
||||
p.add_argument(
|
||||
"--overlap-threshold",
|
||||
type=float,
|
||||
default=0.60,
|
||||
help="Jaccard overlap threshold for token_overlap tier (default 0.60)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--min-shared",
|
||||
type=int,
|
||||
default=2,
|
||||
help="Minimum shared tokens for token_overlap tier (default 2)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--skip-overlap",
|
||||
action="store_true",
|
||||
help="Skip the O(N*M) token_overlap pass (much faster on large CSVs)",
|
||||
)
|
||||
args = p.parse_args()
|
||||
|
||||
count = resolve(
|
||||
left_path=args.left,
|
||||
left_col=args.left_name_col,
|
||||
right_path=args.right,
|
||||
right_col=args.right_name_col,
|
||||
out_path=args.out,
|
||||
overlap_threshold=args.overlap_threshold,
|
||||
min_shared=args.min_shared,
|
||||
skip_overlap=args.skip_overlap,
|
||||
)
|
||||
print(f"Wrote {count} match rows to {args.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,149 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Search court records via CourtListener (Free Law Project).
|
||||
|
||||
Covers ~10M federal and state court opinions, plus PACER docket data
|
||||
where available. Public REST API v4 supports anonymous read access for
|
||||
search; some endpoints require a token (free at courtlistener.com).
|
||||
|
||||
Set COURTLISTENER_TOKEN to authenticate (raises rate limits).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import os
|
||||
import sys
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get_json # noqa: E402
|
||||
|
||||
BASE = "https://www.courtlistener.com/api/rest/v4/search/"
|
||||
|
||||
COLUMNS = [
|
||||
"case_name",
|
||||
"court",
|
||||
"court_id",
|
||||
"date_filed",
|
||||
"docket_number",
|
||||
"judge",
|
||||
"citation",
|
||||
"result_type",
|
||||
"snippet",
|
||||
"absolute_url",
|
||||
]
|
||||
|
||||
SEARCH_TYPES = {
|
||||
"opinions": "o", # Court opinions
|
||||
"dockets": "r", # PACER dockets (may require auth depending on coverage)
|
||||
"oral": "oa", # Oral arguments
|
||||
"people": "p", # Judges / people
|
||||
"recap": "r", # Same as dockets in v4
|
||||
}
|
||||
|
||||
|
||||
def fetch(
|
||||
query: str,
|
||||
search_type: str,
|
||||
court: str | None,
|
||||
date_from: str | None,
|
||||
date_to: str | None,
|
||||
token: str | None,
|
||||
limit: int,
|
||||
out_path: str,
|
||||
) -> int:
|
||||
type_code = SEARCH_TYPES.get(search_type, search_type)
|
||||
params = {
|
||||
"q": query,
|
||||
"type": type_code,
|
||||
}
|
||||
if court:
|
||||
params["court"] = court
|
||||
if date_from:
|
||||
params["filed_after"] = date_from
|
||||
if date_to:
|
||||
params["filed_before"] = date_to
|
||||
headers = {"Authorization": f"Token {token}"} if token else None
|
||||
|
||||
rows: list[dict[str, str]] = []
|
||||
next_url: str | None = f"{BASE}?{urllib.parse.urlencode(params)}"
|
||||
while next_url and len(rows) < limit:
|
||||
try:
|
||||
payload = get_json(next_url, headers=headers)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"CourtListener error: {e}", file=sys.stderr)
|
||||
break
|
||||
if not isinstance(payload, dict):
|
||||
break
|
||||
results = payload.get("results", [])
|
||||
for r in results:
|
||||
if len(rows) >= limit:
|
||||
break
|
||||
rows.append(
|
||||
{
|
||||
"case_name": r.get("caseName", "") or r.get("case_name", "") or "",
|
||||
"court": r.get("court", "") or "",
|
||||
"court_id": r.get("court_id", "") or "",
|
||||
"date_filed": (r.get("dateFiled", "") or r.get("date_filed", "") or "")[:10],
|
||||
"docket_number": r.get("docketNumber", "") or r.get("docket_number", "") or "",
|
||||
"judge": r.get("judge", "") or "",
|
||||
"citation": "; ".join(r.get("citation", []) or []) if isinstance(r.get("citation"), list) else (r.get("citation") or ""),
|
||||
"result_type": search_type,
|
||||
"snippet": (r.get("snippet", "") or "").replace("\n", " ")[:500],
|
||||
"absolute_url": (
|
||||
f"https://www.courtlistener.com{r.get('absolute_url', '')}"
|
||||
if r.get("absolute_url", "").startswith("/")
|
||||
else r.get("absolute_url", "")
|
||||
),
|
||||
}
|
||||
)
|
||||
next_url = payload.get("next")
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
if not rows:
|
||||
print(
|
||||
f"CourtListener: 0 results for type={search_type!r} q={query!r}. "
|
||||
"Most private individuals don't appear in published court records "
|
||||
"unless they were party to a federal or state appellate case.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--query", required=True, help="Search query (party name, case name, keyword)")
|
||||
p.add_argument(
|
||||
"--type",
|
||||
default="opinions",
|
||||
choices=list(SEARCH_TYPES.keys()),
|
||||
help="Search type (default: opinions)",
|
||||
)
|
||||
p.add_argument("--court", help="Court ID filter (e.g. 'nysd' = SDNY, 'scotus' = Supreme Court)")
|
||||
p.add_argument("--date-from", help="Filed-after date YYYY-MM-DD")
|
||||
p.add_argument("--date-to", help="Filed-before date YYYY-MM-DD")
|
||||
p.add_argument("--token", default=os.environ.get("COURTLISTENER_TOKEN"))
|
||||
p.add_argument("--limit", type=int, default=100)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
n = fetch(
|
||||
query=a.query,
|
||||
search_type=a.type,
|
||||
court=a.court,
|
||||
date_from=a.date_from,
|
||||
date_to=a.date_to,
|
||||
token=a.token,
|
||||
limit=a.limit,
|
||||
out_path=a.out,
|
||||
)
|
||||
print(f"Wrote {n} CourtListener rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Search the GDELT 2.0 DOC API for news mentions.
|
||||
|
||||
GDELT monitors world news in 100+ languages and indexes the full text.
|
||||
Free, anonymous, ~15-minute update frequency. Covers ~2015→present.
|
||||
|
||||
Useful for surfacing news mentions of a person, company, or topic across
|
||||
international media — much wider net than Google News.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import sys
|
||||
import time
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get_json # noqa: E402
|
||||
|
||||
BASE = "https://api.gdeltproject.org/api/v2/doc/doc"
|
||||
|
||||
COLUMNS = [
|
||||
"title",
|
||||
"url",
|
||||
"seen_date",
|
||||
"domain",
|
||||
"language",
|
||||
"source_country",
|
||||
"tone",
|
||||
"social_image",
|
||||
]
|
||||
|
||||
|
||||
def fetch(
|
||||
query: str,
|
||||
mode: str,
|
||||
timespan: str | None,
|
||||
start_datetime: str | None,
|
||||
end_datetime: str | None,
|
||||
source_country: str | None,
|
||||
source_lang: str | None,
|
||||
limit: int,
|
||||
out_path: str,
|
||||
) -> int:
|
||||
params: dict[str, str] = {
|
||||
"query": query,
|
||||
"mode": mode,
|
||||
"format": "json",
|
||||
"maxrecords": str(min(limit, 250)),
|
||||
"sort": "datedesc",
|
||||
}
|
||||
if timespan:
|
||||
params["timespan"] = timespan
|
||||
if start_datetime:
|
||||
params["startdatetime"] = start_datetime.replace("-", "").replace(":", "").replace(" ", "")
|
||||
if end_datetime:
|
||||
params["enddatetime"] = end_datetime.replace("-", "").replace(":", "").replace(" ", "")
|
||||
if source_country:
|
||||
params["sourcecountry"] = source_country
|
||||
if source_lang:
|
||||
params["sourcelang"] = source_lang
|
||||
|
||||
url = f"{BASE}?{urllib.parse.urlencode(params)}"
|
||||
payload: dict | list = {}
|
||||
for attempt in range(3):
|
||||
try:
|
||||
payload = get_json(url)
|
||||
break
|
||||
except RuntimeError as e:
|
||||
# GDELT requires 1 request per 5 seconds; back off and retry.
|
||||
if "429" in str(e) and attempt < 2:
|
||||
print(
|
||||
f"GDELT throttle hit; sleeping 6s before retry "
|
||||
f"(attempt {attempt + 1}/3)",
|
||||
file=sys.stderr,
|
||||
)
|
||||
time.sleep(6)
|
||||
continue
|
||||
print(f"GDELT error: {e}", file=sys.stderr)
|
||||
payload = {}
|
||||
break
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"GDELT error: {e}", file=sys.stderr)
|
||||
payload = {}
|
||||
break
|
||||
|
||||
rows: list[dict[str, str]] = []
|
||||
if isinstance(payload, dict):
|
||||
articles = payload.get("articles", []) or []
|
||||
for a in articles[:limit]:
|
||||
seen = (a.get("seendate") or "")
|
||||
# GDELT format: 20260319T083000Z → 2026-03-19 08:30:00Z
|
||||
if len(seen) == 16 and "T" in seen:
|
||||
seen = f"{seen[0:4]}-{seen[4:6]}-{seen[6:8]} {seen[9:11]}:{seen[11:13]}:{seen[13:15]}Z"
|
||||
rows.append(
|
||||
{
|
||||
"title": (a.get("title") or "").replace("\n", " ").strip(),
|
||||
"url": a.get("url") or "",
|
||||
"seen_date": seen,
|
||||
"domain": a.get("domain") or "",
|
||||
"language": a.get("language") or "",
|
||||
"source_country": a.get("sourcecountry") or "",
|
||||
"tone": str(a.get("tone") or ""),
|
||||
"social_image": a.get("socialimage") or "",
|
||||
}
|
||||
)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
if not rows:
|
||||
print(
|
||||
f"GDELT: 0 articles for query={query!r}. "
|
||||
"GDELT indexes ~2015→present. Try widening the timespan or "
|
||||
"checking the query syntax (https://blog.gdeltproject.org/gdelt-doc-2-0-api-debuts/).",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--query", required=True, help='Search query (supports GDELT operators: quoted phrases, AND/OR/NOT, sourcecountry:, theme:)')
|
||||
p.add_argument(
|
||||
"--mode",
|
||||
default="ArtList",
|
||||
choices=["ArtList", "ImageCollage", "TimelineVol", "TimelineTone", "ToneChart"],
|
||||
help="GDELT mode (default ArtList for article list)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--timespan",
|
||||
help="Relative window: e.g. '1d', '1w', '1m', '3m', '1y' (overrides start/end)",
|
||||
)
|
||||
p.add_argument("--start", help="Absolute start YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS")
|
||||
p.add_argument("--end", help="Absolute end YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS")
|
||||
p.add_argument("--source-country", help="2-letter source country (e.g. US, UK)")
|
||||
p.add_argument("--source-lang", help="Source language (e.g. English, Spanish)")
|
||||
p.add_argument("--limit", type=int, default=100)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
n = fetch(
|
||||
query=a.query,
|
||||
mode=a.mode,
|
||||
timespan=a.timespan,
|
||||
start_datetime=a.start,
|
||||
end_datetime=a.end,
|
||||
source_country=a.source_country,
|
||||
source_lang=a.source_lang,
|
||||
limit=a.limit,
|
||||
out_path=a.out,
|
||||
)
|
||||
print(f"Wrote {n} GDELT article rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,234 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Search ICIJ Offshore Leaks via the bulk CSV database.
|
||||
|
||||
The old reconcile endpoint (https://offshoreleaks.icij.org/reconcile) returns
|
||||
404 — ICIJ has removed it. The remaining stable access path is the public
|
||||
bulk download:
|
||||
|
||||
https://offshoreleaks-data.icij.org/offshoreleaks/csv/full-oldb.LATEST.zip
|
||||
|
||||
~70 MB, ~6 CSVs inside (nodes-entities, nodes-officers, nodes-intermediaries,
|
||||
nodes-addresses, relationships, ...). We cache it under
|
||||
$HERMES_OSINT_CACHE/icij/ (default: ~/.cache/hermes-osint/icij/) and search
|
||||
locally so the agent doesn't re-download for every query.
|
||||
|
||||
Output CSV columns match the original `fetch_icij_offshore.py` contract.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import io
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
BULK_URL = "https://offshoreleaks-data.icij.org/offshoreleaks/csv/full-oldb.LATEST.zip"
|
||||
|
||||
COLUMNS = [
|
||||
"node_id",
|
||||
"name",
|
||||
"node_type",
|
||||
"country_codes",
|
||||
"countries",
|
||||
"jurisdiction",
|
||||
"incorporation_date",
|
||||
"inactivation_date",
|
||||
"source",
|
||||
"entity_url",
|
||||
"connections",
|
||||
]
|
||||
|
||||
|
||||
def _cache_dir() -> Path:
|
||||
base = os.environ.get("HERMES_OSINT_CACHE")
|
||||
if base:
|
||||
return Path(base) / "icij"
|
||||
return Path.home() / ".cache" / "hermes-osint" / "icij"
|
||||
|
||||
|
||||
def _download(dest: Path, force: bool = False) -> Path:
|
||||
"""Download (or reuse cached) ICIJ bulk ZIP."""
|
||||
dest.mkdir(parents=True, exist_ok=True)
|
||||
zip_path = dest / "full-oldb.zip"
|
||||
if zip_path.exists() and not force:
|
||||
# Re-check age: refetch if older than 30 days.
|
||||
age_days = (time.time() - zip_path.stat().st_mtime) / 86400
|
||||
if age_days < 30:
|
||||
return zip_path
|
||||
print(f"Downloading ICIJ bulk database (~70 MB) to {zip_path}", file=sys.stderr)
|
||||
req = urllib.request.Request(
|
||||
BULK_URL,
|
||||
headers={"User-Agent": "hermes-agent osint-investigation skill"},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=120) as resp: # noqa: S310
|
||||
tmp = zip_path.with_suffix(".zip.tmp")
|
||||
with open(tmp, "wb") as fh:
|
||||
while True:
|
||||
chunk = resp.read(1 << 16)
|
||||
if not chunk:
|
||||
break
|
||||
fh.write(chunk)
|
||||
tmp.replace(zip_path)
|
||||
return zip_path
|
||||
|
||||
|
||||
def _open_csv(zf: zipfile.ZipFile, name_pattern: str):
|
||||
"""Open the first CSV matching name_pattern (case-insensitive substring)."""
|
||||
for info in zf.infolist():
|
||||
if name_pattern.lower() in info.filename.lower() and info.filename.lower().endswith(".csv"):
|
||||
return zf.open(info), info.filename
|
||||
return None, None
|
||||
|
||||
|
||||
def _match(needle_norm: str, hay: str) -> bool:
|
||||
return needle_norm in (hay or "").upper()
|
||||
|
||||
|
||||
def _normalize_query(s: str) -> str:
|
||||
s = s.upper()
|
||||
s = re.sub(r"[^\w\s]", " ", s)
|
||||
s = re.sub(r"\s+", " ", s).strip()
|
||||
return s
|
||||
|
||||
|
||||
def fetch(
|
||||
entity: str | None,
|
||||
officer: str | None,
|
||||
jurisdiction: str | None,
|
||||
out_path: str,
|
||||
cache_dir: Path,
|
||||
force_refresh: bool = False,
|
||||
limit: int = 500,
|
||||
) -> int:
|
||||
zip_path = _download(cache_dir, force=force_refresh)
|
||||
rows: list[dict[str, str]] = []
|
||||
needles: list[tuple[str, str]] = [] # (kind, normalized needle)
|
||||
if entity:
|
||||
needles.append(("Entity", _normalize_query(entity)))
|
||||
if officer:
|
||||
needles.append(("Officer", _normalize_query(officer)))
|
||||
jur_norm = _normalize_query(jurisdiction) if jurisdiction else None
|
||||
|
||||
targets = [
|
||||
("Entity", "nodes-entities"),
|
||||
("Officer", "nodes-officers"),
|
||||
("Intermediary", "nodes-intermediaries"),
|
||||
]
|
||||
|
||||
with zipfile.ZipFile(zip_path) as zf:
|
||||
for node_type, csv_substring in targets:
|
||||
relevant_needles = [n for (k, n) in needles if k in {node_type, "Entity", "Officer"}] or []
|
||||
# Only scan a CSV if we have a needle that could plausibly match it,
|
||||
# or if we have ONLY a jurisdiction filter.
|
||||
applicable_needles = [n for (k, n) in needles if k == node_type]
|
||||
if needles and not applicable_needles and not jur_norm:
|
||||
continue
|
||||
stream, fname = _open_csv(zf, csv_substring)
|
||||
if not stream:
|
||||
continue
|
||||
with stream:
|
||||
text = io.TextIOWrapper(stream, encoding="utf-8", errors="replace")
|
||||
reader = csv.DictReader(text)
|
||||
for row in reader:
|
||||
name = (row.get("name") or "").strip()
|
||||
if not name:
|
||||
continue
|
||||
name_u = name.upper()
|
||||
matched = False
|
||||
for n in applicable_needles or relevant_needles:
|
||||
if _match(n, name_u):
|
||||
matched = True
|
||||
break
|
||||
if not needles:
|
||||
matched = True # jurisdiction-only sweep
|
||||
if not matched:
|
||||
continue
|
||||
jur = (row.get("jurisdiction_description") or row.get("country_codes") or "").strip()
|
||||
if jur_norm and jur_norm not in jur.upper() and jur_norm not in (row.get("countries") or "").upper():
|
||||
continue
|
||||
node_id = (row.get("node_id") or "").strip()
|
||||
rows.append(
|
||||
{
|
||||
"node_id": node_id,
|
||||
"name": name,
|
||||
"node_type": node_type,
|
||||
"country_codes": row.get("country_codes", "") or "",
|
||||
"countries": row.get("countries", "") or "",
|
||||
"jurisdiction": jur,
|
||||
"incorporation_date": row.get("incorporation_date", "") or "",
|
||||
"inactivation_date": row.get("inactivation_date", "") or "",
|
||||
"source": row.get("sourceID", "") or row.get("source", "") or "",
|
||||
"entity_url": (
|
||||
f"https://offshoreleaks.icij.org/nodes/{node_id}" if node_id else ""
|
||||
),
|
||||
"connections": "",
|
||||
}
|
||||
)
|
||||
if len(rows) >= limit:
|
||||
break
|
||||
if len(rows) >= limit:
|
||||
break
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
if not rows:
|
||||
bits = []
|
||||
if entity:
|
||||
bits.append(f"entity={entity!r}")
|
||||
if officer:
|
||||
bits.append(f"officer={officer!r}")
|
||||
if jurisdiction:
|
||||
bits.append(f"jurisdiction={jurisdiction!r}")
|
||||
print(
|
||||
f"ICIJ: 0 matches for {', '.join(bits)}. "
|
||||
"The bulk database covers offshore leaks (Panama, Paradise, Pandora, "
|
||||
"Bahamas, Offshore Leaks). Most private US individuals are NOT in it.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--entity", help="Search by entity name (substring, case-insensitive)")
|
||||
p.add_argument("--officer", help="Search by officer / individual name (substring, case-insensitive)")
|
||||
p.add_argument("--jurisdiction", help="Filter results by jurisdiction substring")
|
||||
p.add_argument("--limit", type=int, default=500)
|
||||
p.add_argument("--out", required=True)
|
||||
p.add_argument(
|
||||
"--cache-dir",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Override cache directory (default: $HERMES_OSINT_CACHE/icij or ~/.cache/hermes-osint/icij)",
|
||||
)
|
||||
p.add_argument(
|
||||
"--force-refresh",
|
||||
action="store_true",
|
||||
help="Re-download the bulk ZIP even if a recent cached copy exists.",
|
||||
)
|
||||
a = p.parse_args()
|
||||
if not (a.entity or a.officer or a.jurisdiction):
|
||||
p.error("must supply at least one of --entity / --officer / --jurisdiction")
|
||||
n = fetch(
|
||||
entity=a.entity,
|
||||
officer=a.officer,
|
||||
jurisdiction=a.jurisdiction,
|
||||
out_path=a.out,
|
||||
cache_dir=a.cache_dir or _cache_dir(),
|
||||
force_refresh=a.force_refresh,
|
||||
limit=a.limit,
|
||||
)
|
||||
print(f"Wrote {n} ICIJ Offshore Leaks rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,203 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Search NYC property records via ACRIS (Automated City Register Information System).
|
||||
|
||||
Uses the city's Socrata-backed open data API. No auth required for read access.
|
||||
|
||||
Datasets:
|
||||
bnx9-e6tj — Real Property Master (one row per recorded document)
|
||||
636b-3b5g — Real Property Parties (names — grantor, grantee, etc.)
|
||||
8h5j-fqxa — Real Property Legal (lot / property identifiers)
|
||||
uqqa-hym2 — Real Property References
|
||||
|
||||
The Parties dataset has the names. We search by name and optionally join to
|
||||
Master to get the doc type and date.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import sys
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get_json # noqa: E402
|
||||
|
||||
PARTIES_URL = "https://data.cityofnewyork.us/resource/636b-3b5g.json"
|
||||
MASTER_URL = "https://data.cityofnewyork.us/resource/bnx9-e6tj.json"
|
||||
|
||||
PARTY_TYPE = {
|
||||
"1": "grantor (seller / mortgagor / debtor)",
|
||||
"2": "grantee (buyer / mortgagee / creditor)",
|
||||
"3": "other party",
|
||||
}
|
||||
|
||||
BOROUGH = {
|
||||
"1": "Manhattan",
|
||||
"2": "Bronx",
|
||||
"3": "Brooklyn",
|
||||
"4": "Queens",
|
||||
"5": "Staten Island",
|
||||
}
|
||||
|
||||
COLUMNS = [
|
||||
"document_id",
|
||||
"name",
|
||||
"party_type",
|
||||
"party_role",
|
||||
"address_1",
|
||||
"address_2",
|
||||
"city",
|
||||
"state",
|
||||
"zip",
|
||||
"country",
|
||||
"doc_type",
|
||||
"doc_date",
|
||||
"recorded_date",
|
||||
"borough",
|
||||
"amount",
|
||||
"filing_url",
|
||||
]
|
||||
|
||||
|
||||
def _filing_url(document_id: str) -> str:
|
||||
if not document_id:
|
||||
return ""
|
||||
return (
|
||||
f"https://a836-acris.nyc.gov/DS/DocumentSearch/DocumentImageView?doc_id={document_id}"
|
||||
)
|
||||
|
||||
|
||||
def fetch(
|
||||
name: str | None,
|
||||
address: str | None,
|
||||
party_type: str | None,
|
||||
limit: int,
|
||||
out_path: str,
|
||||
enrich: bool = True,
|
||||
) -> int:
|
||||
if not (name or address):
|
||||
raise SystemExit("must supply --name or --address")
|
||||
|
||||
where_clauses: list[str] = []
|
||||
if name:
|
||||
safe = name.upper().replace("'", "''")
|
||||
where_clauses.append(f"upper(name) like '%{safe}%'")
|
||||
if address:
|
||||
safe_addr = address.upper().replace("'", "''")
|
||||
where_clauses.append(f"upper(address_1) like '%{safe_addr}%'")
|
||||
if party_type and party_type in {"1", "2", "3"}:
|
||||
where_clauses.append(f"party_type='{party_type}'")
|
||||
|
||||
params = {
|
||||
"$where": " AND ".join(where_clauses),
|
||||
"$limit": str(limit),
|
||||
}
|
||||
url = f"{PARTIES_URL}?{urllib.parse.urlencode(params)}"
|
||||
parties = get_json(url)
|
||||
if not isinstance(parties, list):
|
||||
raise SystemExit(f"Unexpected ACRIS response: {parties!r}")
|
||||
|
||||
# Enrich with master record (doc_type, dates, borough, amount).
|
||||
doc_ids: list[str] = sorted({
|
||||
d for d in (p.get("document_id") for p in parties) if d
|
||||
})
|
||||
masters: dict[str, dict] = {}
|
||||
if enrich and doc_ids:
|
||||
# Batch up to 100 doc_ids per request (Socrata IN-list is fine for this).
|
||||
for i in range(0, len(doc_ids), 100):
|
||||
chunk = doc_ids[i : i + 100]
|
||||
id_list = ",".join(f"'{d}'" for d in chunk)
|
||||
master_params = {
|
||||
"$where": f"document_id in ({id_list})",
|
||||
"$limit": "100",
|
||||
}
|
||||
url = f"{MASTER_URL}?{urllib.parse.urlencode(master_params)}"
|
||||
try:
|
||||
rows = get_json(url)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"ACRIS master lookup failed for chunk: {e}", file=sys.stderr)
|
||||
continue
|
||||
if isinstance(rows, list):
|
||||
for r in rows:
|
||||
did = r.get("document_id", "")
|
||||
if did:
|
||||
masters[did] = r
|
||||
|
||||
out_rows: list[dict[str, str]] = []
|
||||
for p in parties:
|
||||
did = p.get("document_id", "") or ""
|
||||
m = masters.get(did, {})
|
||||
out_rows.append(
|
||||
{
|
||||
"document_id": did,
|
||||
"name": p.get("name", "") or "",
|
||||
"party_type": p.get("party_type", "") or "",
|
||||
"party_role": PARTY_TYPE.get(p.get("party_type", ""), ""),
|
||||
"address_1": p.get("address_1", "") or "",
|
||||
"address_2": p.get("address_2", "") or "",
|
||||
"city": p.get("city", "") or "",
|
||||
"state": p.get("state", "") or "",
|
||||
"zip": p.get("zip", "") or "",
|
||||
"country": p.get("country", "") or "",
|
||||
"doc_type": m.get("doc_type", "") or "",
|
||||
"doc_date": (m.get("document_date", "") or "")[:10],
|
||||
"recorded_date": (m.get("recorded_datetime", "") or "")[:10],
|
||||
"borough": BOROUGH.get(m.get("recorded_borough", ""), m.get("recorded_borough", "")),
|
||||
"amount": m.get("document_amt", "") or "",
|
||||
"filing_url": _filing_url(did),
|
||||
}
|
||||
)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(out_rows)
|
||||
|
||||
if not out_rows:
|
||||
filters = []
|
||||
if name:
|
||||
filters.append(f"name={name!r}")
|
||||
if address:
|
||||
filters.append(f"address={address!r}")
|
||||
print(
|
||||
f"NYC ACRIS: 0 records for {', '.join(filters)}. "
|
||||
"ACRIS covers ONLY NYC (5 boroughs). For property records elsewhere, "
|
||||
"search the relevant county recorder directly.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(out_rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--name", help="Party name substring (case-insensitive)")
|
||||
p.add_argument("--address", help="Address line 1 substring")
|
||||
p.add_argument(
|
||||
"--party-type",
|
||||
choices=["1", "2", "3"],
|
||||
help="Filter party type: 1=grantor (seller/mortgagor), 2=grantee (buyer/mortgagee), 3=other",
|
||||
)
|
||||
p.add_argument("--limit", type=int, default=200)
|
||||
p.add_argument(
|
||||
"--no-enrich",
|
||||
action="store_true",
|
||||
help="Skip the master-document lookup that adds doc_type/date/amount",
|
||||
)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
n = fetch(
|
||||
name=a.name,
|
||||
address=a.address,
|
||||
party_type=a.party_type,
|
||||
limit=a.limit,
|
||||
out_path=a.out,
|
||||
enrich=not a.no_enrich,
|
||||
)
|
||||
print(f"Wrote {n} NYC ACRIS rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,175 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fetch OFAC SDN list (CSV format) and normalize.
|
||||
|
||||
Public endpoint: https://www.treasury.gov/ofac/downloads/sdn.csv
|
||||
Format reference: https://ofac.treasury.gov/specially-designated-nationals-and-blocked-persons-list-sdn-human-readable-lists
|
||||
|
||||
The SDN CSV uses a specific 12-column format with no header row:
|
||||
ent_num, sdn_name, sdn_type, program, title, call_sign, vess_type,
|
||||
tonnage, grt, vess_flag, vess_owner, remarks
|
||||
Address and AKA records live in separate files. We fetch all three and join.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import io
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get # noqa: E402
|
||||
|
||||
SDN_URL = "https://www.treasury.gov/ofac/downloads/sdn.csv"
|
||||
ADD_URL = "https://www.treasury.gov/ofac/downloads/add.csv"
|
||||
ALT_URL = "https://www.treasury.gov/ofac/downloads/alt.csv"
|
||||
|
||||
SDN_COLS = [
|
||||
"ent_num", "sdn_name", "sdn_type", "program", "title",
|
||||
"call_sign", "vess_type", "tonnage", "grt", "vess_flag",
|
||||
"vess_owner", "remarks",
|
||||
]
|
||||
ADD_COLS = [
|
||||
"ent_num", "add_num", "address", "city_state_zip", "country", "add_remarks",
|
||||
]
|
||||
ALT_COLS = [
|
||||
"ent_num", "alt_num", "alt_type", "alt_name", "alt_remarks",
|
||||
]
|
||||
|
||||
COLUMNS = [
|
||||
"entity_id",
|
||||
"name",
|
||||
"entity_type",
|
||||
"program_list",
|
||||
"title",
|
||||
"nationalities",
|
||||
"aka_list",
|
||||
"addresses",
|
||||
"dob",
|
||||
"pob",
|
||||
"remarks",
|
||||
"last_updated",
|
||||
]
|
||||
|
||||
_TYPE_MAP = {
|
||||
"individual": "individual",
|
||||
"entity": "entity",
|
||||
"vessel": "vessel",
|
||||
"aircraft": "aircraft",
|
||||
}
|
||||
|
||||
|
||||
def _read_csv(url: str, columns: list[str]) -> list[dict[str, str]]:
|
||||
body = get(url, timeout=60).decode("latin-1", errors="replace")
|
||||
reader = csv.reader(io.StringIO(body))
|
||||
out = []
|
||||
for row in reader:
|
||||
if not row:
|
||||
continue
|
||||
# Pad/truncate to expected width.
|
||||
row = row[: len(columns)] + [""] * (len(columns) - len(row))
|
||||
out.append(dict(zip(columns, row)))
|
||||
return out
|
||||
|
||||
|
||||
def _strip_quotes(s: str) -> str:
|
||||
s = s.strip()
|
||||
if s.startswith('"') and s.endswith('"'):
|
||||
s = s[1:-1]
|
||||
if s == "-0-":
|
||||
return ""
|
||||
return s
|
||||
|
||||
|
||||
def fetch(
|
||||
program: str | None,
|
||||
entity_type: str | None,
|
||||
out_path: str,
|
||||
) -> int:
|
||||
sdn = _read_csv(SDN_URL, SDN_COLS)
|
||||
addresses = _read_csv(ADD_URL, ADD_COLS)
|
||||
akas = _read_csv(ALT_URL, ALT_COLS)
|
||||
|
||||
addr_by_ent: dict[str, list[str]] = defaultdict(list)
|
||||
for a in addresses:
|
||||
ent = _strip_quotes(a["ent_num"])
|
||||
parts = [
|
||||
_strip_quotes(a[c])
|
||||
for c in ("address", "city_state_zip", "country")
|
||||
if _strip_quotes(a[c])
|
||||
]
|
||||
if parts:
|
||||
addr_by_ent[ent].append(", ".join(parts))
|
||||
|
||||
aka_by_ent: dict[str, list[str]] = defaultdict(list)
|
||||
for k in akas:
|
||||
ent = _strip_quotes(k["ent_num"])
|
||||
name = _strip_quotes(k["alt_name"])
|
||||
if name:
|
||||
aka_by_ent[ent].append(name)
|
||||
|
||||
rows: list[dict[str, str]] = []
|
||||
for r in sdn:
|
||||
ent_num = _strip_quotes(r["ent_num"])
|
||||
if not ent_num:
|
||||
continue
|
||||
sdn_type = _TYPE_MAP.get(_strip_quotes(r["sdn_type"]).lower(), _strip_quotes(r["sdn_type"]))
|
||||
if entity_type and sdn_type != entity_type:
|
||||
continue
|
||||
progs = _strip_quotes(r["program"])
|
||||
if program and program.upper() not in progs.upper().split(";"):
|
||||
continue
|
||||
remarks = _strip_quotes(r["remarks"])
|
||||
# DOB / POB are commonly embedded in remarks for individuals.
|
||||
dob = ""
|
||||
pob = ""
|
||||
if sdn_type == "individual" and remarks:
|
||||
for chunk in remarks.split(";"):
|
||||
ch = chunk.strip()
|
||||
if ch.upper().startswith("DOB"):
|
||||
dob = ch.split(maxsplit=1)[1] if " " in ch else ""
|
||||
elif ch.upper().startswith("POB"):
|
||||
pob = ch.split(maxsplit=1)[1] if " " in ch else ""
|
||||
rows.append(
|
||||
{
|
||||
"entity_id": ent_num,
|
||||
"name": _strip_quotes(r["sdn_name"]),
|
||||
"entity_type": sdn_type,
|
||||
"program_list": "; ".join(p.strip() for p in progs.split(";") if p.strip()),
|
||||
"title": _strip_quotes(r["title"]),
|
||||
"nationalities": "", # not in this CSV; available in XML format
|
||||
"aka_list": "; ".join(aka_by_ent.get(ent_num, [])),
|
||||
"addresses": "; ".join(addr_by_ent.get(ent_num, [])),
|
||||
"dob": dob,
|
||||
"pob": pob,
|
||||
"remarks": remarks,
|
||||
"last_updated": "",
|
||||
}
|
||||
)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument("--program", help="Filter to specific sanctions program (e.g. SDGT, IRAN)")
|
||||
p.add_argument(
|
||||
"--entity-type",
|
||||
choices=["individual", "entity", "vessel", "aircraft"],
|
||||
help="Filter to a specific entity type",
|
||||
)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
n = fetch(program=a.program, entity_type=a.entity_type, out_path=a.out)
|
||||
print(f"Wrote {n} OFAC SDN rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,191 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Search OpenCorporates company registry data.
|
||||
|
||||
OpenCorporates aggregates ~200M companies from 130+ jurisdictions. The
|
||||
public API requires an API token (free tier: 500 calls/month). Set
|
||||
OPENCORPORATES_API_TOKEN in env or pass --token.
|
||||
|
||||
Without a token, this script falls back to scraping the public HTML
|
||||
search page (limited fields, more brittle, no jurisdiction filter).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get, get_json # noqa: E402
|
||||
|
||||
API_URL = "https://api.opencorporates.com/v0.4/companies/search"
|
||||
HTML_URL = "https://opencorporates.com/companies"
|
||||
|
||||
COLUMNS = [
|
||||
"name",
|
||||
"company_number",
|
||||
"jurisdiction_code",
|
||||
"jurisdiction_name",
|
||||
"incorporation_date",
|
||||
"dissolution_date",
|
||||
"company_type",
|
||||
"status",
|
||||
"registered_address",
|
||||
"opencorporates_url",
|
||||
"officers_count",
|
||||
"source",
|
||||
]
|
||||
|
||||
|
||||
def _via_api(query: str, jurisdiction: str | None, token: str, limit: int) -> list[dict]:
|
||||
params = {
|
||||
"q": query,
|
||||
"api_token": token,
|
||||
"per_page": str(min(limit, 100)),
|
||||
}
|
||||
if jurisdiction:
|
||||
params["jurisdiction_code"] = jurisdiction
|
||||
url = f"{API_URL}?{urllib.parse.urlencode(params)}"
|
||||
payload = get_json(url)
|
||||
if not isinstance(payload, dict):
|
||||
return []
|
||||
results = payload.get("results", {}).get("companies", []) or []
|
||||
return [r.get("company", {}) for r in results if isinstance(r, dict)]
|
||||
|
||||
|
||||
def _via_html(query: str, limit: int) -> list[dict]:
|
||||
"""Best-effort HTML fallback when no API token is available."""
|
||||
params = {"q": query, "utf8": "✓"}
|
||||
url = f"{HTML_URL}?{urllib.parse.urlencode(params)}"
|
||||
body = get(url, user_agent="Mozilla/5.0 hermes-osint").decode("utf-8", errors="replace")
|
||||
# Each result is in <li class="company"> ... </li> with name, url, status
|
||||
pattern = re.compile(
|
||||
r'<li[^>]*class="[^"]*company[^"]*"[^>]*>.*?'
|
||||
r'<a[^>]+href="(?P<url>/companies/[^"]+)"[^>]*>(?P<name>[^<]+)</a>'
|
||||
r'(?:.*?<span[^>]*class="[^"]*jurisdiction[^"]*"[^>]*>(?P<jur>[^<]+)</span>)?'
|
||||
r"(?:.*?<dt[^>]*>(?:Company\s+Number|Number)</dt>\s*<dd[^>]*>(?P<num>[^<]+)</dd>)?",
|
||||
re.DOTALL | re.IGNORECASE,
|
||||
)
|
||||
out = []
|
||||
for m in pattern.finditer(body):
|
||||
if len(out) >= limit:
|
||||
break
|
||||
url_path = m.group("url").strip()
|
||||
out.append(
|
||||
{
|
||||
"name": (m.group("name") or "").strip(),
|
||||
"opencorporates_url": f"https://opencorporates.com{url_path}",
|
||||
"jurisdiction_code": (m.group("jur") or "").strip(),
|
||||
"company_number": (m.group("num") or "").strip(),
|
||||
"_via": "html",
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def fetch(
|
||||
query: str,
|
||||
jurisdiction: str | None,
|
||||
token: str | None,
|
||||
limit: int,
|
||||
out_path: str,
|
||||
) -> int:
|
||||
if token:
|
||||
try:
|
||||
companies = _via_api(query, jurisdiction, token, limit)
|
||||
source_tag = "api"
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(
|
||||
f"OpenCorporates API call failed ({e}); falling back to HTML.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
companies = _via_html(query, limit)
|
||||
source_tag = "html-fallback"
|
||||
else:
|
||||
print(
|
||||
"OPENCORPORATES_API_TOKEN not set — using HTML fallback (limited fields). "
|
||||
"Get a free token at https://opencorporates.com/api_accounts/new",
|
||||
file=sys.stderr,
|
||||
)
|
||||
companies = _via_html(query, limit)
|
||||
source_tag = "html"
|
||||
|
||||
rows: list[dict[str, str]] = []
|
||||
for c in companies[:limit]:
|
||||
if c.get("_via") == "html":
|
||||
rows.append(
|
||||
{
|
||||
"name": c.get("name", ""),
|
||||
"company_number": c.get("company_number", ""),
|
||||
"jurisdiction_code": c.get("jurisdiction_code", ""),
|
||||
"jurisdiction_name": "",
|
||||
"incorporation_date": "",
|
||||
"dissolution_date": "",
|
||||
"company_type": "",
|
||||
"status": "",
|
||||
"registered_address": "",
|
||||
"opencorporates_url": c.get("opencorporates_url", ""),
|
||||
"officers_count": "",
|
||||
"source": source_tag,
|
||||
}
|
||||
)
|
||||
continue
|
||||
addr = c.get("registered_address_in_full") or ""
|
||||
rows.append(
|
||||
{
|
||||
"name": c.get("name", "") or "",
|
||||
"company_number": c.get("company_number", "") or "",
|
||||
"jurisdiction_code": c.get("jurisdiction_code", "") or "",
|
||||
"jurisdiction_name": "",
|
||||
"incorporation_date": c.get("incorporation_date", "") or "",
|
||||
"dissolution_date": c.get("dissolution_date", "") or "",
|
||||
"company_type": c.get("company_type", "") or "",
|
||||
"status": c.get("current_status", "") or c.get("inactive", "") or "",
|
||||
"registered_address": addr,
|
||||
"opencorporates_url": c.get("opencorporates_url", "") or "",
|
||||
"officers_count": str(c.get("officers", {}).get("total_count", "") if c.get("officers") else ""),
|
||||
"source": source_tag,
|
||||
}
|
||||
)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
if not rows:
|
||||
print(
|
||||
f"OpenCorporates: 0 matches for query={query!r}"
|
||||
f"{f' jurisdiction={jurisdiction!r}' if jurisdiction else ''}.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--query", required=True, help="Company name search")
|
||||
p.add_argument(
|
||||
"--jurisdiction",
|
||||
help="Jurisdiction code, e.g. 'us_ny', 'us_de', 'gb', 'sg' (lowercased OpenCorporates style)",
|
||||
)
|
||||
p.add_argument("--limit", type=int, default=50)
|
||||
p.add_argument("--token", default=os.environ.get("OPENCORPORATES_API_TOKEN"))
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
n = fetch(
|
||||
query=a.query,
|
||||
jurisdiction=a.jurisdiction,
|
||||
token=a.token,
|
||||
limit=a.limit,
|
||||
out_path=a.out,
|
||||
)
|
||||
print(f"Wrote {n} OpenCorporates rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,184 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fetch SEC EDGAR filings index for a given CIK or company name.
|
||||
|
||||
SEC requires a User-Agent header with contact info. Set SEC_USER_AGENT,
|
||||
e.g. SEC_USER_AGENT="Research example@example.com".
|
||||
|
||||
Filings JSON is published at:
|
||||
https://data.sec.gov/submissions/CIK<10-digit-padded>.json
|
||||
|
||||
Company lookup uses:
|
||||
https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&company=<name>&output=atom
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get, get_json # noqa: E402
|
||||
|
||||
SUBMISSIONS_URL = "https://data.sec.gov/submissions/CIK{cik}.json"
|
||||
COLUMNS = [
|
||||
"cik",
|
||||
"company_name",
|
||||
"form_type",
|
||||
"filing_date",
|
||||
"accession_number",
|
||||
"primary_document",
|
||||
"filing_url",
|
||||
"reporting_period",
|
||||
]
|
||||
|
||||
|
||||
def _ua() -> str:
|
||||
ua = os.environ.get("SEC_USER_AGENT", "").strip()
|
||||
if not ua:
|
||||
raise SystemExit(
|
||||
"SEC requires a User-Agent with contact info. "
|
||||
"Set SEC_USER_AGENT='Your Name your@email'."
|
||||
)
|
||||
return ua
|
||||
|
||||
|
||||
def _resolve_cik(company: str) -> tuple[str, str]:
|
||||
"""Resolve a company name to a CIK via EDGAR's atom feed.
|
||||
|
||||
Returns (cik, resolved_company_name). The feed entries also reveal whether
|
||||
the match is an individual filer (Form 3/4/5 only) — surfaced in the
|
||||
return value so callers can warn.
|
||||
"""
|
||||
url = "https://www.sec.gov/cgi-bin/browse-edgar"
|
||||
params = {"action": "getcompany", "company": company, "output": "atom", "owner": "include"}
|
||||
body = get(url, params=params, user_agent=_ua()).decode("utf-8", errors="replace")
|
||||
m = re.search(r"CIK=(\d{10})", body)
|
||||
if not m:
|
||||
raise SystemExit(f"Could not resolve CIK for company={company!r}")
|
||||
cik = m.group(1)
|
||||
name_m = re.search(r"<title>([^<]+)\s*\((\d{10})\)</title>", body)
|
||||
resolved = name_m.group(1).strip() if name_m else ""
|
||||
return cik, resolved
|
||||
|
||||
|
||||
def fetch(
|
||||
cik: str | None,
|
||||
company: str | None,
|
||||
types: list[str],
|
||||
since: str | None,
|
||||
out_path: str,
|
||||
) -> int:
|
||||
resolved_name = ""
|
||||
if not cik and company:
|
||||
try:
|
||||
cik, resolved_name = _resolve_cik(company) # type: ignore[assignment]
|
||||
except SystemExit as e:
|
||||
# Write empty CSV with header so downstream tools still work,
|
||||
# and tell the user clearly.
|
||||
print(f"SEC EDGAR: {e}", file=sys.stderr)
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
csv.DictWriter(fh, fieldnames=COLUMNS).writeheader()
|
||||
return 0
|
||||
if resolved_name:
|
||||
print(
|
||||
f"Resolved company={company!r} → CIK {cik} ({resolved_name})",
|
||||
file=sys.stderr,
|
||||
)
|
||||
if not cik:
|
||||
raise SystemExit("must supply --cik or --company")
|
||||
cik = cik.zfill(10)
|
||||
url = SUBMISSIONS_URL.format(cik=cik)
|
||||
payload = get_json(url, user_agent=_ua())
|
||||
if not isinstance(payload, dict):
|
||||
raise SystemExit(f"Unexpected EDGAR response shape for CIK {cik}")
|
||||
name = payload.get("name", "")
|
||||
recent = (payload.get("filings", {}) or {}).get("recent", {}) or {}
|
||||
form = recent.get("form", [])
|
||||
date = recent.get("filingDate", [])
|
||||
accession = recent.get("accessionNumber", [])
|
||||
primary_doc = recent.get("primaryDocument", [])
|
||||
period = recent.get("reportDate", [])
|
||||
|
||||
# Histogram of available filing types — useful for surfacing why a filter
|
||||
# returned 0 (e.g. user asked for 10-K on an individual Form 4 filer).
|
||||
type_hist: dict[str, int] = {}
|
||||
for ftype in form:
|
||||
type_hist[ftype] = type_hist.get(ftype, 0) + 1
|
||||
|
||||
type_set = {t.strip().upper() for t in types} if types else None
|
||||
rows: list[dict[str, str]] = []
|
||||
for i, ftype in enumerate(form):
|
||||
if type_set and ftype.upper() not in type_set:
|
||||
continue
|
||||
fdate = date[i] if i < len(date) else ""
|
||||
if since and fdate and fdate < since:
|
||||
continue
|
||||
acc = accession[i] if i < len(accession) else ""
|
||||
pdoc = primary_doc[i] if i < len(primary_doc) else ""
|
||||
acc_nodash = acc.replace("-", "")
|
||||
filing_url = (
|
||||
f"https://www.sec.gov/Archives/edgar/data/{int(cik)}/{acc_nodash}/{pdoc}"
|
||||
if acc and pdoc
|
||||
else ""
|
||||
)
|
||||
rows.append(
|
||||
{
|
||||
"cik": cik,
|
||||
"company_name": name,
|
||||
"form_type": ftype,
|
||||
"filing_date": fdate,
|
||||
"accession_number": acc,
|
||||
"primary_document": pdoc,
|
||||
"filing_url": filing_url,
|
||||
"reporting_period": period[i] if i < len(period) else "",
|
||||
}
|
||||
)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
|
||||
if not rows and type_hist:
|
||||
top = sorted(type_hist.items(), key=lambda kv: -kv[1])[:8]
|
||||
hist_str = ", ".join(f"{t}={n}" for t, n in top)
|
||||
print(
|
||||
f"Warning: SEC EDGAR CIK {cik} ({name}) has {sum(type_hist.values())} "
|
||||
f"recent filings but NONE match types={types}. "
|
||||
f"Available form types: {hist_str}.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
# Insider-filer heuristic: only Form 3/4/5 → individual person, not a company.
|
||||
company_types = {"10-K", "10-Q", "8-K", "20-F", "DEF 14A", "S-1"}
|
||||
if not (set(type_hist.keys()) & company_types):
|
||||
print(
|
||||
f"Note: CIK {cik} appears to be an INDIVIDUAL filer "
|
||||
f"(insider Form 3/4/5 only), not a corporate registrant. "
|
||||
f"The resolver may have matched an officer/director named "
|
||||
f"{company!r} rather than a company.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument("--cik", help="Central Index Key (will be 10-digit zero-padded)")
|
||||
p.add_argument("--company", help="Resolve to CIK by company name")
|
||||
p.add_argument("--types", default="", help="Comma-separated form types (e.g. 10-K,10-Q,8-K)")
|
||||
p.add_argument("--since", help="Skip filings before YYYY-MM-DD")
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
types = [t for t in (a.types or "").split(",") if t.strip()]
|
||||
n = fetch(cik=a.cik, company=a.company, types=types, since=a.since, out_path=a.out)
|
||||
print(f"Wrote {n} EDGAR filing rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,146 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fetch Senate Lobbying Disclosure (LD-1 / LD-2) filings.
|
||||
|
||||
Anonymous: 120 req/hour. Token (SENATE_LDA_TOKEN): 1200 req/hour.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get_json # noqa: E402
|
||||
|
||||
ENDPOINT = "https://lda.senate.gov/api/v1/filings/"
|
||||
COLUMNS = [
|
||||
"filing_uuid",
|
||||
"filing_type",
|
||||
"filing_year",
|
||||
"filing_period",
|
||||
"registrant_name",
|
||||
"registrant_id",
|
||||
"client_name",
|
||||
"client_id",
|
||||
"client_general_description",
|
||||
"income",
|
||||
"expenses",
|
||||
"lobbyists",
|
||||
"issues",
|
||||
"government_entities",
|
||||
"filing_date",
|
||||
]
|
||||
|
||||
|
||||
def fetch(
|
||||
client: str | None,
|
||||
registrant: str | None,
|
||||
year: int,
|
||||
token: str | None,
|
||||
out_path: str,
|
||||
page_size: int = 100,
|
||||
max_pages: int = 25,
|
||||
) -> int:
|
||||
params: dict = {"filing_year": year, "page_size": page_size}
|
||||
if client:
|
||||
params["client_name"] = client
|
||||
if registrant:
|
||||
params["registrant_name"] = registrant
|
||||
|
||||
headers = {"Authorization": f"Token {token}"} if token else None
|
||||
rows: list[dict[str, str]] = []
|
||||
url = ENDPOINT
|
||||
page = 0
|
||||
while page < max_pages:
|
||||
try:
|
||||
payload = get_json(url, params=params if page == 0 else None, headers=headers)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Senate LDA error on page {page + 1}: {e}", file=sys.stderr)
|
||||
break
|
||||
if not isinstance(payload, dict):
|
||||
break
|
||||
results = payload.get("results", [])
|
||||
for r in results:
|
||||
client_obj = r.get("client") or {}
|
||||
registrant_obj = r.get("registrant") or {}
|
||||
lobbying_activities = r.get("lobbying_activities") or []
|
||||
lobbyists = []
|
||||
issues = []
|
||||
entities = []
|
||||
for la in lobbying_activities:
|
||||
for lob in la.get("lobbyists") or []:
|
||||
lob_obj = lob.get("lobbyist") or {}
|
||||
name = " ".join(
|
||||
x for x in (lob_obj.get("first_name", ""), lob_obj.get("last_name", "")) if x
|
||||
)
|
||||
if name:
|
||||
lobbyists.append(name)
|
||||
desc = la.get("description") or ""
|
||||
if desc:
|
||||
issues.append(desc)
|
||||
for ge in la.get("government_entities") or []:
|
||||
nm = ge.get("name") or ""
|
||||
if nm:
|
||||
entities.append(nm)
|
||||
rows.append(
|
||||
{
|
||||
"filing_uuid": r.get("filing_uuid", "") or "",
|
||||
"filing_type": r.get("filing_type", "") or "",
|
||||
"filing_year": str(r.get("filing_year", "") or year),
|
||||
"filing_period": r.get("filing_period", "") or "",
|
||||
"registrant_name": registrant_obj.get("name", "") or "",
|
||||
"registrant_id": str(registrant_obj.get("id", "") or ""),
|
||||
"client_name": client_obj.get("name", "") or "",
|
||||
"client_id": str(client_obj.get("id", "") or ""),
|
||||
"client_general_description": client_obj.get("general_description", "") or "",
|
||||
"income": str(r.get("income", "") or ""),
|
||||
"expenses": str(r.get("expenses", "") or ""),
|
||||
"lobbyists": "; ".join(sorted(set(lobbyists))),
|
||||
"issues": "; ".join(issues),
|
||||
"government_entities": "; ".join(sorted(set(entities))),
|
||||
"filing_date": (r.get("dt_posted") or "")[:10],
|
||||
}
|
||||
)
|
||||
next_url = payload.get("next")
|
||||
if not next_url:
|
||||
break
|
||||
url = next_url
|
||||
page += 1
|
||||
time.sleep(1.0 if not token else 0.3)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument("--client", help="Client name filter")
|
||||
p.add_argument("--registrant", help="Registrant (lobbying firm) name filter")
|
||||
p.add_argument("--year", type=int, default=2024)
|
||||
p.add_argument("--token", default=os.environ.get("SENATE_LDA_TOKEN"))
|
||||
p.add_argument("--max-pages", type=int, default=25)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
if not (a.client or a.registrant):
|
||||
p.error("must supply at least one of --client / --registrant")
|
||||
n = fetch(
|
||||
client=a.client,
|
||||
registrant=a.registrant,
|
||||
year=a.year,
|
||||
token=a.token,
|
||||
out_path=a.out,
|
||||
max_pages=a.max_pages,
|
||||
)
|
||||
print(f"Wrote {n} Senate LDA rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,170 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fetch federal contracts/awards from USAspending.gov API v2.
|
||||
|
||||
No auth required. POST to /api/v2/search/spending_by_award/ with filters.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ENDPOINT = "https://api.usaspending.gov/api/v2/search/spending_by_award/"
|
||||
COLUMNS = [
|
||||
"award_id",
|
||||
"recipient_name",
|
||||
"recipient_uei",
|
||||
"recipient_duns",
|
||||
"recipient_parent_name",
|
||||
"recipient_state",
|
||||
"awarding_agency",
|
||||
"awarding_sub_agency",
|
||||
"award_type",
|
||||
"award_amount",
|
||||
"award_date",
|
||||
"period_of_performance_start",
|
||||
"period_of_performance_end",
|
||||
"naics_code",
|
||||
"psc_code",
|
||||
"competition_extent",
|
||||
"description",
|
||||
]
|
||||
|
||||
# USAspending result column "code" → human label mapping for output.
|
||||
_FIELDS = [
|
||||
"Award ID",
|
||||
"Recipient Name",
|
||||
"Recipient UEI",
|
||||
"Recipient DUNS Number",
|
||||
"Recipient Parent Name",
|
||||
"Recipient State Code",
|
||||
"Awarding Agency",
|
||||
"Awarding Sub Agency",
|
||||
"Award Type",
|
||||
"Award Amount",
|
||||
"Start Date",
|
||||
"End Date",
|
||||
"NAICS Code",
|
||||
"PSC Code",
|
||||
"Type of Set Aside",
|
||||
"Description",
|
||||
]
|
||||
|
||||
|
||||
def _post(body: dict) -> dict:
|
||||
req = urllib.request.Request(
|
||||
ENDPOINT,
|
||||
data=json.dumps(body).encode("utf-8"),
|
||||
headers={"Content-Type": "application/json", "User-Agent": "hermes-agent osint-investigation"},
|
||||
method="POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=60) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def fetch(
|
||||
recipient: str | None,
|
||||
agency: str | None,
|
||||
fy: int,
|
||||
sole_source_only: bool,
|
||||
out_path: str,
|
||||
page_size: int = 100,
|
||||
max_pages: int = 20,
|
||||
) -> int:
|
||||
filters: dict = {
|
||||
"time_period": [{"start_date": f"{fy - 1}-10-01", "end_date": f"{fy}-09-30"}],
|
||||
# Contracts only by default; adjust award_type_codes for grants/loans.
|
||||
"award_type_codes": ["A", "B", "C", "D"],
|
||||
}
|
||||
if recipient:
|
||||
filters["recipient_search_text"] = [recipient]
|
||||
if agency:
|
||||
filters["agencies"] = [{"type": "awarding", "tier": "toptier", "name": agency}]
|
||||
|
||||
rows: list[dict[str, str]] = []
|
||||
page = 1
|
||||
while page <= max_pages:
|
||||
body = {
|
||||
"filters": filters,
|
||||
"fields": _FIELDS,
|
||||
"page": page,
|
||||
"limit": page_size,
|
||||
"sort": "Award Amount",
|
||||
"order": "desc",
|
||||
}
|
||||
try:
|
||||
payload = _post(body)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"USAspending error on page {page}: {e}", file=sys.stderr)
|
||||
break
|
||||
results = payload.get("results", [])
|
||||
if not results:
|
||||
break
|
||||
for r in results:
|
||||
set_aside = r.get("Type of Set Aside", "") or ""
|
||||
if sole_source_only and "sole" not in set_aside.lower():
|
||||
continue
|
||||
rows.append(
|
||||
{
|
||||
"award_id": r.get("Award ID", "") or "",
|
||||
"recipient_name": r.get("Recipient Name", "") or "",
|
||||
"recipient_uei": r.get("Recipient UEI", "") or "",
|
||||
"recipient_duns": r.get("Recipient DUNS Number", "") or "",
|
||||
"recipient_parent_name": r.get("Recipient Parent Name", "") or "",
|
||||
"recipient_state": r.get("Recipient State Code", "") or "",
|
||||
"awarding_agency": r.get("Awarding Agency", "") or "",
|
||||
"awarding_sub_agency": r.get("Awarding Sub Agency", "") or "",
|
||||
"award_type": r.get("Award Type", "") or "",
|
||||
"award_amount": str(r.get("Award Amount", "") or ""),
|
||||
"award_date": r.get("Start Date", "") or "",
|
||||
"period_of_performance_start": r.get("Start Date", "") or "",
|
||||
"period_of_performance_end": r.get("End Date", "") or "",
|
||||
"naics_code": str(r.get("NAICS Code", "") or ""),
|
||||
"psc_code": str(r.get("PSC Code", "") or ""),
|
||||
"competition_extent": set_aside,
|
||||
"description": r.get("Description", "") or "",
|
||||
}
|
||||
)
|
||||
meta = payload.get("page_metadata", {})
|
||||
if not meta.get("hasNext"):
|
||||
break
|
||||
page += 1
|
||||
time.sleep(0.5)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument("--recipient", help="Recipient name search")
|
||||
p.add_argument("--agency", help="Awarding agency (top-tier)")
|
||||
p.add_argument("--fy", type=int, default=2024, help="Federal fiscal year")
|
||||
p.add_argument("--sole-source-only", action="store_true")
|
||||
p.add_argument("--max-pages", type=int, default=20)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
if not (a.recipient or a.agency):
|
||||
p.error("must supply at least one of --recipient / --agency")
|
||||
n = fetch(
|
||||
recipient=a.recipient,
|
||||
agency=a.agency,
|
||||
fy=a.fy,
|
||||
sole_source_only=a.sole_source_only,
|
||||
out_path=a.out,
|
||||
max_pages=a.max_pages,
|
||||
)
|
||||
print(f"Wrote {n} USAspending rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,142 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Search the Internet Archive Wayback Machine via the CDX server.
|
||||
|
||||
The CDX API indexes ~900B+ archived web pages. Anonymous read access,
|
||||
no auth required. Useful for finding deleted / changed pages by URL,
|
||||
domain, or substring match.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import sys
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get_json # noqa: E402
|
||||
|
||||
BASE = "https://web.archive.org/cdx/search/cdx"
|
||||
|
||||
COLUMNS = [
|
||||
"url",
|
||||
"timestamp",
|
||||
"wayback_url",
|
||||
"mimetype",
|
||||
"status",
|
||||
"digest",
|
||||
"length",
|
||||
]
|
||||
|
||||
|
||||
def fetch(
|
||||
url_or_host: str,
|
||||
match_type: str,
|
||||
from_date: str | None,
|
||||
to_date: str | None,
|
||||
status: str | None,
|
||||
mime: str | None,
|
||||
collapse: str | None,
|
||||
limit: int,
|
||||
out_path: str,
|
||||
) -> int:
|
||||
params: dict[str, str] = {
|
||||
"url": url_or_host,
|
||||
"matchType": match_type,
|
||||
"output": "json",
|
||||
"limit": str(limit),
|
||||
}
|
||||
if from_date:
|
||||
params["from"] = from_date.replace("-", "")
|
||||
if to_date:
|
||||
params["to"] = to_date.replace("-", "")
|
||||
if status:
|
||||
params["filter"] = f"statuscode:{status}"
|
||||
if mime:
|
||||
params.setdefault("filter", "")
|
||||
# Multiple filters: CDX accepts repeated filter params via urlencode list
|
||||
params["filter"] = f"mimetype:{mime}"
|
||||
if collapse:
|
||||
params["collapse"] = collapse
|
||||
|
||||
url = f"{BASE}?{urllib.parse.urlencode(params)}"
|
||||
try:
|
||||
payload = get_json(url)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Wayback CDX error: {e}", file=sys.stderr)
|
||||
payload = []
|
||||
|
||||
rows: list[dict[str, str]] = []
|
||||
if isinstance(payload, list) and len(payload) > 1:
|
||||
header = payload[0]
|
||||
idx = {h: i for i, h in enumerate(header)}
|
||||
for entry in payload[1:]:
|
||||
ts = entry[idx["timestamp"]] if "timestamp" in idx else ""
|
||||
orig = entry[idx["original"]] if "original" in idx else ""
|
||||
rows.append(
|
||||
{
|
||||
"url": orig,
|
||||
"timestamp": ts,
|
||||
"wayback_url": f"https://web.archive.org/web/{ts}/{orig}" if ts and orig else "",
|
||||
"mimetype": entry[idx["mimetype"]] if "mimetype" in idx else "",
|
||||
"status": entry[idx["statuscode"]] if "statuscode" in idx else "",
|
||||
"digest": entry[idx["digest"]] if "digest" in idx else "",
|
||||
"length": entry[idx["length"]] if "length" in idx else "",
|
||||
}
|
||||
)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
if not rows:
|
||||
print(
|
||||
f"Wayback Machine: 0 captures for {url_or_host!r} matchType={match_type}.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--url", required=True, help="URL or host to look up in the archive")
|
||||
p.add_argument(
|
||||
"--match",
|
||||
default="exact",
|
||||
choices=["exact", "prefix", "host", "domain"],
|
||||
help=(
|
||||
"exact: this URL only. "
|
||||
"prefix: this URL's path-prefix. "
|
||||
"host: any URL on this host. "
|
||||
"domain: any URL on this domain or subdomains."
|
||||
),
|
||||
)
|
||||
p.add_argument("--from-date", help="Earliest capture YYYY-MM-DD")
|
||||
p.add_argument("--to-date", help="Latest capture YYYY-MM-DD")
|
||||
p.add_argument("--status", help="HTTP status filter (e.g. 200)")
|
||||
p.add_argument("--mime", help="MIME type filter (e.g. text/html)")
|
||||
p.add_argument(
|
||||
"--collapse",
|
||||
help="Collapse adjacent identical entries (e.g. 'digest' for unique-content captures)",
|
||||
)
|
||||
p.add_argument("--limit", type=int, default=200)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
n = fetch(
|
||||
url_or_host=a.url,
|
||||
match_type=a.match,
|
||||
from_date=a.from_date,
|
||||
to_date=a.to_date,
|
||||
status=a.status,
|
||||
mime=a.mime,
|
||||
collapse=a.collapse,
|
||||
limit=a.limit,
|
||||
out_path=a.out,
|
||||
)
|
||||
print(f"Wrote {n} Wayback capture rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,266 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Search Wikipedia + Wikidata for an entity (person, company, place, concept).
|
||||
|
||||
Two free APIs:
|
||||
- Wikipedia OpenSearch + REST summary endpoint for narrative bio
|
||||
- Wikidata SPARQL endpoint for structured facts (birth, employer, awards, etc.)
|
||||
|
||||
Both are anonymous-access. Useful for resolving who-is-this-entity questions
|
||||
and surfacing cross-references that other sources can join against.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import re
|
||||
import sys
|
||||
import urllib.parse
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _http import get_json # noqa: E402
|
||||
|
||||
WP_OPENSEARCH = "https://en.wikipedia.org/w/api.php"
|
||||
WP_SUMMARY = "https://en.wikipedia.org/api/rest_v1/page/summary/"
|
||||
WD_ACTION = "https://www.wikidata.org/w/api.php"
|
||||
|
||||
COLUMNS = [
|
||||
"source",
|
||||
"label",
|
||||
"description",
|
||||
"qid",
|
||||
"wikipedia_title",
|
||||
"wikipedia_url",
|
||||
"wikidata_url",
|
||||
"instance_of",
|
||||
"country",
|
||||
"occupation",
|
||||
"employer",
|
||||
"date_of_birth",
|
||||
"place_of_birth",
|
||||
"summary",
|
||||
]
|
||||
|
||||
|
||||
def _wp_search(query: str, limit: int) -> list[dict]:
|
||||
params = {
|
||||
"action": "opensearch",
|
||||
"search": query,
|
||||
"limit": str(min(limit, 20)),
|
||||
"format": "json",
|
||||
}
|
||||
url = f"{WP_OPENSEARCH}?{urllib.parse.urlencode(params)}"
|
||||
data = get_json(url)
|
||||
if not isinstance(data, list) or len(data) < 4:
|
||||
return []
|
||||
titles, descs, urls = data[1], data[2], data[3]
|
||||
out = []
|
||||
for i, title in enumerate(titles):
|
||||
out.append(
|
||||
{
|
||||
"title": title,
|
||||
"description": descs[i] if i < len(descs) else "",
|
||||
"url": urls[i] if i < len(urls) else "",
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _wp_summary(title: str) -> dict:
|
||||
"""Pull the REST summary for a title — short bio, image, type."""
|
||||
url = f"{WP_SUMMARY}{urllib.parse.quote(title.replace(' ', '_'))}"
|
||||
try:
|
||||
return get_json(url) # type: ignore[return-value]
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Wikipedia summary lookup for {title!r} failed: {e}", file=sys.stderr)
|
||||
return {}
|
||||
|
||||
|
||||
def _wd_lookup_by_qid(qid: str) -> dict:
|
||||
"""Pull common facts for a QID via Wikidata's Action API (no SPARQL).
|
||||
|
||||
The Action API is far more lenient on rate-limits than the SPARQL Query
|
||||
Service. We get claims as QIDs and then resolve labels in one batch call.
|
||||
"""
|
||||
# Properties of interest. The Action API returns claims as QIDs or
|
||||
# typed literals, so the slot mapping is local-only.
|
||||
interesting = {
|
||||
"P31": "instance_of",
|
||||
"P17": "country", # for orgs / places
|
||||
"P27": "country", # for individuals (country of citizenship)
|
||||
"P106": "occupation",
|
||||
"P108": "employer",
|
||||
"P569": "date_of_birth",
|
||||
"P19": "place_of_birth",
|
||||
}
|
||||
params = {
|
||||
"action": "wbgetentities",
|
||||
"ids": qid,
|
||||
"props": "claims",
|
||||
"format": "json",
|
||||
}
|
||||
url = f"{WD_ACTION}?{urllib.parse.urlencode(params)}"
|
||||
try:
|
||||
data = get_json(url)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Wikidata wbgetentities for {qid} failed: {e}", file=sys.stderr)
|
||||
return {}
|
||||
if not isinstance(data, dict):
|
||||
return {}
|
||||
claims = (data.get("entities", {}).get(qid, {}) or {}).get("claims", {}) or {}
|
||||
|
||||
# Collect raw values (QIDs or literals) and remember which slot each
|
||||
# came from. Date literals come back as ISO strings; QIDs need a label
|
||||
# resolution pass.
|
||||
qid_to_slots: dict[str, list[str]] = {}
|
||||
facts: dict[str, list[str]] = {}
|
||||
for prop_id, slot in interesting.items():
|
||||
for claim in claims.get(prop_id, []) or []:
|
||||
v = (claim.get("mainsnak", {}) or {}).get("datavalue", {}) or {}
|
||||
vtype = v.get("type")
|
||||
value = v.get("value")
|
||||
if vtype == "wikibase-entityid" and isinstance(value, dict):
|
||||
vqid = value.get("id", "")
|
||||
if vqid:
|
||||
qid_to_slots.setdefault(vqid, [])
|
||||
if slot not in qid_to_slots[vqid]:
|
||||
qid_to_slots[vqid].append(slot)
|
||||
elif vtype == "time" and isinstance(value, dict):
|
||||
raw = value.get("time", "") or ""
|
||||
# +1955-10-28T00:00:00Z → 1955-10-28
|
||||
m = re.search(r"[+-]?(\d{4})-(\d{2})-(\d{2})", raw)
|
||||
if m:
|
||||
facts.setdefault(slot, []).append(
|
||||
f"{m.group(1)}-{m.group(2)}-{m.group(3)}"
|
||||
)
|
||||
elif vtype == "string":
|
||||
facts.setdefault(slot, []).append(str(value))
|
||||
|
||||
# Resolve labels for all referenced QIDs in one batch (up to 50 at a time).
|
||||
qids = list(qid_to_slots)
|
||||
for i in range(0, len(qids), 50):
|
||||
batch = qids[i : i + 50]
|
||||
params = {
|
||||
"action": "wbgetentities",
|
||||
"ids": "|".join(batch),
|
||||
"props": "labels",
|
||||
"languages": "en",
|
||||
"format": "json",
|
||||
}
|
||||
url = f"{WD_ACTION}?{urllib.parse.urlencode(params)}"
|
||||
try:
|
||||
data = get_json(url)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Wikidata label batch failed: {e}", file=sys.stderr)
|
||||
continue
|
||||
if not isinstance(data, dict):
|
||||
continue
|
||||
ents = data.get("entities", {}) or {}
|
||||
for vqid, ent in ents.items():
|
||||
label = (ent.get("labels", {}).get("en", {}) or {}).get("value", "") or vqid
|
||||
for slot in qid_to_slots.get(vqid, []):
|
||||
facts.setdefault(slot, []).append(label)
|
||||
|
||||
# Deduplicate per slot, preserving order.
|
||||
deduped: dict[str, list[str]] = {}
|
||||
for slot, vals in facts.items():
|
||||
seen = set()
|
||||
out = []
|
||||
for v in vals:
|
||||
if v in seen:
|
||||
continue
|
||||
seen.add(v)
|
||||
out.append(v)
|
||||
deduped[slot] = out
|
||||
return deduped
|
||||
|
||||
|
||||
def _wd_qid_for_title(title: str) -> str:
|
||||
"""Get the Wikidata QID associated with a Wikipedia article title."""
|
||||
params = {
|
||||
"action": "query",
|
||||
"format": "json",
|
||||
"prop": "pageprops",
|
||||
"ppprop": "wikibase_item",
|
||||
"titles": title,
|
||||
"redirects": 1,
|
||||
}
|
||||
url = f"{WP_OPENSEARCH}?{urllib.parse.urlencode(params)}"
|
||||
try:
|
||||
data = get_json(url)
|
||||
except Exception: # noqa: BLE001
|
||||
return ""
|
||||
if not isinstance(data, dict):
|
||||
return ""
|
||||
pages = data.get("query", {}).get("pages", {}) or {}
|
||||
for page in pages.values():
|
||||
qid = (page.get("pageprops") or {}).get("wikibase_item", "")
|
||||
if qid:
|
||||
return qid
|
||||
return ""
|
||||
|
||||
|
||||
def fetch(query: str, limit: int, no_wikidata: bool, out_path: str) -> int:
|
||||
hits = _wp_search(query, limit)
|
||||
rows: list[dict[str, str]] = []
|
||||
for hit in hits[:limit]:
|
||||
title = hit.get("title", "")
|
||||
if not title:
|
||||
continue
|
||||
summary = _wp_summary(title)
|
||||
qid = _wd_qid_for_title(title) if not no_wikidata else ""
|
||||
facts: dict = {}
|
||||
if qid:
|
||||
facts = _wd_lookup_by_qid(qid)
|
||||
rows.append(
|
||||
{
|
||||
"source": "wikipedia+wikidata" if qid else "wikipedia",
|
||||
"label": title,
|
||||
"description": (summary.get("description") or hit.get("description") or "").strip(),
|
||||
"qid": qid,
|
||||
"wikipedia_title": title,
|
||||
"wikipedia_url": hit.get("url", ""),
|
||||
"wikidata_url": f"https://www.wikidata.org/wiki/{qid}" if qid else "",
|
||||
"instance_of": "; ".join(facts.get("instance_of", [])),
|
||||
"country": "; ".join(facts.get("country", [])),
|
||||
"occupation": "; ".join(facts.get("occupation", [])),
|
||||
"employer": "; ".join(facts.get("employer", [])),
|
||||
"date_of_birth": "; ".join(facts.get("date_of_birth", []))[:10] if facts.get("date_of_birth") else "",
|
||||
"place_of_birth": "; ".join(facts.get("place_of_birth", [])),
|
||||
"summary": (summary.get("extract") or "").replace("\n", " ")[:1000],
|
||||
}
|
||||
)
|
||||
|
||||
Path(out_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(out_path, "w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.DictWriter(fh, fieldnames=COLUMNS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
if not rows:
|
||||
print(
|
||||
f"Wikipedia: 0 articles for query={query!r}. "
|
||||
"Private individuals not notable enough for a Wikipedia article "
|
||||
"won't appear here (the bar is real).",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return len(rows)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--query", required=True, help="Entity name (person, company, place, concept)")
|
||||
p.add_argument("--limit", type=int, default=5)
|
||||
p.add_argument(
|
||||
"--no-wikidata",
|
||||
action="store_true",
|
||||
help="Skip the Wikidata SPARQL enrichment (faster, less detail)",
|
||||
)
|
||||
p.add_argument("--out", required=True)
|
||||
a = p.parse_args()
|
||||
n = fetch(query=a.query, limit=a.limit, no_wikidata=a.no_wikidata, out_path=a.out)
|
||||
print(f"Wrote {n} Wikipedia/Wikidata rows to {a.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,252 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Permutation test for donation/contract timing correlation (stdlib-only).
|
||||
|
||||
For each (donor, vendor) pair, compute the mean number of days between each
|
||||
donation and the nearest contract award. Then shuffle contract award dates
|
||||
N times within the observation window and compute the same statistic. The
|
||||
one-tailed p-value is the fraction of permutations whose mean is <= the
|
||||
observed mean (smaller distance = tighter clustering).
|
||||
|
||||
Adapted from ShinMegamiBoson/OpenPlanter (MIT). Differences:
|
||||
- Pure stdlib (no pandas / numpy)
|
||||
- Domain-agnostic (no snow-vendor / CRITICAL-politician filter)
|
||||
- Configurable column names via flags
|
||||
- Optional --seed for reproducibility
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import datetime as dt
|
||||
import json
|
||||
import random
|
||||
import statistics
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
_DATE_FORMATS = ("%Y-%m-%d", "%m/%d/%Y", "%Y/%m/%d", "%m-%d-%Y", "%Y%m%d")
|
||||
|
||||
|
||||
def parse_date(raw: str) -> dt.date | None:
|
||||
if not raw:
|
||||
return None
|
||||
raw = raw.strip()
|
||||
for fmt in _DATE_FORMATS:
|
||||
try:
|
||||
return dt.datetime.strptime(raw, fmt).date()
|
||||
except ValueError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _read(path: str) -> list[dict[str, str]]:
|
||||
with open(path, newline="", encoding="utf-8") as fh:
|
||||
return list(csv.DictReader(fh))
|
||||
|
||||
|
||||
def _nearest_distance(donation_date: dt.date, awards: list[dt.date]) -> int:
|
||||
"""Absolute days to nearest award date."""
|
||||
return min(abs((donation_date - a).days) for a in awards)
|
||||
|
||||
|
||||
def _permute(
|
||||
awards_count: int,
|
||||
donations: list[dt.date],
|
||||
date_min: dt.date,
|
||||
date_max: dt.date,
|
||||
rng: random.Random,
|
||||
) -> float:
|
||||
"""One permutation: draw uniform random award dates, compute mean nearest-distance."""
|
||||
span_days = (date_max - date_min).days or 1
|
||||
rand_awards = [
|
||||
date_min + dt.timedelta(days=rng.randint(0, span_days))
|
||||
for _ in range(awards_count)
|
||||
]
|
||||
distances = [_nearest_distance(d, rand_awards) for d in donations]
|
||||
return statistics.mean(distances)
|
||||
|
||||
|
||||
def analyze(
|
||||
donations_path: str,
|
||||
donation_date_col: str,
|
||||
donation_amount_col: str,
|
||||
donation_donor_col: str,
|
||||
donation_recipient_col: str,
|
||||
contracts_path: str,
|
||||
contract_date_col: str,
|
||||
contract_vendor_col: str,
|
||||
cross_links_path: str | None,
|
||||
n_permutations: int = 1000,
|
||||
min_donations: int = 3,
|
||||
p_threshold: float = 0.05,
|
||||
seed: int | None = None,
|
||||
out_path: str = "timing.json",
|
||||
) -> dict:
|
||||
rng = random.Random(seed)
|
||||
|
||||
donations = _read(donations_path)
|
||||
contracts = _read(contracts_path)
|
||||
|
||||
# Allow optional join through cross_links — donor (left) ↔ vendor (right).
|
||||
# When present, donor strings get mapped to matched vendor names so the
|
||||
# vendor-date index lookup actually finds the contracts.
|
||||
matched_pairs: set[tuple[str, str]] | None = None
|
||||
donor_to_vendors: dict[str, set[str]] = defaultdict(set)
|
||||
if cross_links_path:
|
||||
matched_pairs = set()
|
||||
for row in _read(cross_links_path):
|
||||
left = row.get("left_name", "")
|
||||
right = row.get("right_name", "")
|
||||
matched_pairs.add((left, right))
|
||||
donor_to_vendors[left].add(right)
|
||||
|
||||
# Index contract dates by vendor name.
|
||||
vendor_to_award_dates: dict[str, list[dt.date]] = defaultdict(list)
|
||||
all_award_dates: list[dt.date] = []
|
||||
for row in contracts:
|
||||
d = parse_date(row.get(contract_date_col, ""))
|
||||
if not d:
|
||||
continue
|
||||
vendor_to_award_dates[row.get(contract_vendor_col, "").strip()].append(d)
|
||||
all_award_dates.append(d)
|
||||
|
||||
if not all_award_dates:
|
||||
raise SystemExit(f"No parseable dates in {contracts_path}/{contract_date_col}")
|
||||
global_min = min(all_award_dates)
|
||||
global_max = max(all_award_dates)
|
||||
|
||||
# Group donations by (donor, recipient).
|
||||
grouped: dict[tuple[str, str], list[tuple[dt.date, float]]] = defaultdict(list)
|
||||
for row in donations:
|
||||
donor = row.get(donation_donor_col, "").strip()
|
||||
recip = row.get(donation_recipient_col, "").strip()
|
||||
d = parse_date(row.get(donation_date_col, ""))
|
||||
try:
|
||||
amt = float(row.get(donation_amount_col, "0") or 0)
|
||||
except ValueError:
|
||||
amt = 0.0
|
||||
if not (donor and recip and d):
|
||||
continue
|
||||
grouped[(donor, recip)].append((d, amt))
|
||||
|
||||
results = []
|
||||
skipped = 0
|
||||
for (donor, recip), records in grouped.items():
|
||||
if len(records) < min_donations:
|
||||
skipped += 1
|
||||
continue
|
||||
# Only test if donor appears in cross-links (when provided). The
|
||||
# (donor, candidate) tuple itself is NOT what's in matched_pairs —
|
||||
# cross_links pairs are (donor, vendor). We use the cross-link to
|
||||
# map donor → vendor name(s) so the vendor-date index resolves.
|
||||
if matched_pairs is not None and donor not in donor_to_vendors:
|
||||
skipped += 1
|
||||
continue
|
||||
# Try direct donor→awards first, then go through cross-link vendor names.
|
||||
award_dates = list(vendor_to_award_dates.get(donor, []))
|
||||
if not award_dates:
|
||||
award_dates = list(vendor_to_award_dates.get(recip, []))
|
||||
if not award_dates and donor_to_vendors.get(donor):
|
||||
for vendor_name in donor_to_vendors[donor]:
|
||||
award_dates.extend(vendor_to_award_dates.get(vendor_name, []))
|
||||
if not award_dates:
|
||||
skipped += 1
|
||||
continue
|
||||
|
||||
donation_dates = [d for (d, _) in records]
|
||||
observed = statistics.mean(
|
||||
_nearest_distance(d, award_dates) for d in donation_dates
|
||||
)
|
||||
|
||||
permuted_means = [
|
||||
_permute(len(award_dates), donation_dates, global_min, global_max, rng)
|
||||
for _ in range(n_permutations)
|
||||
]
|
||||
p_value = sum(1 for m in permuted_means if m <= observed) / n_permutations
|
||||
null_mean = statistics.mean(permuted_means)
|
||||
null_std = statistics.pstdev(permuted_means) or 1.0
|
||||
effect_size = (null_mean - observed) / null_std
|
||||
|
||||
results.append(
|
||||
{
|
||||
"donor": donor,
|
||||
"recipient": recip,
|
||||
"n_donations": len(records),
|
||||
"n_award_dates": len(award_dates),
|
||||
"observed_mean_days": round(observed, 2),
|
||||
"null_mean_days": round(null_mean, 2),
|
||||
"p_value": round(p_value, 4),
|
||||
"effect_size_sd": round(effect_size, 2),
|
||||
"significant": p_value < p_threshold,
|
||||
"total_donation_amount": round(sum(a for (_, a) in records), 2),
|
||||
}
|
||||
)
|
||||
|
||||
results.sort(key=lambda r: r["p_value"])
|
||||
|
||||
payload = {
|
||||
"metadata": {
|
||||
"n_permutations": n_permutations,
|
||||
"min_donations": min_donations,
|
||||
"p_threshold": p_threshold,
|
||||
"seed": seed,
|
||||
"n_pairs_tested": len(results),
|
||||
"n_pairs_skipped": skipped,
|
||||
"n_significant": sum(1 for r in results if r["significant"]),
|
||||
"observation_window": [global_min.isoformat(), global_max.isoformat()],
|
||||
},
|
||||
"results": results,
|
||||
}
|
||||
|
||||
Path(out_path).write_text(json.dumps(payload, indent=2), encoding="utf-8")
|
||||
return payload
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--donations", required=True)
|
||||
p.add_argument("--donation-date-col", required=True)
|
||||
p.add_argument("--donation-amount-col", required=True)
|
||||
p.add_argument("--donation-donor-col", required=True)
|
||||
p.add_argument("--donation-recipient-col", required=True)
|
||||
p.add_argument("--contracts", required=True)
|
||||
p.add_argument("--contract-date-col", required=True)
|
||||
p.add_argument("--contract-vendor-col", required=True)
|
||||
p.add_argument(
|
||||
"--cross-links",
|
||||
help="Optional cross_links.csv to restrict (donor, vendor) pairs",
|
||||
)
|
||||
p.add_argument("--permutations", type=int, default=1000)
|
||||
p.add_argument("--min-donations", type=int, default=3)
|
||||
p.add_argument("--p-threshold", type=float, default=0.05)
|
||||
p.add_argument("--seed", type=int)
|
||||
p.add_argument("--out", default="timing.json")
|
||||
a = p.parse_args()
|
||||
|
||||
payload = analyze(
|
||||
donations_path=a.donations,
|
||||
donation_date_col=a.donation_date_col,
|
||||
donation_amount_col=a.donation_amount_col,
|
||||
donation_donor_col=a.donation_donor_col,
|
||||
donation_recipient_col=a.donation_recipient_col,
|
||||
contracts_path=a.contracts,
|
||||
contract_date_col=a.contract_date_col,
|
||||
contract_vendor_col=a.contract_vendor_col,
|
||||
cross_links_path=a.cross_links,
|
||||
n_permutations=a.permutations,
|
||||
min_donations=a.min_donations,
|
||||
p_threshold=a.p_threshold,
|
||||
seed=a.seed,
|
||||
out_path=a.out,
|
||||
)
|
||||
meta = payload["metadata"]
|
||||
print(
|
||||
f"Tested {meta['n_pairs_tested']} pairs ({meta['n_pairs_skipped']} skipped). "
|
||||
f"Significant (p<{meta['p_threshold']}): {meta['n_significant']}. "
|
||||
f"Wrote {a.out}"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,59 @@
|
||||
# <Source Name>
|
||||
|
||||
## 1. Summary
|
||||
|
||||
What this data source is, who publishes it, why it matters for investigations.
|
||||
|
||||
## 2. Access Methods
|
||||
|
||||
- API endpoint(s)
|
||||
- Bulk download URLs
|
||||
- Auth requirements (none / API key / OAuth)
|
||||
- Rate limits
|
||||
|
||||
## 3. Data Schema
|
||||
|
||||
Key fields, record types, table relationships. List the columns the fetch
|
||||
script emits.
|
||||
|
||||
## 4. Coverage
|
||||
|
||||
- Jurisdiction
|
||||
- Time range
|
||||
- Update frequency
|
||||
- Data volume (rows / GB)
|
||||
|
||||
## 5. Cross-Reference Potential
|
||||
|
||||
Which other sources can be joined and on what keys. Be explicit:
|
||||
|
||||
- `<source>` ↔ `<column>` (join key: <normalized entity name / EIN / CIK / etc.>)
|
||||
|
||||
## 6. Data Quality
|
||||
|
||||
Known issues — formatting inconsistencies, missing fields, duplicates,
|
||||
historical gaps, redaction.
|
||||
|
||||
## 7. Acquisition Script
|
||||
|
||||
Path: `scripts/fetch_<source>.py`
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
python3 SKILL_DIR/scripts/fetch_<source>.py --<filter> <value> --out data/<source>.csv
|
||||
```
|
||||
|
||||
Output CSV columns: `<col1>, <col2>, ...`
|
||||
|
||||
## 8. Legal & Licensing
|
||||
|
||||
- Public records law / FOIA basis
|
||||
- Terms of use / acceptable use
|
||||
- Attribution requirements (if any)
|
||||
|
||||
## 9. References
|
||||
|
||||
- Official docs: <url>
|
||||
- Data dictionary: <url>
|
||||
- Related coverage / journalism: <url>
|
||||
@@ -0,0 +1,391 @@
|
||||
---
|
||||
name: parallel-cli
|
||||
description: Agent-native web search, deep research, and enrichment.
|
||||
version: 1.1.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Research, Web, Search, Deep-Research, Enrichment, CLI]
|
||||
related_skills: [duckduckgo-search, mcporter]
|
||||
---
|
||||
|
||||
# Parallel CLI
|
||||
|
||||
Use `parallel-cli` when the user explicitly wants Parallel, or when a terminal-native workflow would benefit from Parallel's vendor-specific stack for web search, extraction, deep research, enrichment, entity discovery, or monitoring.
|
||||
|
||||
This is an optional third-party workflow, not a Hermes core capability.
|
||||
|
||||
Important expectations:
|
||||
- Parallel is a paid service with a free tier, not a fully free local tool.
|
||||
- It overlaps with Hermes native `web_search` / `web_extract`, so do not prefer it by default for ordinary lookups.
|
||||
- Prefer this skill when the user mentions Parallel specifically or needs capabilities like Parallel's enrichment, FindAll, or monitor workflows.
|
||||
|
||||
`parallel-cli` is designed for agents:
|
||||
- JSON output via `--json`
|
||||
- Non-interactive command execution
|
||||
- Async long-running jobs with `--no-wait`, `status`, and `poll`
|
||||
- Context chaining with `--previous-interaction-id`
|
||||
- Search, extract, research, enrichment, entity discovery, and monitoring in one CLI
|
||||
|
||||
## When to use it
|
||||
|
||||
Prefer this skill when:
|
||||
- The user explicitly mentions Parallel or `parallel-cli`
|
||||
- The task needs richer workflows than a simple one-shot search/extract pass
|
||||
- You need async deep research jobs that can be launched and polled later
|
||||
- You need structured enrichment, FindAll entity discovery, or monitoring
|
||||
|
||||
Prefer Hermes native `web_search` / `web_extract` for quick one-off lookups when Parallel is not specifically requested.
|
||||
|
||||
## Installation
|
||||
|
||||
Try the least invasive install path available for the environment.
|
||||
|
||||
### Homebrew
|
||||
|
||||
```bash
|
||||
brew install parallel-web/tap/parallel-cli
|
||||
```
|
||||
|
||||
### npm
|
||||
|
||||
```bash
|
||||
npm install -g parallel-web-cli
|
||||
```
|
||||
|
||||
### Python package
|
||||
|
||||
```bash
|
||||
pip install "parallel-web-tools[cli]"
|
||||
```
|
||||
|
||||
### Standalone installer
|
||||
|
||||
```bash
|
||||
curl -fsSL https://parallel.ai/install.sh | bash
|
||||
```
|
||||
|
||||
If you want an isolated Python install, `pipx` can also work:
|
||||
|
||||
```bash
|
||||
pipx install "parallel-web-tools[cli]"
|
||||
pipx ensurepath
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
Interactive login:
|
||||
|
||||
```bash
|
||||
parallel-cli login
|
||||
```
|
||||
|
||||
Headless / SSH / CI:
|
||||
|
||||
```bash
|
||||
parallel-cli login --device
|
||||
```
|
||||
|
||||
API key environment variable:
|
||||
|
||||
```bash
|
||||
export PARALLEL_API_KEY="***"
|
||||
```
|
||||
|
||||
Verify current auth status:
|
||||
|
||||
```bash
|
||||
parallel-cli auth
|
||||
```
|
||||
|
||||
If auth requires browser interaction, run with `pty=true`.
|
||||
|
||||
## Core rule set
|
||||
|
||||
1. Always prefer `--json` when you need machine-readable output.
|
||||
2. Prefer explicit arguments and non-interactive flows.
|
||||
3. For long-running jobs, use `--no-wait` and then `status` / `poll`.
|
||||
4. Cite only URLs returned by the CLI output.
|
||||
5. Save large JSON outputs to a temp file when follow-up questions are likely.
|
||||
6. Use background processes only for genuinely long-running workflows; otherwise run in foreground.
|
||||
7. Prefer Hermes native tools unless the user wants Parallel specifically or needs Parallel-only workflows.
|
||||
|
||||
## Quick reference
|
||||
|
||||
```text
|
||||
parallel-cli
|
||||
├── auth
|
||||
├── login
|
||||
├── logout
|
||||
├── search
|
||||
├── extract / fetch
|
||||
├── research run|status|poll|processors
|
||||
├── enrich run|status|poll|plan|suggest|deploy
|
||||
├── findall run|ingest|status|poll|result|enrich|extend|schema|cancel
|
||||
└── monitor create|list|get|update|delete|events|event-group|simulate
|
||||
```
|
||||
|
||||
## Common flags and patterns
|
||||
|
||||
Commonly useful flags:
|
||||
- `--json` for structured output
|
||||
- `--no-wait` for async jobs
|
||||
- `--previous-interaction-id <id>` for follow-up tasks that reuse earlier context
|
||||
- `--max-results <n>` for search result count
|
||||
- `--mode one-shot|agentic` for search behavior
|
||||
- `--include-domains domain1.com,domain2.com`
|
||||
- `--exclude-domains domain1.com,domain2.com`
|
||||
- `--after-date YYYY-MM-DD`
|
||||
|
||||
Read from stdin when convenient:
|
||||
|
||||
```bash
|
||||
echo "What is the latest funding for Anthropic?" | parallel-cli search - --json
|
||||
echo "Research question" | parallel-cli research run - --json
|
||||
```
|
||||
|
||||
## Search
|
||||
|
||||
Use for current web lookups with structured results.
|
||||
|
||||
```bash
|
||||
parallel-cli search "What is Anthropic's latest AI model?" --json
|
||||
parallel-cli search "SEC filings for Apple" --include-domains sec.gov --json
|
||||
parallel-cli search "bitcoin price" --after-date 2026-01-01 --max-results 10 --json
|
||||
parallel-cli search "latest browser benchmarks" --mode one-shot --json
|
||||
parallel-cli search "AI coding agent enterprise reviews" --mode agentic --json
|
||||
```
|
||||
|
||||
Useful constraints:
|
||||
- `--include-domains` to narrow trusted sources
|
||||
- `--exclude-domains` to strip noisy domains
|
||||
- `--after-date` for recency filtering
|
||||
- `--max-results` when you need broader coverage
|
||||
|
||||
If you expect follow-up questions, save output:
|
||||
|
||||
```bash
|
||||
parallel-cli search "latest React 19 changes" --json -o /tmp/react-19-search.json
|
||||
```
|
||||
|
||||
When summarizing results:
|
||||
- lead with the answer
|
||||
- include dates, names, and concrete facts
|
||||
- cite only returned sources
|
||||
- avoid inventing URLs or source titles
|
||||
|
||||
## Extraction
|
||||
|
||||
Use to pull clean content or markdown from a URL.
|
||||
|
||||
```bash
|
||||
parallel-cli extract https://example.com --json
|
||||
parallel-cli extract https://company.com --objective "Find pricing info" --json
|
||||
parallel-cli extract https://example.com --full-content --json
|
||||
parallel-cli fetch https://example.com --json
|
||||
```
|
||||
|
||||
Use `--objective` when the page is broad and you only need one slice of information.
|
||||
|
||||
## Deep research
|
||||
|
||||
Use for deeper multi-step research tasks that may take time.
|
||||
|
||||
Common processor tiers:
|
||||
- `lite` / `base` for faster, cheaper passes
|
||||
- `core` / `pro` for more thorough synthesis
|
||||
- `ultra` for the heaviest research jobs
|
||||
|
||||
### Synchronous
|
||||
|
||||
```bash
|
||||
parallel-cli research run \
|
||||
"Compare the leading AI coding agents by pricing, model support, and enterprise controls" \
|
||||
--processor core \
|
||||
--json
|
||||
```
|
||||
|
||||
### Async launch + poll
|
||||
|
||||
```bash
|
||||
parallel-cli research run \
|
||||
"Compare the leading AI coding agents by pricing, model support, and enterprise controls" \
|
||||
--processor ultra \
|
||||
--no-wait \
|
||||
--json
|
||||
|
||||
parallel-cli research status trun_xxx --json
|
||||
parallel-cli research poll trun_xxx --json
|
||||
parallel-cli research processors --json
|
||||
```
|
||||
|
||||
### Context chaining / follow-up
|
||||
|
||||
```bash
|
||||
parallel-cli research run "What are the top AI coding agents?" --json
|
||||
parallel-cli research run \
|
||||
"What enterprise controls does the top-ranked one offer?" \
|
||||
--previous-interaction-id trun_xxx \
|
||||
--json
|
||||
```
|
||||
|
||||
Recommended Hermes workflow:
|
||||
1. launch with `--no-wait --json`
|
||||
2. capture the returned run/task ID
|
||||
3. if the user wants to continue other work, keep moving
|
||||
4. later call `status` or `poll`
|
||||
5. summarize the final report with citations from the returned sources
|
||||
|
||||
## Enrichment
|
||||
|
||||
Use when the user has CSV/JSON/tabular inputs and wants additional columns inferred from web research.
|
||||
|
||||
### Suggest columns
|
||||
|
||||
```bash
|
||||
parallel-cli enrich suggest "Find the CEO and annual revenue" --json
|
||||
```
|
||||
|
||||
### Plan a config
|
||||
|
||||
```bash
|
||||
parallel-cli enrich plan -o config.yaml
|
||||
```
|
||||
|
||||
### Inline data
|
||||
|
||||
```bash
|
||||
parallel-cli enrich run \
|
||||
--data '[{"company": "Anthropic"}, {"company": "Mistral"}]' \
|
||||
--intent "Find headquarters and employee count" \
|
||||
--json
|
||||
```
|
||||
|
||||
### Non-interactive file run
|
||||
|
||||
```bash
|
||||
parallel-cli enrich run \
|
||||
--source-type csv \
|
||||
--source companies.csv \
|
||||
--target enriched.csv \
|
||||
--source-columns '[{"name": "company", "description": "Company name"}]' \
|
||||
--intent "Find the CEO and annual revenue"
|
||||
```
|
||||
|
||||
### YAML config run
|
||||
|
||||
```bash
|
||||
parallel-cli enrich run config.yaml
|
||||
```
|
||||
|
||||
### Status / polling
|
||||
|
||||
```bash
|
||||
parallel-cli enrich status <task_group_id> --json
|
||||
parallel-cli enrich poll <task_group_id> --json
|
||||
```
|
||||
|
||||
Use explicit JSON arrays for column definitions when operating non-interactively.
|
||||
Validate the output file before reporting success.
|
||||
|
||||
## FindAll
|
||||
|
||||
Use for web-scale entity discovery when the user wants a discovered dataset rather than a short answer.
|
||||
|
||||
```bash
|
||||
parallel-cli findall run "Find AI coding agent startups with enterprise offerings" --json
|
||||
parallel-cli findall run "AI startups in healthcare" -n 25 --json
|
||||
parallel-cli findall status <run_id> --json
|
||||
parallel-cli findall poll <run_id> --json
|
||||
parallel-cli findall result <run_id> --json
|
||||
parallel-cli findall schema <run_id> --json
|
||||
```
|
||||
|
||||
This is a better fit than ordinary search when the user wants a discovered set of entities that can be reviewed, filtered, or enriched later.
|
||||
|
||||
## Monitor
|
||||
|
||||
Use for ongoing change detection over time.
|
||||
|
||||
```bash
|
||||
parallel-cli monitor list --json
|
||||
parallel-cli monitor get <monitor_id> --json
|
||||
parallel-cli monitor events <monitor_id> --json
|
||||
parallel-cli monitor delete <monitor_id> --json
|
||||
```
|
||||
|
||||
Creation is usually the sensitive part because cadence and delivery matter:
|
||||
|
||||
```bash
|
||||
parallel-cli monitor create --help
|
||||
```
|
||||
|
||||
Use this when the user wants recurring tracking of a page or source rather than a one-time fetch.
|
||||
|
||||
## Recommended Hermes usage patterns
|
||||
|
||||
### Fast answer with citations
|
||||
1. Run `parallel-cli search ... --json`
|
||||
2. Parse titles, URLs, dates, excerpts
|
||||
3. Summarize with inline citations from the returned URLs only
|
||||
|
||||
### URL investigation
|
||||
1. Run `parallel-cli extract URL --json`
|
||||
2. If needed, rerun with `--objective` or `--full-content`
|
||||
3. Quote or summarize the extracted markdown
|
||||
|
||||
### Long research workflow
|
||||
1. Run `parallel-cli research run ... --no-wait --json`
|
||||
2. Store the returned ID
|
||||
3. Continue other work or periodically poll
|
||||
4. Summarize the final report with citations
|
||||
|
||||
### Structured enrichment workflow
|
||||
1. Inspect the input file and columns
|
||||
2. Use `enrich suggest` or provide explicit enriched columns
|
||||
3. Run `enrich run`
|
||||
4. Poll for completion if needed
|
||||
5. Validate the output file before reporting success
|
||||
|
||||
## Error handling and exit codes
|
||||
|
||||
The CLI documents these exit codes:
|
||||
- `0` success
|
||||
- `2` bad input
|
||||
- `3` auth error
|
||||
- `4` API error
|
||||
- `5` timeout
|
||||
|
||||
If you hit auth errors:
|
||||
1. check `parallel-cli auth`
|
||||
2. confirm `PARALLEL_API_KEY` or run `parallel-cli login` / `parallel-cli login --device`
|
||||
3. verify `parallel-cli` is on `PATH`
|
||||
|
||||
## Maintenance
|
||||
|
||||
Check current auth / install state:
|
||||
|
||||
```bash
|
||||
parallel-cli auth
|
||||
parallel-cli --help
|
||||
```
|
||||
|
||||
Update commands:
|
||||
|
||||
```bash
|
||||
parallel-cli update
|
||||
pip install --upgrade parallel-web-tools
|
||||
parallel-cli config auto-update-check off
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Do not omit `--json` unless the user explicitly wants human-formatted output.
|
||||
- Do not cite sources not present in the CLI output.
|
||||
- `login` may require PTY/browser interaction.
|
||||
- Prefer foreground execution for short tasks; do not overuse background processes.
|
||||
- For large result sets, save JSON to `/tmp/*.json` instead of stuffing everything into context.
|
||||
- Do not silently choose Parallel when Hermes native tools are already sufficient.
|
||||
- Remember this is a vendor workflow that usually requires account auth and paid usage beyond the free tier.
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
name: pinecone-research
|
||||
description: Agent RAG and long-term memory with Pinecone.
|
||||
version: 1.0.0
|
||||
author: immuhammadfurqan
|
||||
license: MIT
|
||||
dependencies: [pinecone-client, langchain-pinecone]
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [RAG, Pinecone, Memory, Research, Vector Database, Agent, Retrieval]
|
||||
|
||||
---
|
||||
|
||||
# Pinecone Research — Agent RAG & Long-Term Memory
|
||||
|
||||
Use Pinecone as a retrieval-augmented generation (RAG) backend for agent
|
||||
conversations: persist embeddings, retrieve relevant context from past
|
||||
sessions, and build long-term memory.
|
||||
|
||||
## When to use this skill
|
||||
|
||||
**Use when:**
|
||||
- Building agent RAG pipelines with Pinecone as the vector store
|
||||
- Need persistent long-term memory across agent sessions
|
||||
- Combining retrieval with agent tool use
|
||||
- Researching or prototyping semantic search workflows
|
||||
|
||||
**Use the mlops/pinecone skill instead when:**
|
||||
- Need a general Pinecone reference (index management, CRUD, hybrid search)
|
||||
- Working on production infrastructure without agent integration
|
||||
|
||||
## Quick start
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
pip install pinecone-client langchain-pinecone langchain-openai
|
||||
```
|
||||
|
||||
Set your API key:
|
||||
```bash
|
||||
export PINECONE_API_KEY="your-api-key"
|
||||
```
|
||||
|
||||
### Basic RAG pipeline
|
||||
|
||||
```python
|
||||
from pinecone import Pinecone, ServerlessSpec
|
||||
from langchain_pinecone import PineconeVectorStore
|
||||
from langchain_openai import OpenAIEmbeddings
|
||||
|
||||
# Initialize Pinecone
|
||||
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
|
||||
|
||||
# Create or connect to index
|
||||
index_name = "agent-memory"
|
||||
if index_name not in [i.name for i in pc.list_indexes()]:
|
||||
pc.create_index(
|
||||
name=index_name,
|
||||
dimension=1536,
|
||||
metric="cosine",
|
||||
spec=ServerlessSpec(cloud="aws", region="us-east-1"),
|
||||
)
|
||||
|
||||
# Build vector store
|
||||
vectorstore = PineconeVectorStore.from_documents(
|
||||
documents=docs,
|
||||
embedding=OpenAIEmbeddings(),
|
||||
index_name=index_name,
|
||||
)
|
||||
|
||||
# Retrieve relevant context
|
||||
retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
|
||||
results = retriever.invoke("What did the agent discuss yesterday?")
|
||||
```
|
||||
|
||||
### Namespace-based session memory
|
||||
|
||||
```python
|
||||
# Store per-session memory
|
||||
vectorstore = PineconeVectorStore(
|
||||
index=pc.Index(index_name),
|
||||
embedding=OpenAIEmbeddings(),
|
||||
namespace=f"session-{session_id}",
|
||||
)
|
||||
|
||||
# Query across all sessions (no namespace filter)
|
||||
all_memory = PineconeVectorStore(
|
||||
index=pc.Index(index_name),
|
||||
embedding=OpenAIEmbeddings(),
|
||||
)
|
||||
results = all_memory.similarity_search("relevant query", k=10)
|
||||
```
|
||||
|
||||
## Best practices
|
||||
|
||||
1. **Namespace by session or user** — isolate data for multi-tenant agents
|
||||
2. **Batch upserts** — 100–200 vectors per batch for efficiency
|
||||
3. **Metadata filtering** — tag vectors with session ID, timestamp, topic
|
||||
4. **Prune old memory** — delete stale namespaces to control costs
|
||||
5. **Use serverless** — auto-scaling, pay-per-use pricing
|
||||
|
||||
## Resources
|
||||
|
||||
- **Pinecone Docs**: https://docs.pinecone.io
|
||||
- **LangChain Integration**: https://python.langchain.com/docs/integrations/vectorstores/pinecone
|
||||
- **Free Tier**: 1 index, 100K vectors (1536 dimensions)
|
||||
@@ -0,0 +1,155 @@
|
||||
"""Pinecone memory manager — namespace-based session memory for agents.
|
||||
|
||||
Provides helpers for storing and retrieving agent conversation memory
|
||||
using Pinecone namespaces. Each session gets its own namespace for isolation,
|
||||
with cross-session search available via the global namespace.
|
||||
|
||||
Usage:
|
||||
export PINECONE_API_KEY="your-key"
|
||||
export OPENAI_API_KEY="your-key"
|
||||
python memory_manager.py --index-name agent-memory --action store \
|
||||
--session-id sess-001 --text "User discussed project architecture"
|
||||
python memory_manager.py --index-name agent-memory --action recall \
|
||||
--query "architecture decisions"
|
||||
python memory_manager.py --index-name agent-memory --action cleanup \
|
||||
--session-id sess-001
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
|
||||
def get_pinecone_client():
|
||||
"""Initialize Pinecone client from environment."""
|
||||
try:
|
||||
from pinecone import Pinecone
|
||||
except ImportError:
|
||||
print("Error: pinecone-client not installed. Run: pip install pinecone-client", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
api_key = os.environ.get("PINECONE_API_KEY")
|
||||
if not api_key:
|
||||
print("Error: PINECONE_API_KEY environment variable not set.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
return Pinecone(api_key=api_key)
|
||||
|
||||
|
||||
def get_embeddings():
|
||||
"""Get the embedding model."""
|
||||
try:
|
||||
from langchain_openai import OpenAIEmbeddings
|
||||
except ImportError:
|
||||
print("Error: langchain-openai not installed. Run: pip install langchain-openai", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
return OpenAIEmbeddings()
|
||||
|
||||
|
||||
def store_memory(index, session_id: str, text: str, metadata: dict | None = None):
|
||||
"""Store a memory entry in the session namespace."""
|
||||
embeddings = get_embeddings()
|
||||
vector = embeddings.embed_query(text)
|
||||
|
||||
doc_id = hashlib.sha256(f"{session_id}:{text}:{time.time()}".encode()).hexdigest()[:16]
|
||||
entry_metadata = {
|
||||
"text": text[:1000],
|
||||
"session_id": session_id,
|
||||
"timestamp": int(time.time()),
|
||||
}
|
||||
if metadata:
|
||||
entry_metadata.update(metadata)
|
||||
|
||||
index.upsert(
|
||||
vectors=[{"id": doc_id, "values": vector, "metadata": entry_metadata}],
|
||||
namespace=session_id,
|
||||
)
|
||||
print(f"Stored memory [{doc_id}] in namespace '{session_id}'")
|
||||
return doc_id
|
||||
|
||||
|
||||
def recall_memories(index, query: str, session_id: str | None = None, top_k: int = 5):
|
||||
"""Recall memories matching a query, optionally scoped to a session."""
|
||||
embeddings = get_embeddings()
|
||||
query_vector = embeddings.embed_query(query)
|
||||
|
||||
kwargs = {"vector": query_vector, "top_k": top_k, "include_metadata": True}
|
||||
if session_id:
|
||||
kwargs["namespace"] = session_id
|
||||
|
||||
results = index.query(**kwargs)
|
||||
|
||||
print(f"\nRecalling memories for: {query!r}")
|
||||
if session_id:
|
||||
print(f"Scoped to session: {session_id}")
|
||||
print(f"Found {len(results['matches'])} results:\n")
|
||||
|
||||
for match in results["matches"]:
|
||||
score = match["score"]
|
||||
text = match["metadata"].get("text", "")[:200]
|
||||
sess = match["metadata"].get("session_id", "unknown")
|
||||
ts = match["metadata"].get("timestamp", 0)
|
||||
print(f" [{score:.4f}] session={sess} time={ts}")
|
||||
print(f" {text}")
|
||||
print()
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def cleanup_session(index, session_id: str):
|
||||
"""Delete all vectors in a session namespace."""
|
||||
index.delete(delete_all=True, namespace=session_id)
|
||||
print(f"Cleaned up namespace '{session_id}'")
|
||||
|
||||
|
||||
def show_stats(index):
|
||||
"""Show index statistics."""
|
||||
stats = index.describe_index_stats()
|
||||
print(f"Total vectors: {stats['total_vector_count']}")
|
||||
namespaces = stats.get("namespaces", {})
|
||||
if namespaces:
|
||||
print(f"Namespaces ({len(namespaces)}):")
|
||||
for ns, info in sorted(namespaces.items()):
|
||||
print(f" '{ns}': {info['vector_count']} vectors")
|
||||
else:
|
||||
print("No namespaces found.")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Pinecone agent memory manager")
|
||||
parser.add_argument("--index-name", required=True, help="Pinecone index name")
|
||||
parser.add_argument(
|
||||
"--action",
|
||||
choices=["store", "recall", "cleanup", "stats"],
|
||||
required=True,
|
||||
)
|
||||
parser.add_argument("--session-id", help="Session namespace ID")
|
||||
parser.add_argument("--text", help="Text to store as memory")
|
||||
parser.add_argument("--query", help="Query for recall")
|
||||
parser.add_argument("--top-k", type=int, default=5, help="Number of results")
|
||||
args = parser.parse_args()
|
||||
|
||||
pc = get_pinecone_client()
|
||||
index = pc.Index(args.index_name)
|
||||
|
||||
if args.action == "store":
|
||||
if not args.session_id or not args.text:
|
||||
parser.error("--session-id and --text required for store action")
|
||||
store_memory(index, args.session_id, args.text)
|
||||
elif args.action == "recall":
|
||||
if not args.query:
|
||||
parser.error("--query required for recall action")
|
||||
recall_memories(index, args.query, session_id=args.session_id, top_k=args.top_k)
|
||||
elif args.action == "cleanup":
|
||||
if not args.session_id:
|
||||
parser.error("--session-id required for cleanup action")
|
||||
cleanup_session(index, args.session_id)
|
||||
elif args.action == "stats":
|
||||
show_stats(index)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,156 @@
|
||||
"""Pinecone RAG pipeline — index documents and query with retrieval-augmented generation.
|
||||
|
||||
Usage:
|
||||
export PINECONE_API_KEY="your-key"
|
||||
export OPENAI_API_KEY="your-key"
|
||||
python rag_pipeline.py --index-name agent-memory --action index --docs-dir ./docs
|
||||
python rag_pipeline.py --index-name agent-memory --action query --query "How does X work?"
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def get_pinecone_client():
|
||||
"""Initialize Pinecone client from environment."""
|
||||
try:
|
||||
from pinecone import Pinecone
|
||||
except ImportError:
|
||||
print("Error: pinecone-client not installed. Run: pip install pinecone-client", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
api_key = os.environ.get("PINECONE_API_KEY")
|
||||
if not api_key:
|
||||
print("Error: PINECONE_API_KEY environment variable not set.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
return Pinecone(api_key=api_key)
|
||||
|
||||
|
||||
def ensure_index(pc, index_name: str, dimension: int = 1536):
|
||||
"""Create the index if it doesn't exist."""
|
||||
from pinecone import ServerlessSpec
|
||||
|
||||
existing = [idx.name for idx in pc.list_indexes()]
|
||||
if index_name not in existing:
|
||||
pc.create_index(
|
||||
name=index_name,
|
||||
dimension=dimension,
|
||||
metric="cosine",
|
||||
spec=ServerlessSpec(cloud="aws", region="us-east-1"),
|
||||
)
|
||||
print(f"Created index: {index_name}")
|
||||
else:
|
||||
print(f"Index already exists: {index_name}")
|
||||
return pc.Index(index_name)
|
||||
|
||||
|
||||
def load_documents(docs_dir: str) -> list[dict]:
|
||||
"""Load text files from a directory as documents."""
|
||||
docs = []
|
||||
docs_path = Path(docs_dir)
|
||||
if not docs_path.is_dir():
|
||||
print(f"Error: {docs_dir} is not a directory.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
for filepath in sorted(docs_path.rglob("*.txt")):
|
||||
text = filepath.read_text(encoding="utf-8").strip()
|
||||
if text:
|
||||
docs.append({
|
||||
"id": str(filepath.relative_to(docs_path)),
|
||||
"text": text,
|
||||
"metadata": {"source": str(filepath.name)},
|
||||
})
|
||||
return docs
|
||||
|
||||
|
||||
def index_documents(index, docs: list[dict], batch_size: int = 100):
|
||||
"""Embed and upsert documents into Pinecone."""
|
||||
try:
|
||||
from langchain_openai import OpenAIEmbeddings
|
||||
except ImportError:
|
||||
print("Error: langchain-openai not installed. Run: pip install langchain-openai", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
embeddings = OpenAIEmbeddings()
|
||||
vectors = []
|
||||
|
||||
for doc in docs:
|
||||
embedding = embeddings.embed_query(doc["text"])
|
||||
vectors.append({
|
||||
"id": doc["id"],
|
||||
"values": embedding,
|
||||
"metadata": {**doc["metadata"], "text": doc["text"][:1000]},
|
||||
})
|
||||
|
||||
# Batch upsert
|
||||
for i in range(0, len(vectors), batch_size):
|
||||
batch = vectors[i : i + batch_size]
|
||||
index.upsert(vectors=batch)
|
||||
print(f"Upserted batch {i // batch_size + 1} ({len(batch)} vectors)")
|
||||
|
||||
print(f"Total vectors indexed: {len(vectors)}")
|
||||
|
||||
|
||||
def query_index(index, query: str, top_k: int = 5):
|
||||
"""Embed a query and retrieve similar documents from Pinecone."""
|
||||
try:
|
||||
from langchain_openai import OpenAIEmbeddings
|
||||
except ImportError:
|
||||
print("Error: langchain-openai not installed. Run: pip install langchain-openai", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
embeddings = OpenAIEmbeddings()
|
||||
query_vector = embeddings.embed_query(query)
|
||||
|
||||
results = index.query(vector=query_vector, top_k=top_k, include_metadata=True)
|
||||
|
||||
print(f"\nQuery: {query}")
|
||||
print(f"Top {top_k} results:\n")
|
||||
for match in results["matches"]:
|
||||
score = match["score"]
|
||||
source = match["metadata"].get("source", "unknown")
|
||||
text_preview = match["metadata"].get("text", "")[:200]
|
||||
print(f" [{score:.4f}] {source}")
|
||||
print(f" {text_preview}...")
|
||||
print()
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Pinecone RAG pipeline")
|
||||
parser.add_argument("--index-name", required=True, help="Pinecone index name")
|
||||
parser.add_argument("--action", choices=["index", "query", "stats"], required=True)
|
||||
parser.add_argument("--docs-dir", help="Directory of .txt files to index")
|
||||
parser.add_argument("--query", help="Query string for retrieval")
|
||||
parser.add_argument("--top-k", type=int, default=5, help="Number of results to return")
|
||||
args = parser.parse_args()
|
||||
|
||||
pc = get_pinecone_client()
|
||||
index = ensure_index(pc, args.index_name)
|
||||
|
||||
if args.action == "index":
|
||||
if not args.docs_dir:
|
||||
parser.error("--docs-dir required for index action")
|
||||
docs = load_documents(args.docs_dir)
|
||||
if not docs:
|
||||
print("No .txt documents found.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
index_documents(index, docs)
|
||||
elif args.action == "query":
|
||||
if not args.query:
|
||||
parser.error("--query required for query action")
|
||||
query_index(index, args.query, top_k=args.top_k)
|
||||
elif args.action == "stats":
|
||||
stats = index.describe_index_stats()
|
||||
print(f"Total vectors: {stats['total_vector_count']}")
|
||||
for ns, info in stats.get("namespaces", {}).items():
|
||||
print(f" Namespace '{ns}': {info['vector_count']} vectors")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,441 @@
|
||||
---
|
||||
name: qmd
|
||||
description: Hybrid local search over notes, docs, and transcripts.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent + Teknium
|
||||
license: MIT
|
||||
platforms: [macos, linux]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Search, Knowledge-Base, RAG, Notes, MCP, Local-AI]
|
||||
related_skills: [obsidian, hermes-agent, arxiv]
|
||||
---
|
||||
|
||||
# QMD — Query Markup Documents
|
||||
|
||||
Local, on-device search engine for personal knowledge bases. Indexes markdown
|
||||
notes, meeting transcripts, documentation, and any text-based files, then
|
||||
provides hybrid search combining keyword matching, semantic understanding, and
|
||||
LLM-powered reranking — all running locally with no cloud dependencies.
|
||||
|
||||
Created by [Tobi Lütke](https://github.com/tobi/qmd). MIT licensed.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to search their notes, docs, knowledge base, or meeting transcripts
|
||||
- User wants to find something across a large collection of markdown/text files
|
||||
- User wants semantic search ("find notes about X concept") not just keyword grep
|
||||
- User has already set up qmd collections and wants to query them
|
||||
- User asks to set up a local knowledge base or document search system
|
||||
- Keywords: "search my notes", "find in my docs", "knowledge base", "qmd"
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Node.js >= 22 (required)
|
||||
|
||||
```bash
|
||||
# Check version
|
||||
node --version # must be >= 22
|
||||
|
||||
# macOS — install or upgrade via Homebrew
|
||||
brew install node@22
|
||||
|
||||
# Linux — use NodeSource or nvm
|
||||
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
|
||||
sudo apt-get install -y nodejs
|
||||
# or with nvm:
|
||||
nvm install 22 && nvm use 22
|
||||
```
|
||||
|
||||
### SQLite with Extension Support (macOS only)
|
||||
|
||||
macOS system SQLite lacks extension loading. Install via Homebrew:
|
||||
|
||||
```bash
|
||||
brew install sqlite
|
||||
```
|
||||
|
||||
### Install qmd
|
||||
|
||||
```bash
|
||||
npm install -g @tobilu/qmd
|
||||
# or with Bun:
|
||||
bun install -g @tobilu/qmd
|
||||
```
|
||||
|
||||
First run auto-downloads 3 local GGUF models (~2GB total):
|
||||
|
||||
| Model | Purpose | Size |
|
||||
|-------|---------|------|
|
||||
| embeddinggemma-300M-Q8_0 | Vector embeddings | ~300MB |
|
||||
| qwen3-reranker-0.6b-q8_0 | Result reranking | ~640MB |
|
||||
| qmd-query-expansion-1.7B | Query expansion | ~1.1GB |
|
||||
|
||||
### Verify Installation
|
||||
|
||||
```bash
|
||||
qmd --version
|
||||
qmd status
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Command | What It Does | Speed |
|
||||
|---------|-------------|-------|
|
||||
| `qmd search "query"` | BM25 keyword search (no models) | ~0.2s |
|
||||
| `qmd vsearch "query"` | Semantic vector search (1 model) | ~3s |
|
||||
| `qmd query "query"` | Hybrid + reranking (all 3 models) | ~2-3s warm, ~19s cold |
|
||||
| `qmd get <docid>` | Retrieve full document content | instant |
|
||||
| `qmd multi-get "glob"` | Retrieve multiple files | instant |
|
||||
| `qmd collection add <path> --name <n>` | Add a directory as a collection | instant |
|
||||
| `qmd context add <path> "description"` | Add context metadata to improve retrieval | instant |
|
||||
| `qmd embed` | Generate/update vector embeddings | varies |
|
||||
| `qmd status` | Show index health and collection info | instant |
|
||||
| `qmd mcp` | Start MCP server (stdio) | persistent |
|
||||
| `qmd mcp --http --daemon` | Start MCP server (HTTP, warm models) | persistent |
|
||||
|
||||
## Setup Workflow
|
||||
|
||||
### 1. Add Collections
|
||||
|
||||
Point qmd at directories containing your documents:
|
||||
|
||||
```bash
|
||||
# Add a notes directory
|
||||
qmd collection add ~/notes --name notes
|
||||
|
||||
# Add project docs
|
||||
qmd collection add ~/projects/myproject/docs --name project-docs
|
||||
|
||||
# Add meeting transcripts
|
||||
qmd collection add ~/meetings --name meetings
|
||||
|
||||
# List all collections
|
||||
qmd collection list
|
||||
```
|
||||
|
||||
### 2. Add Context Descriptions
|
||||
|
||||
Context metadata helps the search engine understand what each collection
|
||||
contains. This significantly improves retrieval quality:
|
||||
|
||||
```bash
|
||||
qmd context add qmd://notes "Personal notes, ideas, and journal entries"
|
||||
qmd context add qmd://project-docs "Technical documentation for the main project"
|
||||
qmd context add qmd://meetings "Meeting transcripts and action items from team syncs"
|
||||
```
|
||||
|
||||
### 3. Generate Embeddings
|
||||
|
||||
```bash
|
||||
qmd embed
|
||||
```
|
||||
|
||||
This processes all documents in all collections and generates vector
|
||||
embeddings. Re-run after adding new documents or collections.
|
||||
|
||||
### 4. Verify
|
||||
|
||||
```bash
|
||||
qmd status # shows index health, collection stats, model info
|
||||
```
|
||||
|
||||
## Search Patterns
|
||||
|
||||
### Fast Keyword Search (BM25)
|
||||
|
||||
Best for: exact terms, code identifiers, names, known phrases.
|
||||
No models loaded — near-instant results.
|
||||
|
||||
```bash
|
||||
qmd search "authentication middleware"
|
||||
qmd search "handleError async"
|
||||
```
|
||||
|
||||
### Semantic Vector Search
|
||||
|
||||
Best for: natural language questions, conceptual queries.
|
||||
Loads embedding model (~3s first query).
|
||||
|
||||
```bash
|
||||
qmd vsearch "how does the rate limiter handle burst traffic"
|
||||
qmd vsearch "ideas for improving onboarding flow"
|
||||
```
|
||||
|
||||
### Hybrid Search with Reranking (Best Quality)
|
||||
|
||||
Best for: important queries where quality matters most.
|
||||
Uses all 3 models — query expansion, parallel BM25+vector, reranking.
|
||||
|
||||
```bash
|
||||
qmd query "what decisions were made about the database migration"
|
||||
```
|
||||
|
||||
### Structured Multi-Mode Queries
|
||||
|
||||
Combine different search types in a single query for precision:
|
||||
|
||||
```bash
|
||||
# BM25 for exact term + vector for concept
|
||||
qmd query $'lex: rate limiter\nvec: how does throttling work under load'
|
||||
|
||||
# With query expansion
|
||||
qmd query $'expand: database migration plan\nlex: "schema change"'
|
||||
```
|
||||
|
||||
### Query Syntax (lex/BM25 mode)
|
||||
|
||||
| Syntax | Effect | Example |
|
||||
|--------|--------|---------|
|
||||
| `term` | Prefix match | `perf` matches "performance" |
|
||||
| `"phrase"` | Exact phrase | `"rate limiter"` |
|
||||
| `-term` | Exclude term | `performance -sports` |
|
||||
|
||||
### HyDE (Hypothetical Document Embeddings)
|
||||
|
||||
For complex topics, write what you expect the answer to look like:
|
||||
|
||||
```bash
|
||||
qmd query $'hyde: The migration plan involves three phases. First, we add the new columns without dropping the old ones. Then we backfill data. Finally we cut over and remove legacy columns.'
|
||||
```
|
||||
|
||||
### Scoping to Collections
|
||||
|
||||
```bash
|
||||
qmd search "query" --collection notes
|
||||
qmd query "query" --collection project-docs
|
||||
```
|
||||
|
||||
### Output Formats
|
||||
|
||||
```bash
|
||||
qmd search "query" --json # JSON output (best for parsing)
|
||||
qmd search "query" --limit 5 # Limit results
|
||||
qmd get "#abc123" # Get by document ID
|
||||
qmd get "path/to/file.md" # Get by file path
|
||||
qmd get "file.md:50" -l 100 # Get specific line range
|
||||
qmd multi-get "journals/*.md" --json # Batch retrieve by glob
|
||||
```
|
||||
|
||||
## MCP Integration (Recommended)
|
||||
|
||||
qmd exposes an MCP server that provides search tools directly to
|
||||
Hermes Agent via the native MCP client. This is the preferred
|
||||
integration — once configured, the agent gets qmd tools automatically
|
||||
without needing to load this skill.
|
||||
|
||||
### Option A: Stdio Mode (Simple)
|
||||
|
||||
Add to `~/.hermes/config.yaml`:
|
||||
|
||||
```yaml
|
||||
mcp_servers:
|
||||
qmd:
|
||||
command: "qmd"
|
||||
args: ["mcp"]
|
||||
timeout: 30
|
||||
connect_timeout: 45
|
||||
```
|
||||
|
||||
This registers tools: `mcp_qmd_search`, `mcp_qmd_vsearch`,
|
||||
`mcp_qmd_deep_search`, `mcp_qmd_get`, `mcp_qmd_status`.
|
||||
|
||||
**Tradeoff:** Models load on first search call (~19s cold start),
|
||||
then stay warm for the session. Acceptable for occasional use.
|
||||
|
||||
### Option B: HTTP Daemon Mode (Fast, Recommended for Heavy Use)
|
||||
|
||||
Start the qmd daemon separately — it keeps models warm in memory:
|
||||
|
||||
```bash
|
||||
# Start daemon (persists across agent restarts)
|
||||
qmd mcp --http --daemon
|
||||
|
||||
# Runs on http://localhost:8181 by default
|
||||
```
|
||||
|
||||
Then configure Hermes Agent to connect via HTTP:
|
||||
|
||||
```yaml
|
||||
mcp_servers:
|
||||
qmd:
|
||||
url: "http://localhost:8181/mcp"
|
||||
timeout: 30
|
||||
```
|
||||
|
||||
**Tradeoff:** Uses ~2GB RAM while running, but every query is fast
|
||||
(~2-3s). Best for users who search frequently.
|
||||
|
||||
### Keeping the Daemon Running
|
||||
|
||||
#### macOS (launchd)
|
||||
|
||||
```bash
|
||||
cat > ~/Library/LaunchAgents/com.qmd.daemon.plist << 'EOF'
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
|
||||
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key>
|
||||
<string>com.qmd.daemon</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>qmd</string>
|
||||
<string>mcp</string>
|
||||
<string>--http</string>
|
||||
<string>--daemon</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key>
|
||||
<true/>
|
||||
<key>KeepAlive</key>
|
||||
<true/>
|
||||
<key>StandardOutPath</key>
|
||||
<string>/tmp/qmd-daemon.log</string>
|
||||
<key>StandardErrorPath</key>
|
||||
<string>/tmp/qmd-daemon.log</string>
|
||||
</dict>
|
||||
</plist>
|
||||
EOF
|
||||
|
||||
launchctl load ~/Library/LaunchAgents/com.qmd.daemon.plist
|
||||
```
|
||||
|
||||
#### Linux (systemd user service)
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.config/systemd/user
|
||||
|
||||
cat > ~/.config/systemd/user/qmd-daemon.service << 'EOF'
|
||||
[Unit]
|
||||
Description=QMD MCP Daemon
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
ExecStart=qmd mcp --http --daemon
|
||||
Restart=on-failure
|
||||
RestartSec=10
|
||||
Environment=PATH=/usr/local/bin:/usr/bin:/bin
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
EOF
|
||||
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now qmd-daemon
|
||||
systemctl --user status qmd-daemon
|
||||
```
|
||||
|
||||
### MCP Tools Reference
|
||||
|
||||
Once connected, these tools are available as `mcp_qmd_*`:
|
||||
|
||||
| MCP Tool | Maps To | Description |
|
||||
|----------|---------|-------------|
|
||||
| `mcp_qmd_search` | `qmd search` | BM25 keyword search |
|
||||
| `mcp_qmd_vsearch` | `qmd vsearch` | Semantic vector search |
|
||||
| `mcp_qmd_deep_search` | `qmd query` | Hybrid search + reranking |
|
||||
| `mcp_qmd_get` | `qmd get` | Retrieve document by ID or path |
|
||||
| `mcp_qmd_status` | `qmd status` | Index health and stats |
|
||||
|
||||
The MCP tools accept structured JSON queries for multi-mode search:
|
||||
|
||||
```json
|
||||
{
|
||||
"searches": [
|
||||
{"type": "lex", "query": "authentication middleware"},
|
||||
{"type": "vec", "query": "how user login is verified"}
|
||||
],
|
||||
"collections": ["project-docs"],
|
||||
"limit": 10
|
||||
}
|
||||
```
|
||||
|
||||
## CLI Usage (Without MCP)
|
||||
|
||||
When MCP is not configured, use qmd directly via terminal:
|
||||
|
||||
```
|
||||
terminal(command="qmd query 'what was decided about the API redesign' --json", timeout=30)
|
||||
```
|
||||
|
||||
For setup and management tasks, always use terminal:
|
||||
|
||||
```
|
||||
terminal(command="qmd collection add ~/Documents/notes --name notes")
|
||||
terminal(command="qmd context add qmd://notes 'Personal research notes and ideas'")
|
||||
terminal(command="qmd embed")
|
||||
terminal(command="qmd status")
|
||||
```
|
||||
|
||||
## How the Search Pipeline Works
|
||||
|
||||
Understanding the internals helps choose the right search mode:
|
||||
|
||||
1. **Query Expansion** — A fine-tuned 1.7B model generates 2 alternative
|
||||
queries. The original gets 2x weight in fusion.
|
||||
2. **Parallel Retrieval** — BM25 (SQLite FTS5) and vector search run
|
||||
simultaneously across all query variants.
|
||||
3. **RRF Fusion** — Reciprocal Rank Fusion (k=60) merges results.
|
||||
Top-rank bonus: #1 gets +0.05, #2-3 get +0.02.
|
||||
4. **LLM Reranking** — qwen3-reranker scores top 30 candidates (0.0-1.0).
|
||||
5. **Position-Aware Blending** — Ranks 1-3: 75% retrieval / 25% reranker.
|
||||
Ranks 4-10: 60/40. Ranks 11+: 40/60 (trusts reranker more for long tail).
|
||||
|
||||
**Smart Chunking:** Documents are split at natural break points (headings,
|
||||
code blocks, blank lines) targeting ~900 tokens with 15% overlap. Code
|
||||
blocks are never split mid-block.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always add context descriptions** — `qmd context add` dramatically
|
||||
improves retrieval accuracy. Describe what each collection contains.
|
||||
2. **Re-embed after adding documents** — `qmd embed` must be re-run when
|
||||
new files are added to collections.
|
||||
3. **Use `qmd search` for speed** — when you need fast keyword lookup
|
||||
(code identifiers, exact names), BM25 is instant and needs no models.
|
||||
4. **Use `qmd query` for quality** — when the question is conceptual or
|
||||
the user needs the best possible results, use hybrid search.
|
||||
5. **Prefer MCP integration** — once configured, the agent gets native
|
||||
tools without needing to load this skill each time.
|
||||
6. **Daemon mode for frequent users** — if the user searches their
|
||||
knowledge base regularly, recommend the HTTP daemon setup.
|
||||
7. **First query in structured search gets 2x weight** — put the most
|
||||
important/certain query first when combining lex and vec.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Models downloading on first run"
|
||||
Normal — qmd auto-downloads ~2GB of GGUF models on first use.
|
||||
This is a one-time operation.
|
||||
|
||||
### Cold start latency (~19s)
|
||||
This happens when models aren't loaded in memory. Solutions:
|
||||
- Use HTTP daemon mode (`qmd mcp --http --daemon`) to keep warm
|
||||
- Use `qmd search` (BM25 only) when models aren't needed
|
||||
- MCP stdio mode loads models on first search, stays warm for session
|
||||
|
||||
### macOS: "unable to load extension"
|
||||
Install Homebrew SQLite: `brew install sqlite`
|
||||
Then ensure it's on PATH before system SQLite.
|
||||
|
||||
### "No collections found"
|
||||
Run `qmd collection add <path> --name <name>` to add directories,
|
||||
then `qmd embed` to index them.
|
||||
|
||||
### Embedding model override (CJK/multilingual)
|
||||
Set `QMD_EMBED_MODEL` environment variable for non-English content:
|
||||
```bash
|
||||
export QMD_EMBED_MODEL="your-multilingual-model"
|
||||
```
|
||||
|
||||
## Data Storage
|
||||
|
||||
- **Index & vectors:** `~/.cache/qmd/index.sqlite`
|
||||
- **Models:** Auto-downloaded to local cache on first run
|
||||
- **No cloud dependencies** — everything runs locally
|
||||
|
||||
## References
|
||||
|
||||
- [GitHub: tobi/qmd](https://github.com/tobi/qmd)
|
||||
- [QMD Changelog](https://github.com/tobi/qmd/blob/main/CHANGELOG.md)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,394 @@
|
||||
# Autoreason: Iterative Refinement Methodology
|
||||
|
||||
Complete reference for the autoreason iterative refinement method, derived from experimental results across subjective writing tasks, competitive programming, and four model tiers. Use this when any output (paper draft, experiment script, analysis, task definition) needs iterative improvement.
|
||||
|
||||
**Source**: [NousResearch/autoreason](https://github.com/NousResearch/autoreason) — "Autoreason: When Iterative LLM Refinement Works and Why It Fails"
|
||||
|
||||
---
|
||||
|
||||
## Strategy Selection Guide
|
||||
|
||||
### Decision Tree
|
||||
|
||||
```
|
||||
Is the task objectively verifiable (code, math, factual)?
|
||||
├── YES → Does the model solve it on the first attempt?
|
||||
│ ├── YES → Use single pass (no refinement needed)
|
||||
│ └── NO → Use autoreason (structured analysis → reason-informed revision)
|
||||
│
|
||||
└── NO (subjective) → What model tier are you using?
|
||||
├── Weak (Llama 8B, small models)
|
||||
│ → Single pass. Model too weak for refinement to help.
|
||||
│ Invest in generation quality, not iteration.
|
||||
│
|
||||
├── Mid-tier (Haiku 3.5, Gemini Flash)
|
||||
│ → Autoreason with stronger judges. This is the sweet spot.
|
||||
│ Self-refinement DESTROYS weak model outputs — autoreason prevents this.
|
||||
│
|
||||
├── Strong (Sonnet 4)
|
||||
│ → Autoreason for open-ended tasks. Wins 3/5.
|
||||
│ Critique-and-revise for concrete technical tasks (2/5).
|
||||
│
|
||||
└── Frontier (Sonnet 4.6, Opus)
|
||||
├── Constrained scope? → Autoreason. Wins 2/3 constrained tasks.
|
||||
└── Unconstrained? → Critique-and-revise or single pass.
|
||||
Autoreason FAILS on unconstrained frontier tasks (comes last).
|
||||
```
|
||||
|
||||
### Strategy Comparison Table
|
||||
|
||||
| Strategy | Best For | Avoid When | Compute (per iteration) |
|
||||
|----------|----------|------------|------------------------|
|
||||
| **Single pass** | Frontier models, template tasks, tight budgets | Mid-tier models where quality ceiling is low | 1 call |
|
||||
| **Critique-and-revise** | Concrete technical requirements (system design, specifications) | Weak models (degrades output), unconstrained subjective tasks | 2 calls |
|
||||
| **Autoreason** | Mid-tier models, constrained scope, tasks with genuine tradeoffs | Weak models (Llama 8B), frontier + unconstrained | ~6 calls |
|
||||
| **Best-of-N** | Almost never recommended | Weak models especially — worse than single pass | N calls |
|
||||
|
||||
### Why Each Strategy Fails
|
||||
|
||||
| Strategy | Failure Mode | Mechanism |
|
||||
|----------|-------------|-----------|
|
||||
| **Single pass** | Quality ceiling | No mechanism to improve beyond first attempt |
|
||||
| **Critique-and-revise** | Progressive degradation | Model hallucinates problems (sycophancy), scope creeps each pass, never declines to change |
|
||||
| **Best-of-N** | Random selection | Without good ranking signal, more samples = more mediocre options |
|
||||
| **Autoreason (unconstrained)** | Synthesis drift | Stronger models produce syntheses so consistently preferred that incumbent never stabilizes |
|
||||
|
||||
---
|
||||
|
||||
## The Autoreason Loop
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ ITERATION LOOP │
|
||||
│ │
|
||||
│ Incumbent A ──► Critic ──► Author B ──► Synthesizer │
|
||||
│ │ │ │
|
||||
│ │ ┌───────────────────────┘ │
|
||||
│ ▼ ▼ │
|
||||
│ [A] [AB] [B] │
|
||||
│ │ │ │ │
|
||||
│ └──────────────┼────────────┘ │
|
||||
│ ▼ │
|
||||
│ Judge Panel (blind) │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ Winner │
|
||||
│ │ │
|
||||
│ ┌───────┴───────┐ │
|
||||
│ ▼ ▼ │
|
||||
│ A wins k=2 B or AB wins │
|
||||
│ consecutive? → new incumbent │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ CONVERGED │
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Roles
|
||||
|
||||
Every role is a **fresh, isolated agent** with no shared context:
|
||||
|
||||
| Role | Input | Output | Key Rule |
|
||||
|------|-------|--------|----------|
|
||||
| **Critic** | Task + Incumbent A | List of problems | Find problems ONLY. No fixes. No suggestions. |
|
||||
| **Author B** | Task + A + Critique | Revised version B | Address each criticism. State which problem each change fixes. |
|
||||
| **Synthesizer** | Task + X + Y (randomized labels) | Synthesis AB | Take strongest elements of each. Not a compromise. |
|
||||
| **Judge Panel** | Task + A, AB, B (randomized labels + order) | Ranking | Rank best to worst. No authorship stake. |
|
||||
|
||||
### Configuration
|
||||
|
||||
| Parameter | Value | Rationale |
|
||||
|-----------|-------|-----------|
|
||||
| **Convergence k** | 2 | k=1 premature (94% displaced later). k=2 converges 100%, quality plateaus. k=3 fails 24%, 2x cost, no quality gain. |
|
||||
| **Author temperature** | 0.7-0.8 | Encourages diverse revisions |
|
||||
| **Judge temperature** | 0.3 | Encourages consistent evaluation |
|
||||
| **In-loop judges** | 3 | Balance per-pass cost vs evaluation stability |
|
||||
| **Final evaluation judges** | 7 | Higher statistical power for final comparison |
|
||||
| **Max tokens** | 4096 | Standard; 8192 for long-form (papers) |
|
||||
| **Judge type** | Chain-of-thought | 3x faster convergence on some tasks. Always use. |
|
||||
| **Tiebreak** | Conservative (incumbent wins) | Prevents false positives — A must be genuinely beaten |
|
||||
| **Max passes** | 25 (constrained), 50 (remedy) | Safety cap; most converge by pass 10-15 |
|
||||
|
||||
### Prompts
|
||||
|
||||
#### Critic
|
||||
```
|
||||
System: You are a critical reviewer. Your only job is to find real problems.
|
||||
Be specific and concrete. Do not suggest fixes.
|
||||
|
||||
User: Find real problems with this proposal. Focus on:
|
||||
- Things that won't work as described
|
||||
- Complexity that doesn't pay for itself
|
||||
- Assumptions that are wrong
|
||||
- Missing pieces
|
||||
Do NOT propose fixes. Just the problems.
|
||||
```
|
||||
|
||||
#### Author B
|
||||
```
|
||||
System: You are a senior consultant revising a proposal based on specific
|
||||
criticisms. Address each valid criticism directly. Do not make changes not
|
||||
motivated by an identified problem.
|
||||
|
||||
User: [TASK] + [VERSION A] + [CRITIC OUTPUT]
|
||||
Revise to address these problems. For each change, state which problem it fixes.
|
||||
```
|
||||
|
||||
#### Synthesizer
|
||||
```
|
||||
System: You are given two versions as equal inputs. Take the strongest elements
|
||||
from each and produce a coherent synthesis. This is not a compromise.
|
||||
|
||||
User: [TASK] + [VERSION X] + [VERSION Y]
|
||||
(labels randomized — synthesizer doesn't know which is incumbent)
|
||||
```
|
||||
|
||||
#### Judge (Chain-of-Thought) — ALWAYS USE THIS VERSION
|
||||
```
|
||||
System: You are an independent evaluator. Think carefully before deciding.
|
||||
|
||||
User: [TASK] + Three proposals. For each, think step by step:
|
||||
1. What does it get right?
|
||||
2. What does it get wrong or miss?
|
||||
3. Are numbers and claims defensible?
|
||||
4. Is detail appropriate or bloated?
|
||||
After reasoning, rank all three.
|
||||
RANKING: [best], [second], [worst]
|
||||
```
|
||||
|
||||
#### Baseline Prompts (for comparison experiments)
|
||||
|
||||
| Baseline | Prompt |
|
||||
|----------|--------|
|
||||
| **Conservative** | "Make minimal improvements while preserving what works. Do not add new sections or significantly expand scope." |
|
||||
| **Improve this** | "Improve this document." (no further guidance) |
|
||||
| **Harsh critic** | "Critically evaluate and rewrite, fixing all weaknesses you identify." |
|
||||
| **Critique & revise** | Step 1: "Produce a structured critique. List specific weaknesses." Step 2: "Revise to address each criticism." |
|
||||
|
||||
---
|
||||
|
||||
## Scoring: Borda Count
|
||||
|
||||
Judges rank candidates. Points awarded by rank position:
|
||||
|
||||
| Rank | Points (3 candidates) |
|
||||
|------|----------------------|
|
||||
| 1st | 3 |
|
||||
| 2nd | 2 |
|
||||
| 3rd | 1 |
|
||||
|
||||
**Aggregation**: Sum across all judges. Winner = highest total.
|
||||
**Tiebreak**: Incumbent (A) wins any tie.
|
||||
|
||||
**Example** (3 judges):
|
||||
- Judge 1: AB > A > B → AB gets 3, A gets 2, B gets 1
|
||||
- Judge 2: A > AB > B → A gets 3, AB gets 2, B gets 1
|
||||
- Judge 3: AB > B > A → AB gets 3, B gets 2, A gets 1
|
||||
- Totals: AB=8, A=6, B=4 → AB wins, becomes new incumbent
|
||||
|
||||
**Randomization per judge**:
|
||||
- Candidate labels randomized (A might be called "Proposal X" for one judge, "Proposal Z" for another)
|
||||
- Presentation order randomized (AB might appear first or last)
|
||||
- This prevents position bias and label bias
|
||||
|
||||
---
|
||||
|
||||
## Model Selection Guide
|
||||
|
||||
### Empirical Results by Model Tier
|
||||
|
||||
| Model | Autoreason Wins | Autoreason Avg Borda | Best Baseline | Margin | Recommendation |
|
||||
|-------|----------------|---------------------|---------------|--------|----------------|
|
||||
| **Llama 3.1 8B** | 1/3 | 23.7 | 25.0 (single) | -1.3 | Skip autoreason. Model too weak for diverse candidates. |
|
||||
| **Gemini 2.0 Flash** | 2/3 | 25.0 | 20.0 (single) | +5.0 | Good candidate. Moderate gains. |
|
||||
| **Haiku 3.5** | 3/3 | **42.0** | 33.7 (single) | **+8.3** | **Best candidate.** Perfect scores. Baselines actively destroy quality. |
|
||||
| **Sonnet 4** | 3/5 | 27.8 | 22.4 (C&R) | +5.4 | Good candidate for open tasks. C&R better for technical tasks. |
|
||||
| **Sonnet 4.6 (unconstrained)** | 0/1 | 7.0 | 31.0 (C&R) | -24.0 | Do NOT use autoreason without constraints. |
|
||||
| **Sonnet 4.6 (constrained)** | 2/3 | 29.0 | 27.0 (improve) | +2.0 | Use only with scope constraints. |
|
||||
|
||||
### The Generation-Evaluation Gap
|
||||
|
||||
The core insight: **autoreason's value depends on the gap between a model's generation capability and its self-evaluation capability.**
|
||||
|
||||
```
|
||||
Weak models (Llama 8B):
|
||||
Generation: Poor | Self-evaluation: Poor
|
||||
Gap: Small (both bad) → Autoreason can't help, no diverse candidates
|
||||
|
||||
Mid-tier models (Haiku, Flash):
|
||||
Generation: Decent | Self-evaluation: Poor
|
||||
Gap: LARGE → Autoreason's sweet spot. External eval bridges the gap.
|
||||
|
||||
Strong models (Sonnet 4):
|
||||
Generation: Good | Self-evaluation: Decent
|
||||
Gap: Moderate → Autoreason helps on 3/5 tasks
|
||||
|
||||
Frontier models (Sonnet 4.6):
|
||||
Generation: Excellent | Self-evaluation: Good
|
||||
Gap: Small → Simple methods suffice. Autoreason hurts on unconstrained tasks.
|
||||
```
|
||||
|
||||
**Practical rule**: As model costs drop and capabilities improve, today's frontier becomes tomorrow's mid-tier. The generation-evaluation gap is structural, not temporary. Match refinement architecture to the model's position on the capability curve.
|
||||
|
||||
### Judge Selection
|
||||
|
||||
| Author Model | Recommended Judge | Rationale |
|
||||
|-------------|------------------|-----------|
|
||||
| Llama 8B | Don't use autoreason | Model too weak |
|
||||
| Gemini Flash | Sonnet 4 | Cross-model evaluation works |
|
||||
| Haiku 3.5 | Sonnet 4 | Strong external eval is the mechanism |
|
||||
| Haiku 3.5 | Haiku 3.5 (same) | Still works — tournament structure provides value even without strong judges (20.7 vs 18.3 avg Borda) |
|
||||
| Sonnet 4 | Sonnet 4 (same) | Same-model judges work at this tier |
|
||||
| Sonnet 4.6 | Sonnet 4.6 (same) | Only with scope constraints |
|
||||
|
||||
---
|
||||
|
||||
## Scope Constraint Design
|
||||
|
||||
### What Makes Autoreason Work on Constrained Tasks
|
||||
|
||||
The same model (Sonnet 4.6) goes from **last place** (unconstrained) to **first place** (constrained) with scope constraints. The constraints bound the improvement space so synthesis drift can't accumulate.
|
||||
|
||||
### Effective Constraints
|
||||
|
||||
| Constraint Type | Example | Why It Works |
|
||||
|----------------|---------|-------------|
|
||||
| **Fixed facts** | "Use only these 8 data points, add nothing else" | Bounds information space |
|
||||
| **Fixed deliverable** | "500-word startup pitch" (not "improve this") | Defines done condition |
|
||||
| **Fixed structure** | "Exactly 4 sections, each with 3 numbered items" | Prevents structural drift |
|
||||
| **Fixed change items** | "Address exactly these 3 reviewer concerns" | Bounds modification scope |
|
||||
|
||||
### Ineffective Constraints
|
||||
|
||||
| Constraint | Why It Fails | What Happens |
|
||||
|-----------|-------------|-------------|
|
||||
| Word count alone | Not a scope constraint | False convergence — rejected for length, not quality |
|
||||
| "Be concise" | Too vague | Ignored after 2-3 passes |
|
||||
| "Be comprehensive" | Anti-constraint | Invites scope creep |
|
||||
| No constraints at all | Unbounded improvement space | Synthesis dominates, no convergence |
|
||||
|
||||
### Task Categories
|
||||
|
||||
| Task Type | Autoreason Works? | Why |
|
||||
|-----------|-------------------|-----|
|
||||
| Tasks with genuine tradeoffs (strategy, policy) | Yes | Multiple valid approaches for tournament to select between |
|
||||
| Constrained writing (pitch, memo, postmortem) | Mostly (2/3) | Bounded scope, clear evaluation criteria |
|
||||
| Template-filling (incident postmortem) | No | One correct structure, minimal decision space |
|
||||
| Competitive programming | Yes | Naturally scoped, test suite provides external verification |
|
||||
| Open-ended unconstrained + frontier model | No | Synthesis drift, no convergence |
|
||||
|
||||
---
|
||||
|
||||
## Failure Taxonomy
|
||||
|
||||
| Failure Mode | Condition | Detection | Evidence |
|
||||
|-------------|-----------|-----------|----------|
|
||||
| **Self-correction unreliable** | No external evaluation signal | Baselines degrade below single pass | Haiku baselines: 16.3 avg vs 33.7 single pass |
|
||||
| **Drift / synthesis dominance** | Unconstrained scope | A wins <15%, AB dominates | Sonnet 4.6 unconstrained: A wins 12%, AB wins 60%+ |
|
||||
| **Overfitting to visible feedback** | Shallow revision loop (C&R) | High public/private divergence | C&R overfits 32% on hard code problems |
|
||||
| **No convergence** | Broken judge pipeline | Parsing failures, <3 valid judges | Mixed panel parser failure: 11+ passes |
|
||||
| **Model too weak** | Insufficient generation diversity | All candidates look similar | Llama 8B wins only 1/3 tasks |
|
||||
|
||||
### Recovery Patterns
|
||||
|
||||
| Failure | Recovery |
|
||||
|---------|----------|
|
||||
| No convergence (drift) | Add scope constraints to the task |
|
||||
| No convergence (broken judges) | Fix parser, ensure 3 valid judges before continuing |
|
||||
| Quality degrades with iteration | Switch to single pass or add constraints |
|
||||
| Model too weak | Use a stronger model for generation, keep weak model for cheap roles |
|
||||
| Overfitting (code) | Use structured analysis step, not just test feedback |
|
||||
|
||||
---
|
||||
|
||||
## Code Domain Adaptation
|
||||
|
||||
The autoreason method adapts differently for code vs writing:
|
||||
|
||||
### Writing Domain
|
||||
```
|
||||
Call 1: Critic (find problems in incumbent)
|
||||
Call 2: Author B (revise based on critique)
|
||||
Call 3: Synthesizer (merge A and B)
|
||||
Calls 4-6: Judge Panel (3 blind judges rank A, B, AB)
|
||||
```
|
||||
|
||||
### Code Domain (6-call budget)
|
||||
```
|
||||
Call 1: Initial generation
|
||||
Call 2: Structured analysis (5 points — NO CODE):
|
||||
- Problem analysis: what does the problem actually require?
|
||||
- Approach analysis: what approach did we use, is it correct?
|
||||
- Failure analysis: why did tests fail?
|
||||
- Alternative approaches: what else could work?
|
||||
- Edge cases: what inputs might break the solution?
|
||||
Calls 3-6: Reason-informed revisions
|
||||
- Each revision must explain WHY it fixes the issue
|
||||
- Sees test results from public (visible) test cases
|
||||
```
|
||||
|
||||
**Key difference**: The code strategy replaces the judge panel with test-suite evaluation (objective ground truth). The structured analysis step (Call 2) is what drives recovery — it forces reasoning about *why* the approach failed before attempting fixes.
|
||||
|
||||
**Results**: Recovery is the mechanism. Among problems where both autoreason and single-pass failed initially, autoreason recovered 62% vs single-pass's 43% (McNemar p=0.041, Cohen's h=0.32).
|
||||
|
||||
---
|
||||
|
||||
## Applying Autoreason to Paper Writing
|
||||
|
||||
The paper itself was refined using autoreason (Section 8 of the paper):
|
||||
|
||||
### Setup
|
||||
- Model: claude-opus-4
|
||||
- Judges: 3 Opus judges
|
||||
- Enhancement: Ground-truth critic (access to actual experimental data)
|
||||
- Result: Converged in 9 passes
|
||||
|
||||
### Key Findings for Paper Refinement
|
||||
|
||||
1. **Ground-truth critic is essential**: Without ground-truth access, Opus hallucinated a fabricated ablation study, fake confidence intervals, wrong model names, and incorrect role descriptions. With ground-truth access, the critic caught all four on pass 1.
|
||||
|
||||
2. **Judge panel integrity matters**: A broken parser in one judge (Gemini output format mismatch) reduced the panel from 3 to 2 judges. This prevented convergence for 11+ passes. Fixing to 3 working judges, the same incumbent converged in 2 passes. A broken judge doesn't add noise — it prevents equilibrium.
|
||||
|
||||
### Recommended Setup for Paper Refinement
|
||||
|
||||
```
|
||||
Critic prompt: "You are reviewing a research paper draft. You have access to the
|
||||
actual experimental results [GROUND TRUTH DATA]. Find factual errors, unsupported
|
||||
claims, hallucinated results, and structural problems. Do not suggest fixes."
|
||||
|
||||
Author B prompt: "Revise this paper draft to fix the identified problems. For each
|
||||
change, cite the specific problem it addresses. Do not add claims not supported by
|
||||
the provided experimental data."
|
||||
|
||||
Judge prompt (CoT): "Compare three versions of this paper. For each, evaluate:
|
||||
1. Factual accuracy against the provided results
|
||||
2. Clarity of the narrative and contribution
|
||||
3. Whether claims are properly hedged and supported
|
||||
4. Writing quality (concision, precision, no filler)
|
||||
After reasoning, rank all three. RANKING: [best], [second], [worst]"
|
||||
```
|
||||
|
||||
### What to Provide as Ground Truth
|
||||
- All experimental result JSON files
|
||||
- Statistical test outputs
|
||||
- Raw numbers for every table and figure
|
||||
- Configuration files showing exact hyperparameters
|
||||
- Code that generated the results (for method description accuracy)
|
||||
|
||||
---
|
||||
|
||||
## Compute Budget Reference
|
||||
|
||||
| Method | Calls per Pass | Typical Passes | Total Calls | Relative Cost |
|
||||
|--------|---------------|----------------|-------------|---------------|
|
||||
| Single pass | 1 | 1 | 1 | 1x |
|
||||
| Best-of-N | N | 1 | N | Nx |
|
||||
| Critique & revise | 2 | 15 | 30 | 30x |
|
||||
| Autoreason (in-loop) | ~6 | 10-15 | 60-90 | 60-90x |
|
||||
| Autoreason (with final eval) | ~6 + 7 | 10-15 + 1 | 67-97 | ~80x |
|
||||
|
||||
**Cost-quality tradeoff**: Autoreason uses ~6x more compute per pass and typically runs more passes. This is a real tradeoff. The method trades compute for evaluation quality. On constrained tasks with mid-tier models, this tradeoff is strongly positive. On unconstrained tasks with frontier models, it's negative.
|
||||
|
||||
**CoT judges reduce cost**: 1 CoT judge provides evaluation quality comparable to 3 standard judges, at ~40% cost savings. Always use CoT judges.
|
||||
@@ -0,0 +1,434 @@
|
||||
# Conference Paper Checklists
|
||||
|
||||
This reference documents the mandatory checklist requirements for major ML/AI conferences. All major venues now require paper checklists—missing them results in desk rejection.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
|
||||
- [NeurIPS Paper Checklist](#neurips-paper-checklist)
|
||||
- [ICML Paper Checklist](#icml-paper-checklist)
|
||||
- [ICLR Requirements](#iclr-requirements)
|
||||
- [ACL Requirements](#acl-requirements)
|
||||
- [AAAI Requirements](#aaai-requirements)
|
||||
- [COLM Requirements](#colm-requirements)
|
||||
- [Universal Pre-Submission Checklist](#universal-pre-submission-checklist)
|
||||
|
||||
---
|
||||
|
||||
## NeurIPS Paper Checklist
|
||||
|
||||
### Mandatory Components
|
||||
|
||||
All NeurIPS submissions must include a completed paper checklist. Papers lacking this element face **automatic desk rejection**. The checklist appears after references and supplemental material, outside the page limit.
|
||||
|
||||
### 16 Required Checklist Items
|
||||
|
||||
#### 1. Claims Alignment
|
||||
Authors must verify that abstract and introduction claims match theoretical and experimental results, with clearly stated contributions, assumptions, and limitations.
|
||||
|
||||
**What to check:**
|
||||
- [ ] Abstract claims match actual results
|
||||
- [ ] Introduction doesn't overclaim
|
||||
- [ ] Contributions are specific and falsifiable
|
||||
|
||||
#### 2. Limitations Discussion
|
||||
Papers should include a dedicated "Limitations" section addressing strong assumptions, robustness to violations, scope constraints, and performance-influencing factors.
|
||||
|
||||
**What to include:**
|
||||
- [ ] Dedicated Limitations section
|
||||
- [ ] Honest assessment of scope
|
||||
- [ ] Conditions where method may fail
|
||||
|
||||
#### 3. Theory & Proofs
|
||||
Theoretical contributions require full assumption statements and complete proofs (main paper or appendix with proof sketches for intuition).
|
||||
|
||||
**What to check:**
|
||||
- [ ] All assumptions stated formally
|
||||
- [ ] Complete proofs provided (main text or appendix)
|
||||
- [ ] Proof sketches for intuition in main text
|
||||
|
||||
#### 4. Reproducibility
|
||||
Authors must describe steps ensuring results verification through code release, detailed instructions, model access, or checkpoints appropriate to their contribution type.
|
||||
|
||||
**What to provide:**
|
||||
- [ ] Clear reproducibility statement
|
||||
- [ ] Code availability information
|
||||
- [ ] Model checkpoints if applicable
|
||||
|
||||
#### 5. Data & Code Access
|
||||
Instructions for reproducing main experimental results should be provided (supplemental material or URLs), including exact commands and environment specifications.
|
||||
|
||||
**What to include:**
|
||||
- [ ] Exact commands to run experiments
|
||||
- [ ] Environment specifications (requirements.txt, conda env)
|
||||
- [ ] Data access instructions
|
||||
|
||||
#### 6. Experimental Details
|
||||
Papers must specify training details: data splits, hyperparameters, and selection methods in the main paper or supplementary materials.
|
||||
|
||||
**What to document:**
|
||||
- [ ] Train/val/test split details
|
||||
- [ ] All hyperparameters used
|
||||
- [ ] Hyperparameter selection method
|
||||
|
||||
#### 7. Statistical Significance
|
||||
Results require error bars, confidence intervals, or statistical tests with clearly stated calculation methods and underlying assumptions.
|
||||
|
||||
**What to include:**
|
||||
- [ ] Error bars or confidence intervals
|
||||
- [ ] Number of runs/seeds
|
||||
- [ ] Calculation method (std dev vs std error)
|
||||
|
||||
#### 8. Compute Resources
|
||||
Specifications needed: compute worker types (CPU/GPU), memory, storage, execution time per run, and total project compute requirements.
|
||||
|
||||
**What to document:**
|
||||
- [ ] GPU type and count
|
||||
- [ ] Training time per run
|
||||
- [ ] Total compute used
|
||||
|
||||
#### 9. Ethics Code Compliance
|
||||
Authors confirm adherence to the NeurIPS Code of Ethics, noting any necessary deviations.
|
||||
|
||||
**What to verify:**
|
||||
- [ ] Read NeurIPS Code of Ethics
|
||||
- [ ] Confirm compliance
|
||||
- [ ] Note any deviations with justification
|
||||
|
||||
#### 10. Broader Impacts
|
||||
Discussion of potential negative societal applications, fairness concerns, privacy risks, and possible mitigation strategies when applicable.
|
||||
|
||||
**What to address:**
|
||||
- [ ] Potential negative applications
|
||||
- [ ] Fairness considerations
|
||||
- [ ] Privacy implications
|
||||
- [ ] Mitigation strategies
|
||||
|
||||
#### 11. Safeguards
|
||||
High-risk models (language models, internet-scraped datasets) require controlled release mechanisms and usage guidelines.
|
||||
|
||||
**What to consider:**
|
||||
- [ ] Release strategy for sensitive models
|
||||
- [ ] Usage guidelines if needed
|
||||
- [ ] Access controls if appropriate
|
||||
|
||||
#### 12. License Respect
|
||||
All existing assets require creator citations, license names, URLs, version numbers, and terms-of-service acknowledgment.
|
||||
|
||||
**What to document:**
|
||||
- [ ] Dataset licenses cited
|
||||
- [ ] Code licenses respected
|
||||
- [ ] Version numbers included
|
||||
|
||||
#### 13. Asset Documentation
|
||||
New releases need structured templates documenting training details, limitations, consent procedures, and licensing information.
|
||||
|
||||
**For new datasets/models:**
|
||||
- [ ] Datasheet or model card
|
||||
- [ ] Training data documentation
|
||||
- [ ] Known limitations
|
||||
|
||||
#### 14. Human Subjects
|
||||
Crowdsourcing studies must include participant instructions, screenshots, compensation details, and comply with minimum wage requirements.
|
||||
|
||||
**What to include:**
|
||||
- [ ] Task instructions
|
||||
- [ ] Compensation details
|
||||
- [ ] Time estimates
|
||||
|
||||
#### 15. IRB Approvals
|
||||
Human subjects research requires documented institutional review board approval or equivalent, with risk descriptions disclosed (maintaining anonymity at submission).
|
||||
|
||||
**What to verify:**
|
||||
- [ ] IRB approval obtained
|
||||
- [ ] Risk assessment completed
|
||||
- [ ] Anonymized at submission
|
||||
|
||||
#### 16. LLM Declaration
|
||||
Usage of large language models as core methodology components requires disclosure; writing/editing use doesn't require declaration.
|
||||
|
||||
**What to disclose:**
|
||||
- [ ] LLM used as core methodology component
|
||||
- [ ] How LLM was used
|
||||
- [ ] (Writing assistance doesn't require disclosure)
|
||||
|
||||
### Response Format
|
||||
|
||||
Authors select "yes," "no," or "N/A" per question, with optional 1-2 sentence justifications.
|
||||
|
||||
**Important:** Reviewers are explicitly instructed not to penalize honest limitation acknowledgment.
|
||||
|
||||
---
|
||||
|
||||
## ICML Paper Checklist
|
||||
|
||||
### Broader Impact Statement
|
||||
|
||||
ICML requires a Broader Impact Statement at the end of the paper, before references. This does NOT count toward the page limit.
|
||||
|
||||
**Required elements:**
|
||||
- Potential positive impacts
|
||||
- Potential negative impacts
|
||||
- Mitigation strategies
|
||||
- Who may be affected
|
||||
|
||||
### ICML Specific Requirements
|
||||
|
||||
#### Reproducibility Checklist
|
||||
|
||||
- [ ] Data splits clearly specified
|
||||
- [ ] Hyperparameters listed
|
||||
- [ ] Search ranges documented
|
||||
- [ ] Selection method explained
|
||||
- [ ] Compute resources specified
|
||||
- [ ] Code availability stated
|
||||
|
||||
#### Statistical Reporting
|
||||
|
||||
- [ ] Error bars on all figures
|
||||
- [ ] Standard deviation vs standard error specified
|
||||
- [ ] Number of runs stated
|
||||
- [ ] Significance tests if comparing methods
|
||||
|
||||
#### Anonymization
|
||||
|
||||
- [ ] No author names in paper
|
||||
- [ ] No acknowledgments
|
||||
- [ ] No grant numbers
|
||||
- [ ] Prior work cited in third person
|
||||
- [ ] No identifiable repository URLs
|
||||
|
||||
---
|
||||
|
||||
## ICLR Requirements
|
||||
|
||||
### LLM Disclosure Policy (New for 2026)
|
||||
|
||||
ICLR has a specific LLM disclosure requirement:
|
||||
|
||||
> "If LLMs played a significant role in research ideation and/or writing to the extent that they could be regarded as a contributor, authors must describe their precise role in a separate appendix section."
|
||||
|
||||
**When disclosure is required:**
|
||||
- LLM used for significant research ideation
|
||||
- LLM used for substantial writing
|
||||
- LLM could be considered a contributor
|
||||
|
||||
**When disclosure is NOT required:**
|
||||
- Grammar checking
|
||||
- Minor editing assistance
|
||||
- Code completion tools
|
||||
|
||||
**Consequences of non-disclosure:**
|
||||
- Desk rejection
|
||||
- Potential post-publication issues
|
||||
|
||||
### ICLR Specific Requirements
|
||||
|
||||
#### Reproducibility Statement (Optional but Recommended)
|
||||
|
||||
Add a statement referencing:
|
||||
- Supporting materials
|
||||
- Code availability
|
||||
- Data availability
|
||||
- Model checkpoints
|
||||
|
||||
#### Ethics Statement (Optional)
|
||||
|
||||
Address potential concerns in ≤1 page. Does not count toward page limit.
|
||||
|
||||
#### Reciprocal Reviewing
|
||||
|
||||
- Authors on 3+ papers must serve as reviewers for ≥6 papers
|
||||
- Each submission needs ≥1 author registered to review ≥3 papers
|
||||
|
||||
---
|
||||
|
||||
## ACL Requirements
|
||||
|
||||
### Limitations Section (Mandatory)
|
||||
|
||||
ACL specifically requires a Limitations section:
|
||||
|
||||
**What to include:**
|
||||
- Strong assumptions made
|
||||
- Scope limitations
|
||||
- When method may fail
|
||||
- Generalization concerns
|
||||
|
||||
**Important:** The Limitations section does NOT count toward the page limit.
|
||||
|
||||
### ACL Specific Checklist
|
||||
|
||||
#### Responsible NLP
|
||||
|
||||
- [ ] Bias considerations addressed
|
||||
- [ ] Fairness evaluated if applicable
|
||||
- [ ] Dual-use concerns discussed
|
||||
|
||||
#### Multilingual Considerations
|
||||
|
||||
If applicable:
|
||||
- [ ] Language diversity addressed
|
||||
- [ ] Non-English languages included
|
||||
- [ ] Translation quality verified
|
||||
|
||||
#### Human Evaluation
|
||||
|
||||
If applicable:
|
||||
- [ ] Annotator details provided
|
||||
- [ ] Agreement metrics reported
|
||||
- [ ] Compensation documented
|
||||
|
||||
---
|
||||
|
||||
## AAAI Requirements
|
||||
|
||||
### Formatting (Strictest of All Venues)
|
||||
|
||||
AAAI enforces formatting rules more strictly than any other major venue. Papers that deviate from the template are desk-rejected.
|
||||
|
||||
- [ ] Use the **exact** AAAI style file without modification — no `\setlength`, no `\vspace` hacks, no font overrides
|
||||
- [ ] 7 pages main content (8 for camera-ready with author info)
|
||||
- [ ] Two-column format, Times font (set by template)
|
||||
- [ ] References and appendices do not count toward page limit
|
||||
- [ ] Abstract must be a single paragraph
|
||||
- [ ] Do not modify margins, column widths, or font sizes
|
||||
|
||||
### Required Sections
|
||||
|
||||
- [ ] Abstract (single paragraph, no math or citations)
|
||||
- [ ] Introduction with clear contribution statement
|
||||
- [ ] References in AAAI format (uses `aaai2026.bst`)
|
||||
- [ ] Appendix (optional, unlimited)
|
||||
|
||||
### Ethics and Reproducibility
|
||||
|
||||
- [ ] Broader impact statement (encouraged but not always mandatory — check current year's CFP)
|
||||
- [ ] Reproducibility details (datasets, code availability)
|
||||
- [ ] Acknowledge use of AI writing tools if applicable
|
||||
|
||||
### Key Differences from Other Venues
|
||||
|
||||
- **No separate limitations section required** (unlike ACL), but discussing limitations is recommended
|
||||
- **Strictest formatting enforcement** — the style checker will reject non-compliant PDFs
|
||||
- **No paper checklist** like NeurIPS has, but the universal checklist below still applies
|
||||
- **Unified template** covers main paper and supplementary in the same file
|
||||
|
||||
---
|
||||
|
||||
## COLM Requirements
|
||||
|
||||
### Overview
|
||||
|
||||
COLM (Conference on Language Modeling) focuses specifically on language model research. Framing must target this community.
|
||||
|
||||
### Formatting
|
||||
|
||||
- [ ] 9 pages main content (10 for camera-ready)
|
||||
- [ ] Use COLM template (based on ICLR template with modifications)
|
||||
- [ ] Double-blind review
|
||||
- [ ] References and appendices unlimited
|
||||
|
||||
### Required Sections
|
||||
|
||||
- [ ] Abstract
|
||||
- [ ] Introduction framed for language modeling community
|
||||
- [ ] Conclusion
|
||||
- [ ] References
|
||||
|
||||
### Content Expectations
|
||||
|
||||
- [ ] Contribution must be relevant to language models (broadly interpreted: training, evaluation, applications, theory, alignment, safety)
|
||||
- [ ] If the method is general, frame with language model examples
|
||||
- [ ] Baselines should include recent LM-specific methods where applicable
|
||||
|
||||
### Key Differences from Other Venues
|
||||
|
||||
- **Narrower scope** than NeurIPS/ICML — must frame for LM community
|
||||
- **Template derived from ICLR** — similar formatting rules
|
||||
- **Newer venue** — reviewer norms are still establishing; err on the side of thorough evaluation
|
||||
- **No mandatory checklist** like NeurIPS, but broader impact discussion is expected
|
||||
- **LLM disclosure**: If LLMs were used in research (code generation, data annotation, writing assistance), disclose this
|
||||
|
||||
---
|
||||
|
||||
## Universal Pre-Submission Checklist
|
||||
|
||||
### Before Every Submission
|
||||
|
||||
#### Paper Content
|
||||
|
||||
- [ ] Abstract ≤ word limit (usually 250-300 words)
|
||||
- [ ] Main content within page limit
|
||||
- [ ] References complete and verified
|
||||
- [ ] Limitations section included
|
||||
- [ ] All figures/tables have captions
|
||||
- [ ] Captions are self-contained
|
||||
|
||||
#### Formatting
|
||||
|
||||
- [ ] Correct template used (venue + year specific)
|
||||
- [ ] Margins not modified
|
||||
- [ ] Font sizes not modified
|
||||
- [ ] Double-blind requirements met
|
||||
- [ ] Page numbers (for review) or none (camera-ready)
|
||||
|
||||
#### Technical
|
||||
|
||||
- [ ] All claims supported by evidence
|
||||
- [ ] Error bars included
|
||||
- [ ] Baselines appropriate
|
||||
- [ ] Hyperparameters documented
|
||||
- [ ] Compute resources stated
|
||||
|
||||
#### Reproducibility
|
||||
|
||||
- [ ] Code will be available (or justification)
|
||||
- [ ] Data will be available (or justification)
|
||||
- [ ] Environment documented
|
||||
- [ ] Commands to reproduce provided
|
||||
|
||||
#### Ethics
|
||||
|
||||
- [ ] Broader impacts considered
|
||||
- [ ] Limitations honestly stated
|
||||
- [ ] Licenses respected
|
||||
- [ ] IRB obtained if needed
|
||||
|
||||
#### Final Checks
|
||||
|
||||
- [ ] PDF compiles without errors
|
||||
- [ ] All figures render correctly
|
||||
- [ ] All citations resolve
|
||||
- [ ] Supplementary material organized
|
||||
- [ ] Conference checklist completed
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference: Page Limits
|
||||
|
||||
| Conference | Main Content | References | Appendix |
|
||||
|------------|-------------|------------|----------|
|
||||
| NeurIPS 2025 | 9 pages | Unlimited | Unlimited (checklist separate) |
|
||||
| ICML 2026 | 8 pages (+1 camera) | Unlimited | Unlimited |
|
||||
| ICLR 2026 | 9 pages (+1 camera) | Unlimited | Unlimited |
|
||||
| ACL 2025 | 8 pages (long) | Unlimited | Unlimited |
|
||||
| AAAI 2026 | 7 pages (+1 camera) | Unlimited | Unlimited |
|
||||
| COLM 2025 | 9 pages (+1 camera) | Unlimited | Unlimited |
|
||||
|
||||
---
|
||||
|
||||
## Template Locations
|
||||
|
||||
All conference templates are in the `templates/` directory:
|
||||
|
||||
```
|
||||
templates/
|
||||
├── icml2026/ # ICML 2026 official
|
||||
├── iclr2026/ # ICLR 2026 official
|
||||
├── neurips2025/ # NeurIPS 2025
|
||||
├── acl/ # ACL style files
|
||||
├── aaai2026/ # AAAI 2026
|
||||
└── colm2025/ # COLM 2025
|
||||
```
|
||||
@@ -0,0 +1,564 @@
|
||||
# Citation Management & Hallucination Prevention
|
||||
|
||||
This reference provides a complete workflow for managing citations programmatically, preventing AI-generated citation hallucinations, and maintaining clean bibliographies.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
|
||||
- [Why Citation Verification Matters](#why-citation-verification-matters)
|
||||
- [Citation APIs Overview](#citation-apis-overview)
|
||||
- [Verified Citation Workflow](#verified-citation-workflow)
|
||||
- [Python Implementation](#python-implementation)
|
||||
- [BibTeX Management](#bibtex-management)
|
||||
- [Common Citation Formats](#common-citation-formats)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## Why Citation Verification Matters
|
||||
|
||||
### The Hallucination Problem
|
||||
|
||||
Research has documented significant issues with AI-generated citations:
|
||||
- **~40% error rate** in AI-generated citations (Enago Academy research)
|
||||
- NeurIPS 2025 found **100+ hallucinated citations** slipped through review
|
||||
- Common errors include:
|
||||
- Fabricated paper titles with real author names
|
||||
- Wrong publication venues or years
|
||||
- Non-existent papers with plausible metadata
|
||||
- Incorrect DOIs or arXiv IDs
|
||||
|
||||
### Consequences
|
||||
|
||||
- Desk rejection at some venues
|
||||
- Loss of credibility with reviewers
|
||||
- Potential retraction if published
|
||||
- Wasted time chasing non-existent sources
|
||||
|
||||
### Solution
|
||||
|
||||
**Never generate citations from memory—always verify programmatically.**
|
||||
|
||||
---
|
||||
|
||||
## Citation APIs Overview
|
||||
|
||||
### Primary APIs
|
||||
|
||||
| API | Coverage | Rate Limits | Best For |
|
||||
|-----|----------|-------------|----------|
|
||||
| **Semantic Scholar** | 214M papers | 1 RPS (free key) | ML/AI papers, citation graphs |
|
||||
| **CrossRef** | 140M+ DOIs | Polite pool with mailto | DOI lookup, BibTeX retrieval |
|
||||
| **arXiv** | Preprints | 3-second delays | ML preprints, PDF access |
|
||||
| **OpenAlex** | 240M+ works | 100K/day, 10 RPS | Open alternative to MAG |
|
||||
|
||||
### API Selection Guide
|
||||
|
||||
```
|
||||
Need ML paper search? → Semantic Scholar
|
||||
Have DOI, need BibTeX? → CrossRef content negotiation
|
||||
Looking for preprint? → arXiv API
|
||||
Need open data, bulk access? → OpenAlex
|
||||
```
|
||||
|
||||
### No Official Google Scholar API
|
||||
|
||||
Google Scholar has no official API. Scraping violates ToS. Use SerpApi ($75-275/month) only if Semantic Scholar coverage is insufficient.
|
||||
|
||||
---
|
||||
|
||||
## Verified Citation Workflow
|
||||
|
||||
### 5-Step Process
|
||||
|
||||
```
|
||||
1. SEARCH → Query Semantic Scholar with specific keywords
|
||||
↓
|
||||
2. VERIFY → Confirm paper exists in 2+ sources
|
||||
↓
|
||||
3. RETRIEVE → Get BibTeX via DOI content negotiation
|
||||
↓
|
||||
4. VALIDATE → Confirm the claim appears in source
|
||||
↓
|
||||
5. ADD → Add verified entry to .bib file
|
||||
```
|
||||
|
||||
### Step 1: Search
|
||||
|
||||
Use Semantic Scholar for ML/AI papers:
|
||||
|
||||
```python
|
||||
from semanticscholar import SemanticScholar
|
||||
|
||||
sch = SemanticScholar()
|
||||
results = sch.search_paper("transformer attention mechanism", limit=10)
|
||||
|
||||
for paper in results:
|
||||
print(f"Title: {paper.title}")
|
||||
print(f"Year: {paper.year}")
|
||||
print(f"DOI: {paper.externalIds.get('DOI', 'N/A')}")
|
||||
print(f"arXiv: {paper.externalIds.get('ArXiv', 'N/A')}")
|
||||
print(f"Citation count: {paper.citationCount}")
|
||||
print("---")
|
||||
```
|
||||
|
||||
### Step 2: Verify Existence
|
||||
|
||||
Confirm paper exists in at least two sources:
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
def verify_paper(doi=None, arxiv_id=None, title=None):
|
||||
"""Verify paper exists in multiple sources."""
|
||||
sources_found = []
|
||||
|
||||
# Check Semantic Scholar
|
||||
sch = SemanticScholar()
|
||||
if doi:
|
||||
paper = sch.get_paper(f"DOI:{doi}")
|
||||
if paper:
|
||||
sources_found.append("Semantic Scholar")
|
||||
|
||||
# Check CrossRef (via DOI)
|
||||
if doi:
|
||||
resp = requests.get(f"https://api.crossref.org/works/{doi}")
|
||||
if resp.status_code == 200:
|
||||
sources_found.append("CrossRef")
|
||||
|
||||
# Check arXiv
|
||||
if arxiv_id:
|
||||
resp = requests.get(
|
||||
f"http://export.arxiv.org/api/query?id_list={arxiv_id}"
|
||||
)
|
||||
if "<entry>" in resp.text:
|
||||
sources_found.append("arXiv")
|
||||
|
||||
return len(sources_found) >= 2, sources_found
|
||||
```
|
||||
|
||||
### Step 3: Retrieve BibTeX
|
||||
|
||||
Use DOI content negotiation for guaranteed accuracy:
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
def doi_to_bibtex(doi: str) -> str:
|
||||
"""Get verified BibTeX from DOI via CrossRef content negotiation."""
|
||||
response = requests.get(
|
||||
f"https://doi.org/{doi}",
|
||||
headers={"Accept": "application/x-bibtex"},
|
||||
allow_redirects=True
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.text
|
||||
|
||||
# Example: "Attention Is All You Need"
|
||||
bibtex = doi_to_bibtex("10.48550/arXiv.1706.03762")
|
||||
print(bibtex)
|
||||
```
|
||||
|
||||
### Step 4: Validate Claims
|
||||
|
||||
Before citing a paper for a specific claim, verify the claim exists:
|
||||
|
||||
```python
|
||||
def get_paper_abstract(doi):
|
||||
"""Get abstract to verify claims."""
|
||||
sch = SemanticScholar()
|
||||
paper = sch.get_paper(f"DOI:{doi}")
|
||||
return paper.abstract if paper else None
|
||||
|
||||
# Verify claim appears in abstract
|
||||
abstract = get_paper_abstract("10.48550/arXiv.1706.03762")
|
||||
claim = "attention mechanism"
|
||||
if claim.lower() in abstract.lower():
|
||||
print("Claim appears in paper")
|
||||
```
|
||||
|
||||
### Step 5: Add to Bibliography
|
||||
|
||||
Add verified entry to your .bib file with consistent key format:
|
||||
|
||||
```python
|
||||
def generate_citation_key(bibtex: str) -> str:
|
||||
"""Generate consistent citation key: author_year_firstword."""
|
||||
import re
|
||||
|
||||
# Extract author
|
||||
author_match = re.search(r'author\s*=\s*\{([^}]+)\}', bibtex, re.I)
|
||||
if author_match:
|
||||
first_author = author_match.group(1).split(',')[0].split()[-1]
|
||||
else:
|
||||
first_author = "unknown"
|
||||
|
||||
# Extract year
|
||||
year_match = re.search(r'year\s*=\s*\{?(\d{4})\}?', bibtex, re.I)
|
||||
year = year_match.group(1) if year_match else "0000"
|
||||
|
||||
# Extract title first word
|
||||
title_match = re.search(r'title\s*=\s*\{([^}]+)\}', bibtex, re.I)
|
||||
if title_match:
|
||||
first_word = title_match.group(1).split()[0].lower()
|
||||
first_word = re.sub(r'[^a-z]', '', first_word)
|
||||
else:
|
||||
first_word = "paper"
|
||||
|
||||
return f"{first_author.lower()}_{year}_{first_word}"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Python Implementation
|
||||
|
||||
### Complete Citation Manager Class
|
||||
|
||||
{% raw %}
|
||||
```python
|
||||
"""
|
||||
Citation Manager - Verified citation workflow for ML papers.
|
||||
"""
|
||||
|
||||
import requests
|
||||
import time
|
||||
from typing import Optional, List, Dict, Tuple
|
||||
from dataclasses import dataclass
|
||||
|
||||
try:
|
||||
from semanticscholar import SemanticScholar
|
||||
except ImportError:
|
||||
print("Install: pip install semanticscholar")
|
||||
SemanticScholar = None
|
||||
|
||||
@dataclass
|
||||
class Paper:
|
||||
title: str
|
||||
authors: List[str]
|
||||
year: int
|
||||
doi: Optional[str]
|
||||
arxiv_id: Optional[str]
|
||||
venue: Optional[str]
|
||||
citation_count: int
|
||||
abstract: Optional[str]
|
||||
|
||||
class CitationManager:
|
||||
"""Manage citations with verification."""
|
||||
|
||||
def __init__(self, api_key: Optional[str] = None):
|
||||
self.sch = SemanticScholar(api_key=api_key) if SemanticScholar else None
|
||||
self.verified_papers: Dict[str, Paper] = {}
|
||||
|
||||
def search(self, query: str, limit: int = 10) -> List[Paper]:
|
||||
"""Search for papers using Semantic Scholar."""
|
||||
if not self.sch:
|
||||
raise RuntimeError("Semantic Scholar not available")
|
||||
|
||||
results = self.sch.search_paper(query, limit=limit)
|
||||
papers = []
|
||||
|
||||
for r in results:
|
||||
paper = Paper(
|
||||
title=r.title,
|
||||
authors=[a.name for a in (r.authors or [])],
|
||||
year=r.year or 0,
|
||||
doi=r.externalIds.get('DOI') if r.externalIds else None,
|
||||
arxiv_id=r.externalIds.get('ArXiv') if r.externalIds else None,
|
||||
venue=r.venue,
|
||||
citation_count=r.citationCount or 0,
|
||||
abstract=r.abstract
|
||||
)
|
||||
papers.append(paper)
|
||||
|
||||
return papers
|
||||
|
||||
def verify(self, paper: Paper) -> Tuple[bool, List[str]]:
|
||||
"""Verify paper exists in multiple sources."""
|
||||
sources = []
|
||||
|
||||
# Already found in Semantic Scholar via search
|
||||
sources.append("Semantic Scholar")
|
||||
|
||||
# Check CrossRef if DOI available
|
||||
if paper.doi:
|
||||
try:
|
||||
resp = requests.get(
|
||||
f"https://api.crossref.org/works/{paper.doi}",
|
||||
timeout=10
|
||||
)
|
||||
if resp.status_code == 200:
|
||||
sources.append("CrossRef")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Check arXiv if ID available
|
||||
if paper.arxiv_id:
|
||||
try:
|
||||
resp = requests.get(
|
||||
f"http://export.arxiv.org/api/query?id_list={paper.arxiv_id}",
|
||||
timeout=10
|
||||
)
|
||||
if "<entry>" in resp.text and "<title>" in resp.text:
|
||||
sources.append("arXiv")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return len(sources) >= 2, sources
|
||||
|
||||
def get_bibtex(self, paper: Paper) -> Optional[str]:
|
||||
"""Get BibTeX for verified paper."""
|
||||
if paper.doi:
|
||||
try:
|
||||
resp = requests.get(
|
||||
f"https://doi.org/{paper.doi}",
|
||||
headers={"Accept": "application/x-bibtex"},
|
||||
timeout=10,
|
||||
allow_redirects=True
|
||||
)
|
||||
if resp.status_code == 200:
|
||||
return resp.text
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Fallback: generate from paper data
|
||||
return self._generate_bibtex(paper)
|
||||
|
||||
def _generate_bibtex(self, paper: Paper) -> str:
|
||||
"""Generate BibTeX from paper metadata."""
|
||||
# Generate citation key
|
||||
first_author = paper.authors[0].split()[-1] if paper.authors else "unknown"
|
||||
first_word = paper.title.split()[0].lower().replace(',', '').replace(':', '')
|
||||
key = f"{first_author.lower()}_{paper.year}_{first_word}"
|
||||
|
||||
# Format authors
|
||||
authors = " and ".join(paper.authors) if paper.authors else "Unknown"
|
||||
|
||||
bibtex = f"""@article{{{key},
|
||||
title = {{{paper.title}}},
|
||||
author = {{{authors}}},
|
||||
year = {{{paper.year}}},
|
||||
{'doi = {' + paper.doi + '},' if paper.doi else ''}
|
||||
{'eprint = {' + paper.arxiv_id + '},' if paper.arxiv_id else ''}
|
||||
{'journal = {' + paper.venue + '},' if paper.venue else ''}
|
||||
}}"""
|
||||
return bibtex
|
||||
|
||||
def cite(self, query: str) -> Optional[str]:
|
||||
"""Full workflow: search, verify, return BibTeX."""
|
||||
# Search
|
||||
papers = self.search(query, limit=5)
|
||||
if not papers:
|
||||
return None
|
||||
|
||||
# Take top result
|
||||
paper = papers[0]
|
||||
|
||||
# Verify
|
||||
verified, sources = self.verify(paper)
|
||||
if not verified:
|
||||
print(f"Warning: Could only verify in {sources}")
|
||||
|
||||
# Get BibTeX
|
||||
bibtex = self.get_bibtex(paper)
|
||||
|
||||
# Cache
|
||||
if bibtex:
|
||||
self.verified_papers[paper.title] = paper
|
||||
|
||||
return bibtex
|
||||
|
||||
|
||||
# Usage example
|
||||
if __name__ == "__main__":
|
||||
cm = CitationManager()
|
||||
|
||||
# Search and cite
|
||||
bibtex = cm.cite("attention is all you need transformer")
|
||||
if bibtex:
|
||||
print(bibtex)
|
||||
```
|
||||
{% endraw %}
|
||||
|
||||
### Quick Functions
|
||||
|
||||
```python
|
||||
def quick_cite(query: str) -> str:
|
||||
"""One-liner citation."""
|
||||
cm = CitationManager()
|
||||
return cm.cite(query)
|
||||
|
||||
def batch_cite(queries: List[str], output_file: str = "references.bib"):
|
||||
"""Cite multiple papers and save to file."""
|
||||
cm = CitationManager()
|
||||
bibtex_entries = []
|
||||
|
||||
for query in queries:
|
||||
print(f"Processing: {query}")
|
||||
bibtex = cm.cite(query)
|
||||
if bibtex:
|
||||
bibtex_entries.append(bibtex)
|
||||
time.sleep(1) # Rate limiting
|
||||
|
||||
with open(output_file, 'w') as f:
|
||||
f.write("\n\n".join(bibtex_entries))
|
||||
|
||||
print(f"Saved {len(bibtex_entries)} citations to {output_file}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## BibTeX Management
|
||||
|
||||
### BibTeX vs BibLaTeX
|
||||
|
||||
| Feature | BibTeX | BibLaTeX |
|
||||
|---------|--------|----------|
|
||||
| Unicode support | Limited | Full |
|
||||
| Entry types | Standard | Extended (@online, @dataset) |
|
||||
| Customization | Limited | Highly flexible |
|
||||
| Backend | bibtex | Biber (recommended) |
|
||||
|
||||
**Recommendation**: Use natbib with BibTeX for conference submissions — all major venue templates (NeurIPS, ICML, ICLR, ACL, AAAI, COLM) ship with natbib and `.bst` files. BibLaTeX with Biber is an option for journals or personal projects where you control the template.
|
||||
|
||||
### LaTeX Setup
|
||||
|
||||
```latex
|
||||
% In preamble
|
||||
\usepackage[
|
||||
backend=biber,
|
||||
style=numeric,
|
||||
sorting=none
|
||||
]{biblatex}
|
||||
\addbibresource{references.bib}
|
||||
|
||||
% In document
|
||||
\cite{vaswani_2017_attention}
|
||||
|
||||
% At end
|
||||
\printbibliography
|
||||
```
|
||||
|
||||
### Citation Commands
|
||||
|
||||
```latex
|
||||
\cite{key} % Numeric: [1]
|
||||
\citep{key} % Parenthetical: (Author, 2020)
|
||||
\citet{key} % Textual: Author (2020)
|
||||
\citeauthor{key} % Just author name
|
||||
\citeyear{key} % Just year
|
||||
```
|
||||
|
||||
### Consistent Citation Keys
|
||||
|
||||
Use format: `author_year_firstword`
|
||||
|
||||
```
|
||||
vaswani_2017_attention
|
||||
devlin_2019_bert
|
||||
brown_2020_language
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Citation Formats
|
||||
|
||||
### Conference Paper
|
||||
|
||||
```bibtex
|
||||
@inproceedings{vaswani_2017_attention,
|
||||
title = {Attention Is All You Need},
|
||||
author = {Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and
|
||||
Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and
|
||||
Kaiser, Lukasz and Polosukhin, Illia},
|
||||
booktitle = {Advances in Neural Information Processing Systems},
|
||||
volume = {30},
|
||||
year = {2017},
|
||||
publisher = {Curran Associates, Inc.}
|
||||
}
|
||||
```
|
||||
|
||||
### Journal Article
|
||||
|
||||
```bibtex
|
||||
@article{hochreiter_1997_long,
|
||||
title = {Long Short-Term Memory},
|
||||
author = {Hochreiter, Sepp and Schmidhuber, J{\"u}rgen},
|
||||
journal = {Neural Computation},
|
||||
volume = {9},
|
||||
number = {8},
|
||||
pages = {1735--1780},
|
||||
year = {1997},
|
||||
publisher = {MIT Press}
|
||||
}
|
||||
```
|
||||
|
||||
### arXiv Preprint
|
||||
|
||||
```bibtex
|
||||
@misc{brown_2020_language,
|
||||
title = {Language Models are Few-Shot Learners},
|
||||
author = {Brown, Tom and Mann, Benjamin and Ryder, Nick and others},
|
||||
year = {2020},
|
||||
eprint = {2005.14165},
|
||||
archiveprefix = {arXiv},
|
||||
primaryclass = {cs.CL}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Issue: Semantic Scholar returns no results**
|
||||
- Try more specific keywords
|
||||
- Check spelling of author names
|
||||
- Use quotation marks for exact phrases
|
||||
|
||||
**Issue: DOI doesn't resolve to BibTeX**
|
||||
- DOI may be registered but not linked to CrossRef
|
||||
- Try arXiv ID instead if available
|
||||
- Generate BibTeX from metadata manually
|
||||
|
||||
**Issue: Rate limiting errors**
|
||||
- Add delays between requests (1-3 seconds)
|
||||
- Use API key if available
|
||||
- Cache results to avoid repeat queries
|
||||
|
||||
**Issue: Encoding problems in BibTeX**
|
||||
- Use proper LaTeX escaping: `{\"u}` for ü
|
||||
- Ensure file is UTF-8 encoded
|
||||
- Use BibLaTeX with Biber for better Unicode
|
||||
|
||||
### Verification Checklist
|
||||
|
||||
Before adding a citation:
|
||||
|
||||
- [ ] Paper found in at least 2 sources
|
||||
- [ ] DOI or arXiv ID verified
|
||||
- [ ] BibTeX retrieved (not generated from memory)
|
||||
- [ ] Entry type correct (@inproceedings vs @article)
|
||||
- [ ] Author names complete and correctly formatted
|
||||
- [ ] Year and venue verified
|
||||
- [ ] Citation key follows consistent format
|
||||
|
||||
---
|
||||
|
||||
## Additional Resources
|
||||
|
||||
**APIs:**
|
||||
- Semantic Scholar: https://api.semanticscholar.org/api-docs/
|
||||
- CrossRef: https://www.crossref.org/documentation/retrieve-metadata/rest-api/
|
||||
- arXiv: https://info.arxiv.org/help/api/basics.html
|
||||
- OpenAlex: https://docs.openalex.org/
|
||||
|
||||
**Python Libraries:**
|
||||
- `semanticscholar`: https://pypi.org/project/semanticscholar/
|
||||
- `arxiv`: https://pypi.org/project/arxiv/
|
||||
- `habanero` (CrossRef): https://github.com/sckott/habanero
|
||||
|
||||
**Verification Tools:**
|
||||
- Citely: https://citely.ai/citation-checker
|
||||
- ReciteWorks: https://reciteworks.com/
|
||||
@@ -0,0 +1,728 @@
|
||||
# Experiment Design Patterns
|
||||
|
||||
Patterns and best practices distilled from running research experiments at scale with the Hermes agent. These cover experiment infrastructure, evaluation protocols, monitoring, and failure recovery.
|
||||
|
||||
---
|
||||
|
||||
## Experiment Infrastructure
|
||||
|
||||
### Directory Structure
|
||||
|
||||
Organize experiments with a consistent structure:
|
||||
|
||||
```
|
||||
workspace/
|
||||
experiments/
|
||||
run_main.py # Core experiment runner
|
||||
run_baselines.py # Baseline comparison
|
||||
run_ablation.py # Ablation studies
|
||||
strategies.py # Method implementations
|
||||
config.yaml # Shared configuration
|
||||
results/
|
||||
<experiment_name>/
|
||||
<task_or_problem>/
|
||||
<strategy>/
|
||||
result.json # Final metrics
|
||||
final_output.md # Final output artifact
|
||||
history.json # Full trajectory/log
|
||||
pass_01/ # Per-iteration artifacts (if iterative)
|
||||
intermediate.md
|
||||
analysis/
|
||||
analyze_results.py # Statistical analysis
|
||||
compute_stats.py # Significance tests
|
||||
make_charts.py # Visualization
|
||||
paper/
|
||||
paper.tex # LaTeX source
|
||||
fig_*.pdf # Generated figures
|
||||
```
|
||||
|
||||
### Script Design Principles
|
||||
|
||||
**1. Incremental Saving (Crash Recovery)**
|
||||
|
||||
Every experiment script should save results after each unit of work, and skip already-completed work on restart:
|
||||
|
||||
```python
|
||||
import json, os
|
||||
from pathlib import Path
|
||||
|
||||
def run_experiment(problems, strategies, output_dir):
|
||||
for problem in problems:
|
||||
for strategy in strategies:
|
||||
result_path = Path(output_dir) / problem["id"] / strategy / "result.json"
|
||||
if result_path.exists():
|
||||
print(f"Skipping {problem['id']}/{strategy} (already done)")
|
||||
continue
|
||||
|
||||
# Run the experiment
|
||||
result = execute_strategy(problem, strategy)
|
||||
|
||||
# Save immediately
|
||||
result_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(result_path, 'w') as f:
|
||||
json.dump(result, f, indent=2)
|
||||
```
|
||||
|
||||
This pattern makes re-runs safe and efficient. If a process crashes at problem 47/150, restarting skips the first 46.
|
||||
|
||||
**2. Artifact Preservation**
|
||||
|
||||
Save all intermediate outputs, not just final results. This enables post-hoc analysis without re-running:
|
||||
|
||||
```python
|
||||
def save_pass_artifacts(output_dir, pass_num, artifacts):
|
||||
"""Save all artifacts from a single pass of an iterative method."""
|
||||
pass_dir = Path(output_dir) / f"pass_{pass_num:02d}"
|
||||
pass_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for name, content in artifacts.items():
|
||||
with open(pass_dir / f"{name}.md", 'w') as f:
|
||||
f.write(content)
|
||||
```
|
||||
|
||||
**3. Configuration Management**
|
||||
|
||||
Use YAML configs for reproducibility:
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
model: anthropic/claude-sonnet-4-20250514
|
||||
author_temperature: 0.8
|
||||
judge_temperature: 0.3
|
||||
max_tokens: 4096
|
||||
num_judges: 3
|
||||
max_passes: 15
|
||||
convergence_k: 2
|
||||
```
|
||||
|
||||
```python
|
||||
import yaml
|
||||
|
||||
with open("config.yaml") as f:
|
||||
config = yaml.safe_load(f)
|
||||
```
|
||||
|
||||
**4. Separation of Concerns**
|
||||
|
||||
Keep generation, evaluation, and visualization in separate scripts:
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `run_experiment.py` | Core method execution |
|
||||
| `run_baselines.py` | Baseline comparisons at same compute |
|
||||
| `run_eval.py` | Blind evaluation / judge panels |
|
||||
| `analyze_results.py` | Statistical analysis |
|
||||
| `make_charts.py` | Figure generation |
|
||||
|
||||
This lets you re-run evaluation without re-running expensive generation, and regenerate figures without re-running analysis.
|
||||
|
||||
---
|
||||
|
||||
## Evaluation Protocols
|
||||
|
||||
### Blind Judge Panels (for Subjective Tasks)
|
||||
|
||||
When evaluating subjective outputs (writing, analysis, recommendations), use a blind judge panel:
|
||||
|
||||
```python
|
||||
import random
|
||||
|
||||
def run_blind_evaluation(outputs: dict, task_prompt: str, num_judges: int = 7):
|
||||
"""
|
||||
Run blind evaluation of multiple method outputs.
|
||||
|
||||
Args:
|
||||
outputs: {"method_name": "output_text", ...}
|
||||
task_prompt: The original task description
|
||||
num_judges: Number of independent judge evaluations
|
||||
"""
|
||||
rankings = []
|
||||
|
||||
for judge_i in range(num_judges):
|
||||
# Randomize labels and presentation order per judge
|
||||
methods = list(outputs.keys())
|
||||
random.shuffle(methods)
|
||||
labels = {m: chr(65 + i) for i, m in enumerate(methods)} # A, B, C...
|
||||
|
||||
# Present to judge with randomized labels
|
||||
prompt = f"Task: {task_prompt}\n\n"
|
||||
for method in methods:
|
||||
prompt += f"--- Proposal {labels[method]} ---\n{outputs[method]}\n\n"
|
||||
prompt += "Rank all proposals from best to worst. Format: RANKING: [best], [second], [worst]"
|
||||
|
||||
ranking = call_judge(prompt)
|
||||
rankings.append({"labels": labels, "ranking": ranking})
|
||||
|
||||
# Aggregate via Borda count
|
||||
return compute_borda(rankings)
|
||||
|
||||
def compute_borda(rankings, n_methods=3):
|
||||
"""Borda count: 3/2/1 points for 1st/2nd/3rd."""
|
||||
scores = {}
|
||||
points = {0: n_methods, 1: n_methods - 1, 2: n_methods - 2} # Adjust for n_methods
|
||||
|
||||
for r in rankings:
|
||||
for position, method in enumerate(r["ranking"]):
|
||||
scores[method] = scores.get(method, 0) + points.get(position, 0)
|
||||
|
||||
return scores
|
||||
```
|
||||
|
||||
Key design decisions:
|
||||
- **Randomize both labels AND order** per judge to prevent position bias
|
||||
- **Use odd number of judges** (3, 5, 7) to break ties
|
||||
- **Conservative tiebreak**: Incumbent/baseline wins ties (prevents false positives)
|
||||
- **CoT judges** match non-CoT quality at ~40% cost (1 CoT judge ≈ 3 standard judges)
|
||||
|
||||
### Code/Objective Evaluation
|
||||
|
||||
For tasks with ground-truth evaluation (code, math, factual):
|
||||
|
||||
```python
|
||||
import subprocess
|
||||
|
||||
def evaluate_code(solution: str, test_cases: list, timeout: int = 30):
|
||||
"""Run code solution against test cases with sandboxed execution."""
|
||||
results = {"public": [], "private": []}
|
||||
|
||||
for test in test_cases:
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
["python3", "-c", solution],
|
||||
input=test["input"],
|
||||
capture_output=True,
|
||||
timeout=timeout,
|
||||
text=True
|
||||
)
|
||||
actual = proc.stdout.strip()
|
||||
expected = test["expected"].strip()
|
||||
passed = actual == expected
|
||||
except subprocess.TimeoutExpired:
|
||||
passed = False
|
||||
|
||||
category = "public" if test.get("public") else "private"
|
||||
results[category].append(passed)
|
||||
|
||||
return {
|
||||
"public_pass_rate": sum(results["public"]) / max(len(results["public"]), 1),
|
||||
"private_pass_rate": sum(results["private"]) / max(len(results["private"]), 1),
|
||||
}
|
||||
```
|
||||
|
||||
### Compute-Matched Comparison
|
||||
|
||||
Always compare methods at equal compute budget. If your method uses N API calls, baselines get N calls too:
|
||||
|
||||
| Method | Call Budget | Allocation |
|
||||
|--------|-----------|------------|
|
||||
| Single pass | 6 calls | 6 independent generations |
|
||||
| Critique & revise | 6 calls | 1 generate + 5 revise rounds |
|
||||
| Autoreason | 6 calls | 1 generate + 1 analysis + 4 revisions |
|
||||
| Best-of-N | 6 calls | 6 independent, pick best on public test |
|
||||
|
||||
### Human Evaluation Design
|
||||
|
||||
Many ML/NLP papers require human evaluation, especially for subjective tasks (text generation, summarization, dialogue, creative writing). Poorly designed human evals are a common rejection reason.
|
||||
|
||||
#### When Human Evaluation Is Required
|
||||
|
||||
| Task Type | Required? | Notes |
|
||||
|-----------|-----------|-------|
|
||||
| Text generation (open-ended) | Yes | LLM-as-judge alone is insufficient for acceptance at ACL/EMNLP |
|
||||
| Summarization | Usually | At minimum for a subset of outputs |
|
||||
| Dialogue systems | Yes | User studies or annotation |
|
||||
| Code generation | No | Test suites are objective ground truth |
|
||||
| Classification | No | Standard metrics suffice |
|
||||
| Any task with subjective quality | Strongly recommended | Strengthens the paper significantly |
|
||||
|
||||
#### Annotation Protocol Design
|
||||
|
||||
```
|
||||
Human Evaluation Protocol:
|
||||
1. Define the evaluation dimensions (fluency, relevance, factual accuracy, etc.)
|
||||
2. Create annotation guidelines with examples of each score level
|
||||
3. Run a pilot with 2-3 annotators on 20-30 examples
|
||||
4. Compute pilot inter-annotator agreement — if low, revise guidelines
|
||||
5. Run full evaluation
|
||||
6. Report: annotator count, agreement metrics, compensation, time per item
|
||||
```
|
||||
|
||||
**Evaluation dimensions** (pick relevant subset):
|
||||
|
||||
| Dimension | Definition | Scale |
|
||||
|-----------|-----------|-------|
|
||||
| Fluency | Grammaticality and naturalness | 1-5 Likert |
|
||||
| Relevance | Does it address the task? | 1-5 Likert |
|
||||
| Factual accuracy | Are stated facts correct? | Binary or 1-5 |
|
||||
| Coherence | Logical flow and consistency | 1-5 Likert |
|
||||
| Informativeness | Does it provide useful information? | 1-5 Likert |
|
||||
| Overall preference | Which output is better? | A/B/Tie (pairwise) |
|
||||
|
||||
**Pairwise comparison** (preferred over absolute scoring — more reliable):
|
||||
- Present two outputs side-by-side (randomize left/right position)
|
||||
- Ask: "Which is better? A / B / Tie"
|
||||
- More discriminative and less susceptible to annotator calibration drift
|
||||
|
||||
#### Inter-Annotator Agreement
|
||||
|
||||
Always report agreement metrics. Without them, reviewers assume your annotations are unreliable.
|
||||
|
||||
```python
|
||||
# Krippendorff's alpha (preferred — handles missing data, any scale)
|
||||
# pip install krippendorffs-alpha
|
||||
import krippendorff
|
||||
|
||||
# Ratings: rows = annotators, columns = items, values = scores
|
||||
ratings = [
|
||||
[3, 4, 1, 2, 5, None, 3], # Annotator 1
|
||||
[3, 5, 1, 3, 5, 2, 3], # Annotator 2
|
||||
[4, 4, 2, 2, 4, 2, None], # Annotator 3
|
||||
]
|
||||
alpha = krippendorff.alpha(reliability_data=ratings, level_of_measurement="ordinal")
|
||||
print(f"Krippendorff's alpha: {alpha:.3f}")
|
||||
# Interpretation: >0.80 good, 0.67-0.80 acceptable, <0.67 questionable
|
||||
```
|
||||
|
||||
```python
|
||||
# Cohen's kappa (for exactly 2 annotators, categorical data)
|
||||
from sklearn.metrics import cohen_kappa_score
|
||||
|
||||
annotator_1 = [1, 2, 3, 1, 2, 3, 2]
|
||||
annotator_2 = [1, 2, 2, 1, 3, 3, 2]
|
||||
kappa = cohen_kappa_score(annotator_1, annotator_2)
|
||||
print(f"Cohen's kappa: {kappa:.3f}")
|
||||
# Interpretation: >0.80 excellent, 0.60-0.80 substantial, 0.40-0.60 moderate
|
||||
```
|
||||
|
||||
| Metric | When to Use | Annotators | Scale |
|
||||
|--------|------------|-----------|-------|
|
||||
| Krippendorff's alpha | Default choice | Any number | Any (ordinal, nominal, ratio) |
|
||||
| Cohen's kappa | 2 annotators, categorical | Exactly 2 | Nominal/ordinal |
|
||||
| Fleiss' kappa | 3+ annotators, categorical | 3+ | Nominal |
|
||||
| Pearson/Spearman | Continuous scores | 2 | Interval/ratio |
|
||||
|
||||
#### Crowdsourcing Platforms
|
||||
|
||||
| Platform | Best For | Cost | Quality |
|
||||
|----------|----------|------|---------|
|
||||
| **Prolific** | Academic research, higher quality | $8-15/hr | High — academic participant pool |
|
||||
| **MTurk** | Large-scale, fast turnaround | $2-10/hr | Variable — use qualifications |
|
||||
| **Surge AI** | NLP-specific annotations | Premium | High — trained annotators |
|
||||
| **Expert annotators** | Domain-specific (medical, legal) | Highest | Highest — but slow |
|
||||
|
||||
**Ethics requirements**:
|
||||
- Report compensation rate (must be at minimum local minimum wage)
|
||||
- Describe annotator demographics if relevant
|
||||
- Obtain IRB/ethics approval if required by your institution
|
||||
- ACL venues explicitly require compensation documentation
|
||||
|
||||
#### What to Report in the Paper
|
||||
|
||||
```
|
||||
Human Evaluation Section Checklist:
|
||||
- [ ] Number of annotators
|
||||
- [ ] Annotator qualifications / recruitment method
|
||||
- [ ] Number of items evaluated
|
||||
- [ ] Evaluation dimensions with definitions
|
||||
- [ ] Scale used (Likert, pairwise, binary)
|
||||
- [ ] Inter-annotator agreement (Krippendorff's alpha or Cohen's kappa)
|
||||
- [ ] Compensation rate
|
||||
- [ ] Time per annotation item
|
||||
- [ ] Whether annotators saw model identities (should be blind)
|
||||
- [ ] Randomization of presentation order
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Statistical Analysis
|
||||
|
||||
### Required Tests
|
||||
|
||||
| Test | When to Use | Python |
|
||||
|------|------------|--------|
|
||||
| McNemar's test | Comparing two methods on same problems | `scipy.stats.binomtest` for small n |
|
||||
| Two-proportion z-test | Comparing success rates | Custom or `statsmodels` |
|
||||
| Fisher's exact test | Small sample pairwise comparison | `scipy.stats.fisher_exact` |
|
||||
| Bootstrapped CI | Confidence intervals for any metric | Custom bootstrap |
|
||||
| Cohen's h | Effect size for proportions | Manual calculation |
|
||||
|
||||
### Standard Analysis Script
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from scipy import stats
|
||||
from pathlib import Path
|
||||
import json
|
||||
|
||||
def load_all_results(results_dir):
|
||||
"""Load all results into a structured format."""
|
||||
results = {}
|
||||
for result_file in Path(results_dir).rglob("result.json"):
|
||||
parts = result_file.relative_to(results_dir).parts
|
||||
if len(parts) >= 3:
|
||||
experiment, task, strategy = parts[0], parts[1], parts[2]
|
||||
data = json.loads(result_file.read_text())
|
||||
results.setdefault(experiment, {}).setdefault(strategy, {})[task] = data
|
||||
return results
|
||||
|
||||
def pairwise_mcnemar(method_a_results, method_b_results):
|
||||
"""McNemar's test for paired binary outcomes."""
|
||||
a_win_b_lose = sum(1 for a, b in zip(method_a_results, method_b_results) if a and not b)
|
||||
b_win_a_lose = sum(1 for a, b in zip(method_a_results, method_b_results) if b and not a)
|
||||
|
||||
n = a_win_b_lose + b_win_a_lose
|
||||
if n < 25:
|
||||
# Use exact binomial for small samples
|
||||
result = stats.binomtest(a_win_b_lose, n, 0.5)
|
||||
p_value = result.pvalue
|
||||
else:
|
||||
# Chi-squared approximation
|
||||
chi2 = (abs(a_win_b_lose - b_win_a_lose) - 1)**2 / (a_win_b_lose + b_win_a_lose)
|
||||
p_value = 1 - stats.chi2.cdf(chi2, df=1)
|
||||
|
||||
return {
|
||||
"a_wins": a_win_b_lose,
|
||||
"b_wins": b_win_a_lose,
|
||||
"n_discordant": n,
|
||||
"p_value": p_value,
|
||||
"significant": p_value < 0.05
|
||||
}
|
||||
|
||||
def bootstrap_ci(data, n_bootstrap=10000, ci=0.95):
|
||||
"""Bootstrap confidence interval for mean."""
|
||||
means = []
|
||||
for _ in range(n_bootstrap):
|
||||
sample = np.random.choice(data, size=len(data), replace=True)
|
||||
means.append(np.mean(sample))
|
||||
lower = np.percentile(means, (1 - ci) / 2 * 100)
|
||||
upper = np.percentile(means, (1 + ci) / 2 * 100)
|
||||
return {"mean": np.mean(data), "ci_lower": lower, "ci_upper": upper}
|
||||
|
||||
def cohens_h(p1, p2):
|
||||
"""Cohen's h effect size for two proportions."""
|
||||
return 2 * np.arcsin(np.sqrt(p1)) - 2 * np.arcsin(np.sqrt(p2))
|
||||
```
|
||||
|
||||
### Reporting Standards
|
||||
|
||||
Always include in the paper:
|
||||
- **Sample sizes**: n=X problems/tasks
|
||||
- **Number of runs**: K independent runs if applicable
|
||||
- **Error bars**: Specify standard deviation or standard error
|
||||
- **Confidence intervals**: 95% CI for key results
|
||||
- **Significance tests**: p-values for key comparisons
|
||||
- **Effect sizes**: Cohen's d or h for practical significance
|
||||
|
||||
---
|
||||
|
||||
## Monitoring (Cron Pattern)
|
||||
|
||||
### Cron Prompt Template
|
||||
|
||||
For each experiment batch, create a monitoring prompt:
|
||||
|
||||
```
|
||||
Check the status of the [EXPERIMENT_NAME] experiment:
|
||||
|
||||
1. Process check: ps aux | grep [PROCESS_PATTERN]
|
||||
2. Log check: tail -30 [LOG_FILE]
|
||||
3. Results check: ls [RESULT_DIR]/eval/ (or appropriate result location)
|
||||
4. If results are available:
|
||||
- Read the result JSON files
|
||||
- Report metrics in a table (Borda scores, accuracy, etc.)
|
||||
- Compute key comparisons between methods
|
||||
5. If all experiments in this batch are complete:
|
||||
- git add -A && git commit -m "[COMMIT_MESSAGE]" && git push
|
||||
- Report final summary
|
||||
6. Key question: [SPECIFIC ANALYTICAL QUESTION]
|
||||
|
||||
If nothing has changed since the last check, respond with [SILENT].
|
||||
```
|
||||
|
||||
### Monitoring Best Practices
|
||||
|
||||
1. **Check processes first** — don't read results if the experiment is still running and results are incomplete
|
||||
2. **Read the log tail** — look for errors, progress indicators, completion messages
|
||||
3. **Count completed vs expected** — "45/150 problems done" is more useful than "some results exist"
|
||||
4. **Report in structured tables** — always include key metrics in a table
|
||||
5. **Answer the key question** — each experiment should have a specific analytical question to answer when done
|
||||
6. **[SILENT] for no-news** — suppress notifications when nothing has changed
|
||||
7. **Commit on completion** — every completed batch gets committed with a descriptive message
|
||||
|
||||
### Example Monitoring Report
|
||||
|
||||
```
|
||||
## Code Experiments (Haiku 3.5) - COMPLETE
|
||||
|
||||
| Strategy | Pass Rate (150 problems) | vs Single |
|
||||
|----------|------------------------|-----------|
|
||||
| single_pass | 38.0% | — |
|
||||
| critique_revise | 35.2% | -2.8pp |
|
||||
| **autoreason** | **40.0%** | **+2.0pp** |
|
||||
| best_of_6 | 31.0% | -7.0pp |
|
||||
|
||||
Key finding: Autoreason shows +2pp improvement over single pass, while
|
||||
best-of-6 collapses due to single-public-test selection issue.
|
||||
|
||||
Committed: `git commit -m "Add Haiku code results (150 problems, 4 strategies)"`
|
||||
Next: Run significance tests on these results.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Failure Recovery
|
||||
|
||||
### Common Failures and Recovery
|
||||
|
||||
| Failure | Detection | Recovery |
|
||||
|---------|-----------|----------|
|
||||
| **API credit exhaustion** | 402 errors in logs, incomplete results | Top up credits, re-run (skips completed work automatically) |
|
||||
| **Rate limiting** | 429 errors, slow progress | Add retry logic with exponential backoff |
|
||||
| **Process crash** | PID gone, log stops mid-problem | Re-run script (resumes from last checkpoint) |
|
||||
| **Wrong model ID** | Model not found errors | Fix ID (e.g., `claude-opus-4-6` not `claude-opus-4.6`) |
|
||||
| **Parallel slowdown** | Each experiment taking 2x longer | Reduce parallel experiments to 2-3 max |
|
||||
| **Security scan blocks** | Commands blocked by security | Use `execute_code` instead of piped `terminal` commands |
|
||||
| **Delegation failures** | `delegate_task` returns errors | Fall back to doing work directly |
|
||||
| **Timeout on hard problems** | Process stuck, no log progress | Kill, skip problem, note in results |
|
||||
| **Dataset path mismatch** | File not found errors | Verify paths before launching |
|
||||
|
||||
### Retry Naming Convention
|
||||
|
||||
When re-running failed experiments, use a suffix to track rounds:
|
||||
|
||||
```
|
||||
logs/experiment_haiku_0_50.log # Round 1
|
||||
logs/experiment_haiku_0_50_r2.log # Round 2 (after credit exhaustion)
|
||||
logs/experiment_haiku_0_50_r3.log # Round 3 (after bug fix)
|
||||
```
|
||||
|
||||
### Pre-Flight Checklist
|
||||
|
||||
Before launching any experiment batch:
|
||||
|
||||
```
|
||||
Pre-Flight:
|
||||
- [ ] API credits sufficient for estimated calls
|
||||
- [ ] Model IDs correct (test with 1 problem first)
|
||||
- [ ] Output directory exists and is writable
|
||||
- [ ] Resume logic works (re-run won't overwrite existing results)
|
||||
- [ ] Log file path is unique (won't overwrite previous logs)
|
||||
- [ ] Dataset/task files are accessible
|
||||
- [ ] Config matches intended experiment
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task/Benchmark Design
|
||||
|
||||
### Open-Ended Tasks (Subjective Evaluation)
|
||||
|
||||
Design tasks that have clear objectives but subjective quality:
|
||||
|
||||
```markdown
|
||||
# Task: [Title]
|
||||
|
||||
## Context
|
||||
[Specific scenario with concrete details: company size, constraints, timeline]
|
||||
|
||||
## Deliverable
|
||||
[Exact format and structure required]
|
||||
|
||||
## Requirements
|
||||
- [Specific, measurable requirements]
|
||||
- [Not vague — "be comprehensive" is bad, "include exactly 6 sections" is good]
|
||||
```
|
||||
|
||||
### Constrained Tasks (for Testing Scope Effects)
|
||||
|
||||
Constrained tasks test whether methods respect scope boundaries. Design with:
|
||||
|
||||
- **Fixed facts**: "Use only these N data points, add nothing else"
|
||||
- **Fixed deliverable**: Specific format (pitch, postmortem, memo — not "improve this")
|
||||
- **Fixed structure**: "These sections in this order, do not add/remove"
|
||||
- **Fixed change items**: "Address exactly these N points, nothing else"
|
||||
|
||||
**Do NOT use word count as a scope constraint.** Word limits cause false convergence — outputs get rejected for length, not quality. Constrain scope (what to include) not length.
|
||||
|
||||
### Example: Good vs Bad Constraints
|
||||
|
||||
| Bad Constraint | Why | Good Constraint |
|
||||
|---------------|-----|-----------------|
|
||||
| "Max 500 words" | Judges reject for length | "Exactly 4 sections, each with 3 numbered items" |
|
||||
| "Be concise" | Too vague | "Each prohibition must reference a specific base fact" |
|
||||
| "Improve this" | Unbounded scope | "Write a 600-word incident postmortem with this exact structure" |
|
||||
| "Make it better" | No clear criterion | "Address exactly these 3 reviewer concerns" |
|
||||
|
||||
---
|
||||
|
||||
## Visualization Best Practices
|
||||
|
||||
### Setup: SciencePlots + matplotlib
|
||||
|
||||
Install SciencePlots for publication-ready defaults:
|
||||
|
||||
```bash
|
||||
pip install SciencePlots matplotlib numpy
|
||||
```
|
||||
|
||||
**Option A: SciencePlots styles** (recommended — handles most defaults automatically):
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import scienceplots # registers the styles
|
||||
|
||||
# Pick a style:
|
||||
# 'science' — clean, serif fonts, suitable for most venues
|
||||
# 'science+ieee' — IEEE-style (good for two-column papers)
|
||||
# 'science+nature' — Nature-style
|
||||
# Add 'no-latex' if LaTeX is not installed on the machine generating plots
|
||||
|
||||
with plt.style.context(['science', 'no-latex']):
|
||||
fig, ax = plt.subplots(figsize=(3.5, 2.5)) # single-column width
|
||||
# ... plot ...
|
||||
fig.savefig('paper/fig_results.pdf', bbox_inches='tight')
|
||||
```
|
||||
|
||||
**Option B: Manual rcParams** (when you need full control):
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
plt.rcParams.update({
|
||||
'font.size': 10,
|
||||
'font.family': 'serif',
|
||||
'axes.labelsize': 11,
|
||||
'axes.titlesize': 11,
|
||||
'xtick.labelsize': 9,
|
||||
'ytick.labelsize': 9,
|
||||
'legend.fontsize': 9,
|
||||
'figure.figsize': (3.5, 2.5), # single-column default
|
||||
'figure.dpi': 300,
|
||||
'savefig.dpi': 300,
|
||||
'savefig.bbox': 'tight',
|
||||
'savefig.pad_inches': 0.05,
|
||||
'axes.linewidth': 0.8,
|
||||
'lines.linewidth': 1.5,
|
||||
'lines.markersize': 5,
|
||||
'axes.grid': True,
|
||||
'grid.alpha': 0.3,
|
||||
'grid.linewidth': 0.5,
|
||||
})
|
||||
```
|
||||
|
||||
### Standard Figure Sizes (Two-Column Format)
|
||||
|
||||
| Use Case | figsize | Notes |
|
||||
|----------|---------|-------|
|
||||
| Single column | `(3.5, 2.5)` | Fits in one column of two-column layout |
|
||||
| Double column | `(7.0, 3.0)` | Spans full page width |
|
||||
| Square (heatmap, confusion matrix) | `(3.5, 3.5)` | Single column |
|
||||
| Tall single (many rows) | `(3.5, 5.0)` | Use sparingly |
|
||||
|
||||
### Colorblind-Safe Palette (Okabe-Ito)
|
||||
|
||||
Use this palette for all paper figures. It is distinguishable by people with all common forms of color vision deficiency:
|
||||
|
||||
```python
|
||||
COLORS = {
|
||||
'blue': '#0072B2',
|
||||
'orange': '#E69F00',
|
||||
'green': '#009E73',
|
||||
'red': '#D55E00',
|
||||
'purple': '#CC79A7',
|
||||
'cyan': '#56B4E9',
|
||||
'yellow': '#F0E442',
|
||||
'black': '#000000',
|
||||
}
|
||||
|
||||
# As a list for cycling:
|
||||
COLOR_CYCLE = ['#0072B2', '#D55E00', '#009E73', '#E69F00', '#CC79A7', '#56B4E9']
|
||||
```
|
||||
|
||||
Also differentiate lines by **marker and linestyle**, not just color:
|
||||
```python
|
||||
STYLES = [
|
||||
{'color': '#0072B2', 'marker': 'o', 'linestyle': '-'},
|
||||
{'color': '#D55E00', 'marker': 's', 'linestyle': '--'},
|
||||
{'color': '#009E73', 'marker': '^', 'linestyle': '-.'},
|
||||
{'color': '#E69F00', 'marker': 'D', 'linestyle': ':'},
|
||||
]
|
||||
```
|
||||
|
||||
### Complete Example: Method Comparison Bar Chart
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
|
||||
try:
|
||||
import scienceplots
|
||||
style = ['science', 'no-latex']
|
||||
except ImportError:
|
||||
style = 'default'
|
||||
|
||||
with plt.style.context(style):
|
||||
methods = ['Single Pass', 'Critique+Revise', 'Best-of-N', 'Ours']
|
||||
scores = [73.2, 74.1, 68.5, 77.0]
|
||||
errors = [2.1, 1.8, 3.2, 1.5]
|
||||
colors = ['#56B4E9', '#E69F00', '#CC79A7', '#0072B2']
|
||||
|
||||
fig, ax = plt.subplots(figsize=(3.5, 2.5))
|
||||
bars = ax.bar(methods, scores, yerr=errors, capsize=3,
|
||||
color=colors, edgecolor='black', linewidth=0.5)
|
||||
|
||||
# Highlight "Ours"
|
||||
bars[-1].set_edgecolor('#0072B2')
|
||||
bars[-1].set_linewidth(1.5)
|
||||
|
||||
ax.set_ylabel('Pass Rate (%)')
|
||||
ax.set_ylim(60, 85)
|
||||
ax.spines['top'].set_visible(False)
|
||||
ax.spines['right'].set_visible(False)
|
||||
|
||||
fig.savefig('paper/fig_comparison.pdf', bbox_inches='tight')
|
||||
```
|
||||
|
||||
### Complete Example: Convergence/Trajectory Line Chart
|
||||
|
||||
```python
|
||||
with plt.style.context(style):
|
||||
fig, ax = plt.subplots(figsize=(3.5, 2.5))
|
||||
|
||||
passes = np.arange(1, 16)
|
||||
ours = [65, 72, 78, 82, 85, 87, 88, 89, 89.5, 90, 90, 90, 90, 90, 90]
|
||||
baseline = [65, 68, 70, 71, 69, 67, 66, 65, 64, 63, 62, 61, 60, 59, 58]
|
||||
|
||||
ax.plot(passes, ours, **STYLES[0], label='Ours', markersize=4)
|
||||
ax.plot(passes, baseline, **STYLES[1], label='Critique+Revise', markersize=4)
|
||||
|
||||
# Mark convergence point
|
||||
ax.axvline(x=10, color='gray', linestyle=':', alpha=0.5, linewidth=0.8)
|
||||
ax.annotate('Converged', xy=(10, 90), fontsize=8, ha='center',
|
||||
xytext=(10, 93), arrowprops=dict(arrowstyle='->', color='gray'))
|
||||
|
||||
ax.set_xlabel('Iteration')
|
||||
ax.set_ylabel('Quality Score')
|
||||
ax.legend(loc='lower right')
|
||||
ax.spines['top'].set_visible(False)
|
||||
ax.spines['right'].set_visible(False)
|
||||
|
||||
fig.savefig('paper/fig_trajectory.pdf', bbox_inches='tight')
|
||||
```
|
||||
|
||||
### Output Rules
|
||||
|
||||
- **Always save as PDF**: `fig.savefig('fig.pdf')` — vector graphics, sharp at any zoom
|
||||
- **Never save as PNG** for paper figures — raster PNGs look blurry when printed/zoomed
|
||||
- **Exception**: Screenshots, photographs, or pixel-art visualizations → PNG at 600 DPI
|
||||
- **Verify grayscale**: Print to grayscale PDF and check all information is still visible
|
||||
|
||||
### Chart Types for Common Comparisons
|
||||
|
||||
| Comparison Type | Chart | Notes |
|
||||
|----------------|-------|-------|
|
||||
| Method vs method | Grouped bar chart | Include error bars |
|
||||
| Across model sizes | Line chart with CI bands | Log scale for model size axis |
|
||||
| Ablation study | Stacked/grouped bar | Highlight removed component |
|
||||
| Trajectory/convergence | Line chart over iterations | Show winner per iteration |
|
||||
| Per-task breakdown | Heatmap or grouped bar | Show variance across tasks |
|
||||
@@ -0,0 +1,476 @@
|
||||
# Human Evaluation Guide for ML/AI Research
|
||||
|
||||
Comprehensive guide for designing, running, and reporting human evaluations in ML/AI papers. Human evaluation is the primary evidence for many NLP, HCI, and alignment papers, and is increasingly expected as complementary evidence at all ML venues.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
|
||||
- [When Human Evaluation Is Needed](#when-human-evaluation-is-needed)
|
||||
- [Study Design](#study-design)
|
||||
- [Annotation Guidelines](#annotation-guidelines)
|
||||
- [Platforms and Recruitment](#platforms-and-recruitment)
|
||||
- [Quality Control](#quality-control)
|
||||
- [Agreement Metrics](#agreement-metrics)
|
||||
- [Statistical Analysis for Human Eval](#statistical-analysis-for-human-eval)
|
||||
- [Reporting Requirements](#reporting-requirements)
|
||||
- [IRB and Ethics](#irb-and-ethics)
|
||||
- [Common Pitfalls](#common-pitfalls)
|
||||
|
||||
---
|
||||
|
||||
## When Human Evaluation Is Needed
|
||||
|
||||
| Scenario | Human Eval Required? | Notes |
|
||||
|----------|---------------------|-------|
|
||||
| Text generation quality (fluency, coherence) | **Yes** | Automated metrics (BLEU, ROUGE) correlate poorly with human judgment |
|
||||
| Factual accuracy of generated text | **Strongly recommended** | Automated fact-checking is unreliable |
|
||||
| Safety/toxicity evaluation | **Yes for nuanced cases** | Classifiers miss context-dependent harm |
|
||||
| Preference between two systems | **Yes** | Most reliable method for comparing LLM outputs |
|
||||
| Summarization quality | **Yes** | ROUGE doesn't capture faithfulness or relevance well |
|
||||
| Task completion (UI, agents) | **Yes** | User studies are the gold standard |
|
||||
| Classification accuracy | **Usually no** | Ground truth labels suffice; human eval adds cost without insight |
|
||||
| Perplexity or loss comparisons | **No** | Automated metrics are the correct evaluation |
|
||||
|
||||
---
|
||||
|
||||
## Study Design
|
||||
|
||||
### Evaluation Types
|
||||
|
||||
| Type | When to Use | Pros | Cons |
|
||||
|------|-------------|------|------|
|
||||
| **Pairwise comparison** | Comparing two systems | Most reliable, minimizes scale bias | Only compares pairs, quadratic in systems |
|
||||
| **Likert scale** (1-5 or 1-7) | Rating individual outputs | Easy to aggregate | Subjective anchoring, scale compression |
|
||||
| **Ranking** | Ordering 3+ systems | Captures full preference order | Cognitive load increases with items |
|
||||
| **Best-worst scaling** | Comparing many systems efficiently | More reliable than Likert, linear in items | Requires careful item selection |
|
||||
| **Binary judgment** | Yes/no decisions (grammatical? factual?) | Simple, high agreement | Loses nuance |
|
||||
| **Error annotation** | Identifying specific error types | Rich diagnostic information | Expensive, requires trained annotators |
|
||||
|
||||
**Recommendation for most ML papers**: Pairwise comparison is the most defensible. Reviewers rarely question its validity. For Likert scales, always report both mean and distribution.
|
||||
|
||||
### Sample Size Planning
|
||||
|
||||
**Minimum viable sample sizes:**
|
||||
|
||||
| Study Type | Minimum Items | Minimum Annotators | Notes |
|
||||
|------------|--------------|-------------------|-------|
|
||||
| Pairwise comparison | 100 pairs | 3 per pair | Detects ~10% win rate difference at p<0.05 |
|
||||
| Likert rating | 100 items | 3 per item | Enough for meaningful averages |
|
||||
| Ranking | 50 sets | 3 per set | Each set contains all systems being compared |
|
||||
| Error annotation | 200 items | 2 per item | Higher agreement expected for structured schemes |
|
||||
|
||||
**Power analysis** (for planning more precisely):
|
||||
|
||||
```python
|
||||
from scipy import stats
|
||||
import numpy as np
|
||||
|
||||
def sample_size_pairwise(effect_size=0.10, alpha=0.05, power=0.80):
|
||||
"""
|
||||
Estimate sample size for pairwise comparison (sign test).
|
||||
effect_size: expected win rate difference from 0.50
|
||||
"""
|
||||
p_expected = 0.50 + effect_size
|
||||
# Normal approximation to binomial
|
||||
z_alpha = stats.norm.ppf(1 - alpha / 2)
|
||||
z_beta = stats.norm.ppf(power)
|
||||
n = ((z_alpha * np.sqrt(0.25) + z_beta * np.sqrt(p_expected * (1 - p_expected))) ** 2) / (effect_size ** 2)
|
||||
return int(np.ceil(n))
|
||||
|
||||
print(f"Sample size for 10% effect: {sample_size_pairwise(0.10)}") # ~200
|
||||
print(f"Sample size for 15% effect: {sample_size_pairwise(0.15)}") # ~90
|
||||
print(f"Sample size for 20% effect: {sample_size_pairwise(0.20)}") # ~50
|
||||
```
|
||||
|
||||
### Controlling for Bias
|
||||
|
||||
| Bias | Mitigation |
|
||||
|------|-----------|
|
||||
| **Order bias** (first item preferred) | Randomize presentation order for each annotator |
|
||||
| **Length bias** (longer = better) | Control for length or analyze separately |
|
||||
| **Anchoring** (first annotation sets scale) | Include warm-up items (not counted) |
|
||||
| **Fatigue** (quality drops over time) | Limit session length (30-45 min max), randomize item order |
|
||||
| **Annotator expertise** | Report annotator background; use qualification tasks |
|
||||
|
||||
---
|
||||
|
||||
## Annotation Guidelines
|
||||
|
||||
Well-written annotation guidelines are the single biggest factor in evaluation quality. Invest significant time here.
|
||||
|
||||
### Structure of Good Guidelines
|
||||
|
||||
```markdown
|
||||
# [Task Name] Annotation Guidelines
|
||||
|
||||
## Overview
|
||||
[1-2 sentences describing the task]
|
||||
|
||||
## Definitions
|
||||
[Define every term annotators will use in their judgments]
|
||||
- Quality: [specific definition for this study]
|
||||
- Fluency: [specific definition]
|
||||
- Factuality: [specific definition]
|
||||
|
||||
## Rating Scale
|
||||
[For each scale point, provide:]
|
||||
- Numeric value
|
||||
- Label (e.g., "Excellent", "Good", "Acceptable", "Poor", "Unacceptable")
|
||||
- Definition of what qualifies for this rating
|
||||
- 1-2 concrete examples at this level
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: [Rating = 5]
|
||||
Input: [exact input]
|
||||
Output: [exact output]
|
||||
Rating: 5
|
||||
Explanation: [why this is a 5]
|
||||
|
||||
### Example 2: [Rating = 2]
|
||||
Input: [exact input]
|
||||
Output: [exact output]
|
||||
Rating: 2
|
||||
Explanation: [why this is a 2]
|
||||
|
||||
[Include at least 2 examples per rating level, covering edge cases]
|
||||
|
||||
## Edge Cases
|
||||
- If the output is [ambiguous case]: [instruction]
|
||||
- If the input is [unusual case]: [instruction]
|
||||
|
||||
## Common Mistakes
|
||||
- Don't [common annotator error]
|
||||
- Don't let [bias] influence your rating
|
||||
```
|
||||
|
||||
### Pilot Testing
|
||||
|
||||
**Always run a pilot** before the full study:
|
||||
1. 3-5 annotators, 20-30 items
|
||||
2. Compute agreement metrics
|
||||
3. Discuss disagreements in group session
|
||||
4. Revise guidelines based on confusion points
|
||||
5. Run second pilot if agreement was poor (<0.40 kappa)
|
||||
|
||||
---
|
||||
|
||||
## Platforms and Recruitment
|
||||
|
||||
| Platform | Best For | Cost | Quality |
|
||||
|----------|----------|------|---------|
|
||||
| **Prolific** | General annotation, surveys | $8-15/hr | High (academic-focused pool) |
|
||||
| **Amazon MTurk** | Large-scale simple tasks | $5-12/hr | Variable (needs strong QC) |
|
||||
| **Surge AI** | NLP-specific annotation | $15-25/hr | Very high (trained annotators) |
|
||||
| **Scale AI** | Production-quality labeling | Varies | High (managed workforce) |
|
||||
| **Internal team** | Domain expertise required | Varies | Highest for specialized tasks |
|
||||
| **Upwork/contractors** | Long-term annotation projects | $10-30/hr | Depends on hiring |
|
||||
|
||||
**Fair compensation**: Always pay at least the equivalent of local minimum wage for the annotator's location. Many conferences (ACL in particular) now ask about annotator compensation. Paying below minimum wage is an ethics risk.
|
||||
|
||||
**Prolific setup (recommended for most ML papers):**
|
||||
1. Create study on prolific.co
|
||||
2. Set prescreening filters (language, country, approval rate >95%)
|
||||
3. Estimate time per task from pilot → set fair payment
|
||||
4. Use Prolific's built-in attention checks or add your own
|
||||
5. Collect Prolific IDs for quality tracking (but don't share in paper)
|
||||
|
||||
---
|
||||
|
||||
## Quality Control
|
||||
|
||||
### Attention Checks
|
||||
|
||||
Include items where the correct answer is unambiguous:
|
||||
|
||||
```python
|
||||
# Types of attention checks
|
||||
attention_checks = {
|
||||
"instructed_response": "For this item, please select 'Strongly Agree' regardless of content.",
|
||||
"obvious_quality": "Rate this clearly ungrammatical text: 'The cat dog house green yesterday.'", # Should get lowest score
|
||||
"gold_standard": "Items where expert consensus exists (pre-annotated by authors)",
|
||||
"trap_question": "What color is the sky on a clear day? (embedded in annotation interface)"
|
||||
}
|
||||
|
||||
# Recommended: 10-15% of total items should be checks
|
||||
# Exclusion criterion: fail 2+ attention checks → exclude annotator
|
||||
```
|
||||
|
||||
### Annotator Qualification
|
||||
|
||||
For tasks requiring expertise:
|
||||
|
||||
```
|
||||
Qualification Task Design:
|
||||
1. Create a set of 20-30 items with known-correct labels
|
||||
2. Require annotators to complete this before the main task
|
||||
3. Set threshold: ≥80% agreement with gold labels to qualify
|
||||
4. Record qualification scores for reporting
|
||||
```
|
||||
|
||||
### Monitoring During Collection
|
||||
|
||||
```python
|
||||
# Real-time quality monitoring
|
||||
def monitor_quality(annotations):
|
||||
"""Check for annotation quality issues during collection."""
|
||||
issues = []
|
||||
|
||||
# 1. Check for straight-lining (same answer for everything)
|
||||
for annotator_id, items in annotations.groupby('annotator'):
|
||||
if items['rating'].nunique() <= 1:
|
||||
issues.append(f"Annotator {annotator_id}: straight-lining detected")
|
||||
|
||||
# 2. Check time per item (too fast = not reading)
|
||||
median_time = annotations['time_seconds'].median()
|
||||
fast_annotators = annotations.groupby('annotator')['time_seconds'].median()
|
||||
for ann_id, time in fast_annotators.items():
|
||||
if time < median_time * 0.3:
|
||||
issues.append(f"Annotator {ann_id}: suspiciously fast ({time:.0f}s vs median {median_time:.0f}s)")
|
||||
|
||||
# 3. Check attention check performance
|
||||
checks = annotations[annotations['is_attention_check']]
|
||||
for ann_id, items in checks.groupby('annotator'):
|
||||
accuracy = (items['rating'] == items['gold_rating']).mean()
|
||||
if accuracy < 0.80:
|
||||
issues.append(f"Annotator {ann_id}: failing attention checks ({accuracy:.0%})")
|
||||
|
||||
return issues
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agreement Metrics
|
||||
|
||||
### Which Metric to Use
|
||||
|
||||
| Metric | When to Use | Interpretation |
|
||||
|--------|-------------|---------------|
|
||||
| **Cohen's kappa (κ)** | Exactly 2 annotators, categorical | Chance-corrected agreement |
|
||||
| **Fleiss' kappa** | 3+ annotators, all rate same items, categorical | Multi-annotator extension of Cohen's |
|
||||
| **Krippendorff's alpha (α)** | Any number of annotators, handles missing data | Most general; recommended default |
|
||||
| **ICC (Intraclass Correlation)** | Continuous ratings (Likert) | Consistency among raters |
|
||||
| **Percent agreement** | Reporting alongside kappa/alpha | Raw agreement (not chance-corrected) |
|
||||
| **Kendall's W** | Rankings | Concordance among rankers |
|
||||
|
||||
**Always report at least two**: one chance-corrected metric (kappa or alpha) AND raw percent agreement.
|
||||
|
||||
### Interpretation Guide
|
||||
|
||||
| Value | Krippendorff's α / Cohen's κ | Quality |
|
||||
|-------|-------------------------------|---------|
|
||||
| > 0.80 | Excellent agreement | Reliable for most purposes |
|
||||
| 0.67 - 0.80 | Good agreement | Acceptable for most ML papers |
|
||||
| 0.40 - 0.67 | Moderate agreement | Borderline; discuss in paper |
|
||||
| < 0.40 | Poor agreement | Revise guidelines and redo annotation |
|
||||
|
||||
**Note**: Krippendorff recommends α > 0.667 as minimum for tentative conclusions. NLP tasks with subjective judgments (fluency, helpfulness) typically achieve 0.40-0.70.
|
||||
|
||||
### Implementation
|
||||
|
||||
```python
|
||||
import numpy as np
|
||||
from sklearn.metrics import cohen_kappa_score
|
||||
import krippendorff # pip install krippendorff
|
||||
|
||||
def compute_agreement(annotations_matrix):
|
||||
"""
|
||||
annotations_matrix: shape (n_items, n_annotators)
|
||||
Values: ratings (int or float). Use np.nan for missing.
|
||||
"""
|
||||
results = {}
|
||||
|
||||
# Krippendorff's alpha (handles missing data, any number of annotators)
|
||||
results['krippendorff_alpha'] = krippendorff.alpha(
|
||||
annotations_matrix.T, # krippendorff expects (annotators, items)
|
||||
level_of_measurement='ordinal' # or 'nominal', 'interval', 'ratio'
|
||||
)
|
||||
|
||||
# Pairwise Cohen's kappa (for 2 annotators at a time)
|
||||
n_annotators = annotations_matrix.shape[1]
|
||||
kappas = []
|
||||
for i in range(n_annotators):
|
||||
for j in range(i + 1, n_annotators):
|
||||
mask = ~np.isnan(annotations_matrix[:, i]) & ~np.isnan(annotations_matrix[:, j])
|
||||
if mask.sum() > 0:
|
||||
k = cohen_kappa_score(
|
||||
annotations_matrix[mask, i].astype(int),
|
||||
annotations_matrix[mask, j].astype(int)
|
||||
)
|
||||
kappas.append(k)
|
||||
results['mean_pairwise_kappa'] = np.mean(kappas) if kappas else None
|
||||
|
||||
# Raw percent agreement
|
||||
agree_count = 0
|
||||
total_count = 0
|
||||
for item in range(annotations_matrix.shape[0]):
|
||||
ratings = annotations_matrix[item, ~np.isnan(annotations_matrix[item, :])]
|
||||
if len(ratings) >= 2:
|
||||
# All annotators agree
|
||||
if len(set(ratings.astype(int))) == 1:
|
||||
agree_count += 1
|
||||
total_count += 1
|
||||
results['percent_agreement'] = agree_count / total_count if total_count > 0 else None
|
||||
|
||||
return results
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Statistical Analysis for Human Eval
|
||||
|
||||
### Pairwise Comparisons
|
||||
|
||||
```python
|
||||
from scipy import stats
|
||||
|
||||
def analyze_pairwise(wins_a, wins_b, ties=0):
|
||||
"""
|
||||
Analyze pairwise comparison results.
|
||||
wins_a: number of times system A won
|
||||
wins_b: number of times system B won
|
||||
ties: number of ties (excluded from sign test)
|
||||
"""
|
||||
n = wins_a + wins_b # exclude ties
|
||||
|
||||
# Sign test (exact binomial)
|
||||
p_value = stats.binom_test(wins_a, n, 0.5, alternative='two-sided')
|
||||
|
||||
# Win rate with 95% CI (Wilson score interval)
|
||||
win_rate = wins_a / n if n > 0 else 0.5
|
||||
z = 1.96
|
||||
denominator = 1 + z**2 / n
|
||||
center = (win_rate + z**2 / (2 * n)) / denominator
|
||||
margin = z * np.sqrt((win_rate * (1 - win_rate) + z**2 / (4 * n)) / n) / denominator
|
||||
ci_lower = center - margin
|
||||
ci_upper = center + margin
|
||||
|
||||
return {
|
||||
'win_rate_a': win_rate,
|
||||
'win_rate_b': 1 - win_rate,
|
||||
'p_value': p_value,
|
||||
'ci_95': (ci_lower, ci_upper),
|
||||
'significant': p_value < 0.05,
|
||||
'n_comparisons': n,
|
||||
'ties': ties,
|
||||
}
|
||||
```
|
||||
|
||||
### Likert Scale Analysis
|
||||
|
||||
```python
|
||||
def analyze_likert(ratings_a, ratings_b):
|
||||
"""Compare Likert ratings between two systems (paired)."""
|
||||
# Wilcoxon signed-rank test (non-parametric, paired)
|
||||
stat, p_value = stats.wilcoxon(ratings_a, ratings_b, alternative='two-sided')
|
||||
|
||||
# Effect size (rank-biserial correlation)
|
||||
n = len(ratings_a)
|
||||
r = 1 - (2 * stat) / (n * (n + 1))
|
||||
|
||||
return {
|
||||
'mean_a': np.mean(ratings_a),
|
||||
'mean_b': np.mean(ratings_b),
|
||||
'std_a': np.std(ratings_a),
|
||||
'std_b': np.std(ratings_b),
|
||||
'wilcoxon_stat': stat,
|
||||
'p_value': p_value,
|
||||
'effect_size_r': r,
|
||||
'significant': p_value < 0.05,
|
||||
}
|
||||
```
|
||||
|
||||
### Multiple Comparisons Correction
|
||||
|
||||
When comparing more than two systems:
|
||||
|
||||
```python
|
||||
from statsmodels.stats.multitest import multipletests
|
||||
|
||||
# After computing p-values for all pairs
|
||||
p_values = [0.03, 0.001, 0.08, 0.04, 0.15, 0.002]
|
||||
rejected, corrected_p, _, _ = multipletests(p_values, method='holm')
|
||||
# Use corrected p-values in your paper
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Reporting Requirements
|
||||
|
||||
Reviewers at NLP venues (ACL, EMNLP, NAACL) check for all of these. ML venues (NeurIPS, ICML) increasingly expect them too.
|
||||
|
||||
### Mandatory Reporting
|
||||
|
||||
```latex
|
||||
% In your paper's human evaluation section:
|
||||
\paragraph{Annotators.} We recruited [N] annotators via [platform].
|
||||
[Describe qualifications or screening.] Annotators were paid
|
||||
\$[X]/hour, above the [country] minimum wage.
|
||||
|
||||
\paragraph{Agreement.} Inter-annotator agreement was [metric] = [value]
|
||||
(Krippendorff's $\alpha$ = [value]; raw agreement = [value]\%).
|
||||
[If low: explain why the task is subjective and how you handle disagreements.]
|
||||
|
||||
\paragraph{Evaluation Protocol.} Each [item type] was rated by [N]
|
||||
annotators on a [scale description]. We collected [total] annotations
|
||||
across [N items]. [Describe randomization and blinding.]
|
||||
```
|
||||
|
||||
### What Goes in the Appendix
|
||||
|
||||
```
|
||||
Appendix: Human Evaluation Details
|
||||
- Full annotation guidelines (verbatim)
|
||||
- Screenshot of annotation interface
|
||||
- Qualification task details and threshold
|
||||
- Attention check items and failure rates
|
||||
- Per-annotator agreement breakdown
|
||||
- Full results table (not just averages)
|
||||
- Compensation calculation
|
||||
- IRB approval number (if applicable)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## IRB and Ethics
|
||||
|
||||
### When IRB Approval Is Needed
|
||||
|
||||
| Situation | IRB Required? |
|
||||
|-----------|---------------|
|
||||
| Crowdworkers rating text quality | **Usually no** (not "human subjects research" at most institutions) |
|
||||
| User study with real users | **Yes** at most US/EU institutions |
|
||||
| Collecting personal information | **Yes** |
|
||||
| Studying annotator behavior/cognition | **Yes** (they become the subject) |
|
||||
| Using existing annotated data | **Usually no** (secondary data analysis) |
|
||||
|
||||
**Check your institution's policy.** The definition of "human subjects research" varies. When in doubt, submit an IRB protocol — the review is often fast for minimal-risk studies.
|
||||
|
||||
### Ethics Checklist for Human Evaluation
|
||||
|
||||
```
|
||||
- [ ] Annotators informed about task purpose (not deceptive)
|
||||
- [ ] Annotators can withdraw at any time without penalty
|
||||
- [ ] No personally identifiable information collected beyond platform ID
|
||||
- [ ] Content being evaluated does not expose annotators to harm
|
||||
(if it does: content warnings + opt-out + higher compensation)
|
||||
- [ ] Fair compensation (>= equivalent local minimum wage)
|
||||
- [ ] Data stored securely, access limited to research team
|
||||
- [ ] IRB approval obtained if required by institution
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
| Pitfall | Problem | Fix |
|
||||
|---------|---------|-----|
|
||||
| Too few annotators (1-2) | No agreement metric possible | Minimum 3 annotators per item |
|
||||
| No attention checks | Can't detect low-quality annotations | Include 10-15% attention checks |
|
||||
| Not reporting compensation | Reviewers flag as ethics concern | Always report hourly rate |
|
||||
| Using only automated metrics for generation | Reviewers will ask for human eval | Add at least pairwise comparison |
|
||||
| Not piloting guidelines | Low agreement, wasted budget | Always pilot with 3-5 people first |
|
||||
| Reporting only averages | Hides annotator disagreement | Report distribution and agreement |
|
||||
| Not controlling for order/position | Position bias inflates results | Randomize presentation order |
|
||||
| Conflating annotator agreement with ground truth | High agreement doesn't mean correct | Validate against expert judgments |
|
||||
@@ -0,0 +1,481 @@
|
||||
# Paper Types Beyond Empirical ML
|
||||
|
||||
Guide for writing non-standard paper types: theory papers, survey/tutorial papers, benchmark/dataset papers, and position papers. Each type has distinct structure, evidence standards, and venue expectations.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
|
||||
- [Theory Papers](#theory-papers)
|
||||
- [Survey and Tutorial Papers](#survey-and-tutorial-papers)
|
||||
- [Benchmark and Dataset Papers](#benchmark-and-dataset-papers)
|
||||
- [Position Papers](#position-papers)
|
||||
- [Reproducibility and Replication Papers](#reproducibility-and-replication-papers)
|
||||
|
||||
---
|
||||
|
||||
## Theory Papers
|
||||
|
||||
### When to Write a Theory Paper
|
||||
|
||||
Your paper should be a theory paper if:
|
||||
- The main contribution is a theorem, bound, impossibility result, or formal characterization
|
||||
- Experiments are supplementary validation, not the core evidence
|
||||
- The contribution advances understanding rather than achieving state-of-the-art numbers
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
1. Introduction (1-1.5 pages)
|
||||
- Problem statement and motivation
|
||||
- Informal statement of main results
|
||||
- Comparison to prior theoretical work
|
||||
- Contribution bullets (state theorems informally)
|
||||
|
||||
2. Preliminaries (0.5-1 page)
|
||||
- Notation table
|
||||
- Formal definitions
|
||||
- Assumptions (numbered, referenced later)
|
||||
- Known results you build on
|
||||
|
||||
3. Main Results (2-3 pages)
|
||||
- Theorem statements (formal)
|
||||
- Proof sketches (intuition + key steps)
|
||||
- Corollaries and special cases
|
||||
- Discussion of tightness / optimality
|
||||
|
||||
4. Experimental Validation (1-2 pages, optional but recommended)
|
||||
- Do theoretical predictions match empirical behavior?
|
||||
- Synthetic experiments that isolate the phenomenon
|
||||
- Comparison to bounds from prior work
|
||||
|
||||
5. Related Work (1 page)
|
||||
- Theoretical predecessors
|
||||
- Empirical work your theory explains
|
||||
|
||||
6. Discussion & Open Problems (0.5 page)
|
||||
- Limitations of your results
|
||||
- Conjectures suggested by your analysis
|
||||
- Concrete open problems
|
||||
|
||||
Appendix:
|
||||
- Full proofs
|
||||
- Technical lemmas
|
||||
- Extended experimental details
|
||||
```
|
||||
|
||||
### Writing Theorems
|
||||
|
||||
**Template for a well-stated theorem:**
|
||||
|
||||
```latex
|
||||
\begin{assumption}[Bounded Gradients]\label{assum:bounded-grad}
|
||||
There exists $G > 0$ such that $\|\nabla f(x)\| \leq G$ for all $x \in \mathcal{X}$.
|
||||
\end{assumption}
|
||||
|
||||
\begin{theorem}[Convergence Rate]\label{thm:convergence}
|
||||
Under Assumptions~\ref{assum:bounded-grad} and~\ref{assum:smoothness},
|
||||
Algorithm~\ref{alg:method} with step size $\eta = \frac{1}{\sqrt{T}}$ satisfies
|
||||
\[
|
||||
\frac{1}{T}\sum_{t=1}^{T} \mathbb{E}\left[\|\nabla f(x_t)\|^2\right]
|
||||
\leq \frac{2(f(x_1) - f^*)}{\sqrt{T}} + \frac{G^2}{\sqrt{T}}.
|
||||
\]
|
||||
In particular, after $T = O(1/\epsilon^2)$ iterations, we obtain an
|
||||
$\epsilon$-stationary point.
|
||||
\end{theorem}
|
||||
```
|
||||
|
||||
**Rules for theorem statements:**
|
||||
- State all assumptions explicitly (numbered, with names)
|
||||
- Include the formal bound, not just "converges at rate O(·)"
|
||||
- Add a plain-language corollary: "In particular, this means..."
|
||||
- Compare to known bounds: "This improves over [prior work]'s bound of O(·) by a factor of..."
|
||||
|
||||
### Proof Sketches
|
||||
|
||||
The proof sketch is the most important part of the main text for a theory paper. Reviewers evaluate whether you have genuine insight or just mechanical derivation.
|
||||
|
||||
**Good proof sketch pattern:**
|
||||
|
||||
```latex
|
||||
\begin{proof}[Proof Sketch of Theorem~\ref{thm:convergence}]
|
||||
The key insight is that [one sentence describing the main idea].
|
||||
|
||||
The proof proceeds in three steps:
|
||||
\begin{enumerate}
|
||||
\item \textbf{Decomposition.} We decompose the error into [term A]
|
||||
and [term B] using [technique]. This reduces the problem to
|
||||
bounding each term separately.
|
||||
|
||||
\item \textbf{Bounding [term A].} By [assumption/lemma], [term A]
|
||||
is bounded by $O(\cdot)$. The critical observation is that
|
||||
[specific insight that makes this non-trivial].
|
||||
|
||||
\item \textbf{Combining.} Choosing $\eta = 1/\sqrt{T}$ balances
|
||||
the two terms, yielding the stated bound.
|
||||
\end{enumerate}
|
||||
|
||||
The full proof, including the technical lemma for Step 2,
|
||||
appears in Appendix~\ref{app:proofs}.
|
||||
\end{proof}
|
||||
```
|
||||
|
||||
**Bad proof sketch**: Restating the theorem with slightly different notation, or just saying "the proof follows standard techniques."
|
||||
|
||||
### Full Proofs in Appendix
|
||||
|
||||
```latex
|
||||
\appendix
|
||||
\section{Proofs}\label{app:proofs}
|
||||
|
||||
\subsection{Proof of Theorem~\ref{thm:convergence}}
|
||||
|
||||
We first establish two technical lemmas.
|
||||
|
||||
\begin{lemma}[Descent Lemma]\label{lem:descent}
|
||||
Under Assumption~\ref{assum:smoothness}, for any step size $\eta \leq 1/L$:
|
||||
\[
|
||||
f(x_{t+1}) \leq f(x_t) - \frac{\eta}{2}\|\nabla f(x_t)\|^2 + \frac{\eta^2 L}{2}\|\nabla f(x_t)\|^2.
|
||||
\]
|
||||
\end{lemma}
|
||||
|
||||
\begin{proof}
|
||||
[Complete proof with all steps]
|
||||
\end{proof}
|
||||
|
||||
% Continue with remaining lemmas and main theorem proof
|
||||
```
|
||||
|
||||
### Common Theory Paper Pitfalls
|
||||
|
||||
| Pitfall | Problem | Fix |
|
||||
|---------|---------|-----|
|
||||
| Assumptions too strong | Trivializes the result | Discuss which assumptions are necessary; prove lower bounds |
|
||||
| No comparison to existing bounds | Reviewers can't assess contribution | Add a comparison table of bounds |
|
||||
| Proof sketch is just the full proof shortened | Doesn't convey insight | Focus on the 1-2 key ideas; defer mechanics to appendix |
|
||||
| No experimental validation | Reviewers question practical relevance | Add synthetic experiments testing predictions |
|
||||
| Notation inconsistency | Confuses reviewers | Create a notation table in Preliminaries |
|
||||
| Overly complex proofs where simple ones exist | Reviewers suspect error | Prefer clarity over generality |
|
||||
|
||||
### Venues for Theory Papers
|
||||
|
||||
| Venue | Theory Acceptance Rate | Notes |
|
||||
|-------|----------------------|-------|
|
||||
| **NeurIPS** | Moderate | Values theory with practical implications |
|
||||
| **ICML** | High | Strong theory track |
|
||||
| **ICLR** | Moderate | Prefers theory with empirical validation |
|
||||
| **COLT** | High | Theory-focused venue |
|
||||
| **ALT** | High | Algorithmic learning theory |
|
||||
| **STOC/FOCS** | For TCS-flavored results | If contribution is primarily combinatorial/algorithmic |
|
||||
| **JMLR** | High | No page limit; good for long proofs |
|
||||
|
||||
---
|
||||
|
||||
## Survey and Tutorial Papers
|
||||
|
||||
### When to Write a Survey
|
||||
|
||||
- A subfield has matured enough that synthesis is valuable
|
||||
- You've identified connections between works that individual papers don't make
|
||||
- Newcomers to the area have no good entry point
|
||||
- The landscape has changed significantly since the last survey
|
||||
|
||||
**Warning**: Surveys require genuine expertise. A survey by someone outside the field, however comprehensive, will miss nuances and mischaracterize work.
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
1. Introduction (1-2 pages)
|
||||
- Scope definition (what's included and excluded, and why)
|
||||
- Motivation for the survey now
|
||||
- Overview of organization (often with a figure)
|
||||
|
||||
2. Background / Problem Formulation (1-2 pages)
|
||||
- Formal problem definition
|
||||
- Notation (used consistently throughout)
|
||||
- Historical context
|
||||
|
||||
3. Taxonomy (the core contribution)
|
||||
- Organize methods along meaningful axes
|
||||
- Present taxonomy as a figure or table
|
||||
- Each category gets a subsection
|
||||
|
||||
4. Detailed Coverage (bulk of paper)
|
||||
- For each category: representative methods, key ideas, strengths/weaknesses
|
||||
- Comparison tables within and across categories
|
||||
- Don't just describe — analyze and compare
|
||||
|
||||
5. Experimental Comparison (if applicable)
|
||||
- Standardized benchmark comparison
|
||||
- Fair hyperparameter tuning for all methods
|
||||
- Not always feasible but significantly strengthens the survey
|
||||
|
||||
6. Open Problems & Future Directions (1-2 pages)
|
||||
- Unsolved problems the field should tackle
|
||||
- Promising but underexplored directions
|
||||
- This section is what makes a survey a genuine contribution
|
||||
|
||||
7. Conclusion
|
||||
```
|
||||
|
||||
### Taxonomy Design
|
||||
|
||||
The taxonomy is the core intellectual contribution of a survey. It should:
|
||||
|
||||
- **Be meaningful**: Categories should correspond to real methodological differences, not arbitrary groupings
|
||||
- **Be exhaustive**: Every relevant paper should fit somewhere
|
||||
- **Be mutually exclusive** (ideally): Each paper belongs to one primary category
|
||||
- **Have informative names**: "Attention-based methods" > "Category 3"
|
||||
- **Be visualized**: A figure showing the taxonomy is almost always helpful
|
||||
|
||||
**Example taxonomy axes for "LLM Reasoning" survey:**
|
||||
- By technique: chain-of-thought, tree-of-thought, self-consistency, tool use
|
||||
- By training requirement: prompting-only, fine-tuned, RLHF
|
||||
- By reasoning type: mathematical, commonsense, logical, causal
|
||||
|
||||
### Writing Standards
|
||||
|
||||
- **Cite every relevant paper** — authors will check if their work is included
|
||||
- **Be fair** — don't dismiss methods you don't prefer
|
||||
- **Synthesize, don't just list** — identify patterns, trade-offs, open questions
|
||||
- **Include a comparison table** — even if qualitative (features/properties checklist)
|
||||
- **Update before submission** — check arXiv for papers published since you started writing
|
||||
|
||||
### Venues for Surveys
|
||||
|
||||
| Venue | Notes |
|
||||
|-------|-------|
|
||||
| **TMLR** (Survey track) | Dedicated survey submissions; no page limit |
|
||||
| **JMLR** | Long format, well-respected |
|
||||
| **Foundations and Trends in ML** | Invited, but can be proposed |
|
||||
| **ACM Computing Surveys** | Broad CS audience |
|
||||
| **arXiv** (standalone) | No peer review but high visibility if well-done |
|
||||
| **Conference tutorials** | Present as tutorial at NeurIPS/ICML/ACL; write up as paper |
|
||||
|
||||
---
|
||||
|
||||
## Benchmark and Dataset Papers
|
||||
|
||||
### When to Write a Benchmark Paper
|
||||
|
||||
- Existing benchmarks don't measure what you think matters
|
||||
- A new capability has emerged with no standard evaluation
|
||||
- Existing benchmarks are saturated (all methods score >95%)
|
||||
- You want to standardize evaluation in a fragmented subfield
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
1. Introduction
|
||||
- What evaluation gap does this benchmark fill?
|
||||
- Why existing benchmarks are insufficient
|
||||
|
||||
2. Task Definition
|
||||
- Formal task specification
|
||||
- Input/output format
|
||||
- Evaluation criteria (what makes a good answer?)
|
||||
|
||||
3. Dataset Construction
|
||||
- Data source and collection methodology
|
||||
- Annotation process (if human-annotated)
|
||||
- Quality control measures
|
||||
- Dataset statistics (size, distribution, splits)
|
||||
|
||||
4. Baseline Evaluation
|
||||
- Run strong baselines (don't just report random/majority)
|
||||
- Show the benchmark is challenging but not impossible
|
||||
- Human performance baseline (if feasible)
|
||||
|
||||
5. Analysis
|
||||
- Error analysis on baselines
|
||||
- What makes items hard/easy?
|
||||
- Construct validity: does the benchmark measure what you claim?
|
||||
|
||||
6. Intended Use & Limitations
|
||||
- What should this benchmark be used for?
|
||||
- What should it NOT be used for?
|
||||
- Known biases or limitations
|
||||
|
||||
7. Datasheet (Appendix)
|
||||
- Full datasheet for datasets (Gebru et al.)
|
||||
```
|
||||
|
||||
### Evidence Standards
|
||||
|
||||
Reviewers evaluate benchmarks on different criteria than methods papers:
|
||||
|
||||
| Criterion | What Reviewers Check |
|
||||
|-----------|---------------------|
|
||||
| **Novelty of evaluation** | Does this measure something existing benchmarks don't? |
|
||||
| **Construct validity** | Does the benchmark actually measure the stated capability? |
|
||||
| **Difficulty calibration** | Not too easy (saturated) or too hard (random performance) |
|
||||
| **Annotation quality** | Agreement metrics, annotator qualifications, guidelines |
|
||||
| **Documentation** | Datasheet, license, maintenance plan |
|
||||
| **Reproducibility** | Can others use this benchmark easily? |
|
||||
| **Ethical considerations** | Bias analysis, consent, sensitive content handling |
|
||||
|
||||
### Dataset Documentation (Required)
|
||||
|
||||
Follow the Datasheets for Datasets framework (Gebru et al., 2021):
|
||||
|
||||
```
|
||||
Datasheet Questions:
|
||||
1. Motivation
|
||||
- Why was this dataset created?
|
||||
- Who created it and on behalf of whom?
|
||||
- Who funded the creation?
|
||||
|
||||
2. Composition
|
||||
- What do the instances represent?
|
||||
- How many instances are there?
|
||||
- Does it contain all possible instances or a sample?
|
||||
- Is there a label? If so, how was it determined?
|
||||
- Are there recommended data splits?
|
||||
|
||||
3. Collection Process
|
||||
- How was the data collected?
|
||||
- Who was involved in collection?
|
||||
- Over what timeframe?
|
||||
- Was ethical review conducted?
|
||||
|
||||
4. Preprocessing
|
||||
- What preprocessing was done?
|
||||
- Was the "raw" data saved?
|
||||
|
||||
5. Uses
|
||||
- What tasks has this been used for?
|
||||
- What should it NOT be used for?
|
||||
- Are there other tasks it could be used for?
|
||||
|
||||
6. Distribution
|
||||
- How is it distributed?
|
||||
- Under what license?
|
||||
- Are there any restrictions?
|
||||
|
||||
7. Maintenance
|
||||
- Who maintains it?
|
||||
- How can users contact the maintainer?
|
||||
- Will it be updated? How?
|
||||
- Is there an erratum?
|
||||
```
|
||||
|
||||
### Venues for Benchmark Papers
|
||||
|
||||
| Venue | Notes |
|
||||
|-------|-------|
|
||||
| **NeurIPS Datasets & Benchmarks** | Dedicated track; best venue for this |
|
||||
| **ACL** (Resource papers) | NLP-focused datasets |
|
||||
| **LREC-COLING** | Language resources |
|
||||
| **TMLR** | Good for benchmarks with analysis |
|
||||
|
||||
---
|
||||
|
||||
## Position Papers
|
||||
|
||||
### When to Write a Position Paper
|
||||
|
||||
- You have an argument about how the field should develop
|
||||
- You want to challenge a widely-held assumption
|
||||
- You want to propose a research agenda based on analysis
|
||||
- You've identified a systematic problem in current methodology
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
1. Introduction
|
||||
- State your thesis clearly in the first paragraph
|
||||
- Why this matters now
|
||||
|
||||
2. Background
|
||||
- Current state of the field
|
||||
- Prevailing assumptions you're challenging
|
||||
|
||||
3. Argument
|
||||
- Present your thesis with supporting evidence
|
||||
- Evidence can be: empirical data, theoretical analysis, logical argument,
|
||||
case studies, historical precedent
|
||||
- Be rigorous — this isn't an opinion piece
|
||||
|
||||
4. Counterarguments
|
||||
- Engage seriously with the strongest objections
|
||||
- Explain why they don't undermine your thesis
|
||||
- Concede where appropriate — it strengthens credibility
|
||||
|
||||
5. Implications
|
||||
- What should the field do differently?
|
||||
- Concrete research directions your thesis suggests
|
||||
- How should evaluation/methodology change?
|
||||
|
||||
6. Conclusion
|
||||
- Restate thesis
|
||||
- Call to action
|
||||
```
|
||||
|
||||
### Writing Standards
|
||||
|
||||
- **Lead with the strongest version of your argument** — don't hedge in the first paragraph
|
||||
- **Engage with counterarguments honestly** — the best position papers address the strongest objections, not the weakest
|
||||
- **Provide evidence** — a position paper without evidence is an editorial
|
||||
- **Be concrete** — "the field should do X" is better than "more work is needed"
|
||||
- **Don't straw-man existing work** — characterize opposing positions fairly
|
||||
|
||||
### Venues for Position Papers
|
||||
|
||||
| Venue | Notes |
|
||||
|-------|-------|
|
||||
| **ICML** (Position track) | Dedicated track for position papers |
|
||||
| **NeurIPS** (Workshop papers) | Workshops often welcome position pieces |
|
||||
| **ACL** (Theme papers) | When your position aligns with the conference theme |
|
||||
| **TMLR** | Accepts well-argued position papers |
|
||||
| **CACM** | For broader CS audience |
|
||||
|
||||
---
|
||||
|
||||
## Reproducibility and Replication Papers
|
||||
|
||||
### When to Write a Reproducibility Paper
|
||||
|
||||
- You attempted to reproduce a published result and succeeded/failed
|
||||
- You want to verify claims under different conditions
|
||||
- You've identified that a popular method's performance depends on unreported details
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
1. Introduction
|
||||
- What paper/result are you reproducing?
|
||||
- Why is this reproduction valuable?
|
||||
|
||||
2. Original Claims
|
||||
- State the exact claims from the original paper
|
||||
- What evidence was provided?
|
||||
|
||||
3. Methodology
|
||||
- Your reproduction approach
|
||||
- Differences from original (if any) and why
|
||||
- What information was missing from the original paper?
|
||||
|
||||
4. Results
|
||||
- Side-by-side comparison with original results
|
||||
- Statistical comparison (confidence intervals overlap?)
|
||||
- What reproduced and what didn't?
|
||||
|
||||
5. Analysis
|
||||
- If results differ: why? What's sensitive?
|
||||
- Hidden hyperparameters or implementation details?
|
||||
- Robustness to seed, hardware, library versions?
|
||||
|
||||
6. Recommendations
|
||||
- For original authors: what should be clarified?
|
||||
- For practitioners: what to watch out for?
|
||||
- For the field: what reproducibility lessons emerge?
|
||||
```
|
||||
|
||||
### Venues
|
||||
|
||||
| Venue | Notes |
|
||||
|-------|-------|
|
||||
| **ML Reproducibility Challenge** | Annual challenge at NeurIPS |
|
||||
| **ReScience** | Journal dedicated to replications |
|
||||
| **TMLR** | Accepts reproductions with analysis |
|
||||
| **Workshops** | Reproducibility workshops at major conferences |
|
||||
@@ -0,0 +1,757 @@
|
||||
# Phase 5: Paper Drafting (full procedure)
|
||||
|
||||
**Goal**: Write a complete, publication-ready paper.
|
||||
|
||||
### Context Management for Large Projects
|
||||
|
||||
A paper project with 50+ experiment files, multiple result directories, and extensive literature notes can easily exceed the agent's context window. Manage this proactively:
|
||||
|
||||
**What to load into context per drafting task:**
|
||||
|
||||
| Drafting Task | Load Into Context | Do NOT Load |
|
||||
|---------------|------------------|-------------|
|
||||
| Writing Introduction | `experiment_log.md`, contribution statement, 5-10 most relevant paper abstracts | Raw result JSONs, full experiment scripts, all literature notes |
|
||||
| Writing Methods | Experiment configs, pseudocode, architecture description | Raw logs, results from other experiments |
|
||||
| Writing Results | `experiment_log.md`, result summary tables, figure list | Full analysis scripts, intermediate data |
|
||||
| Writing Related Work | Organized citation notes (Step 1.4 output), .bib file | Experiment files, raw PDFs |
|
||||
| Revision pass | Full paper draft, specific reviewer concerns | Everything else |
|
||||
|
||||
**Principles:**
|
||||
- **`experiment_log.md` is the primary context bridge** — it summarizes everything needed for writing without loading raw data files (see Step 4.6)
|
||||
- **Load one section's context at a time** when delegating. A sub-agent drafting Methods doesn't need the literature review notes.
|
||||
- **Summarize, don't include raw files.** For a 200-line result JSON, load a 10-line summary table. For a 50-page related paper, load the 5-sentence abstract + your 2-line note about its relevance.
|
||||
- **For very large projects**: Create a `context/` directory with pre-compressed summaries:
|
||||
```
|
||||
context/
|
||||
contribution.md # 1 sentence
|
||||
experiment_summary.md # Key results table (from experiment_log.md)
|
||||
literature_map.md # Organized citation notes
|
||||
figure_inventory.md # List of figures with descriptions
|
||||
```
|
||||
|
||||
### The Narrative Principle
|
||||
|
||||
**The single most critical insight**: Your paper is not a collection of experiments — it's a story with one clear contribution supported by evidence.
|
||||
|
||||
Every successful ML paper centers on what Neel Nanda calls "the narrative": a short, rigorous, evidence-based technical story with a takeaway readers care about.
|
||||
|
||||
**Three Pillars (must be crystal clear by end of introduction):**
|
||||
|
||||
| Pillar | Description | Test |
|
||||
|--------|-------------|------|
|
||||
| **The What** | 1-3 specific novel claims | Can you state them in one sentence? |
|
||||
| **The Why** | Rigorous empirical evidence | Do experiments distinguish your hypothesis from alternatives? |
|
||||
| **The So What** | Why readers should care | Does this connect to a recognized community problem? |
|
||||
|
||||
**If you cannot state your contribution in one sentence, you don't yet have a paper.**
|
||||
|
||||
### The Sources Behind This Guidance
|
||||
|
||||
This skill synthesizes writing philosophy from researchers who have published extensively at top venues. The writing philosophy layer was originally compiled by [Orchestra Research](https://github.com/orchestra-research) as the `ml-paper-writing` skill.
|
||||
|
||||
| Source | Key Contribution | Link |
|
||||
|--------|-----------------|------|
|
||||
| **Neel Nanda** (Google DeepMind) | The Narrative Principle, What/Why/So What framework | [How to Write ML Papers](https://www.alignmentforum.org/posts/eJGptPbbFPZGLpjsp/highly-opinionated-advice-on-how-to-write-ml-papers) |
|
||||
| **Sebastian Farquhar** (DeepMind) | 5-sentence abstract formula | [How to Write ML Papers](https://sebastianfarquhar.com/on-research/2024/11/04/how_to_write_ml_papers/) |
|
||||
| **Gopen & Swan** | 7 principles of reader expectations | [Science of Scientific Writing](https://cseweb.ucsd.edu/~swanson/papers/science-of-writing.pdf) |
|
||||
| **Zachary Lipton** | Word choice, eliminating hedging | [Heuristics for Scientific Writing](https://www.approximatelycorrect.com/2018/01/29/heuristics-technical-scientific-writing-machine-learning-perspective/) |
|
||||
| **Jacob Steinhardt** (UC Berkeley) | Precision, consistent terminology | [Writing Tips](https://bounded-regret.ghost.io/) |
|
||||
| **Ethan Perez** (Anthropic) | Micro-level clarity tips | [Easy Paper Writing Tips](https://ethanperez.net/easy-paper-writing-tips/) |
|
||||
| **Andrej Karpathy** | Single contribution focus | Various lectures |
|
||||
|
||||
**For deeper dives into any of these, see:**
|
||||
- [references/writing-guide.md](references/writing-guide.md) — Full explanations with examples
|
||||
- [references/sources.md](references/sources.md) — Complete bibliography
|
||||
|
||||
### Time Allocation
|
||||
|
||||
Spend approximately **equal time** on each of:
|
||||
1. The abstract
|
||||
2. The introduction
|
||||
3. The figures
|
||||
4. Everything else combined
|
||||
|
||||
**Why?** Most reviewers form judgments before reaching your methods. Readers encounter your paper as: title → abstract → introduction → figures → maybe the rest.
|
||||
|
||||
### Writing Workflow
|
||||
|
||||
```
|
||||
Paper Writing Checklist:
|
||||
- [ ] Step 1: Define the one-sentence contribution
|
||||
- [ ] Step 2: Draft Figure 1 (core idea or most compelling result)
|
||||
- [ ] Step 3: Draft abstract (5-sentence formula)
|
||||
- [ ] Step 4: Draft introduction (1-1.5 pages max)
|
||||
- [ ] Step 5: Draft methods
|
||||
- [ ] Step 6: Draft experiments & results
|
||||
- [ ] Step 7: Draft related work
|
||||
- [ ] Step 8: Draft conclusion & discussion
|
||||
- [ ] Step 9: Draft limitations (REQUIRED by all venues)
|
||||
- [ ] Step 10: Plan appendix (proofs, extra experiments, details)
|
||||
- [ ] Step 11: Complete paper checklist
|
||||
- [ ] Step 12: Final review
|
||||
```
|
||||
|
||||
### Two-Pass Refinement Pattern
|
||||
|
||||
When drafting with an AI agent, use a **two-pass** approach (proven effective in SakanaAI's AI-Scientist pipeline):
|
||||
|
||||
**Pass 1 — Write + immediate refine per section:**
|
||||
For each section, write a complete draft, then immediately refine it in the same context. This catches local issues (clarity, flow, completeness) while the section is fresh.
|
||||
|
||||
**Pass 2 — Global refinement with full-paper context:**
|
||||
After all sections are drafted, revisit each section with awareness of the complete paper. This catches cross-section issues: redundancy, inconsistent terminology, narrative flow, and gaps where one section promises something another doesn't deliver.
|
||||
|
||||
```
|
||||
Second-pass refinement prompt (per section):
|
||||
"Review the [SECTION] in the context of the complete paper.
|
||||
- Does it fit with the rest of the paper? Are there redundancies with other sections?
|
||||
- Is terminology consistent with Introduction and Methods?
|
||||
- Can anything be cut without weakening the message?
|
||||
- Does the narrative flow from the previous section and into the next?
|
||||
Make minimal, targeted edits. Do not rewrite from scratch."
|
||||
```
|
||||
|
||||
### LaTeX Error Checklist
|
||||
|
||||
Append this checklist to every refinement prompt. These are the most common errors when LLMs write LaTeX:
|
||||
|
||||
```
|
||||
LaTeX Quality Checklist (verify after every edit):
|
||||
- [ ] No unenclosed math symbols ($ signs balanced)
|
||||
- [ ] Only reference figures/tables that exist (\ref matches \label)
|
||||
- [ ] No fabricated citations (\cite matches entries in .bib)
|
||||
- [ ] Every \begin{env} has matching \end{env} (especially figure, table, algorithm)
|
||||
- [ ] No HTML contamination (</end{figure}> instead of \end{figure})
|
||||
- [ ] No unescaped underscores outside math mode (use \_ in text)
|
||||
- [ ] No duplicate \label definitions
|
||||
- [ ] No duplicate section headers
|
||||
- [ ] Numbers in text match actual experimental results
|
||||
- [ ] All figures have captions and labels
|
||||
- [ ] No overly long lines that cause overfull hbox warnings
|
||||
```
|
||||
|
||||
### Step 5.0: Title
|
||||
|
||||
The title is the single most-read element of the paper. It determines whether anyone clicks through to the abstract.
|
||||
|
||||
**Good titles**:
|
||||
- State the contribution or finding: "Autoreason: When Iterative LLM Refinement Works and Why It Fails"
|
||||
- Highlight a surprising result: "Scaling Data-Constrained Language Models" (implies you can)
|
||||
- Name the method + what it does: "DPO: Direct Preference Optimization of Language Models"
|
||||
|
||||
**Bad titles**:
|
||||
- Too generic: "An Approach to Improving Language Model Outputs"
|
||||
- Too long: anything over ~15 words
|
||||
- Jargon-only: "Asymptotic Convergence of Iterative Stochastic Policy Refinement" (who is this for?)
|
||||
|
||||
**Rules**:
|
||||
- Include your method name if you have one (for citability)
|
||||
- Include 1-2 keywords reviewers will search for
|
||||
- Avoid colons unless both halves carry meaning
|
||||
- Test: would a reviewer know the domain and contribution from the title alone?
|
||||
|
||||
### Step 5.1: Abstract (5-Sentence Formula)
|
||||
|
||||
From Sebastian Farquhar (DeepMind):
|
||||
|
||||
```
|
||||
1. What you achieved: "We introduce...", "We prove...", "We demonstrate..."
|
||||
2. Why this is hard and important
|
||||
3. How you do it (with specialist keywords for discoverability)
|
||||
4. What evidence you have
|
||||
5. Your most remarkable number/result
|
||||
```
|
||||
|
||||
**Delete** generic openings like "Large language models have achieved remarkable success..."
|
||||
|
||||
### Step 5.2: Figure 1
|
||||
|
||||
Figure 1 is the second thing most readers look at (after abstract). Draft it before writing the introduction — it forces you to clarify the core idea.
|
||||
|
||||
| Figure 1 Type | When to Use | Example |
|
||||
|---------------|-------------|---------|
|
||||
| **Method diagram** | New architecture or pipeline | TikZ flowchart showing your system |
|
||||
| **Results teaser** | One compelling result tells the whole story | Bar chart: "Ours vs baselines" with clear gap |
|
||||
| **Problem illustration** | The problem is unintuitive | Before/after showing failure mode you fix |
|
||||
| **Conceptual diagram** | Abstract contribution needs visual grounding | 2x2 matrix of method properties |
|
||||
|
||||
**Rules**: Figure 1 must be understandable without reading any text. The caption alone should communicate the core idea. Use color purposefully — don't just decorate.
|
||||
|
||||
### Step 5.3: Introduction (1-1.5 pages max)
|
||||
|
||||
Must include:
|
||||
- Clear problem statement
|
||||
- Brief approach overview
|
||||
- 2-4 bullet contribution list (max 1-2 lines each in two-column format)
|
||||
- Methods should start by page 2-3
|
||||
|
||||
### Step 5.4: Methods
|
||||
|
||||
Enable reimplementation:
|
||||
- Conceptual outline or pseudocode
|
||||
- All hyperparameters listed
|
||||
- Architectural details sufficient for reproduction
|
||||
- Present final design decisions; ablations go in experiments
|
||||
|
||||
### Step 5.5: Experiments & Results
|
||||
|
||||
For each experiment, explicitly state:
|
||||
- **What claim it supports**
|
||||
- How it connects to main contribution
|
||||
- What to observe: "the blue line shows X, which demonstrates Y"
|
||||
|
||||
Requirements:
|
||||
- Error bars with methodology (std dev vs std error)
|
||||
- Hyperparameter search ranges
|
||||
- Compute infrastructure (GPU type, total hours)
|
||||
- Seed-setting methods
|
||||
|
||||
### Step 5.6: Related Work
|
||||
|
||||
Organize methodologically, not paper-by-paper. Cite generously — reviewers likely authored relevant papers.
|
||||
|
||||
### Step 5.7: Limitations (REQUIRED)
|
||||
|
||||
All major conferences require this. Honesty helps:
|
||||
- Reviewers are instructed not to penalize honest limitation acknowledgment
|
||||
- Pre-empt criticisms by identifying weaknesses first
|
||||
- Explain why limitations don't undermine core claims
|
||||
|
||||
### Step 5.8: Conclusion & Discussion
|
||||
|
||||
**Conclusion** (required, 0.5-1 page):
|
||||
- Restate the contribution in one sentence (different wording from abstract)
|
||||
- Summarize key findings (2-3 sentences, not a list)
|
||||
- Implications: what does this mean for the field?
|
||||
- Future work: 2-3 concrete next steps (not vague "we leave X for future work")
|
||||
|
||||
**Discussion** (optional, sometimes combined with conclusion):
|
||||
- Broader implications beyond immediate results
|
||||
- Connections to other subfields
|
||||
- Honest assessment of when the method does and doesn't work
|
||||
- Practical deployment considerations
|
||||
|
||||
**Do NOT** introduce new results or claims in the conclusion.
|
||||
|
||||
### Step 5.9: Appendix Strategy
|
||||
|
||||
Appendices are unlimited at all major venues and are essential for reproducibility. Structure:
|
||||
|
||||
| Appendix Section | What Goes Here |
|
||||
|-----------------|---------------|
|
||||
| **Proofs & Derivations** | Full proofs too long for main text. Main text can state theorems with "proof in Appendix A." |
|
||||
| **Additional Experiments** | Ablations, scaling curves, per-dataset breakdowns, hyperparameter sensitivity |
|
||||
| **Implementation Details** | Full hyperparameter tables, training details, hardware specs, random seeds |
|
||||
| **Dataset Documentation** | Data collection process, annotation guidelines, licensing, preprocessing |
|
||||
| **Prompts & Templates** | Exact prompts used (for LLM-based methods), evaluation templates |
|
||||
| **Human Evaluation** | Annotation interface screenshots, instructions given to annotators, IRB details |
|
||||
| **Additional Figures** | Per-task breakdowns, trajectory visualizations, failure case examples |
|
||||
|
||||
**Rules**:
|
||||
- The main paper must be self-contained — reviewers are not required to read appendices
|
||||
- Never put critical evidence only in the appendix
|
||||
- Cross-reference: "Full results in Table 5 (Appendix B)" not just "see appendix"
|
||||
- Use `\appendix` command, then `\section{A: Proofs}` etc.
|
||||
|
||||
### Page Budget Management
|
||||
|
||||
When over the page limit:
|
||||
|
||||
| Cut Strategy | Saves | Risk |
|
||||
|-------------|-------|------|
|
||||
| Move proofs to appendix | 0.5-2 pages | Low — standard practice |
|
||||
| Condense related work | 0.5-1 page | Medium — may miss key citations |
|
||||
| Combine tables with subfigures | 0.25-0.5 page | Low — often improves readability |
|
||||
| Use `\vspace{-Xpt}` sparingly | 0.1-0.3 page | Low if subtle, high if obvious |
|
||||
| Remove qualitative examples | 0.5-1 page | Medium — reviewers like examples |
|
||||
| Reduce figure sizes | 0.25-0.5 page | High — figures must remain readable |
|
||||
|
||||
**Do NOT**: reduce font size, change margins, remove required sections (limitations, broader impact), or use `\small`/`\footnotesize` for main text.
|
||||
|
||||
### Step 5.10: Ethics & Broader Impact Statement
|
||||
|
||||
Most venues now require or strongly encourage an ethics/broader impact statement. This is not boilerplate — reviewers read it and can flag ethics concerns that trigger desk rejection.
|
||||
|
||||
**What to include:**
|
||||
|
||||
| Component | Content | Required By |
|
||||
|-----------|---------|-------------|
|
||||
| **Positive societal impact** | How your work benefits society | NeurIPS, ICML |
|
||||
| **Potential negative impact** | Misuse risks, dual-use concerns, failure modes | NeurIPS, ICML |
|
||||
| **Fairness & bias** | Does your method/data have known biases? | All venues (implicitly) |
|
||||
| **Environmental impact** | Compute carbon footprint for large-scale training | ICML, increasingly NeurIPS |
|
||||
| **Privacy** | Does your work use or enable processing of personal data? | ACL, NeurIPS |
|
||||
| **LLM disclosure** | Was AI used in writing or experiments? | ICLR (mandatory), ACL |
|
||||
|
||||
**Writing the statement:**
|
||||
|
||||
```latex
|
||||
\section*{Broader Impact Statement}
|
||||
% NeurIPS/ICML: after conclusion, does not count toward page limit
|
||||
|
||||
% 1. Positive applications (1-2 sentences)
|
||||
This work enables [specific application] which may benefit [specific group].
|
||||
|
||||
% 2. Risks and mitigations (1-3 sentences, be specific)
|
||||
[Method/model] could potentially be misused for [specific risk]. We mitigate
|
||||
this by [specific mitigation, e.g., releasing only model weights above size X,
|
||||
including safety filters, documenting failure modes].
|
||||
|
||||
% 3. Limitations of impact claims (1 sentence)
|
||||
Our evaluation is limited to [specific domain]; broader deployment would
|
||||
require [specific additional work].
|
||||
```
|
||||
|
||||
**Common mistakes:**
|
||||
- Writing "we foresee no negative impacts" (almost never true — reviewers distrust this)
|
||||
- Being vague: "this could be misused" without specifying how
|
||||
- Ignoring compute costs for large-scale work
|
||||
- Forgetting to disclose LLM use at venues that require it
|
||||
|
||||
**Compute carbon footprint** (for training-heavy papers):
|
||||
```python
|
||||
# Estimate using ML CO2 Impact tool methodology
|
||||
gpu_hours = 1000 # total GPU hours
|
||||
gpu_tdp_watts = 400 # e.g., A100 = 400W
|
||||
pue = 1.1 # Power Usage Effectiveness (data center overhead)
|
||||
carbon_intensity = 0.429 # kg CO2/kWh (US average; varies by region)
|
||||
|
||||
energy_kwh = (gpu_hours * gpu_tdp_watts * pue) / 1000
|
||||
carbon_kg = energy_kwh * carbon_intensity
|
||||
print(f"Energy: {energy_kwh:.0f} kWh, Carbon: {carbon_kg:.0f} kg CO2eq")
|
||||
```
|
||||
|
||||
### Step 5.11: Datasheets & Model Cards (If Applicable)
|
||||
|
||||
If your paper introduces a **new dataset** or **releases a model**, include structured documentation. Reviewers increasingly expect this, and NeurIPS Datasets & Benchmarks track requires it.
|
||||
|
||||
**Datasheets for Datasets** (Gebru et al., 2021) — include in appendix:
|
||||
|
||||
```
|
||||
Dataset Documentation (Appendix):
|
||||
- Motivation: Why was this dataset created? What task does it support?
|
||||
- Composition: What are the instances? How many? What data types?
|
||||
- Collection: How was data collected? What was the source?
|
||||
- Preprocessing: What cleaning/filtering was applied?
|
||||
- Distribution: How is the dataset distributed? Under what license?
|
||||
- Maintenance: Who maintains it? How to report issues?
|
||||
- Ethical considerations: Contains personal data? Consent obtained?
|
||||
Potential for harm? Known biases?
|
||||
```
|
||||
|
||||
**Model Cards** (Mitchell et al., 2019) — include in appendix for model releases:
|
||||
|
||||
```
|
||||
Model Card (Appendix):
|
||||
- Model details: Architecture, training data, training procedure
|
||||
- Intended use: Primary use cases, out-of-scope uses
|
||||
- Metrics: Evaluation metrics and results on benchmarks
|
||||
- Ethical considerations: Known biases, fairness evaluations
|
||||
- Limitations: Known failure modes, domains where model underperforms
|
||||
```
|
||||
|
||||
### Writing Style
|
||||
|
||||
**Sentence-level clarity (Gopen & Swan's 7 Principles):**
|
||||
|
||||
| Principle | Rule |
|
||||
|-----------|------|
|
||||
| Subject-verb proximity | Keep subject and verb close |
|
||||
| Stress position | Place emphasis at sentence ends |
|
||||
| Topic position | Put context first, new info after |
|
||||
| Old before new | Familiar info → unfamiliar info |
|
||||
| One unit, one function | Each paragraph makes one point |
|
||||
| Action in verb | Use verbs, not nominalizations |
|
||||
| Context before new | Set stage before presenting |
|
||||
|
||||
**Word choice (Lipton, Steinhardt):**
|
||||
- Be specific: "accuracy" not "performance"
|
||||
- Eliminate hedging: drop "may" unless genuinely uncertain
|
||||
- Consistent terminology throughout
|
||||
- Avoid incremental vocabulary: "develop", not "combine"
|
||||
|
||||
**Full writing guide with examples**: See [references/writing-guide.md](references/writing-guide.md)
|
||||
|
||||
### Using LaTeX Templates
|
||||
|
||||
**Always copy the entire template directory first, then write within it.**
|
||||
|
||||
```
|
||||
Template Setup Checklist:
|
||||
- [ ] Step 1: Copy entire template directory to new project
|
||||
- [ ] Step 2: Verify template compiles as-is (before any changes)
|
||||
- [ ] Step 3: Read the template's example content to understand structure
|
||||
- [ ] Step 4: Replace example content section by section
|
||||
- [ ] Step 5: Use template macros (check preamble for \newcommand definitions)
|
||||
- [ ] Step 6: Clean up template artifacts only at the end
|
||||
```
|
||||
|
||||
**Step 1: Copy the Full Template**
|
||||
|
||||
```bash
|
||||
cp -r templates/neurips2025/ ~/papers/my-paper/
|
||||
cd ~/papers/my-paper/
|
||||
ls -la # Should see: main.tex, neurips.sty, Makefile, etc.
|
||||
```
|
||||
|
||||
Copy the ENTIRE directory, not just the .tex file. Templates include style files (.sty), bibliography styles (.bst), example content, and Makefiles.
|
||||
|
||||
**Step 2: Verify Template Compiles First**
|
||||
|
||||
Before making ANY changes:
|
||||
```bash
|
||||
latexmk -pdf main.tex
|
||||
# Or manual: pdflatex main.tex && bibtex main && pdflatex main.tex && pdflatex main.tex
|
||||
```
|
||||
|
||||
If the unmodified template doesn't compile, fix that first (usually missing TeX packages — install via `tlmgr install <package>`).
|
||||
|
||||
**Step 3: Keep Template Content as Reference**
|
||||
|
||||
Don't immediately delete example content. Comment it out and use as formatting reference:
|
||||
```latex
|
||||
% Template example (keep for reference):
|
||||
% \begin{figure}[t]
|
||||
% \centering
|
||||
% \includegraphics[width=0.8\linewidth]{example-image}
|
||||
% \caption{Template shows caption style}
|
||||
% \end{figure}
|
||||
|
||||
% Your actual figure:
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.8\linewidth]{your-figure.pdf}
|
||||
\caption{Your caption following the same style.}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Step 4: Replace Content Section by Section**
|
||||
|
||||
Work through systematically: title/authors → abstract → introduction → methods → experiments → related work → conclusion → references → appendix. Compile after each section.
|
||||
|
||||
**Step 5: Use Template Macros**
|
||||
|
||||
```latex
|
||||
\newcommand{\method}{YourMethodName} % Consistent method naming
|
||||
\newcommand{\eg}{e.g.,\xspace} % Proper abbreviations
|
||||
\newcommand{\ie}{i.e.,\xspace}
|
||||
```
|
||||
|
||||
### Template Pitfalls
|
||||
|
||||
| Pitfall | Problem | Solution |
|
||||
|---------|---------|----------|
|
||||
| Copying only `.tex` file | Missing `.sty`, won't compile | Copy entire directory |
|
||||
| Modifying `.sty` files | Breaks conference formatting | Never edit style files |
|
||||
| Adding random packages | Conflicts, breaks template | Only add if necessary |
|
||||
| Deleting template content early | Lose formatting reference | Keep as comments until done |
|
||||
| Not compiling frequently | Errors accumulate | Compile after each section |
|
||||
| Raster PNGs for figures | Blurry in paper | Always use vector PDF via `savefig('fig.pdf')` |
|
||||
|
||||
### Quick Template Reference
|
||||
|
||||
| Conference | Main File | Style File | Page Limit |
|
||||
|------------|-----------|------------|------------|
|
||||
| NeurIPS 2025 | `main.tex` | `neurips.sty` | 9 pages |
|
||||
| ICML 2026 | `example_paper.tex` | `icml2026.sty` | 8 pages |
|
||||
| ICLR 2026 | `iclr2026_conference.tex` | `iclr2026_conference.sty` | 9 pages |
|
||||
| ACL 2025 | `acl_latex.tex` | `acl.sty` | 8 pages (long) |
|
||||
| AAAI 2026 | `aaai2026-unified-template.tex` | `aaai2026.sty` | 7 pages |
|
||||
| COLM 2025 | `colm2025_conference.tex` | `colm2025_conference.sty` | 9 pages |
|
||||
|
||||
**Universal**: Double-blind, references don't count, appendices unlimited, LaTeX required.
|
||||
|
||||
Templates in `templates/` directory. See [templates/README.md](templates/README.md) for compilation setup (VS Code, CLI, Overleaf, other IDEs).
|
||||
|
||||
### Tables and Figures
|
||||
|
||||
**Tables** — use `booktabs` for professional formatting:
|
||||
|
||||
```latex
|
||||
\usepackage{booktabs}
|
||||
\begin{tabular}{lcc}
|
||||
\toprule
|
||||
Method & Accuracy $\uparrow$ & Latency $\downarrow$ \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 45ms \\
|
||||
\textbf{Ours} & \textbf{92.1} & 38ms \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Bold best value per metric
|
||||
- Include direction symbols ($\uparrow$ higher better, $\downarrow$ lower better)
|
||||
- Right-align numerical columns
|
||||
- Consistent decimal precision
|
||||
|
||||
**Figures**:
|
||||
- **Vector graphics** (PDF, EPS) for all plots and diagrams — `plt.savefig('fig.pdf')`
|
||||
- **Raster** (PNG 600 DPI) only for photographs
|
||||
- **Colorblind-safe palettes** (Okabe-Ito or Paul Tol)
|
||||
- Verify **grayscale readability** (8% of men have color vision deficiency)
|
||||
- **No title inside figure** — the caption serves this function
|
||||
- **Self-contained captions** — reader should understand without main text
|
||||
|
||||
### Conference Resubmission
|
||||
|
||||
For converting between venues, see Phase 7 (Submission Preparation) — it covers the full conversion workflow, page-change table, and post-rejection guidance.
|
||||
|
||||
### Professional LaTeX Preamble
|
||||
|
||||
Add these packages to any paper for professional quality. They are compatible with all major conference style files:
|
||||
|
||||
```latex
|
||||
% --- Professional Packages (add after conference style file) ---
|
||||
|
||||
% Typography
|
||||
\usepackage{microtype} % Microtypographic improvements (protrusion, expansion)
|
||||
% Makes text noticeably more polished — always include
|
||||
|
||||
% Tables
|
||||
\usepackage{booktabs} % Professional table rules (\toprule, \midrule, \bottomrule)
|
||||
\usepackage{siunitx} % Consistent number formatting, decimal alignment
|
||||
% Usage: \num{12345} → 12,345; \SI{3.5}{GHz} → 3.5 GHz
|
||||
% Table alignment: S column type for decimal-aligned numbers
|
||||
|
||||
% Figures
|
||||
\usepackage{graphicx} % Include graphics (\includegraphics)
|
||||
\usepackage{subcaption} % Subfigures with (a), (b), (c) labels
|
||||
% Usage: \begin{subfigure}{0.48\textwidth} ... \end{subfigure}
|
||||
|
||||
% Diagrams and Algorithms
|
||||
\usepackage{tikz} % Programmable vector diagrams
|
||||
\usetikzlibrary{arrows.meta, positioning, shapes.geometric, calc, fit, backgrounds}
|
||||
\usepackage[ruled,vlined]{algorithm2e} % Professional pseudocode
|
||||
% Alternative: \usepackage{algorithmicx} if template bundles it
|
||||
|
||||
% Cross-references
|
||||
\usepackage{cleveref} % Smart references: \cref{fig:x} → "Figure 1"
|
||||
% MUST be loaded AFTER hyperref
|
||||
% Handles: figures, tables, sections, equations, algorithms
|
||||
|
||||
% Math (usually included by conference .sty, but verify)
|
||||
\usepackage{amsmath,amssymb} % AMS math environments and symbols
|
||||
\usepackage{mathtools} % Extends amsmath (dcases, coloneqq, etc.)
|
||||
|
||||
% Colors (for figures and diagrams)
|
||||
\usepackage{xcolor} % Color management
|
||||
% Okabe-Ito colorblind-safe palette:
|
||||
\definecolor{okblue}{HTML}{0072B2}
|
||||
\definecolor{okorange}{HTML}{E69F00}
|
||||
\definecolor{okgreen}{HTML}{009E73}
|
||||
\definecolor{okred}{HTML}{D55E00}
|
||||
\definecolor{okpurple}{HTML}{CC79A7}
|
||||
\definecolor{okcyan}{HTML}{56B4E9}
|
||||
\definecolor{okyellow}{HTML}{F0E442}
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- `microtype` is the single highest-impact package for visual quality. It adjusts character spacing at a sub-pixel level. Always include it.
|
||||
- `siunitx` handles decimal alignment in tables via the `S` column type — eliminates manual spacing.
|
||||
- `cleveref` must be loaded **after** `hyperref`. Most conference .sty files load hyperref, so put cleveref last.
|
||||
- Check if the conference template already loads any of these (especially `algorithm`, `amsmath`, `graphicx`). Don't double-load.
|
||||
|
||||
### siunitx Table Alignment
|
||||
|
||||
`siunitx` makes number-heavy tables significantly more readable:
|
||||
|
||||
```latex
|
||||
\begin{tabular}{l S[table-format=2.1] S[table-format=2.1] S[table-format=2.1]}
|
||||
\toprule
|
||||
Method & {Accuracy $\uparrow$} & {F1 $\uparrow$} & {Latency (ms) $\downarrow$} \\
|
||||
\midrule
|
||||
Baseline & 85.2 & 83.7 & 45.3 \\
|
||||
Ablation (no X) & 87.1 & 85.4 & 42.1 \\
|
||||
\textbf{Ours} & \textbf{92.1} & \textbf{90.8} & \textbf{38.7} \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
```
|
||||
|
||||
The `S` column type auto-aligns on the decimal point. Headers in `{}` escape the alignment.
|
||||
|
||||
### Subfigures
|
||||
|
||||
Standard pattern for side-by-side figures:
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_a.pdf}
|
||||
\caption{Results on Dataset A.}
|
||||
\label{fig:results-a}
|
||||
\end{subfigure}
|
||||
\hfill
|
||||
\begin{subfigure}[b]{0.48\textwidth}
|
||||
\centering
|
||||
\includegraphics[width=\textwidth]{fig_results_b.pdf}
|
||||
\caption{Results on Dataset B.}
|
||||
\label{fig:results-b}
|
||||
\end{subfigure}
|
||||
\caption{Comparison of our method across two datasets. (a) shows the scaling
|
||||
behavior and (b) shows the ablation results. Both use 5 random seeds.}
|
||||
\label{fig:results}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
Use `\cref{fig:results}` → "Figure 1", `\cref{fig:results-a}` → "Figure 1a".
|
||||
|
||||
### Pseudocode with algorithm2e
|
||||
|
||||
```latex
|
||||
\begin{algorithm}[t]
|
||||
\caption{Iterative Refinement with Judge Panel}
|
||||
\label{alg:method}
|
||||
\KwIn{Task $T$, model $M$, judges $J_1 \ldots J_n$, convergence threshold $k$}
|
||||
\KwOut{Final output $A^*$}
|
||||
$A \gets M(T)$ \tcp*{Initial generation}
|
||||
$\text{streak} \gets 0$\;
|
||||
\While{$\text{streak} < k$}{
|
||||
$C \gets \text{Critic}(A, T)$ \tcp*{Identify weaknesses}
|
||||
$B \gets M(T, C)$ \tcp*{Revised version addressing critique}
|
||||
$AB \gets \text{Synthesize}(A, B)$ \tcp*{Merge best elements}
|
||||
\ForEach{judge $J_i$}{
|
||||
$\text{rank}_i \gets J_i(\text{shuffle}(A, B, AB))$ \tcp*{Blind ranking}
|
||||
}
|
||||
$\text{winner} \gets \text{BordaCount}(\text{ranks})$\;
|
||||
\eIf{$\text{winner} = A$}{
|
||||
$\text{streak} \gets \text{streak} + 1$\;
|
||||
}{
|
||||
$A \gets \text{winner}$; $\text{streak} \gets 0$\;
|
||||
}
|
||||
}
|
||||
\Return{$A$}\;
|
||||
\end{algorithm}
|
||||
```
|
||||
|
||||
### TikZ Diagram Patterns
|
||||
|
||||
TikZ is the standard for method diagrams in ML papers. Common patterns:
|
||||
|
||||
**Pipeline/Flow Diagram** (most common in ML papers):
|
||||
|
||||
```latex
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\begin{tikzpicture}[
|
||||
node distance=1.8cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=1cm,
|
||||
minimum width=2cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
]
|
||||
\node[box, fill=okcyan!20] (input) {Input\\$x$};
|
||||
\node[box, fill=okblue!20, right of=input] (encoder) {Encoder\\$f_\theta$};
|
||||
\node[box, fill=okgreen!20, right of=encoder] (latent) {Latent\\$z$};
|
||||
\node[box, fill=okorange!20, right of=latent] (decoder) {Decoder\\$g_\phi$};
|
||||
\node[box, fill=okred!20, right of=decoder] (output) {Output\\$\hat{x}$};
|
||||
|
||||
\draw[arrow] (input) -- (encoder);
|
||||
\draw[arrow] (encoder) -- (latent);
|
||||
\draw[arrow] (latent) -- (decoder);
|
||||
\draw[arrow] (decoder) -- (output);
|
||||
\end{tikzpicture}
|
||||
\caption{Architecture overview. The encoder maps input $x$ to latent
|
||||
representation $z$, which the decoder reconstructs.}
|
||||
\label{fig:architecture}
|
||||
\end{figure}
|
||||
```
|
||||
|
||||
**Comparison/Matrix Diagram** (for showing method variants):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
cell/.style={rectangle, draw, minimum width=2.5cm, minimum height=1cm,
|
||||
align=center, font=\small},
|
||||
header/.style={cell, fill=gray!20, font=\small\bfseries},
|
||||
]
|
||||
% Headers
|
||||
\node[header] at (0, 0) {Method};
|
||||
\node[header] at (3, 0) {Converges?};
|
||||
\node[header] at (6, 0) {Quality?};
|
||||
% Rows
|
||||
\node[cell] at (0, -1) {Single Pass};
|
||||
\node[cell, fill=okgreen!15] at (3, -1) {N/A};
|
||||
\node[cell, fill=okorange!15] at (6, -1) {Baseline};
|
||||
\node[cell] at (0, -2) {Critique+Revise};
|
||||
\node[cell, fill=okred!15] at (3, -2) {No};
|
||||
\node[cell, fill=okred!15] at (6, -2) {Degrades};
|
||||
\node[cell] at (0, -3) {Ours};
|
||||
\node[cell, fill=okgreen!15] at (3, -3) {Yes ($k$=2)};
|
||||
\node[cell, fill=okgreen!15] at (6, -3) {Improves};
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
**Iterative Loop Diagram** (for methods with feedback):
|
||||
|
||||
```latex
|
||||
\begin{tikzpicture}[
|
||||
node distance=2cm,
|
||||
box/.style={rectangle, draw, rounded corners, minimum height=0.8cm,
|
||||
minimum width=1.8cm, align=center, font=\small},
|
||||
arrow/.style={-{Stealth[length=3mm]}, thick},
|
||||
label/.style={font=\scriptsize, midway, above},
|
||||
]
|
||||
\node[box, fill=okblue!20] (gen) {Generator};
|
||||
\node[box, fill=okred!20, right=2.5cm of gen] (critic) {Critic};
|
||||
\node[box, fill=okgreen!20, below=1.5cm of $(gen)!0.5!(critic)$] (judge) {Judge Panel};
|
||||
|
||||
\draw[arrow] (gen) -- node[label] {output $A$} (critic);
|
||||
\draw[arrow] (critic) -- node[label, right] {critique $C$} (judge);
|
||||
\draw[arrow] (judge) -| node[label, left, pos=0.3] {winner} (gen);
|
||||
\end{tikzpicture}
|
||||
```
|
||||
|
||||
### latexdiff for Revision Tracking
|
||||
|
||||
Essential for rebuttals — generates a marked-up PDF showing changes between versions:
|
||||
|
||||
```bash
|
||||
# Install
|
||||
# macOS: brew install latexdiff (or comes with TeX Live)
|
||||
# Linux: sudo apt install latexdiff
|
||||
|
||||
# Generate diff
|
||||
latexdiff paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
pdflatex paper_diff.tex
|
||||
|
||||
# For multi-file projects (with \input{} or \include{})
|
||||
latexdiff --flatten paper_v1.tex paper_v2.tex > paper_diff.tex
|
||||
```
|
||||
|
||||
This produces a PDF with deletions in red strikethrough and additions in blue — standard format for rebuttal supplements.
|
||||
|
||||
### SciencePlots for matplotlib
|
||||
|
||||
Install and use for publication-quality plots:
|
||||
|
||||
```bash
|
||||
pip install SciencePlots
|
||||
```
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import scienceplots # registers styles
|
||||
|
||||
# Use science style (IEEE-like, clean)
|
||||
with plt.style.context(['science', 'no-latex']):
|
||||
fig, ax = plt.subplots(figsize=(3.5, 2.5)) # Single-column width
|
||||
ax.plot(x, y, label='Ours', color='#0072B2')
|
||||
ax.plot(x, y2, label='Baseline', color='#D55E00', linestyle='--')
|
||||
ax.set_xlabel('Training Steps')
|
||||
ax.set_ylabel('Accuracy')
|
||||
ax.legend()
|
||||
fig.savefig('paper/fig_results.pdf', bbox_inches='tight')
|
||||
|
||||
# Available styles: 'science', 'ieee', 'nature', 'science+ieee'
|
||||
# Add 'no-latex' if LaTeX is not installed on the machine generating plots
|
||||
```
|
||||
|
||||
**Standard figure sizes** (two-column format):
|
||||
- Single column: `figsize=(3.5, 2.5)` — fits in one column
|
||||
- Double column: `figsize=(7.0, 3.0)` — spans both columns
|
||||
- Square: `figsize=(3.5, 3.5)` — for heatmaps, confusion matrices
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,433 @@
|
||||
# Reviewer Guidelines & Evaluation Criteria
|
||||
|
||||
This reference documents how reviewers evaluate papers at major ML/AI conferences, helping authors anticipate and address reviewer concerns.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
|
||||
- [Universal Evaluation Dimensions](#universal-evaluation-dimensions)
|
||||
- [NeurIPS Reviewer Guidelines](#neurips-reviewer-guidelines)
|
||||
- [ICML Reviewer Guidelines](#icml-reviewer-guidelines)
|
||||
- [ICLR Reviewer Guidelines](#iclr-reviewer-guidelines)
|
||||
- [ACL Reviewer Guidelines](#acl-reviewer-guidelines)
|
||||
- [What Makes Reviews Strong](#what-makes-reviews-strong)
|
||||
- [Common Reviewer Concerns](#common-reviewer-concerns)
|
||||
- [How to Address Reviewer Feedback](#how-to-address-reviewer-feedback)
|
||||
|
||||
---
|
||||
|
||||
## Universal Evaluation Dimensions
|
||||
|
||||
All major ML conferences assess papers across four core dimensions:
|
||||
|
||||
### 1. Quality (Technical Soundness)
|
||||
|
||||
**What reviewers ask:**
|
||||
- Are claims well-supported by theoretical analysis or experimental results?
|
||||
- Are the proofs correct? Are the experiments properly controlled?
|
||||
- Are baselines appropriate and fairly compared?
|
||||
- Is the methodology sound?
|
||||
|
||||
**How to ensure high quality:**
|
||||
- Include complete proofs (main paper or appendix with sketches)
|
||||
- Use appropriate baselines (not strawmen)
|
||||
- Report variance/error bars with methodology
|
||||
- Document hyperparameter selection process
|
||||
|
||||
### 2. Clarity (Writing & Organization)
|
||||
|
||||
**What reviewers ask:**
|
||||
- Is the paper clearly written and well organized?
|
||||
- Can an expert in the field reproduce the results?
|
||||
- Is notation consistent? Are terms defined?
|
||||
- Is the paper self-contained?
|
||||
|
||||
**How to ensure clarity:**
|
||||
- Use consistent terminology throughout
|
||||
- Define all notation at first use
|
||||
- Include reproducibility details (appendix acceptable)
|
||||
- Have non-authors read before submission
|
||||
|
||||
### 3. Significance (Impact & Importance)
|
||||
|
||||
**What reviewers ask:**
|
||||
- Are the results impactful for the community?
|
||||
- Will others build upon this work?
|
||||
- Does it address an important problem?
|
||||
- What is the potential for real-world impact?
|
||||
|
||||
**How to demonstrate significance:**
|
||||
- Clearly articulate the problem's importance
|
||||
- Connect to broader research themes
|
||||
- Discuss potential applications
|
||||
- Compare to existing approaches meaningfully
|
||||
|
||||
### 4. Originality (Novelty & Contribution)
|
||||
|
||||
**What reviewers ask:**
|
||||
- Does this provide new insights?
|
||||
- How does it differ from prior work?
|
||||
- Is the contribution non-trivial?
|
||||
|
||||
**Key insight from NeurIPS guidelines:**
|
||||
> "Originality does not necessarily require introducing an entirely new method. Papers that provide novel insights from evaluating existing approaches or shed light on why methods succeed can also be highly original."
|
||||
|
||||
---
|
||||
|
||||
## NeurIPS Reviewer Guidelines
|
||||
|
||||
### Scoring System (1-6 Scale)
|
||||
|
||||
| Score | Label | Description |
|
||||
|-------|-------|-------------|
|
||||
| **6** | Strong Accept | Groundbreaking, flawless work; top 2-3% of submissions |
|
||||
| **5** | Accept | Technically solid, high impact; would benefit the community |
|
||||
| **4** | Borderline Accept | Solid work with limited evaluation; leans accept |
|
||||
| **3** | Borderline Reject | Solid but weaknesses outweigh strengths; leans reject |
|
||||
| **2** | Reject | Technical flaws or weak evaluation |
|
||||
| **1** | Strong Reject | Well-known results or unaddressed ethics concerns |
|
||||
|
||||
### Reviewer Instructions
|
||||
|
||||
Reviewers are explicitly instructed to:
|
||||
|
||||
1. **Evaluate the paper as written** - not what it could be with revisions
|
||||
2. **Provide constructive feedback** - 3-5 actionable points
|
||||
3. **Not penalize honest limitations** - acknowledging weaknesses is encouraged
|
||||
4. **Assess reproducibility** - can the work be verified?
|
||||
5. **Consider ethical implications** - potential misuse or harm
|
||||
|
||||
### What Reviewers Should Avoid
|
||||
|
||||
- Superficial, uninformed reviews
|
||||
- Demanding unreasonable additional experiments
|
||||
- Penalizing authors for honest limitation acknowledgment
|
||||
- Rejecting for missing citations to reviewer's own work
|
||||
|
||||
### Timeline (NeurIPS 2025 — verify dates for current year)
|
||||
|
||||
- Bidding: May 17-21
|
||||
- Reviewing period: May 29 - July 2
|
||||
- Author rebuttals: July 24-30
|
||||
- Discussion period: July 31 - August 13
|
||||
- Final notifications: September 18
|
||||
|
||||
> **Note**: These dates are from the 2025 cycle. Always check the current year's call for papers at the venue website.
|
||||
|
||||
---
|
||||
|
||||
## ICML Reviewer Guidelines
|
||||
|
||||
### Review Structure
|
||||
|
||||
ICML reviewers provide:
|
||||
|
||||
1. **Summary** - Brief description of contributions
|
||||
2. **Strengths** - Positive aspects
|
||||
3. **Weaknesses** - Areas for improvement
|
||||
4. **Questions** - Clarifications for authors
|
||||
5. **Limitations** - Assessment of stated limitations
|
||||
6. **Ethics** - Any concerns
|
||||
7. **Overall Score** - Recommendation
|
||||
|
||||
### Scoring Guidelines
|
||||
|
||||
ICML uses a similar 1-6 scale with calibration:
|
||||
- Top 25% of accepted papers: Score 5-6
|
||||
- Typical accepted paper: Score 4-5
|
||||
- Borderline: Score 3-4
|
||||
- Clear reject: Score 1-2
|
||||
|
||||
### Key Evaluation Points
|
||||
|
||||
1. **Reproducibility** - Are there enough details?
|
||||
2. **Experimental rigor** - Multiple seeds, proper baselines?
|
||||
3. **Writing quality** - Clear, organized, well-structured?
|
||||
4. **Novelty** - Non-trivial contribution?
|
||||
|
||||
---
|
||||
|
||||
## ICLR Reviewer Guidelines
|
||||
|
||||
### OpenReview Process
|
||||
|
||||
ICLR uses OpenReview with:
|
||||
- Public reviews (after acceptance decisions)
|
||||
- Author responses visible to reviewers
|
||||
- Discussion between reviewers and ACs
|
||||
|
||||
### Scoring
|
||||
|
||||
ICLR reviews include:
|
||||
- **Soundness**: 1-4 scale
|
||||
- **Presentation**: 1-4 scale
|
||||
- **Contribution**: 1-4 scale
|
||||
- **Overall**: 1-10 scale
|
||||
- **Confidence**: 1-5 scale
|
||||
|
||||
### Unique ICLR Considerations
|
||||
|
||||
1. **LLM Disclosure** - Reviewers assess whether LLM use is properly disclosed
|
||||
2. **Reproducibility** - Emphasis on code availability
|
||||
3. **Reciprocal Reviewing** - Authors must also serve as reviewers
|
||||
|
||||
---
|
||||
|
||||
## ACL Reviewer Guidelines
|
||||
|
||||
### ACL-Specific Criteria
|
||||
|
||||
ACL adds NLP-specific evaluation:
|
||||
|
||||
1. **Linguistic soundness** - Are linguistic claims accurate?
|
||||
2. **Resource documentation** - Are datasets/models properly documented?
|
||||
3. **Multilingual consideration** - If applicable, is language diversity addressed?
|
||||
|
||||
### Limitations Section
|
||||
|
||||
ACL specifically requires a Limitations section. Reviewers check:
|
||||
- Are limitations honest and comprehensive?
|
||||
- Do limitations undermine core claims?
|
||||
- Are potential negative impacts addressed?
|
||||
|
||||
### Ethics Review
|
||||
|
||||
ACL has a dedicated ethics review process for:
|
||||
- Dual-use concerns
|
||||
- Data privacy issues
|
||||
- Bias and fairness implications
|
||||
|
||||
---
|
||||
|
||||
## AAAI Reviewer Guidelines
|
||||
|
||||
### Evaluation Criteria
|
||||
|
||||
AAAI reviewers evaluate along similar axes to NeurIPS/ICML but with some differences:
|
||||
|
||||
| Criterion | Weight | Notes |
|
||||
|-----------|--------|-------|
|
||||
| **Technical quality** | High | Soundness of approach, correctness of results |
|
||||
| **Significance** | High | Importance of the problem and contribution |
|
||||
| **Novelty** | Medium-High | New ideas, methods, or insights |
|
||||
| **Clarity** | Medium | Clear writing, well-organized presentation |
|
||||
| **Reproducibility** | Medium | Sufficient detail to reproduce results |
|
||||
|
||||
### AAAI-Specific Considerations
|
||||
|
||||
- **Broader AI scope**: AAAI covers all of AI, not just ML. Papers on planning, reasoning, knowledge representation, NLP, vision, robotics, and multi-agent systems are all in scope. Reviewers may not be deep ML specialists.
|
||||
- **Formatting strictness**: AAAI reviewers are instructed to flag formatting violations. Non-compliant papers may be desk-rejected before review.
|
||||
- **Application papers**: AAAI is more receptive to application-focused work than NeurIPS/ICML. Framing a strong application contribution is viable.
|
||||
- **Senior Program Committee**: AAAI uses SPCs (Senior Program Committee members) who mediate between reviewers and make accept/reject recommendations.
|
||||
|
||||
### Scoring (AAAI Scale)
|
||||
|
||||
- **Strong Accept**: Clearly above threshold, excellent contribution
|
||||
- **Accept**: Above threshold, good contribution with minor issues
|
||||
- **Weak Accept**: Borderline, merits outweigh concerns
|
||||
- **Weak Reject**: Borderline, concerns outweigh merits
|
||||
- **Reject**: Below threshold, significant issues
|
||||
- **Strong Reject**: Well below threshold
|
||||
|
||||
---
|
||||
|
||||
## COLM Reviewer Guidelines
|
||||
|
||||
### Evaluation Criteria
|
||||
|
||||
COLM reviews focus on relevance to language modeling in addition to standard criteria:
|
||||
|
||||
| Criterion | Weight | Notes |
|
||||
|-----------|--------|-------|
|
||||
| **Relevance** | High | Must be relevant to language modeling community |
|
||||
| **Technical quality** | High | Sound methodology, well-supported claims |
|
||||
| **Novelty** | Medium-High | New insights about language models |
|
||||
| **Clarity** | Medium | Clear presentation, reproducible |
|
||||
| **Significance** | Medium-High | Impact on LM research and practice |
|
||||
|
||||
### COLM-Specific Considerations
|
||||
|
||||
- **Language model focus**: Reviewers will assess whether the contribution advances understanding of language models. General ML contributions need explicit LM framing.
|
||||
- **Newer venue norms**: COLM is newer than NeurIPS/ICML, so reviewer calibration varies more. Write more defensively — anticipate a wider range of reviewer expertise.
|
||||
- **ICLR-derived process**: Review process is modeled on ICLR (open reviews, author response period, discussion among reviewers).
|
||||
- **Broad interpretation of "language modeling"**: Includes training, evaluation, alignment, safety, efficiency, applications, theory, multimodality (if language is central), and social impact of LMs.
|
||||
|
||||
### Scoring
|
||||
|
||||
COLM uses an ICLR-style scoring system:
|
||||
- **8-10**: Strong accept (top papers)
|
||||
- **6-7**: Weak accept (solid contribution)
|
||||
- **5**: Borderline
|
||||
- **3-4**: Weak reject (below threshold)
|
||||
- **1-2**: Strong reject
|
||||
|
||||
---
|
||||
|
||||
## What Makes Reviews Strong
|
||||
|
||||
### Following Daniel Dennett's Rules
|
||||
|
||||
Good reviewers follow these principles:
|
||||
|
||||
1. **Re-express the position fairly** - Show you understand the paper
|
||||
2. **List agreements** - Acknowledge what works well
|
||||
3. **List what you learned** - Credit the contribution
|
||||
4. **Only then critique** - After establishing understanding
|
||||
|
||||
### Review Structure Best Practices
|
||||
|
||||
**Strong Review Structure:**
|
||||
```
|
||||
Summary (1 paragraph):
|
||||
- What the paper does
|
||||
- Main contribution claimed
|
||||
|
||||
Strengths (3-5 bullets):
|
||||
- Specific positive aspects
|
||||
- Why these matter
|
||||
|
||||
Weaknesses (3-5 bullets):
|
||||
- Specific concerns
|
||||
- Why these matter
|
||||
- Suggestions for addressing
|
||||
|
||||
Questions (2-4 items):
|
||||
- Clarifications needed
|
||||
- Things that would change assessment
|
||||
|
||||
Minor Issues (optional):
|
||||
- Typos, unclear sentences
|
||||
- Formatting issues
|
||||
|
||||
Overall Assessment:
|
||||
- Clear recommendation with reasoning
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Reviewer Concerns
|
||||
|
||||
### Technical Concerns
|
||||
|
||||
| Concern | How to Pre-empt |
|
||||
|---------|-----------------|
|
||||
| "Baselines too weak" | Use state-of-the-art baselines, cite recent work |
|
||||
| "Missing ablations" | Include systematic ablation study |
|
||||
| "No error bars" | Report std dev/error, multiple runs |
|
||||
| "Hyperparameters not tuned" | Document tuning process, search ranges |
|
||||
| "Claims not supported" | Ensure every claim has evidence |
|
||||
|
||||
### Novelty Concerns
|
||||
|
||||
| Concern | How to Pre-empt |
|
||||
|---------|-----------------|
|
||||
| "Incremental contribution" | Clearly articulate what's new vs prior work |
|
||||
| "Similar to [paper X]" | Explicitly compare to X in Related Work |
|
||||
| "Straightforward extension" | Highlight non-obvious aspects |
|
||||
|
||||
### Clarity Concerns
|
||||
|
||||
| Concern | How to Pre-empt |
|
||||
|---------|-----------------|
|
||||
| "Hard to follow" | Use clear structure, signposting |
|
||||
| "Notation inconsistent" | Review all notation, create notation table |
|
||||
| "Missing details" | Include reproducibility appendix |
|
||||
| "Figures unclear" | Self-contained captions, proper sizing |
|
||||
|
||||
### Significance Concerns
|
||||
|
||||
| Concern | How to Pre-empt |
|
||||
|---------|-----------------|
|
||||
| "Limited impact" | Discuss broader implications |
|
||||
| "Narrow evaluation" | Evaluate on multiple benchmarks |
|
||||
| "Only works in restricted setting" | Acknowledge scope, explain why still valuable |
|
||||
|
||||
---
|
||||
|
||||
## How to Address Reviewer Feedback
|
||||
|
||||
### Rebuttal Best Practices
|
||||
|
||||
**Do:**
|
||||
- Thank reviewers for their time
|
||||
- Address each concern specifically
|
||||
- Provide evidence (new experiments if possible)
|
||||
- Be concise—reviewers are busy
|
||||
- Acknowledge valid criticisms
|
||||
|
||||
**Don't:**
|
||||
- Be defensive or dismissive
|
||||
- Make promises you can't keep
|
||||
- Ignore difficult criticisms
|
||||
- Write excessively long rebuttals
|
||||
- Argue about subjective assessments
|
||||
|
||||
### Rebuttal Template
|
||||
|
||||
```markdown
|
||||
We thank the reviewers for their thoughtful feedback.
|
||||
|
||||
## Reviewer 1
|
||||
|
||||
**R1-Q1: [Quoted concern]**
|
||||
[Direct response with evidence]
|
||||
|
||||
**R1-Q2: [Quoted concern]**
|
||||
[Direct response with evidence]
|
||||
|
||||
## Reviewer 2
|
||||
|
||||
...
|
||||
|
||||
## Summary of Changes
|
||||
If accepted, we will:
|
||||
1. [Specific change]
|
||||
2. [Specific change]
|
||||
3. [Specific change]
|
||||
```
|
||||
|
||||
### When to Accept Criticism
|
||||
|
||||
Some reviewer feedback should simply be accepted:
|
||||
- Valid technical errors
|
||||
- Missing important related work
|
||||
- Unclear explanations
|
||||
- Missing experimental details
|
||||
|
||||
Acknowledge these gracefully: "The reviewer is correct that... We will revise to..."
|
||||
|
||||
### When to Push Back
|
||||
|
||||
You can respectfully disagree when:
|
||||
- Reviewer misunderstood the paper
|
||||
- Requested experiments are out of scope
|
||||
- Criticism is factually incorrect
|
||||
|
||||
Frame disagreements constructively: "We appreciate this perspective. However, [explanation]..."
|
||||
|
||||
---
|
||||
|
||||
## Pre-Submission Reviewer Simulation
|
||||
|
||||
Before submitting, ask yourself:
|
||||
|
||||
**Quality:**
|
||||
- [ ] Would I trust these results if I saw them?
|
||||
- [ ] Are all claims supported by evidence?
|
||||
- [ ] Are baselines fair and recent?
|
||||
|
||||
**Clarity:**
|
||||
- [ ] Can someone reproduce this from the paper?
|
||||
- [ ] Is the writing clear to non-experts in this subfield?
|
||||
- [ ] Are all terms and notation defined?
|
||||
|
||||
**Significance:**
|
||||
- [ ] Why should the community care about this?
|
||||
- [ ] What can people do with this work?
|
||||
- [ ] Is the problem important?
|
||||
|
||||
**Originality:**
|
||||
- [ ] What specifically is new here?
|
||||
- [ ] How does this differ from closest related work?
|
||||
- [ ] Is the contribution non-trivial?
|
||||
@@ -0,0 +1,191 @@
|
||||
# Source Bibliography
|
||||
|
||||
This document lists all authoritative sources used to build this skill, organized by topic.
|
||||
|
||||
---
|
||||
|
||||
## Origin & Attribution
|
||||
|
||||
The writing philosophy, citation verification workflow, and conference reference materials in this skill were originally compiled by **[Orchestra Research](https://github.com/orchestra-research)** as the `ml-paper-writing` skill (January 2026), drawing on Neel Nanda's blog post and other researcher guides listed below. The skill was integrated into hermes-agent by teknium (January 2026), then expanded into the current `research-paper-writing` pipeline by SHL0MS (April 2026, PR #4654), which added experiment design, execution monitoring, iterative refinement, and submission phases while preserving the original writing philosophy and reference files.
|
||||
|
||||
---
|
||||
|
||||
## Writing Philosophy & Guides
|
||||
|
||||
### Primary Sources (Must-Read)
|
||||
|
||||
| Source | Author | URL | Key Contribution |
|
||||
|--------|--------|-----|------------------|
|
||||
| **Highly Opinionated Advice on How to Write ML Papers** | Neel Nanda | [Alignment Forum](https://www.alignmentforum.org/posts/eJGptPbbFPZGLpjsp/highly-opinionated-advice-on-how-to-write-ml-papers) | Narrative framework, "What/Why/So What", time allocation |
|
||||
| **How to Write ML Papers** | Sebastian Farquhar (DeepMind) | [Blog](https://sebastianfarquhar.com/on-research/2024/11/04/how_to_write_ml_papers/) | 5-sentence abstract formula, structure templates |
|
||||
| **A Survival Guide to a PhD** | Andrej Karpathy | [Blog](http://karpathy.github.io/2016/09/07/phd/) | Paper structure recipe, contribution framing |
|
||||
| **Heuristics for Scientific Writing** | Zachary Lipton (CMU) | [Blog](https://www.approximatelycorrect.com/2018/01/29/heuristics-technical-scientific-writing-machine-learning-perspective/) | Word choice, section balance, intensifier warnings |
|
||||
| **Advice for Authors** | Jacob Steinhardt (UC Berkeley) | [Blog](https://jsteinhardt.stat.berkeley.edu/blog/advice-for-authors) | Precision over brevity, consistent terminology |
|
||||
| **Easy Paper Writing Tips** | Ethan Perez (Anthropic) | [Blog](https://ethanperez.net/easy-paper-writing-tips/) | Micro-level tips, apostrophe unfolding, clarity tricks |
|
||||
|
||||
### Foundational Scientific Writing
|
||||
|
||||
| Source | Author | URL | Key Contribution |
|
||||
|--------|--------|-----|------------------|
|
||||
| **The Science of Scientific Writing** | Gopen & Swan | [PDF](https://cseweb.ucsd.edu/~swanson/papers/science-of-writing.pdf) | Topic/stress positions, old-before-new, 7 principles |
|
||||
| **Summary of Science of Scientific Writing** | Lawrence Crowl | [Summary](https://www.crowl.org/Lawrence/writing/GopenSwan90.html) | Condensed version of Gopen & Swan |
|
||||
|
||||
### Additional Resources
|
||||
|
||||
| Source | URL | Key Contribution |
|
||||
|--------|-----|------------------|
|
||||
| How To Write A Research Paper In ML | [Blog](https://grigorisg9gr.github.io/machine%20learning/research%20paper/how-to-write-a-research-paper-in-machine-learning/) | Practical walkthrough, LaTeX tips |
|
||||
| A Recipe for Training Neural Networks | [Karpathy Blog](http://karpathy.github.io/2019/04/25/recipe/) | Debugging methodology that translates to paper structure |
|
||||
| ICML Paper Writing Best Practices | [ICML](https://icml.cc/Conferences/2022/BestPractices) | Official venue guidance |
|
||||
| Bill Freeman's Writing Slides | [MIT](https://billf.mit.edu/sites/default/files/documents/cvprPapers.pdf) | Visual guide to paper structure |
|
||||
|
||||
---
|
||||
|
||||
## Official Conference Guidelines
|
||||
|
||||
### NeurIPS
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| Paper Checklist Guidelines | [NeurIPS](https://neurips.cc/public/guides/PaperChecklist) | 16-item mandatory checklist |
|
||||
| Reviewer Guidelines 2025 | [NeurIPS](https://neurips.cc/Conferences/2025/ReviewerGuidelines) | Evaluation criteria, scoring |
|
||||
| Style Files | [NeurIPS](https://neurips.cc/Conferences/2025/PaperInformation/StyleFiles) | LaTeX templates |
|
||||
|
||||
### ICML
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| Paper Guidelines | [ICML](https://icml.cc/Conferences/2024/PaperGuidelines) | Submission requirements |
|
||||
| Reviewer Instructions 2025 | [ICML](https://icml.cc/Conferences/2025/ReviewerInstructions) | Review form, evaluation |
|
||||
| Style & Author Instructions | [ICML](https://icml.cc/Conferences/2022/StyleAuthorInstructions) | Formatting specifications |
|
||||
|
||||
### ICLR
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| Author Guide 2026 | [ICLR](https://iclr.cc/Conferences/2026/AuthorGuide) | Submission requirements, LLM disclosure |
|
||||
| Reviewer Guide 2025 | [ICLR](https://iclr.cc/Conferences/2025/ReviewerGuide) | Review process, evaluation |
|
||||
|
||||
### ACL/EMNLP
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| ACL Style Files | [GitHub](https://github.com/acl-org/acl-style-files) | LaTeX templates |
|
||||
| ACL Rolling Review | [ARR](https://aclrollingreview.org/) | Submission process |
|
||||
|
||||
### AAAI
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| Author Kit 2026 | [AAAI](https://aaai.org/authorkit26/) | Templates and guidelines |
|
||||
|
||||
### COLM
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| Template | [GitHub](https://github.com/COLM-org/Template) | LaTeX templates |
|
||||
|
||||
---
|
||||
|
||||
## Citation APIs & Tools
|
||||
|
||||
### APIs
|
||||
|
||||
| API | Documentation | Best For |
|
||||
|-----|---------------|----------|
|
||||
| **Semantic Scholar** | [Docs](https://api.semanticscholar.org/api-docs/) | ML/AI papers, citation graphs |
|
||||
| **CrossRef** | [Docs](https://www.crossref.org/documentation/retrieve-metadata/rest-api/) | DOI lookup, BibTeX retrieval |
|
||||
| **arXiv** | [Docs](https://info.arxiv.org/help/api/basics.html) | Preprints, PDF access |
|
||||
| **OpenAlex** | [Docs](https://docs.openalex.org/) | Open alternative, bulk access |
|
||||
|
||||
### Python Libraries
|
||||
|
||||
| Library | Install | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `semanticscholar` | `pip install semanticscholar` | Semantic Scholar wrapper |
|
||||
| `arxiv` | `pip install arxiv` | arXiv search and download |
|
||||
| `habanero` | `pip install habanero` | CrossRef client |
|
||||
|
||||
### Citation Verification
|
||||
|
||||
| Tool | URL | Purpose |
|
||||
|------|-----|---------|
|
||||
| Citely | [citely.ai](https://citely.ai/citation-checker) | Batch verification |
|
||||
| ReciteWorks | [reciteworks.com](https://reciteworks.com/) | In-text citation checking |
|
||||
|
||||
---
|
||||
|
||||
## Visualization & Formatting
|
||||
|
||||
### Figure Creation
|
||||
|
||||
| Tool | URL | Purpose |
|
||||
|------|-----|---------|
|
||||
| PlotNeuralNet | [GitHub](https://github.com/HarisIqbal88/PlotNeuralNet) | TikZ neural network diagrams |
|
||||
| SciencePlots | [GitHub](https://github.com/garrettj403/SciencePlots) | Publication-ready matplotlib |
|
||||
| Okabe-Ito Palette | [Reference](https://jfly.uni-koeln.de/color/) | Colorblind-safe colors |
|
||||
|
||||
### LaTeX Resources
|
||||
|
||||
| Resource | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| Overleaf Templates | [Overleaf](https://www.overleaf.com/latex/templates) | Online LaTeX editor |
|
||||
| BibLaTeX Guide | [CTAN](https://ctan.org/pkg/biblatex) | Modern citation management |
|
||||
|
||||
---
|
||||
|
||||
## Research on AI Writing & Hallucination
|
||||
|
||||
| Source | URL | Key Finding |
|
||||
|--------|-----|-------------|
|
||||
| AI Hallucinations in Citations | [Enago](https://www.enago.com/academy/ai-hallucinations-research-citations/) | ~40% error rate |
|
||||
| Hallucination in AI Writing | [PMC](https://pmc.ncbi.nlm.nih.gov/articles/PMC10726751/) | Types of citation errors |
|
||||
| NeurIPS 2025 AI Report | [ByteIota](https://byteiota.com/neurips-2025-100-ai-hallucinations-slip-through-review/) | 100+ hallucinated citations |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference by Topic
|
||||
|
||||
### For Narrative & Structure
|
||||
→ Start with: Neel Nanda, Sebastian Farquhar, Andrej Karpathy
|
||||
|
||||
### For Sentence-Level Clarity
|
||||
→ Start with: Gopen & Swan, Ethan Perez, Zachary Lipton
|
||||
|
||||
### For Word Choice & Style
|
||||
→ Start with: Zachary Lipton, Jacob Steinhardt
|
||||
|
||||
### For Conference-Specific Requirements
|
||||
→ Start with: Official venue guidelines (NeurIPS, ICML, ICLR, ACL)
|
||||
|
||||
### For Citation Management
|
||||
→ Start with: Semantic Scholar API, CrossRef, citation-workflow.md
|
||||
|
||||
### For Reviewer Expectations
|
||||
→ Start with: Venue reviewer guidelines, reviewer-guidelines.md
|
||||
|
||||
### For Human Evaluation
|
||||
→ Start with: human-evaluation.md, Prolific/MTurk documentation
|
||||
|
||||
### For Non-Empirical Papers (Theory, Survey, Benchmark, Position)
|
||||
→ Start with: paper-types.md
|
||||
|
||||
---
|
||||
|
||||
## Human Evaluation & Annotation
|
||||
|
||||
| Source | URL | Key Contribution |
|
||||
|--------|-----|------------------|
|
||||
| **Datasheets for Datasets** | Gebru et al., 2021 ([arXiv](https://arxiv.org/abs/1803.09010)) | Structured dataset documentation framework |
|
||||
| **Model Cards for Model Reporting** | Mitchell et al., 2019 ([arXiv](https://arxiv.org/abs/1810.03993)) | Structured model documentation framework |
|
||||
| **Crowdsourcing and Human Computation** | [Survey](https://arxiv.org/abs/2202.06516) | Best practices for crowdsourced annotation |
|
||||
| **Krippendorff's Alpha** | [Wikipedia](https://en.wikipedia.org/wiki/Krippendorff%27s_alpha) | Inter-annotator agreement metric reference |
|
||||
| **Prolific** | [prolific.co](https://www.prolific.co/) | Recommended crowdsourcing platform for research |
|
||||
|
||||
## Ethics & Broader Impact
|
||||
|
||||
| Source | URL | Key Contribution |
|
||||
|--------|-----|------------------|
|
||||
| **ML CO2 Impact** | [mlco2.github.io](https://mlco2.github.io/impact/) | Compute carbon footprint calculator |
|
||||
| **NeurIPS Broader Impact Guide** | [NeurIPS](https://neurips.cc/public/guides/PaperChecklist) | Official guidance on impact statements |
|
||||
| **ACL Ethics Policy** | [ACL](https://www.aclweb.org/portal/content/acl-code-ethics) | Ethics requirements for NLP research |
|
||||
@@ -0,0 +1,474 @@
|
||||
# ML Paper Writing Philosophy & Best Practices
|
||||
|
||||
This reference compiles writing advice from prominent ML researchers including Neel Nanda, Andrej Karpathy, Sebastian Farquhar, Zachary Lipton, and Jacob Steinhardt.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
|
||||
- [The Narrative Principle](#the-narrative-principle)
|
||||
- [Time Allocation](#time-allocation)
|
||||
- [Abstract Writing Formula](#abstract-writing-formula)
|
||||
- [Introduction Structure](#introduction-structure)
|
||||
- [Sentence-Level Clarity](#sentence-level-clarity)
|
||||
- [Word Choice and Precision](#word-choice-and-precision)
|
||||
- [Mathematical Writing](#mathematical-writing)
|
||||
- [Figure Design](#figure-design)
|
||||
- [Common Mistakes to Avoid](#common-mistakes-to-avoid)
|
||||
|
||||
---
|
||||
|
||||
## The Narrative Principle
|
||||
|
||||
### From Neel Nanda
|
||||
|
||||
"A paper is a short, rigorous, evidence-based technical story with a takeaway readers care about."
|
||||
|
||||
The narrative rests on three pillars that must be crystal clear by the end of your introduction:
|
||||
|
||||
**The "What"**: One to three specific novel claims fitting within a cohesive theme. Vague contributions like "we study X" fail immediately—reviewers need precise, falsifiable claims.
|
||||
|
||||
**The "Why"**: Rigorous empirical evidence that convincingly supports those claims, including strong baselines honestly tuned and experiments that distinguish between competing hypotheses rather than merely showing "decent results."
|
||||
|
||||
**The "So What"**: Why readers should care, connecting your contribution to problems the community recognizes as important.
|
||||
|
||||
### From Andrej Karpathy
|
||||
|
||||
"A paper is not a random collection of experiments you report on. The paper sells a single thing that was not obvious or present before. The entire paper is organized around this core contribution with surgical precision."
|
||||
|
||||
This applies whether you're presenting a new architecture, a theoretical result, or improved understanding of existing methods—NeurIPS explicitly notes that "originality does not necessarily require an entirely new method."
|
||||
|
||||
**Practical Implication**: If you cannot state your contribution in one sentence, you don't yet have a paper. Everything else—experiments, related work, discussion—exists only to support that core claim.
|
||||
|
||||
---
|
||||
|
||||
## Time Allocation
|
||||
|
||||
### From Neel Nanda
|
||||
|
||||
Spend approximately **the same amount of time** on each of:
|
||||
1. The abstract
|
||||
2. The introduction
|
||||
3. The figures
|
||||
4. Everything else combined
|
||||
|
||||
This isn't hyperbole—most reviewers form preliminary judgments before reaching your methods section. Readers encounter your paper in a predictable pattern: **title → abstract → introduction → figures → maybe the rest.**
|
||||
|
||||
### Reviewer Reading Patterns
|
||||
|
||||
Studies of reviewer behavior show:
|
||||
- Abstract is read 100% of the time
|
||||
- Introduction is skimmed by 90%+ of reviewers
|
||||
- Figures are examined before methods by most reviewers
|
||||
- Full methods are read only if interest is established
|
||||
|
||||
**Implication**: Front-load your paper's value. Don't bury the contribution.
|
||||
|
||||
---
|
||||
|
||||
## Abstract Writing Formula
|
||||
|
||||
### Sebastian Farquhar's 5-Sentence Formula
|
||||
|
||||
1. **What you achieved**: "We introduce...", "We prove...", "We demonstrate..."
|
||||
2. **Why this is hard and important**
|
||||
3. **How you do it** (with specialist keywords for discoverability)
|
||||
4. **What evidence you have**
|
||||
5. **Your most remarkable number/result**
|
||||
|
||||
### Example (Good Abstract)
|
||||
|
||||
```
|
||||
We prove that gradient descent on overparameterized neural networks
|
||||
converges to global minima at a linear rate. [What]
|
||||
This resolves a fundamental question about why deep learning works
|
||||
despite non-convex optimization landscapes. [Why hard/important]
|
||||
Our proof relies on showing that the Neural Tangent Kernel remains
|
||||
approximately constant during training, reducing the problem to
|
||||
kernel regression. [How with keywords]
|
||||
We validate our theory on CIFAR-10 and ImageNet, showing that
|
||||
predicted convergence rates match experiments within 5%. [Evidence]
|
||||
This is the first polynomial-time convergence guarantee for
|
||||
networks with practical depth and width. [Remarkable result]
|
||||
```
|
||||
|
||||
### What to Avoid
|
||||
|
||||
From Zachary Lipton: "If the first sentence can be pre-pended to any ML paper, delete it."
|
||||
|
||||
**Delete these openings**:
|
||||
- "Large language models have achieved remarkable success..."
|
||||
- "Deep learning has revolutionized..."
|
||||
- "In recent years, neural networks have..."
|
||||
|
||||
**Start with your specific contribution instead.**
|
||||
|
||||
---
|
||||
|
||||
## Introduction Structure
|
||||
|
||||
### Requirements
|
||||
|
||||
- **1-1.5 pages maximum** (in two-column format)
|
||||
- **Methods should start by page 2-3**
|
||||
- Must include **2-4 bullet contribution list** (max 1-2 lines each)
|
||||
|
||||
### Structure Template
|
||||
|
||||
```markdown
|
||||
1. Opening Hook (2-3 sentences)
|
||||
- State the problem your paper addresses
|
||||
- Why it matters RIGHT NOW
|
||||
|
||||
2. Background/Challenge (1 paragraph)
|
||||
- What makes this problem hard?
|
||||
- What have others tried? Why is it insufficient?
|
||||
|
||||
3. Your Approach (1 paragraph)
|
||||
- What do you do differently?
|
||||
- Key insight that enables your contribution
|
||||
|
||||
4. Contribution Bullets (2-4 items)
|
||||
- Be specific and falsifiable
|
||||
- Each bullet: 1-2 lines maximum
|
||||
|
||||
5. Results Preview (2-3 sentences)
|
||||
- Most impressive numbers
|
||||
- Scope of evaluation
|
||||
|
||||
6. Paper Organization (optional, 1-2 sentences)
|
||||
- "Section 2 presents... Section 3 describes..."
|
||||
```
|
||||
|
||||
### Contribution Bullets: Good vs Bad
|
||||
|
||||
**Good:**
|
||||
- We prove that X converges in O(n log n) time under assumption Y
|
||||
- We introduce Z, a 3-layer architecture that reduces memory by 40%
|
||||
- We demonstrate that A outperforms B by 15% on benchmark C
|
||||
|
||||
**Bad:**
|
||||
- We study the problem of X (not a contribution)
|
||||
- We provide extensive experiments (too vague)
|
||||
- We make several contributions to the field (says nothing)
|
||||
|
||||
---
|
||||
|
||||
## Sentence-Level Clarity
|
||||
|
||||
### From Gopen & Swan: "The Science of Scientific Writing"
|
||||
|
||||
The seminal 1990 paper by George Gopen and Judith Swan establishes that **readers have structural expectations** about where information appears in prose. Violating these expectations forces readers to spend energy on structure rather than content.
|
||||
|
||||
> "If the reader is to grasp what the writer means, the writer must understand what the reader needs."
|
||||
|
||||
#### The 7 Principles of Reader Expectations
|
||||
|
||||
**Principle 1: Subject-Verb Proximity**
|
||||
|
||||
Keep grammatical subject and verb close together. Anything intervening reads as interruption of lesser importance.
|
||||
|
||||
**Weak**: "The model, which was trained on 100M tokens and fine-tuned on domain-specific data using LoRA with rank 16, achieves state-of-the-art results"
|
||||
|
||||
**Strong**: "The model achieves state-of-the-art results after training on 100M tokens and fine-tuning with LoRA (rank 16)"
|
||||
|
||||
**Principle 2: Stress Position (Save the Best for Last)**
|
||||
|
||||
Readers naturally emphasize the **last words of a sentence**. Place your most important information there.
|
||||
|
||||
**Weak**: "Accuracy improves by 15% when using attention"
|
||||
**Strong**: "When using attention, accuracy improves by **15%**"
|
||||
|
||||
**Principle 3: Topic Position (First Things First)**
|
||||
|
||||
The beginning of a sentence establishes perspective. Put the "whose story" element first—readers expect the sentence to be about whoever shows up first.
|
||||
|
||||
**Weak**: "A novel attention mechanism that computes alignment scores is introduced"
|
||||
**Strong**: "To address the alignment problem, we introduce a novel attention mechanism"
|
||||
|
||||
**Principle 4: Old Information Before New**
|
||||
|
||||
Put familiar information (old) in the topic position for backward linkage; put new information in the stress position for emphasis.
|
||||
|
||||
**Weak**: "Sparse attention was introduced by Child et al. The quadratic complexity of standard attention motivates this work."
|
||||
**Strong**: "Standard attention has quadratic complexity. To address this, Child et al. introduced sparse attention."
|
||||
|
||||
**Principle 5: One Unit, One Function**
|
||||
|
||||
Each unit of discourse (sentence, paragraph, section) should serve a single function. If you have two points, use two units.
|
||||
|
||||
**Principle 6: Articulate Action in the Verb**
|
||||
|
||||
Express the action of each sentence in its verb, not in nominalized nouns.
|
||||
|
||||
**Weak**: "We performed an analysis of the results" (nominalization)
|
||||
**Strong**: "We analyzed the results" (action in verb)
|
||||
|
||||
**Principle 7: Context Before New Information**
|
||||
|
||||
Provide context before asking the reader to consider anything new. This applies at all levels—sentence, paragraph, section.
|
||||
|
||||
**Weak**: "Equation 3 shows that convergence is guaranteed when the learning rate satisfies..."
|
||||
**Strong**: "For convergence to be guaranteed, the learning rate must satisfy the condition in Equation 3..."
|
||||
|
||||
#### Summary Table
|
||||
|
||||
| Principle | Rule | Mnemonic |
|
||||
|-----------|------|----------|
|
||||
| Subject-Verb Proximity | Keep subject and verb close | "Don't interrupt yourself" |
|
||||
| Stress Position | Emphasis at sentence end | "Save the best for last" |
|
||||
| Topic Position | Context at sentence start | "First things first" |
|
||||
| Old Before New | Familiar → unfamiliar | "Build on known ground" |
|
||||
| One Unit, One Function | Each paragraph = one point | "One idea per container" |
|
||||
| Action in Verb | Use verbs, not nominalizations | "Verbs do, nouns sit" |
|
||||
| Context Before New | Explain before presenting | "Set the stage first" |
|
||||
|
||||
---
|
||||
|
||||
## Micro-Level Writing Tips
|
||||
|
||||
### From Ethan Perez (Anthropic)
|
||||
|
||||
These practical micro-level tips improve clarity at the sentence and word level.
|
||||
|
||||
#### Pronoun Management
|
||||
|
||||
**Minimize pronouns** ("this," "it," "these," "that"). When pronouns are necessary, use them as adjectives with a noun:
|
||||
|
||||
**Weak**: "This shows that the model converges."
|
||||
**Strong**: "This result shows that the model converges."
|
||||
|
||||
**Weak**: "It improves performance."
|
||||
**Strong**: "This modification improves performance."
|
||||
|
||||
#### Verb Placement
|
||||
|
||||
**Position verbs early** in sentences for better parsing:
|
||||
|
||||
**Weak**: "The gradient, after being computed and normalized, updates the weights."
|
||||
**Strong**: "The gradient updates the weights after being computed and normalized."
|
||||
|
||||
#### Apostrophe Unfolding
|
||||
|
||||
Transform possessive constructions for clarity:
|
||||
|
||||
**Original**: "X's Y" → **Unfolded**: "The Y of X"
|
||||
|
||||
**Before**: "The model's accuracy on the test set"
|
||||
**After**: "The accuracy of the model on the test set"
|
||||
|
||||
This isn't always better, but when sentences feel awkward, try unfolding.
|
||||
|
||||
#### Words to Eliminate
|
||||
|
||||
Delete these filler words in almost all cases:
|
||||
- "actually"
|
||||
- "a bit"
|
||||
- "fortunately" / "unfortunately"
|
||||
- "very" / "really"
|
||||
- "quite"
|
||||
- "basically"
|
||||
- "essentially"
|
||||
- Excessive connectives ("however," "moreover," "furthermore" when not needed)
|
||||
|
||||
#### Sentence Construction Rules
|
||||
|
||||
1. **One idea per sentence** - If struggling to express an idea in one sentence, it needs two
|
||||
2. **No repeated sounds** - Avoid similar-sounding words in the same sentence
|
||||
3. **Every sentence adds information** - Delete sentences that merely restate
|
||||
4. **Active voice always** - Specify the actor ("We find..." not "It is found...")
|
||||
5. **Expand contractions** - "don't" → "do not" for formality
|
||||
|
||||
#### Paragraph Architecture
|
||||
|
||||
- **First sentence**: State the point clearly
|
||||
- **Middle sentences**: Support with evidence
|
||||
- **Last sentence**: Reinforce or transition
|
||||
|
||||
Don't bury key information in the middle of paragraphs.
|
||||
|
||||
---
|
||||
|
||||
## Word Choice and Precision
|
||||
|
||||
### From Zachary Lipton
|
||||
|
||||
**Eliminate hedging** unless genuine uncertainty exists:
|
||||
- Delete "may" and "can" unless necessary
|
||||
- "provides *very* tight approximation" drips with insecurity
|
||||
- "provides tight approximation" is confident
|
||||
|
||||
**Avoid vacuous intensifiers**:
|
||||
- Delete: very, extremely, highly, significantly (unless statistical)
|
||||
- These words signal insecurity, not strength
|
||||
|
||||
### From Jacob Steinhardt
|
||||
|
||||
**Precision over brevity**: Replace vague terms with specific ones.
|
||||
|
||||
| Vague | Specific |
|
||||
|-------|----------|
|
||||
| performance | accuracy, latency, throughput |
|
||||
| improves | increases accuracy by X%, reduces latency by Y |
|
||||
| large | 1B parameters, 100M tokens |
|
||||
| fast | 3x faster, 50ms latency |
|
||||
| good results | 92% accuracy, 0.85 F1 |
|
||||
|
||||
**Consistent terminology**: Referring to the same concept with different terms creates confusion.
|
||||
|
||||
**Choose one and stick with it**:
|
||||
- "model" vs "network" vs "architecture"
|
||||
- "training" vs "learning" vs "optimization"
|
||||
- "sample" vs "example" vs "instance"
|
||||
|
||||
### Vocabulary Signaling
|
||||
|
||||
**Avoid words signaling incremental work**:
|
||||
- Never: "combine," "modify," "expand," "extend"
|
||||
- Instead: "develop," "propose," "introduce"
|
||||
|
||||
**Why**: "We combine X and Y" sounds like you stapled two existing ideas together. "We develop a method that leverages X for Y" sounds like genuine contribution.
|
||||
|
||||
---
|
||||
|
||||
## Mathematical Writing
|
||||
|
||||
### From Ethan Perez
|
||||
|
||||
**Unfold apostrophes** for clarity:
|
||||
- Weak: "X's Y"
|
||||
- Strong: "The Y of X"
|
||||
|
||||
Example: "the model's accuracy" → "the accuracy of the model"
|
||||
|
||||
### General Principles
|
||||
|
||||
1. **State all assumptions formally** before theorems
|
||||
2. **Provide intuitive explanations** alongside proofs
|
||||
3. **Use consistent notation** throughout the paper
|
||||
4. **Define symbols at first use**
|
||||
|
||||
### Notation Conventions
|
||||
|
||||
```latex
|
||||
% Scalars: lowercase italic
|
||||
$x$, $y$, $\alpha$, $\beta$
|
||||
|
||||
% Vectors: lowercase bold
|
||||
$\mathbf{x}$, $\mathbf{v}$
|
||||
|
||||
% Matrices: uppercase bold
|
||||
$\mathbf{W}$, $\mathbf{X}$
|
||||
|
||||
% Sets: uppercase calligraphic
|
||||
$\mathcal{X}$, $\mathcal{D}$
|
||||
|
||||
% Functions: roman for named functions
|
||||
$\mathrm{softmax}$, $\mathrm{ReLU}$
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Figure Design
|
||||
|
||||
### From Neel Nanda
|
||||
|
||||
Figures should tell a coherent story even if the reader skips the text. Many readers DO skip the text initially.
|
||||
|
||||
### Design Principles
|
||||
|
||||
1. **Figure 1 is crucial**: Often the first thing readers examine after abstract
|
||||
2. **Self-contained captions**: Reader should understand figure without main text
|
||||
3. **No title inside figure**: The caption serves this function (ICML/NeurIPS rule)
|
||||
4. **Vector graphics**: PDF/EPS for plots, PNG (600 DPI) only for photographs
|
||||
|
||||
### Accessibility Requirements
|
||||
|
||||
8% of men have color vision deficiency. Your figures must work for them.
|
||||
|
||||
**Solutions**:
|
||||
- Use colorblind-safe palettes: Okabe-Ito or Paul Tol
|
||||
- Avoid red-green combinations
|
||||
- Verify figures work in grayscale
|
||||
- Use different line styles (solid, dashed, dotted) in addition to colors
|
||||
|
||||
### Tools
|
||||
|
||||
```python
|
||||
# SciencePlots: Publication-ready styles
|
||||
import matplotlib.pyplot as plt
|
||||
plt.style.use(['science', 'ieee'])
|
||||
|
||||
# Or for Nature-style
|
||||
plt.style.use(['science', 'nature'])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Mistakes to Avoid
|
||||
|
||||
### Structure Mistakes
|
||||
|
||||
| Mistake | Solution |
|
||||
|---------|----------|
|
||||
| Introduction too long (>1.5 pages) | Move background to Related Work |
|
||||
| Methods buried (after page 3) | Front-load contribution, cut intro |
|
||||
| Missing contribution bullets | Add 2-4 specific, falsifiable claims |
|
||||
| Experiments without explicit claims | State what each experiment tests |
|
||||
|
||||
### Writing Mistakes
|
||||
|
||||
| Mistake | Solution |
|
||||
|---------|----------|
|
||||
| Generic abstract opening | Start with your specific contribution |
|
||||
| Inconsistent terminology | Choose one term per concept |
|
||||
| Passive voice overuse | Use active voice: "We show" not "It is shown" |
|
||||
| Hedging everywhere | Be confident unless genuinely uncertain |
|
||||
|
||||
### Figure Mistakes
|
||||
|
||||
| Mistake | Solution |
|
||||
|---------|----------|
|
||||
| Raster graphics for plots | Use vector (PDF/EPS) |
|
||||
| Red-green color scheme | Use colorblind-safe palette |
|
||||
| Title inside figure | Put title in caption |
|
||||
| Captions require main text | Make captions self-contained |
|
||||
|
||||
### Citation Mistakes
|
||||
|
||||
| Mistake | Solution |
|
||||
|---------|----------|
|
||||
| Paper-by-paper Related Work | Organize methodologically |
|
||||
| Missing relevant citations | Reviewers authored papers—cite generously |
|
||||
| AI-generated citations | Always verify via APIs |
|
||||
| Inconsistent citation format | Use BibLaTeX with consistent keys |
|
||||
|
||||
---
|
||||
|
||||
## Pre-Submission Checklist
|
||||
|
||||
Before submitting, verify:
|
||||
|
||||
**Narrative**:
|
||||
- [ ] Can state contribution in one sentence
|
||||
- [ ] Three pillars (What/Why/So What) clear in intro
|
||||
- [ ] Every experiment supports a specific claim
|
||||
|
||||
**Structure**:
|
||||
- [ ] Abstract follows 5-sentence formula
|
||||
- [ ] Introduction ≤1.5 pages
|
||||
- [ ] Methods start by page 2-3
|
||||
- [ ] 2-4 contribution bullets included
|
||||
- [ ] Limitations section present
|
||||
|
||||
**Writing**:
|
||||
- [ ] Consistent terminology throughout
|
||||
- [ ] No generic opening sentences
|
||||
- [ ] Hedging removed unless necessary
|
||||
- [ ] All figures have self-contained captions
|
||||
|
||||
**Technical**:
|
||||
- [ ] All citations verified via API
|
||||
- [ ] Error bars included with methodology
|
||||
- [ ] Compute resources documented
|
||||
- [ ] Code/data availability stated
|
||||
@@ -0,0 +1,251 @@
|
||||
# LaTeX Templates for ML/AI Conferences
|
||||
|
||||
This directory contains official LaTeX templates for major machine learning and AI conferences.
|
||||
|
||||
---
|
||||
|
||||
## Compiling LaTeX to PDF
|
||||
|
||||
### Option 1: VS Code with LaTeX Workshop (Recommended)
|
||||
|
||||
**Setup:**
|
||||
1. Install [TeX Live](https://www.tug.org/texlive/) (full distribution recommended)
|
||||
- macOS: `brew install --cask mactex`
|
||||
- Ubuntu: `sudo apt install texlive-full`
|
||||
- Windows: Download from [tug.org/texlive](https://www.tug.org/texlive/)
|
||||
|
||||
2. Install VS Code extension: **LaTeX Workshop** by James Yu
|
||||
- Open VS Code → Extensions (Cmd/Ctrl+Shift+X) → Search "LaTeX Workshop" → Install
|
||||
|
||||
**Usage:**
|
||||
- Open any `.tex` file in VS Code
|
||||
- Save the file (Cmd/Ctrl+S) → Auto-compiles to PDF
|
||||
- Click the green play button or use `Cmd/Ctrl+Alt+B` to build
|
||||
- View PDF: Click "View LaTeX PDF" icon or `Cmd/Ctrl+Alt+V`
|
||||
- Side-by-side view: `Cmd/Ctrl+Alt+V` then drag tab
|
||||
|
||||
**Settings** (add to VS Code `settings.json`):
|
||||
```json
|
||||
{
|
||||
"latex-workshop.latex.autoBuild.run": "onSave",
|
||||
"latex-workshop.view.pdf.viewer": "tab",
|
||||
"latex-workshop.latex.recipes": [
|
||||
{
|
||||
"name": "pdflatex → bibtex → pdflatex × 2",
|
||||
"tools": ["pdflatex", "bibtex", "pdflatex", "pdflatex"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Option 2: Command Line
|
||||
|
||||
```bash
|
||||
# Basic compilation
|
||||
pdflatex main.tex
|
||||
|
||||
# With bibliography (full workflow)
|
||||
pdflatex main.tex
|
||||
bibtex main
|
||||
pdflatex main.tex
|
||||
pdflatex main.tex
|
||||
|
||||
# Using latexmk (handles dependencies automatically)
|
||||
latexmk -pdf main.tex
|
||||
|
||||
# Continuous compilation (watches for changes)
|
||||
latexmk -pdf -pvc main.tex
|
||||
```
|
||||
|
||||
### Option 3: Overleaf (Online)
|
||||
|
||||
1. Go to [overleaf.com](https://www.overleaf.com)
|
||||
2. New Project → Upload Project → Upload the template folder as ZIP
|
||||
3. Edit online with real-time PDF preview
|
||||
4. No local installation needed
|
||||
|
||||
### Option 4: Other IDEs
|
||||
|
||||
| IDE | Extension/Plugin | Notes |
|
||||
|-----|------------------|-------|
|
||||
| **Cursor** | LaTeX Workshop | Same as VS Code |
|
||||
| **Sublime Text** | LaTeXTools | Popular, well-maintained |
|
||||
| **Vim/Neovim** | VimTeX | Powerful, keyboard-driven |
|
||||
| **Emacs** | AUCTeX | Comprehensive LaTeX environment |
|
||||
| **TeXstudio** | Built-in | Dedicated LaTeX IDE |
|
||||
| **Texmaker** | Built-in | Cross-platform LaTeX editor |
|
||||
|
||||
### Troubleshooting Compilation
|
||||
|
||||
**"File not found" errors:**
|
||||
```bash
|
||||
# Ensure you're in the template directory
|
||||
cd templates/icml2026
|
||||
pdflatex example_paper.tex
|
||||
```
|
||||
|
||||
**Bibliography not appearing:**
|
||||
```bash
|
||||
# Run bibtex after first pdflatex
|
||||
pdflatex main.tex
|
||||
bibtex main # Uses main.aux to find citations
|
||||
pdflatex main.tex # Incorporates bibliography
|
||||
pdflatex main.tex # Resolves references
|
||||
```
|
||||
|
||||
**Missing packages:**
|
||||
```bash
|
||||
# TeX Live package manager
|
||||
tlmgr install <package-name>
|
||||
|
||||
# Or install full distribution to avoid this
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Available Templates
|
||||
|
||||
| Conference | Directory | Year | Source |
|
||||
|------------|-----------|------|--------|
|
||||
| ICML | `icml2026/` | 2026 | [Official ICML](https://icml.cc/Conferences/2026/AuthorInstructions) |
|
||||
| ICLR | `iclr2026/` | 2026 | [Official GitHub](https://github.com/ICLR/Master-Template) |
|
||||
| NeurIPS | `neurips2025/` | 2025 | Community template |
|
||||
| ACL | `acl/` | 2025+ | [Official ACL](https://github.com/acl-org/acl-style-files) |
|
||||
| AAAI | `aaai2026/` | 2026 | [AAAI Author Kit](https://aaai.org/authorkit26/) |
|
||||
| COLM | `colm2025/` | 2025 | [Official COLM](https://github.com/COLM-org/Template) |
|
||||
|
||||
## Usage
|
||||
|
||||
### ICML 2026
|
||||
|
||||
```latex
|
||||
\documentclass{article}
|
||||
\usepackage{icml2026} % For submission
|
||||
% \usepackage[accepted]{icml2026} % For camera-ready
|
||||
|
||||
\begin{document}
|
||||
% Your paper content
|
||||
\end{document}
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `icml2026.sty` - Style file
|
||||
- `icml2026.bst` - Bibliography style
|
||||
- `example_paper.tex` - Example document
|
||||
|
||||
### ICLR 2026
|
||||
|
||||
```latex
|
||||
\documentclass{article}
|
||||
\usepackage[submission]{iclr2026_conference} % For submission
|
||||
% \usepackage[final]{iclr2026_conference} % For camera-ready
|
||||
|
||||
\begin{document}
|
||||
% Your paper content
|
||||
\end{document}
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `iclr2026_conference.sty` - Style file
|
||||
- `iclr2026_conference.bst` - Bibliography style
|
||||
- `iclr2026_conference.tex` - Example document
|
||||
|
||||
### ACL Venues (ACL, EMNLP, NAACL)
|
||||
|
||||
```latex
|
||||
\documentclass[11pt]{article}
|
||||
\usepackage[review]{acl} % For review
|
||||
% \usepackage{acl} % For camera-ready
|
||||
|
||||
\begin{document}
|
||||
% Your paper content
|
||||
\end{document}
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `acl.sty` - Style file
|
||||
- `acl_natbib.bst` - Bibliography style
|
||||
- `acl_latex.tex` - Example document
|
||||
|
||||
### AAAI 2026
|
||||
|
||||
```latex
|
||||
\documentclass[letterpaper]{article}
|
||||
\usepackage[submission]{aaai2026} % For submission
|
||||
% \usepackage{aaai2026} % For camera-ready
|
||||
|
||||
\begin{document}
|
||||
% Your paper content
|
||||
\end{document}
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `aaai2026.sty` - Style file
|
||||
- `aaai2026.bst` - Bibliography style
|
||||
|
||||
### COLM 2025
|
||||
|
||||
```latex
|
||||
\documentclass{article}
|
||||
\usepackage[submission]{colm2025_conference} % For submission
|
||||
% \usepackage[final]{colm2025_conference} % For camera-ready
|
||||
|
||||
\begin{document}
|
||||
% Your paper content
|
||||
\end{document}
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `colm2025_conference.sty` - Style file
|
||||
- `colm2025_conference.bst` - Bibliography style
|
||||
|
||||
## Page Limits Summary
|
||||
|
||||
| Conference | Submission | Camera-Ready | Notes |
|
||||
|------------|-----------|--------------|-------|
|
||||
| ICML 2026 | 8 pages | 9 pages | +unlimited refs/appendix |
|
||||
| ICLR 2026 | 9 pages | 10 pages | +unlimited refs/appendix |
|
||||
| NeurIPS 2025 | 9 pages | 9 pages | +checklist outside limit |
|
||||
| ACL 2025 | 8 pages (long) | varies | +unlimited refs/appendix |
|
||||
| AAAI 2026 | 7 pages | 8 pages | +unlimited refs/appendix |
|
||||
| COLM 2025 | 9 pages | 10 pages | +unlimited refs/appendix |
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Compilation Errors
|
||||
|
||||
1. **Missing packages**: Install full TeX distribution (TeX Live Full or MikTeX)
|
||||
2. **Bibliography errors**: Use the provided `.bst` file with `\bibliographystyle{}`
|
||||
3. **Font warnings**: Install `cm-super` or use `\usepackage{lmodern}`
|
||||
|
||||
### Anonymization
|
||||
|
||||
For submission, ensure:
|
||||
- No author names in `\author{}`
|
||||
- No acknowledgments section
|
||||
- No grant numbers
|
||||
- Use anonymous repositories
|
||||
- Cite own work in third person
|
||||
|
||||
### Common LaTeX Packages
|
||||
|
||||
```latex
|
||||
% Recommended packages (check compatibility with venue style)
|
||||
\usepackage{amsmath,amsthm,amssymb} % Math
|
||||
\usepackage{graphicx} % Figures
|
||||
\usepackage{booktabs} % Tables
|
||||
\usepackage{hyperref} % Links
|
||||
\usepackage{algorithm,algorithmic} % Algorithms
|
||||
\usepackage{natbib} % Citations
|
||||
```
|
||||
|
||||
## Updating Templates
|
||||
|
||||
Templates are updated annually. Check official sources before each submission:
|
||||
|
||||
- ICML: https://icml.cc/
|
||||
- ICLR: https://iclr.cc/
|
||||
- NeurIPS: https://neurips.cc/
|
||||
- ACL: https://github.com/acl-org/acl-style-files
|
||||
- AAAI: https://aaai.org/
|
||||
- COLM: https://colmweb.org/
|
||||
@@ -0,0 +1,534 @@
|
||||
# AAAI 2026 统一LaTeX模板使用说明 / AAAI 2026 Unified LaTeX Template Guide
|
||||
|
||||
> **📝 重要说明 / Important Notice**: 本仓库借助Cursor在AAAI 2026官方模板基础上改进得到。如果遇到不满足或有冲突的情况，请积极提issues。
|
||||
>
|
||||
> **📝 Important Notice**: This repository is improved based on the official AAAI 2026 template with the assistance of Cursor. If you encounter any issues or conflicts, please actively submit issues.
|
||||
|
||||
[中文](#中文版本) | [English](#english-version)
|
||||
|
||||
---
|
||||
|
||||
## 🌐 在线查看 / Online Access
|
||||
|
||||
**📖 在线阅读和测试模板**: [https://cn.overleaf.com/read/wyhcnvcrtpyt#cd4a07](https://cn.overleaf.com/read/wyhcnvcrtpyt#cd4a07)
|
||||
|
||||
**📖 Online View and Test Template**: [https://cn.overleaf.com/read/wyhcnvcrtpyt#cd4a07](https://cn.overleaf.com/read/wyhcnvcrtpyt#cd4a07)
|
||||
|
||||
💡 **提示 / Tips**:
|
||||
- 中文：您可以通过上述链接在Overleaf中直接查看、编辑和编译模板，无需本地安装LaTeX环境
|
||||
- English: You can view, edit, and compile the template directly in Overleaf using the link above, without needing a local LaTeX installation
|
||||
|
||||
---
|
||||
|
||||
## 中文版本
|
||||
|
||||
### 概述 ✅
|
||||
|
||||
我已经将AAAI 2026的两个版本（匿名投稿版本和camera-ready版本）**完整合并**成一个统一的模板文件 `aaai2026-unified-template.tex`。
|
||||
|
||||
该模板包含了原始两个模板的**所有完整内容**（共886行，比原始文件更全面），包括：
|
||||
- 所有格式化说明和要求
|
||||
- 完整的示例代码和表格
|
||||
- 图片处理指南
|
||||
- 参考文献格式要求
|
||||
- 所有章节和附录内容
|
||||
- 版本特定的Acknowledgments部分
|
||||
|
||||
### 主要差异分析
|
||||
|
||||
通过比较原始的两个模板，我发现主要差异在于：
|
||||
|
||||
#### 1. 包的加载方式
|
||||
- **匿名版本**: `\usepackage[submission]{aaai2026}`
|
||||
- **Camera-ready版本**: `\usepackage{aaai2026}`
|
||||
|
||||
#### 2. 标题差异
|
||||
- **匿名版本**: "AAAI Press Anonymous Submission Instructions for Authors Using LaTeX"
|
||||
- **Camera-ready版本**: "AAAI Press Formatting Instructions for Authors Using LaTeX --- A Guide"
|
||||
|
||||
#### 3. Links环境的处理
|
||||
- **匿名版本**: Links环境被注释掉，防止泄露作者身份
|
||||
- **Camera-ready版本**: Links环境正常显示
|
||||
|
||||
#### 4. 内容部分差异
|
||||
- **匿名版本**: 包含"Preparing an Anonymous Submission"部分的特殊说明
|
||||
- **Camera-ready版本**: 包含完整的格式说明和版权信息
|
||||
|
||||
### 依赖文件检查结果
|
||||
|
||||
✅ **已验证并复制到主目录的文件**：
|
||||
|
||||
- `aaai2026.sty` - AAAI 2026 样式文件（两个版本完全相同）
|
||||
- `aaai2026.bst` - 参考文献样式文件（两个版本完全相同）
|
||||
- `aaai2026.bib` - 示例参考文献文件
|
||||
- `figure1.pdf` 和 `figure2.pdf` - 示例图片文件
|
||||
|
||||
所有这些文件在两个版本中都是相同的，因此统一模板可以正常工作。
|
||||
|
||||
### 如何使用统一模板
|
||||
|
||||
#### 切换到匿名投稿版本
|
||||
在模板文件第11行，**取消注释**这一行：
|
||||
```latex
|
||||
\def\aaaianonymous{true}
|
||||
```
|
||||
|
||||
#### 切换到Camera-ready版本
|
||||
在模板文件第11行，**注释掉**或**删除**这一行：
|
||||
```latex
|
||||
% \def\aaaianonymous{true}
|
||||
```
|
||||
|
||||
### 一键切换的核心机制
|
||||
|
||||
统一模板使用了LaTeX的条件编译功能：
|
||||
|
||||
```latex
|
||||
% 条件包加载
|
||||
\ifdefined\aaaianonymous
|
||||
\usepackage[submission]{aaai2026} % 匿名版本
|
||||
\else
|
||||
\usepackage{aaai2026} % Camera-ready版本
|
||||
\fi
|
||||
|
||||
% 条件标题设置
|
||||
\ifdefined\aaaianonymous
|
||||
\title{AAAI Press Anonymous Submission\\Instructions for Authors Using \LaTeX{}}
|
||||
\else
|
||||
\title{AAAI Press Formatting Instructions \\for Authors Using \LaTeX{} --- A Guide}
|
||||
\fi
|
||||
|
||||
% 条件内容显示
|
||||
\ifdefined\aaaianonymous
|
||||
% 匿名版本特有内容
|
||||
\else
|
||||
% Camera-ready版本特有内容
|
||||
\fi
|
||||
```
|
||||
|
||||
### 文件清单
|
||||
|
||||
主目录现在包含以下文件：
|
||||
|
||||
- `aaai2026-unified-template.tex` - 统一主论文模板文件
|
||||
- `aaai2026-unified-supp.tex` - 统一补充材料模板文件
|
||||
- `aaai2026.sty` - AAAI 2026 LaTeX 样式文件
|
||||
- `aaai2026.bst` - 参考文献样式文件
|
||||
- `aaai2026.bib` - 示例参考文献文件
|
||||
- `figure1.pdf` - 示例图片1
|
||||
- `figure2.pdf` - 示例图片2
|
||||
- `README.md` - 本说明文档
|
||||
|
||||
### 补充材料模板 (Supplementary Material Template)
|
||||
|
||||
#### 概述
|
||||
`aaai2026-unified-supp.tex` 是专门为AAAI 2026补充材料设计的统一模板，与主论文模板使用相同的版本切换机制。
|
||||
|
||||
#### 主要功能
|
||||
- **版本切换**: 通过修改一行代码在匿名投稿和camera-ready版本间切换
|
||||
- **补充内容支持**: 支持额外的实验、推导、数据、图表、算法等
|
||||
- **格式一致性**: 与主论文模板保持完全一致的格式要求
|
||||
- **代码示例**: 包含算法、代码列表等补充材料的示例
|
||||
|
||||
#### 使用方法
|
||||
与主论文模板相同，只需修改第11行：
|
||||
```latex
|
||||
% 匿名投稿版本
|
||||
\def\aaaianonymous{true}
|
||||
|
||||
% Camera-ready版本
|
||||
% \def\aaaianonymous{true}
|
||||
```
|
||||
|
||||
#### 补充材料内容建议
|
||||
- 额外的实验结果和消融研究
|
||||
- 详细的数学推导和证明
|
||||
- 更多的图表和可视化
|
||||
- 算法伪代码和实现细节
|
||||
- 数据集描述和预处理步骤
|
||||
- 超参数设置和实验配置
|
||||
- 失败案例分析
|
||||
- 计算复杂度分析
|
||||
|
||||
### 使用检查清单 (Usage Checklist)
|
||||
|
||||
#### 📋 投稿前检查清单 (Pre-Submission Checklist)
|
||||
|
||||
**版本设置**:
|
||||
- [ ] 已设置 `\def\aaaianonymous{true}` (匿名投稿)
|
||||
- [ ] 已注释掉所有可能暴露身份的信息
|
||||
- [ ] 已匿名化参考文献（移除作者姓名）
|
||||
|
||||
**内容完整性**:
|
||||
- [ ] 标题、摘要、关键词已填写
|
||||
- [ ] 所有章节内容完整
|
||||
- [ ] 图表编号连续且正确
|
||||
- [ ] 参考文献格式正确
|
||||
- [ ] 补充材料（如有）已准备
|
||||
|
||||
**格式检查**:
|
||||
- [ ] 页面边距符合要求
|
||||
- [ ] 字体和字号正确
|
||||
- [ ] 行间距符合标准
|
||||
- [ ] 图表位置和大小合适
|
||||
- [ ] 数学公式格式正确
|
||||
|
||||
**技术检查**:
|
||||
- [ ] LaTeX编译无错误
|
||||
- [ ] 参考文献正确生成
|
||||
- [ ] PDF输出正常
|
||||
- [ ] 文件大小在限制范围内
|
||||
|
||||
#### 📋 录用后检查清单 (Post-Acceptance Checklist)
|
||||
|
||||
**版本切换**:
|
||||
- [ ] 已注释掉 `\def\aaaianonymous{true}` (camera-ready)
|
||||
- [ ] 已添加完整的作者信息
|
||||
- [ ] 已添加所有作者单位信息
|
||||
- [ ] 已恢复所有被注释的内容
|
||||
|
||||
**内容更新**:
|
||||
- [ ] 已根据审稿意见修改内容
|
||||
- [ ] 已更新所有图表和实验
|
||||
- [ ] 已完善补充材料
|
||||
- [ ] 已检查所有链接和引用
|
||||
|
||||
**最终检查**:
|
||||
- [ ] 最终PDF质量检查
|
||||
- [ ] 所有文件已备份
|
||||
- [ ] 符合会议最终提交要求
|
||||
- [ ] 补充材料已单独提交（如需要）
|
||||
|
||||
#### 📋 补充材料检查清单 (Supplementary Material Checklist)
|
||||
|
||||
**内容组织**:
|
||||
- [ ] 补充材料与主论文内容对应
|
||||
- [ ] 章节结构清晰合理
|
||||
- [ ] 图表编号与主论文不冲突
|
||||
- [ ] 参考文献格式一致
|
||||
|
||||
**技术细节**:
|
||||
- [ ] 算法伪代码清晰完整
|
||||
- [ ] 实验设置详细说明
|
||||
- [ ] 数据预处理步骤明确
|
||||
- [ ] 超参数配置完整
|
||||
|
||||
**格式要求**:
|
||||
- [ ] 使用统一的supp模板
|
||||
- [ ] 页面设置与主论文一致
|
||||
- [ ] 字体和格式符合要求
|
||||
- [ ] 文件大小在限制范围内
|
||||
|
||||
### 实际使用建议
|
||||
|
||||
1. **投稿阶段**:
|
||||
- 取消注释 `\def\aaaianonymous{true}`
|
||||
- 确保不包含任何可能暴露身份的信息
|
||||
- 检查参考文献是否已匿名化
|
||||
|
||||
2. **录用后准备final版本**:
|
||||
- 注释掉或删除 `\def\aaaianonymous{true}` 这一行
|
||||
- 添加完整的作者信息和affiliations
|
||||
- 取消注释links环境（如果需要）
|
||||
|
||||
3. **编译测试**:
|
||||
- 分别在两种模式下编译，确保都能正常工作
|
||||
- 检查输出的PDF是否符合要求
|
||||
- 验证参考文献格式是否正确
|
||||
|
||||
4. **依赖文件确认**:
|
||||
- 确保所有依赖文件都在同一目录下
|
||||
- 如果移动模板文件，记得同时移动依赖文件
|
||||
|
||||
### 重要注意事项
|
||||
|
||||
⚠️ **关于Bibliography Style**:
|
||||
- `aaai2026.sty`文件已经自动设置了`\bibliographystyle{aaai2026}`
|
||||
- **不要**在文档中再次添加`\bibliographystyle{aaai2026}`命令
|
||||
- 否则会出现"`Illegal, another \bibstyle command`"错误
|
||||
- 只需要使用`\bibliography{aaai2026}`命令即可
|
||||
|
||||
### 编译命令示例
|
||||
|
||||
```bash
|
||||
# 编译LaTeX文档
|
||||
pdflatex aaai2026-unified-template.tex
|
||||
bibtex aaai2026-unified-template
|
||||
pdflatex aaai2026-unified-template.tex
|
||||
pdflatex aaai2026-unified-template.tex
|
||||
```
|
||||
|
||||
### 常见问题解决
|
||||
|
||||
#### 1. "Illegal, another \bibstyle command"错误
|
||||
**原因**: 重复设置了bibliography style
|
||||
**解决方案**: 删除文档中的`\bibliographystyle{aaai2026}`命令，`aaai2026.sty`会自动处理
|
||||
|
||||
#### 2. 参考文献格式不正确
|
||||
**原因**: 可能缺少natbib包或者BibTeX文件问题
|
||||
**解决方案**: 确保按照标准的LaTeX编译流程：pdflatex → bibtex → pdflatex → pdflatex
|
||||
|
||||
---
|
||||
|
||||
## English Version
|
||||
|
||||
### Overview ✅
|
||||
|
||||
I have **completely merged** the two AAAI 2026 versions (anonymous submission and camera-ready) into a single unified template file `aaai2026-unified-template.tex`.
|
||||
|
||||
This template contains **all complete content** from both original templates (886 lines total, more comprehensive than the original files), including:
|
||||
- All formatting instructions and requirements
|
||||
- Complete example codes and tables
|
||||
- Image processing guidelines
|
||||
- Reference formatting requirements
|
||||
- All sections and appendix content
|
||||
- Version-specific Acknowledgments sections
|
||||
|
||||
### Key Differences Analysis
|
||||
|
||||
By comparing the two original templates, the main differences are:
|
||||
|
||||
#### 1. Package Loading Method
|
||||
- **Anonymous version**: `\usepackage[submission]{aaai2026}`
|
||||
- **Camera-ready version**: `\usepackage{aaai2026}`
|
||||
|
||||
#### 2. Title Differences
|
||||
- **Anonymous version**: "AAAI Press Anonymous Submission Instructions for Authors Using LaTeX"
|
||||
- **Camera-ready version**: "AAAI Press Formatting Instructions for Authors Using LaTeX --- A Guide"
|
||||
|
||||
#### 3. Links Environment Handling
|
||||
- **Anonymous version**: Links environment commented out to prevent identity disclosure
|
||||
- **Camera-ready version**: Links environment displayed normally
|
||||
|
||||
#### 4. Content Section Differences
|
||||
- **Anonymous version**: Contains special instructions in "Preparing an Anonymous Submission" section
|
||||
- **Camera-ready version**: Contains complete formatting instructions and copyright information
|
||||
|
||||
### Dependency Files Verification
|
||||
|
||||
✅ **Files verified and copied to main directory**:
|
||||
|
||||
- `aaai2026.sty` - AAAI 2026 style file (identical in both versions)
|
||||
- `aaai2026.bst` - Bibliography style file (identical in both versions)
|
||||
- `aaai2026.bib` - Sample bibliography file
|
||||
- `figure1.pdf` and `figure2.pdf` - Sample image files
|
||||
|
||||
All these files are identical in both versions, so the unified template works properly.
|
||||
|
||||
### How to Use the Unified Template
|
||||
|
||||
#### Switch to Anonymous Submission Version
|
||||
On line 11 of the template file, **uncomment** this line:
|
||||
```latex
|
||||
\def\aaaianonymous{true}
|
||||
```
|
||||
|
||||
#### Switch to Camera-ready Version
|
||||
On line 11 of the template file, **comment out** or **delete** this line:
|
||||
```latex
|
||||
% \def\aaaianonymous{true}
|
||||
```
|
||||
|
||||
### Core Mechanism of One-Click Switching
|
||||
|
||||
The unified template uses LaTeX conditional compilation:
|
||||
|
||||
```latex
|
||||
% Conditional package loading
|
||||
\ifdefined\aaaianonymous
|
||||
\usepackage[submission]{aaai2026} % Anonymous version
|
||||
\else
|
||||
\usepackage{aaai2026} % Camera-ready version
|
||||
\fi
|
||||
|
||||
% Conditional title setting
|
||||
\ifdefined\aaaianonymous
|
||||
\title{AAAI Press Anonymous Submission\\Instructions for Authors Using \LaTeX{}}
|
||||
\else
|
||||
\title{AAAI Press Formatting Instructions \\for Authors Using \LaTeX{} --- A Guide}
|
||||
\fi
|
||||
|
||||
% Conditional content display
|
||||
\ifdefined\aaaianonymous
|
||||
% Anonymous version specific content
|
||||
\else
|
||||
% Camera-ready version specific content
|
||||
\fi
|
||||
```
|
||||
|
||||
### File List
|
||||
|
||||
The main directory now contains the following files:
|
||||
|
||||
- `aaai2026-unified-template.tex` - Unified main paper template file
|
||||
- `aaai2026-unified-supp.tex` - Unified supplementary material template file
|
||||
- `aaai2026.sty` - AAAI 2026 LaTeX style file
|
||||
- `aaai2026.bst` - Bibliography style file
|
||||
- `aaai2026.bib` - Sample bibliography file
|
||||
- `figure1.pdf` - Sample image 1
|
||||
- `figure2.pdf` - Sample image 2
|
||||
- `README.md` - This documentation
|
||||
|
||||
### Supplementary Material Template
|
||||
|
||||
#### Overview
|
||||
`aaai2026-unified-supp.tex` is a unified template specifically designed for AAAI 2026 supplementary materials, using the same version switching mechanism as the main paper template.
|
||||
|
||||
#### Key Features
|
||||
- **Version Switching**: Switch between anonymous submission and camera-ready versions by modifying one line of code
|
||||
- **Supplementary Content Support**: Supports additional experiments, derivations, data, figures, algorithms, etc.
|
||||
- **Format Consistency**: Maintains complete format consistency with the main paper template
|
||||
- **Code Examples**: Includes examples for algorithms, code listings, and other supplementary materials
|
||||
|
||||
#### Usage
|
||||
Same as the main paper template, just modify line 11:
|
||||
```latex
|
||||
% Anonymous submission version
|
||||
\def\aaaianonymous{true}
|
||||
|
||||
% Camera-ready version
|
||||
% \def\aaaianonymous{true}
|
||||
```
|
||||
|
||||
#### Supplementary Material Content Suggestions
|
||||
- Additional experimental results and ablation studies
|
||||
- Detailed mathematical derivations and proofs
|
||||
- More figures and visualizations
|
||||
- Algorithm pseudocode and implementation details
|
||||
- Dataset descriptions and preprocessing steps
|
||||
- Hyperparameter settings and experimental configurations
|
||||
- Failure case analysis
|
||||
- Computational complexity analysis
|
||||
|
||||
### Usage Checklist
|
||||
|
||||
#### 📋 Pre-Submission Checklist
|
||||
|
||||
**Version Setup**:
|
||||
- [ ] Set `\def\aaaianonymous{true}` (anonymous submission)
|
||||
- [ ] Commented out all information that could reveal identity
|
||||
- [ ] Anonymized references (removed author names)
|
||||
|
||||
**Content Completeness**:
|
||||
- [ ] Title, abstract, and keywords filled
|
||||
- [ ] All sections complete
|
||||
- [ ] Figure and table numbers consecutive and correct
|
||||
- [ ] Reference format correct
|
||||
- [ ] Supplementary materials prepared (if any)
|
||||
|
||||
**Format Check**:
|
||||
- [ ] Page margins meet requirements
|
||||
- [ ] Font and font size correct
|
||||
- [ ] Line spacing meets standards
|
||||
- [ ] Figure and table positions and sizes appropriate
|
||||
- [ ] Mathematical formula format correct
|
||||
|
||||
**Technical Check**:
|
||||
- [ ] LaTeX compilation error-free
|
||||
- [ ] References generated correctly
|
||||
- [ ] PDF output normal
|
||||
- [ ] File size within limits
|
||||
|
||||
#### 📋 Post-Acceptance Checklist
|
||||
|
||||
**Version Switch**:
|
||||
- [ ] Commented out `\def\aaaianonymous{true}` (camera-ready)
|
||||
- [ ] Added complete author information
|
||||
- [ ] Added all author affiliation information
|
||||
- [ ] Restored all commented content
|
||||
|
||||
**Content Updates**:
|
||||
- [ ] Modified content according to reviewer comments
|
||||
- [ ] Updated all figures and experiments
|
||||
- [ ] Completed supplementary materials
|
||||
- [ ] Checked all links and citations
|
||||
|
||||
**Final Check**:
|
||||
- [ ] Final PDF quality check
|
||||
- [ ] All files backed up
|
||||
- [ ] Meets conference final submission requirements
|
||||
- [ ] Supplementary materials submitted separately (if needed)
|
||||
|
||||
#### 📋 Supplementary Material Checklist
|
||||
|
||||
**Content Organization**:
|
||||
- [ ] Supplementary materials correspond to main paper content
|
||||
- [ ] Chapter structure clear and reasonable
|
||||
- [ ] Figure and table numbers don't conflict with main paper
|
||||
- [ ] Reference format consistent
|
||||
|
||||
**Technical Details**:
|
||||
- [ ] Algorithm pseudocode clear and complete
|
||||
- [ ] Experimental setup explained in detail
|
||||
- [ ] Data preprocessing steps clear
|
||||
- [ ] Hyperparameter configuration complete
|
||||
|
||||
**Format Requirements**:
|
||||
- [ ] Using unified supp template
|
||||
- [ ] Page settings consistent with main paper
|
||||
- [ ] Font and format meet requirements
|
||||
- [ ] File size within limits
|
||||
|
||||
### Practical Usage Recommendations
|
||||
|
||||
1. **Submission Stage**:
|
||||
- Uncomment `\def\aaaianonymous{true}`
|
||||
- Ensure no information that could reveal identity is included
|
||||
- Check that references are anonymized
|
||||
|
||||
2. **Preparing final version after acceptance**:
|
||||
- Comment out or delete the `\def\aaaianonymous{true}` line
|
||||
- Add complete author information and affiliations
|
||||
- Uncomment links environment (if needed)
|
||||
|
||||
3. **Compilation Testing**:
|
||||
- Compile in both modes to ensure proper functionality
|
||||
- Check if the output PDF meets requirements
|
||||
- Verify reference formatting is correct
|
||||
|
||||
4. **Dependency File Confirmation**:
|
||||
- Ensure all dependency files are in the same directory
|
||||
- Remember to move dependency files when moving the template file
|
||||
|
||||
### Important Notes
|
||||
|
||||
⚠️ **About Bibliography Style**:
|
||||
- The `aaai2026.sty` file automatically sets `\bibliographystyle{aaai2026}`
|
||||
- **Do NOT** add `\bibliographystyle{aaai2026}` command again in your document
|
||||
- Otherwise you'll get "`Illegal, another \bibstyle command`" error
|
||||
- Just use the `\bibliography{aaai2026}` command
|
||||
|
||||
### Compilation Commands Example
|
||||
|
||||
```bash
|
||||
# Compile LaTeX document
|
||||
pdflatex aaai2026-unified-template.tex
|
||||
bibtex aaai2026-unified-template
|
||||
pdflatex aaai2026-unified-template.tex
|
||||
pdflatex aaai2026-unified-template.tex
|
||||
```
|
||||
|
||||
### Common Issues and Solutions
|
||||
|
||||
#### 1. "Illegal, another \bibstyle command" Error
|
||||
**Cause**: Duplicate bibliography style setting
|
||||
**Solution**: Remove the `\bibliographystyle{aaai2026}` command from your document, `aaai2026.sty` handles it automatically
|
||||
|
||||
#### 2. Incorrect Reference Format
|
||||
**Cause**: Missing natbib package or BibTeX file issues
|
||||
**Solution**: Follow the standard LaTeX compilation process: pdflatex → bibtex → pdflatex → pdflatex
|
||||
|
||||
---
|
||||
|
||||
## 版本信息 / Version Information
|
||||
|
||||
- **模板版本 / Template Version**: AAAI 2026 Unified (Main + Supplementary)
|
||||
- **创建日期 / Created**: 2024年12月
|
||||
- **支持格式 / Supported Formats**: Anonymous Submission & Camera-Ready
|
||||
- **模板类型 / Template Types**: Main Paper Template & Supplementary Material Template
|
||||
- **兼容性 / Compatibility**: LaTeX 2020+ / TeXLive 2024+
|
||||
|
||||
---
|
||||
|
||||
🎉 **现在您只需要修改一行代码就可以在两个版本之间切换，同时所有必要的依赖文件都已经准备就绪！**
|
||||
🎉 **Now you only need to modify one line of code to switch between the two versions, with all necessary dependency files ready to use!**
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
%File: aaai2026-unified-supp.tex
|
||||
%
|
||||
% UNIFIED AAAI 2026 SUPPLEMENTARY MATERIAL TEMPLATE
|
||||
% To switch between anonymous submission and camera-ready versions,
|
||||
% simply change the next line:
|
||||
%
|
||||
% For ANONYMOUS SUBMISSION: uncomment the next line
|
||||
% \def\aaaianonymous{true}
|
||||
%
|
||||
% For CAMERA-READY VERSION: comment out or delete the next line
|
||||
% \def\aaaianonymous{true}
|
||||
%
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
|
||||
\documentclass[letterpaper]{article} % DO NOT CHANGE THIS
|
||||
|
||||
% Conditional package loading based on version
|
||||
\ifdefined\aaaianonymous
|
||||
\usepackage[submission]{aaai2026} % Anonymous submission version
|
||||
\else
|
||||
\usepackage{aaai2026} % Camera-ready version
|
||||
\fi
|
||||
|
||||
\usepackage{times} % DO NOT CHANGE THIS
|
||||
\usepackage{helvet} % DO NOT CHANGE THIS
|
||||
\usepackage{courier} % DO NOT CHANGE THIS
|
||||
\usepackage[hyphens]{url} % DO NOT CHANGE THIS
|
||||
\usepackage{graphicx} % DO NOT CHANGE THIS
|
||||
\urlstyle{rm} % DO NOT CHANGE THIS
|
||||
\def\UrlFont{\rm} % DO NOT CHANGE THIS
|
||||
\usepackage{natbib} % DO NOT CHANGE THIS AND DO NOT ADD ANY OPTIONS TO IT
|
||||
\usepackage{caption} % DO NOT CHANGE THIS AND DO NOT ADD ANY OPTIONS TO IT
|
||||
\frenchspacing % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpagewidth}{8.5in} % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpageheight}{11in} % DO NOT CHANGE THIS
|
||||
|
||||
% These are recommended to typeset algorithms but not required.
|
||||
\usepackage{algorithm}
|
||||
\usepackage{algorithmic}
|
||||
|
||||
% These are recommended to typeset listings but not required.
|
||||
\usepackage{newfloat}
|
||||
\usepackage{listings}
|
||||
\DeclareCaptionStyle{ruled}{labelfont=normalfont,labelsep=colon,strut=off} % DO NOT CHANGE THIS
|
||||
\lstset{%
|
||||
basicstyle={\footnotesize\ttfamily},
|
||||
numbers=left,numberstyle=\footnotesize,xleftmargin=2em,
|
||||
aboveskip=0pt,belowskip=0pt,
|
||||
showstringspaces=false,tabsize=2,breaklines=true}
|
||||
\floatstyle{ruled}
|
||||
\newfloat{listing}{tb}{lst}{}
|
||||
\floatname{listing}{Listing}
|
||||
|
||||
\pdfinfo{
|
||||
/TemplateVersion (2026.1)
|
||||
}
|
||||
|
||||
\setcounter{secnumdepth}{0} %May be changed to 1 or 2 if section numbers are desired.
|
||||
|
||||
% Title - conditionally set based on version
|
||||
\ifdefined\aaaianonymous
|
||||
\title{AAAI 2026 Supplementary Material\\Anonymous Submission}
|
||||
\else
|
||||
\title{AAAI 2026 Supplementary Material\\Camera Ready}
|
||||
\fi
|
||||
|
||||
% Author and affiliation information
|
||||
\ifdefined\aaaianonymous
|
||||
\author{
|
||||
Anonymous Submission
|
||||
}
|
||||
\affiliations{
|
||||
% Leave affiliations empty for anonymous submission
|
||||
}
|
||||
\else
|
||||
\author{
|
||||
%Authors
|
||||
Written by AAAI Press Staff\textsuperscript{\rm 1}\thanks{With help from the AAAI Publications Committee.}\\
|
||||
AAAI Style Contributions by Pater Patel Schneider,
|
||||
Sunil Issar,\\
|
||||
J. Scott Penberthy,
|
||||
George Ferguson,
|
||||
Hans Guesgen,
|
||||
Francisco Cruz\equalcontrib,
|
||||
Marc Pujol-Gonzalez\equalcontrib
|
||||
}
|
||||
\affiliations{
|
||||
\textsuperscript{\rm 1}Association for the Advancement of Artificial Intelligence\\
|
||||
1101 Pennsylvania Ave, NW Suite 300\\
|
||||
Washington, DC 20004 USA\\
|
||||
proceedings-questions@aaai.org
|
||||
}
|
||||
\fi
|
||||
|
||||
\begin{document}
|
||||
|
||||
\maketitle
|
||||
|
||||
\begin{abstract}
|
||||
This document provides supplementary material for the main paper, including additional experiments, derivations, data, figures, algorithms, and other relevant content. Please add detailed information as needed. This supplementary material is submitted together with the main paper to further support and complement the main findings.
|
||||
\end{abstract}
|
||||
|
||||
% ----------- Supplementary Content Starts Here -----------
|
||||
|
||||
\section{Example Supplementary Content}
|
||||
|
||||
This is the main body of the supplementary material. You may add extra experimental results, ablation studies, detailed derivations, additional figures, pseudocode, dataset descriptions, etc.
|
||||
|
||||
\subsection{Additional Experiments}
|
||||
|
||||
% Example: Insert a figure
|
||||
% Uncomment and modify the following lines to add your own figures:
|
||||
% \begin{figure}[h]
|
||||
% \centering
|
||||
% \includegraphics[width=0.9\columnwidth]{your-figure-name}
|
||||
% \caption{Your figure caption here.}
|
||||
% \label{fig:supp1}
|
||||
% \end{figure}
|
||||
|
||||
\subsection{Detailed Derivations}
|
||||
|
||||
You may provide detailed mathematical derivations, proofs, or other technical details here.
|
||||
|
||||
\subsection{Pseudocode}
|
||||
|
||||
\begin{algorithm}[h]
|
||||
\caption{Example Supplementary Algorithm}
|
||||
\begin{algorithmic}[1]
|
||||
\STATE Initialize parameters
|
||||
\FOR{each sample}
|
||||
\STATE Compute loss
|
||||
\STATE Update parameters
|
||||
\ENDFOR
|
||||
\STATE \textbf{return} optimal parameters
|
||||
\end{algorithmic}
|
||||
\end{algorithm}
|
||||
|
||||
% ----------- Supplementary Content Ends Here -----------
|
||||
|
||||
% References and End of Paper
|
||||
% These lines must be placed at the end of your paper
|
||||
\bibliography{aaai2026}
|
||||
|
||||
\end{document}
|
||||
+952
@@ -0,0 +1,952 @@
|
||||
%File: aaai2026-unified-template.tex
|
||||
%
|
||||
% UNIFIED AAAI 2026 TEMPLATE
|
||||
% To switch between anonymous submission and camera-ready versions,
|
||||
% simply change the next line:
|
||||
%
|
||||
% For ANONYMOUS SUBMISSION: uncomment the next line
|
||||
% \def\aaaianonymous{true}
|
||||
%
|
||||
% For CAMERA-READY VERSION: comment out or delete the next line
|
||||
% \def\aaaianonymous{true}
|
||||
%
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
|
||||
\documentclass[letterpaper]{article} % DO NOT CHANGE THIS
|
||||
|
||||
% Conditional package loading based on version
|
||||
\ifdefined\aaaianonymous
|
||||
\usepackage[submission]{aaai2026} % Anonymous submission version
|
||||
\else
|
||||
\usepackage{aaai2026} % Camera-ready version
|
||||
\fi
|
||||
|
||||
\usepackage{times} % DO NOT CHANGE THIS
|
||||
\usepackage{helvet} % DO NOT CHANGE THIS
|
||||
\usepackage{courier} % DO NOT CHANGE THIS
|
||||
\usepackage[hyphens]{url} % DO NOT CHANGE THIS
|
||||
\usepackage{graphicx} % DO NOT CHANGE THIS
|
||||
\urlstyle{rm} % DO NOT CHANGE THIS
|
||||
\def\UrlFont{\rm} % DO NOT CHANGE THIS
|
||||
\usepackage{natbib} % DO NOT CHANGE THIS AND DO NOT ADD ANY OPTIONS TO IT
|
||||
\usepackage{caption} % DO NOT CHANGE THIS AND DO NOT ADD ANY OPTIONS TO IT
|
||||
\frenchspacing % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpagewidth}{8.5in} % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpageheight}{11in} % DO NOT CHANGE THIS
|
||||
|
||||
%
|
||||
% These are recommended to typeset algorithms but not required. See the subsubsection on algorithms. Remove them if you don't have algorithms in your paper.
|
||||
\usepackage{algorithm}
|
||||
\usepackage{algorithmic}
|
||||
|
||||
%
|
||||
% These are are recommended to typeset listings but not required. See the subsubsection on listing. Remove this block if you don't have listings in your paper.
|
||||
\usepackage{newfloat}
|
||||
\usepackage{listings}
|
||||
\DeclareCaptionStyle{ruled}{labelfont=normalfont,labelsep=colon,strut=off} % DO NOT CHANGE THIS
|
||||
\lstset{%
|
||||
basicstyle={\footnotesize\ttfamily},% footnotesize acceptable for monospace
|
||||
numbers=left,numberstyle=\footnotesize,xleftmargin=2em,% show line numbers, remove this entire line if you don't want the numbers.
|
||||
aboveskip=0pt,belowskip=0pt,%
|
||||
showstringspaces=false,tabsize=2,breaklines=true}
|
||||
\floatstyle{ruled}
|
||||
\newfloat{listing}{tb}{lst}{}
|
||||
\floatname{listing}{Listing}
|
||||
|
||||
%
|
||||
% Keep the \pdfinfo as shown here. There's no need
|
||||
% for you to add the /Title and /Author tags.
|
||||
\pdfinfo{
|
||||
/TemplateVersion (2026.1)
|
||||
}
|
||||
|
||||
% DISALLOWED PACKAGES
|
||||
% \usepackage{authblk} -- This package is specifically forbidden
|
||||
% \usepackage{balance} -- This package is specifically forbidden
|
||||
% \usepackage{color (if used in text)
|
||||
% \usepackage{CJK} -- This package is specifically forbidden
|
||||
% \usepackage{float} -- This package is specifically forbidden
|
||||
% \usepackage{flushend} -- This package is specifically forbidden
|
||||
% \usepackage{fontenc} -- This package is specifically forbidden
|
||||
% \usepackage{fullpage} -- This package is specifically forbidden
|
||||
% \usepackage{geometry} -- This package is specifically forbidden
|
||||
% \usepackage{grffile} -- This package is specifically forbidden
|
||||
% \usepackage{hyperref} -- This package is specifically forbidden
|
||||
% \usepackage{navigator} -- This package is specifically forbidden
|
||||
% (or any other package that embeds links such as navigator or hyperref)
|
||||
% \indentfirst} -- This package is specifically forbidden
|
||||
% \layout} -- This package is specifically forbidden
|
||||
% \multicol} -- This package is specifically forbidden
|
||||
% \nameref} -- This package is specifically forbidden
|
||||
% \usepackage{savetrees} -- This package is specifically forbidden
|
||||
% \usepackage{setspace} -- This package is specifically forbidden
|
||||
% \usepackage{stfloats} -- This package is specifically forbidden
|
||||
% \usepackage{tabu} -- This package is specifically forbidden
|
||||
% \usepackage{titlesec} -- This package is specifically forbidden
|
||||
% \usepackage{tocbibind} -- This package is specifically forbidden
|
||||
% \usepackage{ulem} -- This package is specifically forbidden
|
||||
% \usepackage{wrapfig} -- This package is specifically forbidden
|
||||
|
||||
% DISALLOWED COMMANDS
|
||||
% \nocopyright -- Your paper will not be published if you use this command
|
||||
% \addtolength -- This command may not be used
|
||||
% \balance -- This command may not be used
|
||||
% \baselinestretch -- Your paper will not be published if you use this command
|
||||
% \clearpage -- No page breaks of any kind may be used for the final version of your paper
|
||||
% \columnsep -- This command may not be used
|
||||
% \newpage -- No page breaks of any kind may be used for the final version of your paper
|
||||
% \pagebreak -- No page breaks of any kind may be used for the final version of your paperr
|
||||
% \pagestyle -- This command may not be used
|
||||
% \tiny -- This is not an acceptable font size.
|
||||
% \vspace{- -- No negative value may be used in proximity of a caption, figure, table, section, subsection, subsubsection, or reference
|
||||
% \vskip{- -- No negative value may be used to alter spacing above or below a caption, figure, table, section, subsection, subsubsection, or reference
|
||||
|
||||
\setcounter{secnumdepth}{0} %May be changed to 1 or 2 if section numbers are desired.
|
||||
|
||||
% The file aaai2026.sty is the style file for AAAI Press
|
||||
% proceedings, working notes, and technical reports.
|
||||
%
|
||||
|
||||
% Title - conditionally set based on version
|
||||
\ifdefined\aaaianonymous
|
||||
\title{AAAI Press Anonymous Submission\\Instructions for Authors Using \LaTeX{}}
|
||||
\else
|
||||
\title{AAAI Press Formatting Instructions \\for Authors Using \LaTeX{} --- A Guide}
|
||||
\fi
|
||||
|
||||
% Author and affiliation information
|
||||
\author{
|
||||
%Authors
|
||||
% All authors must be in the same font size and format.
|
||||
Written by AAAI Press Staff\textsuperscript{\rm 1}\thanks{With help from the AAAI Publications Committee.}\\
|
||||
AAAI Style Contributions by Pater Patel Schneider,
|
||||
Sunil Issar,\\
|
||||
J. Scott Penberthy,
|
||||
George Ferguson,
|
||||
Hans Guesgen,
|
||||
Francisco Cruz\equalcontrib,
|
||||
Marc Pujol-Gonzalez\equalcontrib
|
||||
}
|
||||
\affiliations{
|
||||
%Afiliations
|
||||
\textsuperscript{\rm 1}Association for the Advancement of Artificial Intelligence\\
|
||||
% If you have multiple authors and multiple affiliations
|
||||
% use superscripts in text and roman font to identify them.
|
||||
% For example,
|
||||
|
||||
% Sunil Issar\textsuperscript{\rm 2},
|
||||
% J. Scott Penberthy\textsuperscript{\rm 3},
|
||||
% George Ferguson\textsuperscript{\rm 4},
|
||||
% Hans Guesgen\textsuperscript{\rm 5}
|
||||
% Note that the comma should be placed after the superscript
|
||||
|
||||
1101 Pennsylvania Ave, NW Suite 300\\
|
||||
Washington, DC 20004 USA\\
|
||||
% email address must be in roman text type, not monospace or sans serif
|
||||
proceedings-questions@aaai.org
|
||||
%
|
||||
% See more examples next
|
||||
}
|
||||
|
||||
%Example, Single Author, ->> remove \iffalse,\fi and place them surrounding AAAI title to use it
|
||||
\iffalse
|
||||
\title{My Publication Title --- Single Author}
|
||||
\author {
|
||||
Author Name
|
||||
}
|
||||
\affiliations{
|
||||
Affiliation\\
|
||||
Affiliation Line 2\\
|
||||
name@example.com
|
||||
}
|
||||
\fi
|
||||
|
||||
\iffalse
|
||||
%Example, Multiple Authors, ->> remove \iffalse,\fi and place them surrounding AAAI title to use it
|
||||
\title{My Publication Title --- Multiple Authors}
|
||||
\author {
|
||||
% Authors
|
||||
First Author Name\textsuperscript{\rm 1},
|
||||
Second Author Name\textsuperscript{\rm 2},
|
||||
Third Author Name\textsuperscript{\rm 1}
|
||||
}
|
||||
\affiliations {
|
||||
% Affiliations
|
||||
\textsuperscript{\rm 1}Affiliation 1\\
|
||||
\textsuperscript{\rm 2}Affiliation 2\\
|
||||
firstAuthor@affiliation1.com, secondAuthor@affilation2.com, thirdAuthor@affiliation1.com
|
||||
}
|
||||
\fi
|
||||
|
||||
% REMOVE THIS: bibentry
|
||||
% This is only needed to show inline citations in the guidelines document. You should not need it and can safely delete it.
|
||||
\usepackage{bibentry}
|
||||
% END REMOVE bibentry
|
||||
|
||||
\begin{document}
|
||||
|
||||
\maketitle
|
||||
|
||||
\begin{abstract}
|
||||
AAAI creates proceedings, working notes, and technical reports directly from electronic source furnished by the authors. To ensure that all papers in the publication have a uniform appearance, authors must adhere to the following instructions.
|
||||
\end{abstract}
|
||||
|
||||
% Links section - only shown in camera-ready version
|
||||
\ifdefined\aaaianonymous
|
||||
% Uncomment the following to link to your code, datasets, an extended version or similar.
|
||||
% You must keep this block between (not within) the abstract and the main body of the paper.
|
||||
% NOTE: For anonymous submissions, do not include links that could reveal your identity
|
||||
% \begin{links}
|
||||
% \link{Code}{https://aaai.org/example/code}
|
||||
% \link{Datasets}{https://aaai.org/example/datasets}
|
||||
% \link{Extended version}{https://aaai.org/example/extended-version}
|
||||
% \end{links}
|
||||
\else
|
||||
% Uncomment the following to link to your code, datasets, an extended version or similar.
|
||||
% You must keep this block between (not within) the abstract and the main body of the paper.
|
||||
\begin{links}
|
||||
\link{Code}{https://aaai.org/example/code}
|
||||
\link{Datasets}{https://aaai.org/example/datasets}
|
||||
\link{Extended version}{https://aaai.org/example/extended-version}
|
||||
\end{links}
|
||||
\fi
|
||||
|
||||
% Version-specific content
|
||||
\ifdefined\aaaianonymous
|
||||
\section{Preparing an Anonymous Submission}
|
||||
|
||||
This document details the formatting requirements for anonymous submissions. The requirements are the same as for camera ready papers but with a few notable differences:
|
||||
|
||||
\begin{itemize}
|
||||
\item Anonymous submissions must not include the author names and affiliations. Write ``Anonymous Submission'' as the ``sole author'' and leave the affiliations empty.
|
||||
\item The PDF document's metadata should be cleared with a metadata-cleaning tool before submitting it. This is to prevent leaked information from revealing your identity.
|
||||
\item References must be anonymized whenever the reader can infer that they are to the authors' previous work.
|
||||
\item AAAI's copyright notice should not be included as a footer in the first page.
|
||||
\item Only the PDF version is required at this stage. No source versions will be requested, nor any copyright transfer form.
|
||||
\end{itemize}
|
||||
|
||||
You can remove the copyright notice and ensure that your names aren't shown by including \texttt{submission} option when loading the \texttt{aaai2026} package:
|
||||
|
||||
\begin{quote}\begin{scriptsize}\begin{verbatim}
|
||||
\documentclass[letterpaper]{article}
|
||||
\usepackage[submission]{aaai2026}
|
||||
\end{verbatim}\end{scriptsize}\end{quote}
|
||||
|
||||
The remainder of this document are the original camera-ready instructions. Any contradiction of the above points ought to be ignored while preparing anonymous submissions.
|
||||
|
||||
\section{Camera-Ready Guidelines}
|
||||
\else
|
||||
\section{Introduction}
|
||||
\fi
|
||||
|
||||
Congratulations on having a paper selected for inclusion in an AAAI Press proceedings or technical report! This document details the requirements necessary to get your accepted paper published using PDF\LaTeX{}. If you are using Microsoft Word, instructions are provided in a different document. AAAI Press does not support any other formatting software.
|
||||
|
||||
The instructions herein are provided as a general guide for experienced \LaTeX{} users. If you do not know how to use \LaTeX{}, please obtain assistance locally. AAAI cannot provide you with support and the accompanying style files are \textbf{not} guaranteed to work. If the results you obtain are not in accordance with the specifications you received, you must correct your source file to achieve the correct result.
|
||||
|
||||
These instructions are generic. Consequently, they do not include specific dates, page charges, and so forth. Please consult your specific written conference instructions for details regarding your submission. Please review the entire document for specific instructions that might apply to your particular situation. All authors must comply with the following:
|
||||
|
||||
\begin{itemize}
|
||||
\item You must use the 2026 AAAI Press \LaTeX{} style file and the aaai2026.bst bibliography style files, which are located in the 2026 AAAI Author Kit (aaai2026.sty, aaai2026.bst).
|
||||
\item You must complete, sign, and return by the deadline the AAAI copyright form (unless directed by AAAI Press to use the AAAI Distribution License instead).
|
||||
\item You must read and format your paper source and PDF according to the formatting instructions for authors.
|
||||
\item You must submit your electronic files and abstract using our electronic submission form \textbf{on time.}
|
||||
\item You must pay any required page or formatting charges to AAAI Press so that they are received by the deadline.
|
||||
\item You must check your paper before submitting it, ensuring that it compiles without error, and complies with the guidelines found in the AAAI Author Kit.
|
||||
\end{itemize}
|
||||
|
||||
\ifdefined\aaaianonymous
|
||||
\else
|
||||
\section{Copyright}
|
||||
All papers submitted for publication by AAAI Press must be accompanied by a valid signed copyright form. They must also contain the AAAI copyright notice at the bottom of the first page of the paper. There are no exceptions to these requirements. If you fail to provide us with a signed copyright form or disable the copyright notice, we will be unable to publish your paper. There are \textbf{no exceptions} to this policy. You will find a PDF version of the AAAI copyright form in the AAAI AuthorKit. Please see the specific instructions for your conference for submission details.
|
||||
\fi
|
||||
|
||||
\section{Formatting Requirements in Brief}
|
||||
We need source and PDF files that can be used in a variety of ways and can be output on a variety of devices. The design and appearance of the paper is \ifdefined\aaaianonymous governed by the aaai2026.sty file (aaai2026.bst for the bibliography style).\else strictly governed by the aaai style file (aaai2026.sty).\fi
|
||||
\ifdefined\aaaianonymous
|
||||
\begin{itemize}
|
||||
\item You must not modify the aaai2026.sty file or change the TeX commands.
|
||||
\item You must not use any commands that alter the layout or formatting of your document (i.e., you cannot change the default margins, line spacing, etc.).
|
||||
\item You may include other font size changes, color changes, or other formatting commands in your own source, but the paper has to be able to compile, and the styling commands are ignored.
|
||||
\end{itemize}
|
||||
\else
|
||||
\textbf{You must not make any changes to the aaai style file, nor use any commands, packages, style files, or macros within your own paper that alter that design, including, but not limited to spacing, floats, margins, fonts, font size, and appearance.} AAAI imposes requirements on your source and PDF files that must be followed. Most of these requirements are based on our efforts to standardize conference manuscript properties and layout. All papers submitted to AAAI for publication will be recompiled for standardization purposes. Consequently, every paper submission must comply with the following requirements:
|
||||
|
||||
\begin{itemize}
|
||||
\item Your .tex file must compile in PDF\LaTeX{} --- (you may not include .ps or .eps figure files.)
|
||||
\item All fonts must be embedded in the PDF file --- including your figures.
|
||||
\item Modifications to the style file, whether directly or via commands in your document may not ever be made, most especially when made in an effort to avoid extra page charges or make your paper fit in a specific number of pages.
|
||||
\item No type 3 fonts may be used (even in illustrations).
|
||||
\item You may not alter the spacing above and below captions, figures, headings, and subheadings.
|
||||
\item You may not alter the font sizes of text elements, footnotes, heading elements, captions, or title information (for references and mathematics, please see the limited exceptions provided herein).
|
||||
\item You may not alter the line spacing of text.
|
||||
\item Your title must follow Title Case capitalization rules (not sentence case).
|
||||
\item \LaTeX{} documents must use the Times or Nimbus font package (you may not use Computer Modern for the text of your paper).
|
||||
\item No \LaTeX{} 209 documents may be used or submitted.
|
||||
\item Your source must not require use of fonts for non-Roman alphabets within the text itself. If your paper includes symbols in other languages (such as, but not limited to, Arabic, Chinese, Hebrew, Japanese, Thai, Russian and other Cyrillic languages), you must restrict their use to bit-mapped figures. Fonts that require non-English language support (CID and Identity-H) must be converted to outlines or 300 dpi bitmap or removed from the document (even if they are in a graphics file embedded in the document).
|
||||
\item Two-column format in AAAI style is required for all papers.
|
||||
\item The paper size for final submission must be US letter without exception.
|
||||
\item The source file must exactly match the PDF.
|
||||
\item The document margins may not be exceeded (no overfull boxes).
|
||||
\item The number of pages and the file size must be as specified for your event.
|
||||
\item No document may be password protected.
|
||||
\item Neither the PDFs nor the source may contain any embedded links or bookmarks (no hyperref or navigator packages).
|
||||
\item Your source and PDF must not have any page numbers, footers, or headers (no pagestyle commands).
|
||||
\item Your PDF must be compatible with Acrobat 5 or higher.
|
||||
\item Your \LaTeX{} source file (excluding references) must consist of a \textbf{single} file (use of the ``input" command is not allowed.
|
||||
\item Your graphics must be sized appropriately outside of \LaTeX{} (do not use the ``clip" or ``trim'' command) .
|
||||
\end{itemize}
|
||||
|
||||
If you do not follow these requirements, your paper will be returned to you to correct the deficiencies.
|
||||
\fi
|
||||
|
||||
\section{What Files to Submit}
|
||||
You must submit the following items to ensure that your paper is published:
|
||||
\begin{itemize}
|
||||
\item A fully-compliant PDF file.
|
||||
\item Your \LaTeX{} source file submitted as a \textbf{single} .tex file (do not use the ``input" command to include sections of your paper --- every section must be in the single source file). (The only allowable exception is .bib file, which should be included separately).
|
||||
\item The bibliography (.bib) file(s).
|
||||
\item Your source must compile on our system, which includes only standard \LaTeX{} 2020 TeXLive support files.
|
||||
\item Only the graphics files used in compiling paper.
|
||||
\item The \LaTeX{}-generated files (e.g. .aux, .bbl file, PDF, etc.).
|
||||
\end{itemize}
|
||||
|
||||
Your \LaTeX{} source will be reviewed and recompiled on our system (if it does not compile, your paper will be returned to you. \textbf{Do not submit your source in multiple text files.} Your single \LaTeX{} source file must include all your text, your bibliography (formatted using aaai2026.bst), and any custom macros.
|
||||
|
||||
Your files should work without any supporting files (other than the program itself) on any computer with a standard \LaTeX{} distribution.
|
||||
|
||||
\textbf{Do not send files that are not actually used in the paper.} Avoid including any files not needed for compiling your paper, including, for example, this instructions file, unused graphics files, style files, additional material sent for the purpose of the paper review, intermediate build files and so forth.
|
||||
|
||||
\textbf{Obsolete style files.} The commands for some common packages (such as some used for algorithms), may have changed. Please be certain that you are not compiling your paper using old or obsolete style files.
|
||||
|
||||
\textbf{Final Archive.} Place your source files in a single archive which should be compressed using .zip. The final file size may not exceed 10 MB.
|
||||
Name your source file with the last (family) name of the first author, even if that is not you.
|
||||
|
||||
\section{Using \LaTeX{} to Format Your Paper}
|
||||
|
||||
The latest version of the AAAI style file is available on AAAI's website. Download this file and place it in the \TeX\ search path. Placing it in the same directory as the paper should also work. You must download the latest version of the complete AAAI Author Kit so that you will have the latest instruction set and style file.
|
||||
|
||||
\subsection{Document Preamble}
|
||||
|
||||
In the \LaTeX{} source for your paper, you \textbf{must} place the following lines as shown in the example in this subsection. This command set-up is for three authors. Add or subtract author and address lines as necessary, and uncomment the portions that apply to you. In most instances, this is all you need to do to format your paper in the Times font. The helvet package will cause Helvetica to be used for sans serif. These files are part of the PSNFSS2e package, which is freely available from many Internet sites (and is often part of a standard installation).
|
||||
|
||||
Leave the setcounter for section number depth commented out and set at 0 unless you want to add section numbers to your paper. If you do add section numbers, you must uncomment this line and change the number to 1 (for section numbers), or 2 (for section and subsection numbers). The style file will not work properly with numbering of subsubsections, so do not use a number higher than 2.
|
||||
|
||||
\subsubsection{The Following Must Appear in Your Preamble}
|
||||
\ifdefined\aaaianonymous
|
||||
\begin{quote}
|
||||
\begin{scriptsize}\begin{verbatim}
|
||||
\documentclass[letterpaper]{article}
|
||||
% DO NOT CHANGE THIS
|
||||
\usepackage[submission]{aaai2026} % DO NOT CHANGE THIS
|
||||
\usepackage{times} % DO NOT CHANGE THIS
|
||||
\usepackage{helvet} % DO NOT CHANGE THIS
|
||||
\usepackage{courier} % DO NOT CHANGE THIS
|
||||
\usepackage[hyphens]{url} % DO NOT CHANGE THIS
|
||||
\usepackage{graphicx} % DO NOT CHANGE THIS
|
||||
\urlstyle{rm} % DO NOT CHANGE THIS
|
||||
\def\UrlFont{\rm} % DO NOT CHANGE THIS
|
||||
\usepackage{graphicx} % DO NOT CHANGE THIS
|
||||
\usepackage{natbib} % DO NOT CHANGE THIS
|
||||
\usepackage{caption} % DO NOT CHANGE THIS
|
||||
\frenchspacing % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpagewidth}{8.5in} % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpageheight}{11in} % DO NOT CHANGE THIS
|
||||
%
|
||||
% Keep the \pdfinfo as shown here. There's no need
|
||||
% for you to add the /Title and /Author tags.
|
||||
\pdfinfo{
|
||||
/TemplateVersion (2026.1)
|
||||
}
|
||||
\end{verbatim}\end{scriptsize}
|
||||
\end{quote}
|
||||
\else
|
||||
\begin{quote}
|
||||
\begin{scriptsize}\begin{verbatim}
|
||||
\documentclass[letterpaper]{article}
|
||||
% DO NOT CHANGE THIS
|
||||
\usepackage{aaai2026} % DO NOT CHANGE THIS
|
||||
\usepackage{times} % DO NOT CHANGE THIS
|
||||
\usepackage{helvet} % DO NOT CHANGE THIS
|
||||
\usepackage{courier} % DO NOT CHANGE THIS
|
||||
\usepackage[hyphens]{url} % DO NOT CHANGE THIS
|
||||
\usepackage{graphicx} % DO NOT CHANGE THIS
|
||||
\urlstyle{rm} % DO NOT CHANGE THIS
|
||||
\def\UrlFont{\rm} % DO NOT CHANGE THIS
|
||||
\usepackage{graphicx} % DO NOT CHANGE THIS
|
||||
\usepackage{natbib} % DO NOT CHANGE THIS
|
||||
\usepackage{caption} % DO NOT CHANGE THIS
|
||||
\frenchspacing % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpagewidth}{8.5in} % DO NOT CHANGE THIS
|
||||
\setlength{\pdfpageheight}{11in} % DO NOT CHANGE THIS
|
||||
%
|
||||
% Keep the \pdfinfo as shown here. There's no need
|
||||
% for you to add the /Title and /Author tags.
|
||||
\pdfinfo{
|
||||
/TemplateVersion (2026.1)
|
||||
}
|
||||
\end{verbatim}\end{scriptsize}
|
||||
\end{quote}
|
||||
\fi
|
||||
|
||||
\subsection{Preparing Your Paper}
|
||||
|
||||
After the preamble above, you should prepare your paper as follows:
|
||||
\begin{quote}
|
||||
\begin{scriptsize}\begin{verbatim}
|
||||
\begin{document}
|
||||
\maketitle
|
||||
\begin{abstract}
|
||||
%...
|
||||
\end{abstract}\end{verbatim}\end{scriptsize}
|
||||
\end{quote}
|
||||
|
||||
\noindent If you want to add links to the paper's code, dataset(s), and extended version or similar this is the place to add them, within a \emph{links} environment:
|
||||
\begin{quote}%
|
||||
\begin{scriptsize}\begin{verbatim}
|
||||
\begin{links}
|
||||
\link{Code}{https://aaai.org/example/guidelines}
|
||||
\link{Datasets}{https://aaai.org/example/datasets}
|
||||
\link{Extended version}{https://aaai.org/example}
|
||||
\end{links}\end{verbatim}\end{scriptsize}
|
||||
\end{quote}
|
||||
\ifdefined\aaaianonymous
|
||||
\noindent Make sure that you do not de-anonymize yourself with these links.
|
||||
\fi
|
||||
|
||||
\noindent You should then continue with the body of your paper. Your paper must conclude with the references, which should be inserted as follows:
|
||||
\begin{quote}
|
||||
\begin{scriptsize}\begin{verbatim}
|
||||
% References and End of Paper
|
||||
% These lines must be placed at the end of your paper
|
||||
\bibliography{Bibliography-File}
|
||||
\end{document}
|
||||
\end{verbatim}\end{scriptsize}
|
||||
\end{quote}
|
||||
|
||||
\begin{quote}
|
||||
\begin{scriptsize}\begin{verbatim}
|
||||
\begin{document}\\
|
||||
\maketitle\\
|
||||
...\\
|
||||
\bibliography{Bibliography-File}\\
|
||||
\end{document}\\
|
||||
\end{verbatim}\end{scriptsize}
|
||||
\end{quote}
|
||||
|
||||
\subsection{Commands and Packages That May Not Be Used}
|
||||
\begin{table*}[t]
|
||||
\centering
|
||||
\begin{tabular}{l|l|l|l}
|
||||
\textbackslash abovecaption &
|
||||
\textbackslash abovedisplay &
|
||||
\textbackslash addevensidemargin &
|
||||
\textbackslash addsidemargin \\
|
||||
\textbackslash addtolength &
|
||||
\textbackslash baselinestretch &
|
||||
\textbackslash belowcaption &
|
||||
\textbackslash belowdisplay \\
|
||||
\textbackslash break &
|
||||
\textbackslash clearpage &
|
||||
\textbackslash clip &
|
||||
\textbackslash columnsep \\
|
||||
\textbackslash float &
|
||||
\textbackslash input &
|
||||
\textbackslash input &
|
||||
\textbackslash linespread \\
|
||||
\textbackslash newpage &
|
||||
\textbackslash pagebreak &
|
||||
\textbackslash renewcommand &
|
||||
\textbackslash setlength \\
|
||||
\textbackslash text height &
|
||||
\textbackslash tiny &
|
||||
\textbackslash top margin &
|
||||
\textbackslash trim \\
|
||||
\textbackslash vskip\{- &
|
||||
\textbackslash vspace\{- \\
|
||||
\end{tabular}
|
||||
\caption{Commands that must not be used}
|
||||
\label{table1}
|
||||
\end{table*}
|
||||
|
||||
\begin{table}[t]
|
||||
\centering
|
||||
\begin{tabular}{l|l|l|l}
|
||||
authblk & babel & cjk & dvips \\
|
||||
epsf & epsfig & euler & float \\
|
||||
fullpage & geometry & graphics & hyperref \\
|
||||
layout & linespread & lmodern & maltepaper \\
|
||||
navigator & pdfcomment & pgfplots & psfig \\
|
||||
pstricks & t1enc & titlesec & tocbind \\
|
||||
ulem
|
||||
\end{tabular}
|
||||
\caption{LaTeX style packages that must not be used.}
|
||||
\label{table2}
|
||||
\end{table}
|
||||
|
||||
There are a number of packages, commands, scripts, and macros that are incompatable with aaai2026.sty. The common ones are listed in tables \ref{table1} and \ref{table2}. Generally, if a command, package, script, or macro alters floats, margins, fonts, sizing, linespacing, or the presentation of the references and citations, it is unacceptable. Note that negative vskip and vspace may not be used except in certain rare occurances, and may never be used around tables, figures, captions, sections, subsections, subsubsections, or references.
|
||||
|
||||
\subsection{Page Breaks}
|
||||
For your final camera ready copy, you must not use any page break commands. References must flow directly after the text without breaks. Note that some conferences require references to be on a separate page during the review process. AAAI Press, however, does not require this condition for the final paper.
|
||||
|
||||
\subsection{Paper Size, Margins, and Column Width}
|
||||
Papers must be formatted to print in two-column format on 8.5 x 11 inch US letter-sized paper. The margins must be exactly as follows:
|
||||
\begin{itemize}
|
||||
\ifdefined\aaaianonymous
|
||||
\item Top margin: 1.25 inches (first page), .75 inches (others)
|
||||
\else
|
||||
\item Top margin: .75 inches
|
||||
\fi
|
||||
\item Left margin: .75 inches
|
||||
\item Right margin: .75 inches
|
||||
\item Bottom margin: 1.25 inches
|
||||
\end{itemize}
|
||||
|
||||
The default paper size in most installations of \LaTeX{} is A4. However, because we require that your electronic paper be formatted in US letter size, the preamble we have provided includes commands that alter the default to US letter size. Please note that using any other package to alter page size (such as, but not limited to the Geometry package) will result in your final paper being returned to you for correction.
|
||||
|
||||
\subsubsection{Column Width and Margins.}
|
||||
To ensure maximum readability, your paper must include two columns. Each column should be 3.3 inches wide (slightly more than 3.25 inches), with a .375 inch (.952 cm) gutter of white space between the two columns. The aaai2026.sty file will automatically create these columns for you.
|
||||
|
||||
\subsection{Overlength Papers}
|
||||
If your paper is too long and you resort to formatting tricks to make it fit, it is quite likely that it will be returned to you. The best way to retain readability if the paper is overlength is to cut text, figures, or tables. There are a few acceptable ways to reduce paper size that don't affect readability. First, turn on \textbackslash frenchspacing, which will reduce the space after periods. Next, move all your figures and tables to the top of the page. Consider removing less important portions of a figure. If you use \textbackslash centering instead of \textbackslash begin\{center\} in your figure environment, you can also buy some space. For mathematical environments, you may reduce fontsize {\bf but not below 6.5 point}.
|
||||
|
||||
Commands that alter page layout are forbidden. These include \textbackslash columnsep, \textbackslash float, \textbackslash topmargin, \textbackslash topskip, \textbackslash textheight, \textbackslash textwidth, \textbackslash oddsidemargin, and \textbackslash evensizemargin (this list is not exhaustive). If you alter page layout, you will be required to pay the page fee. Other commands that are questionable and may cause your paper to be rejected include \textbackslash parindent, and \textbackslash parskip. Commands that alter the space between sections are forbidden. The title sec package is not allowed. Regardless of the above, if your paper is obviously ``squeezed" it is not going to to be accepted. Options for reducing the length of a paper include reducing the size of your graphics, cutting text, or paying the extra page charge (if it is offered).
|
||||
|
||||
\subsection{Type Font and Size}
|
||||
Your paper must be formatted in Times Roman or Nimbus. We will not accept papers formatted using Computer Modern or Palatino or some other font as the text or heading typeface. Sans serif, when used, should be Courier. Use Symbol or Lucida or Computer Modern for \textit{mathematics only. }
|
||||
|
||||
Do not use type 3 fonts for any portion of your paper, including graphics. Type 3 bitmapped fonts are designed for fixed resolution printers. Most print at 300 dpi even if the printer resolution is 1200 dpi or higher. They also often cause high resolution imagesetter devices to crash. Consequently, AAAI will not accept electronic files containing obsolete type 3 fonts. Files containing those fonts (even in graphics) will be rejected. (Authors using blackboard symbols must avoid packages that use type 3 fonts.)
|
||||
|
||||
Fortunately, there are effective workarounds that will prevent your file from embedding type 3 bitmapped fonts. The easiest workaround is to use the required times, helvet, and courier packages with \LaTeX{}2e. (Note that papers formatted in this way will still use Computer Modern for the mathematics. To make the math look good, you'll either have to use Symbol or Lucida, or you will need to install type 1 Computer Modern fonts --- for more on these fonts, see the section ``Obtaining Type 1 Computer Modern.")
|
||||
|
||||
If you are unsure if your paper contains type 3 fonts, view the PDF in Acrobat Reader. The Properties/Fonts window will display the font name, font type, and encoding properties of all the fonts in the document. If you are unsure if your graphics contain type 3 fonts (and they are PostScript or encapsulated PostScript documents), create PDF versions of them, and consult the properties window in Acrobat Reader.
|
||||
|
||||
The default size for your type must be ten-point with twelve-point leading (line spacing). Start all pages (except the first) directly under the top margin. (See the next section for instructions on formatting the title page.) Indent ten points when beginning a new paragraph, unless the paragraph begins directly below a heading or subheading.
|
||||
|
||||
\subsubsection{Obtaining Type 1 Computer Modern for \LaTeX{}.}
|
||||
If you use Computer Modern for the mathematics in your paper (you cannot use it for the text) you may need to download type 1 Computer fonts. They are available without charge from the American Mathematical Society:
|
||||
http://www.ams.org/tex/type1-fonts.html.
|
||||
|
||||
\subsubsection{Nonroman Fonts.}
|
||||
If your paper includes symbols in other languages (such as, but not limited to, Arabic, Chinese, Hebrew, Japanese, Thai, Russian and other Cyrillic languages), you must restrict their use to bit-mapped figures.
|
||||
|
||||
\subsection{Title and Authors}
|
||||
Your title must appear centered over both text columns in sixteen-point bold type (twenty-four point leading). The title must be written in Title Case capitalization rules (not sentence case). The rules are a bit involved, but in general verbs (including short verbs like be, is, using, and go), nouns, adverbs, adjectives, and pronouns should be capitalized, (including both words in hyphenated terms), while articles, conjunctions, and prepositions are lower case unless they directly follow a colon or long dash. You can use the online tool \url{https://titlecaseconverter.com/} to double-check the proper capitalization (select the "Chicago" style and mark the "Show explanations" checkbox).
|
||||
|
||||
Author's names should appear below the title of the paper, centered in twelve-point type (with fifteen point leading), along with affiliation(s) and complete address(es) (including electronic mail address if available) in nine-point roman type (the twelve point leading). You should begin the two-column format when you come to the abstract.
|
||||
|
||||
\subsubsection{Formatting Author Information.}
|
||||
Author information has to be set according to the following specification depending if you have one or more than one affiliation. You may not use a table nor may you employ the \textbackslash authorblk.sty package. For one or several authors from the same institution, please separate them with commas and write all affiliation directly below (one affiliation per line) using the macros \textbackslash author and \textbackslash affiliations:
|
||||
|
||||
\begin{quote}\begin{scriptsize}\begin{verbatim}
|
||||
\author{
|
||||
Author 1, ..., Author n\\
|
||||
}
|
||||
\affiliations {
|
||||
Address line\\
|
||||
... \\
|
||||
Address line\\
|
||||
}
|
||||
\end{verbatim}\end{scriptsize}\end{quote}
|
||||
|
||||
\noindent For authors from different institutions, use \textbackslash textsuperscript \{\textbackslash rm x \} to match authors and affiliations. Notice that there should not be any spaces between the author name (or comma following it) and the superscript.
|
||||
|
||||
\begin{quote}\begin{scriptsize}\begin{verbatim}
|
||||
\author{
|
||||
AuthorOne\equalcontrib\textsuperscript{\rm 1,\rm 2},
|
||||
AuthorTwo\equalcontrib\textsuperscript{\rm 2},
|
||||
AuthorThree\textsuperscript{\rm 3},\\
|
||||
AuthorFour\textsuperscript{\rm 4},
|
||||
AuthorFive \textsuperscript{\rm 5}}
|
||||
}
|
||||
\affiliations {
|
||||
\textsuperscript{\rm 1}AffiliationOne,\\
|
||||
\textsuperscript{\rm 2}AffiliationTwo,\\
|
||||
\textsuperscript{\rm 3}AffiliationThree,\\
|
||||
\textsuperscript{\rm 4}AffiliationFour,\\
|
||||
\textsuperscript{\rm 5}AffiliationFive\\
|
||||
\{email, email\}@affiliation.com,
|
||||
email@affiliation.com,
|
||||
email@affiliation.com,
|
||||
email@affiliation.com
|
||||
}
|
||||
\end{verbatim}\end{scriptsize}\end{quote}
|
||||
|
||||
You can indicate that some authors contributed equally using the \textbackslash equalcontrib command. This will add a marker after the author names and a footnote on the first page.
|
||||
|
||||
Note that you may want to break the author list for better visualization. You can achieve this using a simple line break (\textbackslash \textbackslash).
|
||||
|
||||
\subsection{\LaTeX{} Copyright Notice}
|
||||
The copyright notice automatically appears if you use aaai2026.sty. It has been hardcoded and may not be disabled.
|
||||
|
||||
\subsection{Credits}
|
||||
Any credits to a sponsoring agency should appear in the acknowledgments section, unless the agency requires different placement. If it is necessary to include this information on the front page, use
|
||||
\textbackslash thanks in either the \textbackslash author or \textbackslash title commands.
|
||||
For example:
|
||||
\begin{quote}
|
||||
\begin{small}
|
||||
\textbackslash title\{Very Important Results in AI\textbackslash thanks\{This work is
|
||||
supported by everybody.\}\}
|
||||
\end{small}
|
||||
\end{quote}
|
||||
Multiple \textbackslash thanks commands can be given. Each will result in a separate footnote indication in the author or title with the corresponding text at the botton of the first column of the document. Note that the \textbackslash thanks command is fragile. You will need to use \textbackslash protect.
|
||||
|
||||
Please do not include \textbackslash pubnote commands in your document.
|
||||
|
||||
\subsection{Abstract}
|
||||
Follow the example commands in this document for creation of your abstract. The command \textbackslash begin\{abstract\} will automatically indent the text block. Please do not indent it further. {Do not include references in your abstract!}
|
||||
|
||||
\subsection{Page Numbers}
|
||||
Do not print any page numbers on your paper. The use of \textbackslash pagestyle is forbidden.
|
||||
|
||||
\subsection{Text}
|
||||
The main body of the paper must be formatted in black, ten-point Times Roman with twelve-point leading (line spacing). You may not reduce font size or the linespacing. Commands that alter font size or line spacing (including, but not limited to baselinestretch, baselineshift, linespread, and others) are expressly forbidden. In addition, you may not use color in the text.
|
||||
|
||||
\subsection{Citations}
|
||||
Citations within the text should include the author's last name and year, for example (Newell 1980). Append lower-case letters to the year in cases of ambiguity. Multiple authors should be treated as follows: (Feigenbaum and Engelmore 1988) or (Ford, Hayes, and Glymour 1992). In the case of four or more authors, list only the first author, followed by et al. (Ford et al. 1997).
|
||||
|
||||
\subsection{Extracts}
|
||||
Long quotations and extracts should be indented ten points from the left and right margins.
|
||||
|
||||
\begin{quote}
|
||||
This is an example of an extract or quotation. Note the indent on both sides. Quotation marks are not necessary if you offset the text in a block like this, and properly identify and cite the quotation in the text.
|
||||
\end{quote}
|
||||
|
||||
\subsection{Footnotes}
|
||||
Use footnotes judiciously, taking into account that they interrupt the reading of the text. When required, they should be consecutively numbered throughout with superscript Arabic numbers. Footnotes should appear at the bottom of the page, separated from the text by a blank line space and a thin, half-point rule.
|
||||
|
||||
\subsection{Headings and Sections}
|
||||
When necessary, headings should be used to separate major sections of your paper. Remember, you are writing a short paper, not a lengthy book! An overabundance of headings will tend to make your paper look more like an outline than a paper. The aaai2026.sty package will create headings for you. Do not alter their size nor their spacing above or below.
|
||||
|
||||
\subsubsection{Section Numbers.}
|
||||
The use of section numbers in AAAI Press papers is optional. To use section numbers in \LaTeX{}, uncomment the setcounter line in your document preamble and change the 0 to a 1. Section numbers should not be used in short poster papers and/or extended abstracts.
|
||||
|
||||
\subsubsection{Section Headings.}
|
||||
Sections should be arranged and headed as follows:
|
||||
\begin{enumerate}
|
||||
\item Main content sections
|
||||
\item Appendices (optional)
|
||||
\item Ethical Statement (optional, unnumbered)
|
||||
\item Acknowledgements (optional, unnumbered)
|
||||
\item References (unnumbered)
|
||||
\end{enumerate}
|
||||
|
||||
\subsubsection{Appendices.}
|
||||
Any appendices must appear after the main content. If your main sections are numbered, appendix sections must use letters instead of arabic numerals. In \LaTeX{} you can use the \texttt{\textbackslash appendix} command to achieve this effect and then use \texttt{\textbackslash section\{Heading\}} normally for your appendix sections.
|
||||
|
||||
\subsubsection{Ethical Statement.}
|
||||
You can write a statement about the potential ethical impact of your work, including its broad societal implications, both positive and negative. If included, such statement must be written in an unnumbered section titled \emph{Ethical Statement}.
|
||||
|
||||
\subsubsection{Acknowledgments.}
|
||||
The acknowledgments section, if included, appears right before the references and is headed ``Acknowledgments". It must not be numbered even if other sections are (use \texttt{\textbackslash section*\{Acknowledgements\}} in \LaTeX{}). This section includes acknowledgments of help from associates and colleagues, credits to sponsoring agencies, financial support, and permission to publish. Please acknowledge other contributors, grant support, and so forth, in this section. Do not put acknowledgments in a footnote on the first page. If your grant agency requires acknowledgment of the grant on page 1, limit the footnote to the required statement, and put the remaining acknowledgments at the back. Please try to limit acknowledgments to no more than three sentences.
|
||||
|
||||
\subsubsection{References.}
|
||||
The references section should be labeled ``References" and must appear at the very end of the paper (don't end the paper with references, and then put a figure by itself on the last page). A sample list of references is given later on in these instructions. Please use a consistent format for references. Poorly prepared or sloppy references reflect badly on the quality of your paper and your research. Please prepare complete and accurate citations.
|
||||
|
||||
\subsection{Illustrations and Figures}
|
||||
|
||||
\begin{figure}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.9\columnwidth]{figure1} % Reduce the figure size so that it is slightly narrower than the column. Don't use precise values for figure width.This setup will avoid overfull boxes.
|
||||
\caption{Using the trim and clip commands produces fragile layers that can result in disasters (like this one from an actual paper) when the color space is corrected or the PDF combined with others for the final proceedings. Crop your figures properly in a graphics program -- not in LaTeX.}
|
||||
\label{fig1}
|
||||
\end{figure}
|
||||
|
||||
\begin{figure*}[t]
|
||||
\centering
|
||||
\includegraphics[width=0.8\textwidth]{figure2} % Reduce the figure size so that it is slightly narrower than the column.
|
||||
\caption{Adjusting the bounding box instead of actually removing the unwanted data resulted multiple layers in this paper. It also needlessly increased the PDF size. In this case, the size of the unwanted layer doubled the paper's size, and produced the following surprising results in final production. Crop your figures properly in a graphics program. Don't just alter the bounding box.}
|
||||
\label{fig2}
|
||||
\end{figure*}
|
||||
|
||||
Your paper must compile in PDF\LaTeX{}. Consequently, all your figures must be .jpg, .png, or .pdf. You may not use the .gif (the resolution is too low), .ps, or .eps file format for your figures.
|
||||
|
||||
Figures, drawings, tables, and photographs should be placed throughout the paper on the page (or the subsequent page) where they are first discussed. Do not group them together at the end of the paper. If placed at the top of the paper, illustrations may run across both columns. Figures must not invade the top, bottom, or side margin areas. Figures must be inserted using the \textbackslash usepackage\{graphicx\}. Number figures sequentially, for example, figure 1, and so on. Do not use minipage to group figures.
|
||||
|
||||
If you normally create your figures using pgfplots, please create the figures first, and then import them as pdfs with proper bounding boxes, as the bounding and trim boxes created by pfgplots are fragile and not valid.
|
||||
|
||||
When you include your figures, you must crop them \textbf{outside} of \LaTeX{}. The command \textbackslash includegraphics*[clip=true, viewport 0 0 10 10]{...} might result in a PDF that looks great, but the image is \textbf{not really cropped.} The full image can reappear (and obscure whatever it is overlapping) when page numbers are applied or color space is standardized. Figures \ref{fig1}, and \ref{fig2} display some unwanted results that often occur.
|
||||
|
||||
If your paper includes illustrations that are not compatible with PDF\TeX{} (such as .eps or .ps documents), you will need to convert them. The epstopdf package will usually work for eps files. You will need to convert your ps files to PDF in either case.
|
||||
|
||||
\subsubsection {Figure Captions.}The illustration number and caption must appear \textit{under} the illustration. Labels and other text with the actual illustration must be at least nine-point type. However, the font and size of figure captions must be 10 point roman. Do not make them smaller, bold, or italic. (Individual words may be italicized if the context requires differentiation.)
|
||||
|
||||
\subsection{Tables}
|
||||
Tables should be presented in 10 point roman type. If necessary, they may be altered to 9 point type. You must not use \texttt{\textbackslash resizebox} or other commands that resize the entire table to make it smaller, because you can't control the final font size this way.
|
||||
If your table is too large you can use \texttt{\textbackslash setlength\{\textbackslash tabcolsep\}\{1mm\}} to compress the columns a bit or you can adapt the content (e.g.: reduce the decimal precision when presenting numbers, use shortened column titles, make some column duble-line to get it narrower).
|
||||
|
||||
Tables that do not fit in a single column must be placed across double columns. If your table won't fit within the margins even when spanning both columns and using the above techniques, you must split it in two separate tables.
|
||||
|
||||
\subsubsection {Table Captions.} The number and caption for your table must appear \textit{under} (not above) the table. Additionally, the font and size of table captions must be 10 point roman and must be placed beneath the figure. Do not make them smaller, bold, or italic. (Individual words may be italicized if the context requires differentiation.)
|
||||
|
||||
\subsubsection{Low-Resolution Bitmaps.}
|
||||
You may not use low-resolution (such as 72 dpi) screen-dumps and GIF files---these files contain so few pixels that they are always blurry, and illegible when printed. If they are color, they will become an indecipherable mess when converted to black and white. This is always the case with gif files, which should never be used. The resolution of screen dumps can be increased by reducing the print size of the original file while retaining the same number of pixels. You can also enlarge files by manipulating them in software such as PhotoShop. Your figures should be 300 dpi when incorporated into your document.
|
||||
|
||||
\subsubsection{\LaTeX{} Overflow.}
|
||||
\LaTeX{} users please beware: \LaTeX{} will sometimes put portions of the figure or table or an equation in the margin. If this happens, you need to make the figure or table span both columns. If absolutely necessary, you may reduce the figure, or reformat the equation, or reconfigure the table.{ \bf Check your log file!} You must fix any overflow into the margin (that means no overfull boxes in \LaTeX{}). \textbf{Nothing is permitted to intrude into the margin or gutter.}
|
||||
|
||||
\subsubsection{Using Color.}
|
||||
Use of color is restricted to figures only. It must be WACG 2.0 compliant. (That is, the contrast ratio must be greater than 4.5:1 no matter the font size.) It must be CMYK, NOT RGB. It may never be used for any portion of the text of your paper. The archival version of your paper will be printed in black and white and grayscale. The web version must be readable by persons with disabilities. Consequently, because conversion to grayscale can cause undesirable effects (red changes to black, yellow can disappear, and so forth), we strongly suggest you avoid placing color figures in your document. If you do include color figures, you must (1) use the CMYK (not RGB) colorspace and (2) be mindful of readers who may happen to have trouble distinguishing colors. Your paper must be decipherable without using color for distinction.
|
||||
|
||||
\subsubsection{Drawings.}
|
||||
We suggest you use computer drawing software (such as Adobe Illustrator or, (if unavoidable), the drawing tools in Microsoft Word) to create your illustrations. Do not use Microsoft Publisher. These illustrations will look best if all line widths are uniform (half- to two-point in size), and you do not create labels over shaded areas. Shading should be 133 lines per inch if possible. Use Times Roman or Helvetica for all figure call-outs. \textbf{Do not use hairline width lines} --- be sure that the stroke width of all lines is at least .5 pt. Zero point lines will print on a laser printer, but will completely disappear on the high-resolution devices used by our printers.
|
||||
|
||||
\subsubsection{Photographs and Images.}
|
||||
Photographs and other images should be in grayscale (color photographs will not reproduce well; for example, red tones will reproduce as black, yellow may turn to white, and so forth) and set to a minimum of 300 dpi. Do not prescreen images.
|
||||
|
||||
\subsubsection{Resizing Graphics.}
|
||||
Resize your graphics \textbf{before} you include them with LaTeX. You may \textbf{not} use trim or clip options as part of your \textbackslash includegraphics command. Resize the media box of your PDF using a graphics program instead.
|
||||
|
||||
\subsubsection{Fonts in Your Illustrations.}
|
||||
You must embed all fonts in your graphics before including them in your LaTeX document.
|
||||
|
||||
\subsubsection{Algorithms.}
|
||||
Algorithms and/or programs are a special kind of figures. Like all illustrations, they should appear floated to the top (preferably) or bottom of the page. However, their caption should appear in the header, left-justified and enclosed between horizontal lines, as shown in Algorithm~\ref{alg:algorithm}. The algorithm body should be terminated with another horizontal line. It is up to the authors to decide whether to show line numbers or not, how to format comments, etc.
|
||||
|
||||
In \LaTeX{} algorithms may be typeset using the {\tt algorithm} and {\tt algorithmic} packages, but you can also use one of the many other packages for the task.
|
||||
|
||||
\begin{algorithm}[tb]
|
||||
\caption{Example algorithm}
|
||||
\label{alg:algorithm}
|
||||
\textbf{Input}: Your algorithm's input\\
|
||||
\textbf{Parameter}: Optional list of parameters\\
|
||||
\textbf{Output}: Your algorithm's output
|
||||
\begin{algorithmic}[1] %[1] enables line numbers
|
||||
\STATE Let $t=0$.
|
||||
\WHILE{condition}
|
||||
\STATE Do some action.
|
||||
\IF {conditional}
|
||||
\STATE Perform task A.
|
||||
\ELSE
|
||||
\STATE Perform task B.
|
||||
\ENDIF
|
||||
\ENDWHILE
|
||||
\STATE \textbf{return} solution
|
||||
\end{algorithmic}
|
||||
\end{algorithm}
|
||||
|
||||
\subsubsection{Listings.}
|
||||
Listings are much like algorithms and programs. They should also appear floated to the top (preferably) or bottom of the page. Listing captions should appear in the header, left-justified and enclosed between horizontal lines as shown in Listing~\ref{lst:listing}. Terminate the body with another horizontal line and avoid any background color. Line numbers, if included, must appear within the text column.
|
||||
|
||||
\begin{listing}[tb]%
|
||||
\caption{Example listing {\tt quicksort.hs}}%
|
||||
\label{lst:listing}%
|
||||
\begin{lstlisting}[language=Haskell]
|
||||
quicksort :: Ord a => [a] -> [a]
|
||||
quicksort [] = []
|
||||
quicksort (p:xs) = (quicksort lesser) ++ [p] ++ (quicksort greater)
|
||||
where
|
||||
lesser = filter (< p) xs
|
||||
greater = filter (>= p) xs
|
||||
\end{lstlisting}
|
||||
\end{listing}
|
||||
|
||||
\subsection{References}
|
||||
The AAAI style includes a set of definitions for use in formatting references with BibTeX. These definitions make the bibliography style fairly close to the ones specified in the Reference Examples appendix below. To use these definitions, you also need the BibTeX style file ``aaai2026.bst," available in the AAAI Author Kit on the AAAI web site. Then, at the end of your paper but before \textbackslash end{document}, you need to put the following lines:
|
||||
|
||||
\begin{quote}
|
||||
\begin{small}
|
||||
\textbackslash bibliography\{bibfile1,bibfile2,...\}
|
||||
\end{small}
|
||||
\end{quote}
|
||||
|
||||
Please note that the aaai2026.sty class already sets the bibliographystyle for you, so you do not have to place any \textbackslash bibliographystyle command in the document yourselves. The aaai2026.sty file is incompatible with the hyperref and navigator packages. If you use either, your references will be garbled and your paper will be returned to you.
|
||||
|
||||
References may be the same size as surrounding text.
|
||||
However, in this section (only), you may reduce the size to {\em \textbackslash small} (9pt) if your paper exceeds the allowable number of pages. Making it any smaller than 9 point with 10 point linespacing, however, is not allowed.
|
||||
|
||||
The list of files in the \textbackslash bibliography command should be the names of your BibTeX source files (that is, the .bib files referenced in your paper).
|
||||
|
||||
The following commands are available for your use in citing references:
|
||||
\begin{quote}
|
||||
{\em \textbackslash cite:} Cites the given reference(s) with a full citation. This appears as ``(Author Year)'' for one reference, or ``(Author Year; Author Year)'' for multiple references.\smallskip\\
|
||||
{\em \textbackslash shortcite:} Cites the given reference(s) with just the year. This appears as ``(Year)'' for one reference, or ``(Year; Year)'' for multiple references.\smallskip\\
|
||||
{\em \textbackslash citeauthor:} Cites the given reference(s) with just the author name(s) and no parentheses.\smallskip\\
|
||||
{\em \textbackslash citeyear:} Cites the given reference(s) with just the date(s) and no parentheses.
|
||||
\end{quote}
|
||||
You may also use any of the \emph{natbib} citation commands.
|
||||
|
||||
\section{Proofreading Your PDF}
|
||||
Please check all the pages of your PDF file. The most commonly forgotten element is the acknowledgements --- especially the correct grant number. Authors also commonly forget to add the metadata to the source, use the wrong reference style file, or don't follow the capitalization rules or comma placement for their author-title information properly. A final common problem is text (expecially equations) that runs into the margin. You will need to fix these common errors before submitting your file.
|
||||
|
||||
\section{Improperly Formatted Files }
|
||||
In the past, AAAI has corrected improperly formatted files submitted by the authors. Unfortunately, this has become an increasingly burdensome expense that we can no longer absorb). Consequently, if your file is improperly formatted, it will be returned to you for correction.
|
||||
|
||||
\section{Naming Your Electronic File}
|
||||
We require that you name your \LaTeX{} source file with the last name (family name) of the first author so that it can easily be differentiated from other submissions. Complete file-naming instructions will be provided to you in the submission instructions.
|
||||
|
||||
\section{Submitting Your Electronic Files to AAAI}
|
||||
Instructions on paper submittal will be provided to you in your acceptance letter.
|
||||
|
||||
\section{Inquiries}
|
||||
If you have any questions about the preparation or submission of your paper as instructed in this document, please contact AAAI Press at the address given below. If you have technical questions about implementation of the aaai style file, please contact an expert at your site. We do not provide technical support for \LaTeX{} or any other software package. To avoid problems, please keep your paper simple, and do not incorporate complicated macros and style files.
|
||||
|
||||
\begin{quote}
|
||||
\noindent AAAI Press\\
|
||||
1101 Pennsylvania Ave, NW Suite 300\\
|
||||
Washington, DC 20004 USA\\
|
||||
\textit{Telephone:} 1-202-360-4062\\
|
||||
\textit{E-mail:} See the submission instructions for your particular conference or event.
|
||||
\end{quote}
|
||||
|
||||
\section{Additional Resources}
|
||||
\LaTeX{} is a difficult program to master. If you've used that software, and this document didn't help or some items were not explained clearly, we recommend you read Michael Shell's excellent document (testflow doc.txt V1.0a 2002/08/13) about obtaining correct PS/PDF output on \LaTeX{} systems. (It was written for another purpose, but it has general application as well). It is available at www.ctan.org in the tex-archive.
|
||||
|
||||
\appendix
|
||||
\section{Reference Examples}
|
||||
\label{sec:reference_examples}
|
||||
|
||||
\nobibliography*
|
||||
Formatted bibliographies should look like the following examples. You should use BibTeX to generate the references. Missing fields are unacceptable when compiling references, and usually indicate that you are using the wrong type of entry (BibTeX class).
|
||||
|
||||
\paragraph{Book with multiple authors~\nocite{em:86}} Use the \texttt{@book} class.\\[.2em]
|
||||
\bibentry{em:86}.
|
||||
|
||||
\paragraph{Journal and magazine articles~\nocite{r:80, hcr:83}} Use the \texttt{@article} class.\\[.2em]
|
||||
\bibentry{r:80}.\\[.2em]
|
||||
\bibentry{hcr:83}.
|
||||
|
||||
\paragraph{Proceedings paper published by a society, press or publisher~\nocite{c:83, c:84}} Use the \texttt{@inproceedings} class. You may abbreviate the \emph{booktitle} field, but make sure that the conference edition is clear.\\[.2em]
|
||||
\bibentry{c:84}.\\[.2em]
|
||||
\bibentry{c:83}.
|
||||
|
||||
\paragraph{University technical report~\nocite{r:86}} Use the \texttt{@techreport} class.\\[.2em]
|
||||
\bibentry{r:86}.
|
||||
|
||||
\paragraph{Dissertation or thesis~\nocite{c:79}} Use the \texttt{@phdthesis} class.\\[.2em]
|
||||
\bibentry{c:79}.
|
||||
|
||||
\paragraph{Forthcoming publication~\nocite{c:21}} Use the \texttt{@misc} class with a \texttt{note="Forthcoming"} annotation.
|
||||
\begin{quote}
|
||||
\begin{footnotesize}
|
||||
\begin{verbatim}
|
||||
@misc(key,
|
||||
[...]
|
||||
note="Forthcoming",
|
||||
)
|
||||
\end{verbatim}
|
||||
\end{footnotesize}
|
||||
\end{quote}
|
||||
\bibentry{c:21}.
|
||||
|
||||
\paragraph{ArXiv paper~\nocite{c:22}} Fetch the BibTeX entry from the "Export Bibtex Citation" link in the arXiv website. Notice it uses the \texttt{@misc} class instead of the \texttt{@article} one, and that it includes the \texttt{eprint} and \texttt{archivePrefix} keys.
|
||||
\begin{quote}
|
||||
\begin{footnotesize}
|
||||
\begin{verbatim}
|
||||
@misc(key,
|
||||
[...]
|
||||
eprint="xxxx.yyyy",
|
||||
archivePrefix="arXiv",
|
||||
)
|
||||
\end{verbatim}
|
||||
\end{footnotesize}
|
||||
\end{quote}
|
||||
\bibentry{c:22}.
|
||||
|
||||
\paragraph{Website or online resource~\nocite{c:23}} Use the \texttt{@misc} class. Add the url in the \texttt{howpublished} field and the date of access in the \texttt{note} field:
|
||||
\begin{quote}
|
||||
\begin{footnotesize}
|
||||
\begin{verbatim}
|
||||
@misc(key,
|
||||
[...]
|
||||
howpublished="\url{http://...}",
|
||||
note="Accessed: YYYY-mm-dd",
|
||||
)
|
||||
\end{verbatim}
|
||||
\end{footnotesize}
|
||||
\end{quote}
|
||||
\bibentry{c:23}.
|
||||
|
||||
\vspace{.2em}
|
||||
For the most up to date version of the AAAI reference style, please consult the \textit{AI Magazine} Author Guidelines at \url{https://aaai.org/ojs/index.php/aimagazine/about/submissions#authorGuidelines}
|
||||
|
||||
\section{Acknowledgments}
|
||||
|
||||
% Anonymous submission version - shorter acknowledgments
|
||||
AAAI is especially grateful to Peter Patel Schneider for his work in implementing the aaai2026.sty file, liberally using the ideas of other style hackers, including Barbara Beeton. We also acknowledge with thanks the work of George Ferguson for his guide to using the style and BibTeX files --- which has been incorporated into this document --- and Hans Guesgen, who provided several timely modifications, as well as the many others who have, from time to time, sent in suggestions on improvements to the AAAI style. We are especially grateful to Francisco Cruz, Marc Pujol-Gonzalez, and Mico Loretan for the improvements to the Bib\TeX{} and \LaTeX{} files made in 2020.
|
||||
|
||||
The preparation of the \LaTeX{} and Bib\TeX{} files that implement these instructions was supported by Schlumberger Palo Alto Research, AT\&T Bell Laboratories, Morgan Kaufmann Publishers, The Live Oak Press, LLC, and AAAI Press. Bibliography style changes were added by Sunil Issar. \verb+\+pubnote was added by J. Scott Penberthy. George Ferguson added support for printing the AAAI copyright slug. Additional changes to aaai2026.sty and aaai2026.bst have been made by Francisco Cruz and Marc Pujol-Gonzalez.
|
||||
|
||||
\bigskip
|
||||
\noindent Thank you for reading these instructions carefully. We look forward to receiving your electronic files!
|
||||
|
||||
|
||||
|
||||
% Note: \bibliographystyle{aaai2026} is automatically set by aaai2026.sty
|
||||
% Do not add \bibliographystyle{aaai2026} here as it will cause "Illegal, another \bibstyle command" error
|
||||
\bibliography{aaai2026}
|
||||
|
||||
\section{Reproducibility Checklist}
|
||||
|
||||
Unless specified otherwise, please answer ``yes'' to each question if the relevant information is described either in the paper itself or in a technical appendix with an explicit reference from the main paper. If you wish to explain an answer further, please do so in a section titled ``Reproducibility Checklist'' at the end of the technical appendix.
|
||||
|
||||
This paper:
|
||||
|
||||
Includes a conceptual outline and/or pseudocode description of AI methods introduced (yes/partial/no/NA)
|
||||
|
||||
Clearly delineates statements that are opinions, hypothesis, and speculation from objective facts and results (yes/no)
|
||||
|
||||
Provides well marked pedagogical references for less-familiare readers to gain background necessary to replicate the paper (yes/no)
|
||||
|
||||
Does this paper make theoretical contributions? (yes/no)
|
||||
|
||||
If yes, please complete the list below.
|
||||
|
||||
All assumptions and restrictions are stated clearly and formally. (yes/partial/no)
|
||||
|
||||
All novel claims are stated formally (e.g., in theorem statements). (yes/partial/no)
|
||||
|
||||
Proofs of all novel claims are included. (yes/partial/no)
|
||||
|
||||
Proof sketches or intuitions are given for complex and/or novel results. (yes/partial/no)
|
||||
|
||||
Appropriate citations to theoretical tools used are given. (yes/partial/no)
|
||||
|
||||
All theoretical claims are demonstrated empirically to hold. (yes/partial/no/NA)
|
||||
|
||||
All experimental code used to eliminate or disprove claims is included. (yes/no/NA)
|
||||
|
||||
Does this paper rely on one or more datasets? (yes/no)
|
||||
|
||||
If yes, please complete the list below.
|
||||
|
||||
A motivation is given for why the experiments are conducted on the selected datasets (yes/partial/no/NA)
|
||||
|
||||
All novel datasets introduced in this paper are included in a data appendix. (yes/partial/no/NA)
|
||||
|
||||
All novel datasets introduced in this paper will be made publicly available upon publication of the paper with a license that allows free usage for research purposes. (yes/partial/no/NA)
|
||||
|
||||
All datasets drawn from the existing literature (potentially including authors' own previously published work) are accompanied by appropriate citations. (yes/no/NA)
|
||||
|
||||
All datasets drawn from the existing literature (potentially including authors' own previously published work) are publicly available. (yes/partial/no/NA)
|
||||
|
||||
All datasets that are not publicly available are described in detail, with explanation why publicly available alternatives are not scientifically satisficing. (yes/partial/no/NA)
|
||||
|
||||
Does this paper include computational experiments? (yes/no)
|
||||
|
||||
If yes, please complete the list below.
|
||||
|
||||
This paper states the number and range of values tried per (hyper-) parameter during development of the paper, along with the criterion used for selecting the final parameter setting. (yes/partial/no/NA)
|
||||
|
||||
Any code required for pre-processing data is included in the appendix. (yes/partial/no).
|
||||
|
||||
All source code required for conducting and analyzing the experiments is included in a code appendix. (yes/partial/no)
|
||||
|
||||
All source code required for conducting and analyzing the experiments will be made publicly available upon publication of the paper with a license that allows free usage for research purposes. (yes/partial/no)
|
||||
|
||||
All source code implementing new methods have comments detailing the implementation, with references to the paper where each step comes from (yes/partial/no)
|
||||
|
||||
If an algorithm depends on randomness, then the method used for setting seeds is described in a way sufficient to allow replication of results. (yes/partial/no/NA)
|
||||
|
||||
This paper specifies the computing infrastructure used for running experiments (hardware and software), including GPU/CPU models; amount of memory; operating system; names and versions of relevant software libraries and frameworks. (yes/partial/no)
|
||||
|
||||
This paper formally describes evaluation metrics used and explains the motivation for choosing these metrics. (yes/partial/no)
|
||||
|
||||
This paper states the number of algorithm runs used to compute each reported result. (yes/no)
|
||||
|
||||
Analysis of experiments goes beyond single-dimensional summaries of performance (e.g., average; median) to include measures of variation, confidence, or other distributional information. (yes/no)
|
||||
|
||||
The significance of any improvement or decrease in performance is judged using appropriate statistical tests (e.g., Wilcoxon signed-rank). (yes/partial/no)
|
||||
|
||||
This paper lists all final (hyper-)parameters used for each model/algorithm in the paper's experiments. (yes/partial/no/NA).
|
||||
|
||||
\end{document}
|
||||
@@ -0,0 +1,111 @@
|
||||
@book{em:86,
|
||||
editor = "Engelmore, Robert and Morgan, Anthony",
|
||||
title = "Blackboard Systems",
|
||||
year = 1986,
|
||||
address = "Reading, Mass.",
|
||||
publisher = "Addison-Wesley",
|
||||
}
|
||||
|
||||
@inproceedings{c:83,
|
||||
author = "Clancey, William J.",
|
||||
year = 1983,
|
||||
title = "{Communication, Simulation, and Intelligent
|
||||
Agents: Implications of Personal Intelligent Machines
|
||||
for Medical Education}",
|
||||
booktitle="Proceedings of the Eighth International Joint Conference on Artificial Intelligence {(IJCAI-83)}",
|
||||
pages = "556-560",
|
||||
address = "Menlo Park, Calif",
|
||||
publisher = "{IJCAI Organization}",
|
||||
}
|
||||
@inproceedings{c:84,
|
||||
author = "Clancey, William J.",
|
||||
year = 1984,
|
||||
title = "{Classification Problem Solving}",
|
||||
booktitle = "Proceedings of the Fourth National
|
||||
Conference on Artificial Intelligence",
|
||||
pages = "45-54",
|
||||
address = "Menlo Park, Calif.",
|
||||
publisher="AAAI Press",
|
||||
}
|
||||
@article{r:80,
|
||||
author = {Robinson, Arthur L.},
|
||||
title = {New Ways to Make Microcircuits Smaller},
|
||||
volume = {208},
|
||||
number = {4447},
|
||||
pages = {1019--1022},
|
||||
year = {1980},
|
||||
doi = {10.1126/science.208.4447.1019},
|
||||
publisher = {American Association for the Advancement of Science},
|
||||
issn = {0036-8075},
|
||||
URL = {https://science.sciencemag.org/content/208/4447/1019},
|
||||
eprint = {https://science.sciencemag.org/content/208/4447/1019.full.pdf},
|
||||
journal = {Science},
|
||||
}
|
||||
@article{r:80x,
|
||||
author = "Robinson, Arthur L.",
|
||||
year = 1980,
|
||||
title = "{New Ways to Make Microcircuits Smaller---Duplicate Entry}",
|
||||
journal = "Science",
|
||||
volume = 208,
|
||||
pages = "1019-1026",
|
||||
}
|
||||
@article{hcr:83,
|
||||
title = {Strategic explanations for a diagnostic consultation system},
|
||||
journal = {International Journal of Man-Machine Studies},
|
||||
volume = {20},
|
||||
number = {1},
|
||||
pages = {3-19},
|
||||
year = {1984},
|
||||
issn = {0020-7373},
|
||||
doi = {https://doi.org/10.1016/S0020-7373(84)80003-6},
|
||||
url = {https://www.sciencedirect.com/science/article/pii/S0020737384800036},
|
||||
author = {Diane Warner Hasling and William J. Clancey and Glenn Rennels},
|
||||
abstract = {This article examines the problem of automatte explanation of reasoning, especially as it relates to expert systems. By explanation we mean the ability of a program to discuss what it is doing in some understandable way. We first present a general framework in which to view explanation and review some of the research done in this area. We then focus on the explanation system for NEOMYCIN, a medical consultation program. A consultation program interactively helps a user to solve a problem. Our goal is to have NEOMYCIN explain its problem-solving strategies. An explanation of strategy describes the plan the program is using to reach a solution. Such an explanation is usually concrete, referring to aspects of the current problem situation. Abstract explanations articulate a general principle, which can be applied in different situations; such explanations are useful in teaching and in explaining by analogy. We describe the aspects of NEOMYCIN that make abstract strategic explanations possible—the representation of strategic knowledge explicitly and separately from domain knowledge— and demonstrate how this representation can be used to generate explanations.}
|
||||
}
|
||||
@article{hcrt:83,
|
||||
author = "Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas",
|
||||
year = 1983,
|
||||
title = "{Strategic Explanations in Consultation---Duplicate}",
|
||||
journal = "The International Journal of Man-Machine Studies",
|
||||
volume = 20,
|
||||
number = 1,
|
||||
pages = "3-19",
|
||||
}
|
||||
@techreport{r:86,
|
||||
author = "Rice, James",
|
||||
year = 1986,
|
||||
title = "{Poligon: A System for Parallel Problem Solving}",
|
||||
type = "Technical Report",
|
||||
number = "KSL-86-19",
|
||||
institution = "Dept.\ of Computer Science, Stanford Univ.",
|
||||
}
|
||||
@phdthesis{c:79,
|
||||
author = "Clancey, William J.",
|
||||
year = 1979,
|
||||
title = "{Transfer of Rule-Based Expertise
|
||||
through a Tutorial Dialogue}",
|
||||
type = "{Ph.D.} diss.",
|
||||
school = "Dept.\ of Computer Science, Stanford Univ.",
|
||||
address = "Stanford, Calif.",
|
||||
}
|
||||
@unpublished{c:21,
|
||||
author = "Clancey, William J.",
|
||||
title = "{The Engineering of Qualitative Models}",
|
||||
year = 2021,
|
||||
note = "Forthcoming",
|
||||
}
|
||||
@misc{c:22,
|
||||
title={Attention Is All You Need},
|
||||
author={Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin},
|
||||
year={2017},
|
||||
eprint={1706.03762},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CL}
|
||||
}
|
||||
@misc{c:23,
|
||||
title = "Pluto: The 'Other' Red Planet",
|
||||
author = "{NASA}",
|
||||
howpublished = "\url{https://www.nasa.gov/nh/pluto-the-other-red-planet}",
|
||||
year = 2015,
|
||||
note = "Accessed: 2018-12-06"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,315 @@
|
||||
\NeedsTeXFormat{LaTeX2e}%
|
||||
\ProvidesPackage{aaai2026}[2026/04/29 AAAI 2026 Submission format]%
|
||||
\def\year{2026}%
|
||||
\typeout{Conference Style for AAAI for LaTeX 2e -- version for submission}%
|
||||
%
|
||||
\def\copyright@on{T}
|
||||
\def\showauthors@on{T}
|
||||
\def\nocopyright{\gdef\copyright@on{}} % Copyright notice is required for camera-ready only.
|
||||
\DeclareOption{submission}{%
|
||||
\gdef\copyright@on{}%
|
||||
\gdef\showauthors@on{}%
|
||||
\long\gdef\pdfinfo #1{\relax}%
|
||||
}%
|
||||
\DeclareOption{draft}{%
|
||||
\gdef\copyright@on{}%
|
||||
}%
|
||||
\ProcessOptions\relax%
|
||||
% WARNING: IF YOU ARE USING THIS STYLE SHEET FOR AN AAAI PUBLICATION, YOU
|
||||
% MAY NOT MODIFY IT FOR ANY REASON. MODIFICATIONS (IN YOUR SOURCE
|
||||
% OR IN THIS STYLE SHEET WILL RESULT IN REJECTION OF YOUR PAPER).
|
||||
%
|
||||
% WARNING: This style is NOT guaranteed to work. It is provided in the
|
||||
% hope that it might make the preparation of papers easier, but this style
|
||||
% file is provided "as is" without warranty of any kind, either express or
|
||||
% implied, including but not limited to the implied warranties of
|
||||
% merchantability, fitness for a particular purpose, or noninfringement.
|
||||
% You use this style file at your own risk. Standard disclaimers apply.
|
||||
% There are undoubtably bugs in this style. If you would like to submit
|
||||
% bug fixes, improvements, etc. please let us know. Please use the contact form
|
||||
% at www.aaai.org.
|
||||
%
|
||||
% Do not use this file unless you are an experienced LaTeX user.
|
||||
%
|
||||
% PHYSICAL PAGE LAYOUT
|
||||
\setlength\topmargin{-0.25in} \setlength\oddsidemargin{-0.25in}
|
||||
\setlength\textheight{9.0in} \setlength\textwidth{7.0in}
|
||||
\setlength\columnsep{0.375in} \newlength\titlebox \setlength\titlebox{2.25in}
|
||||
\setlength\headheight{0pt} \setlength\headsep{0pt}
|
||||
%\setlength\footheight{0pt} \setlength\footskip{0pt}
|
||||
\thispagestyle{empty} \pagestyle{empty}
|
||||
\flushbottom \twocolumn \sloppy
|
||||
% We're never going to need a table of contents, so just flush it to
|
||||
% save space --- suggested by drstrip@sandia-2
|
||||
\def\addcontentsline#1#2#3{}
|
||||
% gf: PRINT COPYRIGHT NOTICE
|
||||
\def\copyright@year{\number\year}
|
||||
\def\copyright@text{Copyright \copyright\space \copyright@year,
|
||||
Association for the Advancement of Artificial Intelligence (www.aaai.org).
|
||||
All rights reserved.}
|
||||
\def\copyrighttext#1{\gdef\copyright@on{T}\gdef\copyright@text{#1}}
|
||||
\def\copyrightyear#1{\gdef\copyright@on{T}\gdef\copyright@year{#1}}
|
||||
% gf: End changes for copyright notice (used in \maketitle, below)
|
||||
% Title stuff, taken from deproc.
|
||||
%
|
||||
\def\maketitle{%
|
||||
\par%
|
||||
\begingroup % to make the footnote style local to the title
|
||||
\def\thefootnote{\fnsymbol{footnote}}
|
||||
\twocolumn[\@maketitle] \@thanks%
|
||||
\endgroup%
|
||||
% Insert copyright slug unless turned off
|
||||
\if T\copyright@on\insert\footins{\noindent\footnotesize\copyright@text}\fi%
|
||||
%
|
||||
\setcounter{footnote}{0}%
|
||||
\let\maketitle\relax%
|
||||
\let\@maketitle\relax%
|
||||
\gdef\@thanks{}%
|
||||
\gdef\@author{}%
|
||||
\gdef\@title{}%
|
||||
\let\thanks\relax%
|
||||
}%
|
||||
\long\gdef\affiliations #1{ \def \affiliations_{\if T\showauthors@on#1\fi}}%
|
||||
%
|
||||
\def\@maketitle{%
|
||||
\def\theauthors{\if T\showauthors@on\@author\else Anonymous submission\fi}
|
||||
\newcounter{eqfn}\setcounter{eqfn}{0}%
|
||||
\newsavebox{\titlearea}
|
||||
\sbox{\titlearea}{
|
||||
\let\footnote\relax\let\thanks\relax%
|
||||
\setcounter{footnote}{0}%
|
||||
\def\equalcontrib{%
|
||||
\ifnum\value{eqfn}=0%
|
||||
\footnote{These authors contributed equally.}%
|
||||
\setcounter{eqfn}{\value{footnote}}%
|
||||
\else%
|
||||
\footnotemark[\value{eqfn}]%
|
||||
\fi%
|
||||
}%
|
||||
\vbox{%
|
||||
\hsize\textwidth%
|
||||
\linewidth\hsize%
|
||||
\vskip 0.625in minus 0.125in%
|
||||
\centering%
|
||||
{\LARGE\bf \@title \par}%
|
||||
\vskip 0.1in plus 0.5fil minus 0.05in%
|
||||
{\Large{\textbf{\theauthors\ifhmode\\\fi}}}%
|
||||
\vskip .2em plus 0.25fil%
|
||||
{\normalsize \affiliations_\ifhmode\\\fi}%
|
||||
\vskip 1em plus 2fil%
|
||||
}%
|
||||
}%
|
||||
%
|
||||
\newlength\actualheight%
|
||||
\settoheight{\actualheight}{\usebox{\titlearea}}%
|
||||
\ifdim\actualheight>\titlebox%
|
||||
\setlength{\titlebox}{\actualheight}%
|
||||
\fi%
|
||||
%
|
||||
\vbox to \titlebox {%
|
||||
\let\footnote\thanks\relax%
|
||||
\setcounter{footnote}{0}%
|
||||
\def\equalcontrib{%
|
||||
\ifnum\value{eqfn}=0%
|
||||
\footnote{These authors contributed equally.}%
|
||||
\setcounter{eqfn}{\value{footnote}}%
|
||||
\else%
|
||||
\footnotemark[\value{eqfn}]%
|
||||
\fi%
|
||||
}%
|
||||
\hsize\textwidth%
|
||||
\linewidth\hsize%
|
||||
\vskip 0.625in minus 0.125in%
|
||||
\centering%
|
||||
{\LARGE\bf \@title \par}%
|
||||
\vskip 0.1in plus 0.5fil minus 0.05in%
|
||||
{\Large{\textbf{\theauthors\ifhmode\\\fi}}}%
|
||||
\vskip .2em plus 0.25fil%
|
||||
{\normalsize \affiliations_\ifhmode\\\fi}%
|
||||
\vskip 1em plus 2fil%
|
||||
}%
|
||||
}%
|
||||
%
|
||||
\renewenvironment{abstract}{%
|
||||
\centerline{\bf Abstract}%
|
||||
\vspace{0.5ex}%
|
||||
\setlength{\leftmargini}{10pt}%
|
||||
\begin{quote}%
|
||||
\small%
|
||||
}{%
|
||||
\par%
|
||||
\end{quote}%
|
||||
\vskip 1ex%
|
||||
}%
|
||||
\newenvironment{links}{%
|
||||
\newcommand{\link}[2]{\par\textbf{##1} --- \url{##2}}%
|
||||
\setlength{\hangindent}{10pt}%
|
||||
\setlength{\parskip}{2pt}%
|
||||
\begin{flushleft}%
|
||||
}{%
|
||||
\end{flushleft}%
|
||||
\vskip 1ex%
|
||||
}%
|
||||
% jsp added:
|
||||
\def\pubnote#1{
|
||||
\thispagestyle{myheadings}%
|
||||
\pagestyle{myheadings}%
|
||||
\markboth{#1}{#1}%
|
||||
\setlength\headheight{10pt}%
|
||||
\setlength\headsep{10pt}%
|
||||
}%
|
||||
%
|
||||
% SECTIONS with less space
|
||||
\def\section{\@startsection {section}{1}{\z@}{-2.0ex plus
|
||||
-0.5ex minus -.2ex}{3pt plus 2pt minus 1pt}{\Large\bf\centering}}
|
||||
\def\subsection{\@startsection{subsection}{2}{\z@}{-2.0ex plus
|
||||
-0.5ex minus -.2ex}{3pt plus 2pt minus 1pt}{\large\bf\raggedright}}
|
||||
\def\subsubsection{\@startsection{subparagraph}{3}{\z@}{-6pt plus
|
||||
%%% DIEGO changed: 29/11/2009
|
||||
%% 2pt minus 1pt}{-1em}{\normalsize\bf}}
|
||||
-2pt minus -1pt}{-1em}{\normalsize\bf}}
|
||||
%%% END changed
|
||||
\renewcommand\paragraph{\@startsection{paragraph}{4}{\z@}{-6pt plus -2pt minus -1pt}{-1em}{\normalsize\bf}}%
|
||||
\setcounter{secnumdepth}{0}
|
||||
% add period to section (but not subsection) numbers, reduce space after
|
||||
%\renewcommand{\thesection}
|
||||
% {\arabic{section}.\hskip-0.6em}
|
||||
%\renewcommand{\thesubsection}
|
||||
% {\arabic{section}.\arabic{subsection}\hskip-0.6em}
|
||||
% FOOTNOTES
|
||||
\footnotesep 6.65pt %
|
||||
\skip\footins 9pt plus 4pt minus 2pt
|
||||
\def\footnoterule{\kern-3pt \hrule width 5pc \kern 2.6pt }
|
||||
\setcounter{footnote}{0}
|
||||
% LISTS AND PARAGRAPHS
|
||||
\parindent 10pt
|
||||
\topsep 4pt plus 1pt minus 2pt
|
||||
\partopsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep 0.5pt plus 1pt minus 0.5pt
|
||||
\parsep 2pt plus 1pt minus 0.5pt
|
||||
\leftmargin 10pt \leftmargini 13pt \leftmarginii 10pt \leftmarginiii 5pt \leftmarginiv 5pt \leftmarginv 5pt \leftmarginvi 5pt
|
||||
\labelwidth\leftmargini\advance\labelwidth-\labelsep \labelsep 5pt
|
||||
\def\@listi{\leftmargin\leftmargini}
|
||||
\def\@listii{\leftmargin\leftmarginii
|
||||
\labelwidth\leftmarginii\advance\labelwidth-\labelsep
|
||||
\topsep 2pt plus 1pt minus 0.5pt
|
||||
\parsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep \parsep}
|
||||
\def\@listiii{\leftmargin\leftmarginiii
|
||||
\labelwidth\leftmarginiii\advance\labelwidth-\labelsep
|
||||
\topsep 1pt plus 0.5pt minus 0.5pt
|
||||
\parsep \z@
|
||||
\partopsep 0.5pt plus 0pt minus 0.5pt
|
||||
\itemsep \topsep}
|
||||
\def\@listiv{\leftmargin\leftmarginiv
|
||||
\labelwidth\leftmarginiv\advance\labelwidth-\labelsep}
|
||||
\def\@listv{\leftmargin\leftmarginv
|
||||
\labelwidth\leftmarginv\advance\labelwidth-\labelsep}
|
||||
\def\@listvi{\leftmargin\leftmarginvi
|
||||
\labelwidth\leftmarginvi\advance\labelwidth-\labelsep}
|
||||
\abovedisplayskip 7pt plus2pt minus5pt%
|
||||
\belowdisplayskip \abovedisplayskip
|
||||
\abovedisplayshortskip 0pt plus3pt%
|
||||
\belowdisplayshortskip 4pt plus3pt minus3pt%
|
||||
% Less leading in most fonts (due to the narrow columns)
|
||||
% The choices were between 1-pt and 1.5-pt leading
|
||||
\def\normalsize{\@setfontsize\normalsize\@xpt{11}} % 10 point on 11
|
||||
\def\small{\@setfontsize\small\@ixpt{10}} % 9 point on 10
|
||||
\def\footnotesize{\@setfontsize\footnotesize\@ixpt{10}} % 9 point on 10
|
||||
\def\scriptsize{\@setfontsize\scriptsize\@viipt{10}} % 7 point on 8
|
||||
\def\tiny{\@setfontsize\tiny\@vipt{7}} % 6 point on 7
|
||||
\def\large{\@setfontsize\large\@xipt{12}} % 11 point on 12
|
||||
\def\Large{\@setfontsize\Large\@xiipt{14}} % 12 point on 14
|
||||
\def\LARGE{\@setfontsize\LARGE\@xivpt{16}} % 14 point on 16
|
||||
\def\huge{\@setfontsize\huge\@xviipt{20}} % 17 point on 20
|
||||
\def\Huge{\@setfontsize\Huge\@xxpt{23}} % 20 point on 23
|
||||
|
||||
\AtBeginDocument{%
|
||||
\@ifpackageloaded{natbib}%
|
||||
{%
|
||||
% When natbib is in use, set the proper style and fix a few things
|
||||
\let\cite\citep
|
||||
\let\shortcite\citeyearpar
|
||||
\setcitestyle{aysep={}}
|
||||
\setlength\bibhang{0pt}
|
||||
\bibliographystyle{aaai2026}
|
||||
}{}%
|
||||
\@ifpackageloaded{hyperref}%
|
||||
{%
|
||||
\PackageError{aaai}{You must not use hyperref in AAAI papers.}{You (or one of the packages you imported) are importing the hyperref package, which is forbidden in AAAI papers. You must remove it from the paper to proceed.}
|
||||
}{}%
|
||||
\@ifpackageloaded{bbm}%
|
||||
{%
|
||||
\PackageError{aaai}{You must not use bbm package in AAAI papers because it introduces Type 3 fonts which are forbidden.}{See https://tex.stackexchange.com/questions/479160/a-replacement-to-mathbbm1-with-type-1-fonts for possible alternatives.}
|
||||
}{}%
|
||||
\@ifpackageloaded{authblk}%
|
||||
{%
|
||||
\PackageError{aaai}{Package authblk is forbbidden.}{Package authblk is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{balance}%
|
||||
{%
|
||||
\PackageError{aaai}{Package balance is forbbidden.}{Package balance is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{CJK}%
|
||||
{%
|
||||
\PackageError{aaai}{Package CJK is forbbidden.}{Package CJK is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{flushend}%
|
||||
{%
|
||||
\PackageError{aaai}{Package flushend is forbbidden.}{Package flushend is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{fontenc}%
|
||||
{%
|
||||
\PackageError{aaai}{Package fontenc is forbbidden.}{Package fontenc is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{fullpage}%
|
||||
{%
|
||||
\PackageError{aaai}{Package fullpage is forbbidden.}{Package fullpage is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{geometry}%
|
||||
{%
|
||||
\PackageError{aaai}{Package geometry is forbbidden.}{Package geometry is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{grffile}%
|
||||
{%
|
||||
\PackageError{aaai}{Package grffile is forbbidden.}{Package grffile is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{navigator}%
|
||||
{%
|
||||
\PackageError{aaai}{Package navigator is forbbidden.}{Package navigator is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{savetrees}%
|
||||
{%
|
||||
\PackageError{aaai}{Package savetrees is forbbidden.}{Package savetrees is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{setspace}%
|
||||
{%
|
||||
\PackageError{aaai}{Package setspace is forbbidden.}{Package setspace is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{stfloats}%
|
||||
{%
|
||||
\PackageError{aaai}{Package stfloats is forbbidden.}{Package stfloats is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{tabu}%
|
||||
{%
|
||||
\PackageError{aaai}{Package tabu is forbbidden.}{Package tabu is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{titlesec}%
|
||||
{%
|
||||
\PackageError{aaai}{Package titlesec is forbbidden.}{Package titlesec is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{tocbibind}%
|
||||
{%
|
||||
\PackageError{aaai}{Package tocbibind is forbbidden.}{Package tocbibind is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{ulem}%
|
||||
{%
|
||||
\PackageError{aaai}{Package ulem is forbbidden.}{Package ulem is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
\@ifpackageloaded{wrapfig}%
|
||||
{%
|
||||
\PackageError{aaai}{Package wrapfig is forbbidden.}{Package wrapfig is forbbiden. You must find an alternative.}
|
||||
}{}%
|
||||
}
|
||||
|
||||
\let\endthebibliography=\endlist
|
||||
@@ -0,0 +1,50 @@
|
||||
# *ACL Paper Styles
|
||||
|
||||
This directory contains the latest LaTeX templates for *ACL conferences.
|
||||
|
||||
## Instructions for authors
|
||||
|
||||
Paper submissions to *ACL conferences must use the official ACL style
|
||||
templates.
|
||||
|
||||
The LaTeX style files are available
|
||||
|
||||
- as an [Overleaf template](https://www.overleaf.com/latex/templates/association-for-computational-linguistics-acl-conference/jvxskxpnznfj)
|
||||
- in this repository
|
||||
- as a [.zip file](https://github.com/acl-org/acl-style-files/archive/refs/heads/master.zip)
|
||||
|
||||
Please see [`acl_latex.tex`](https://github.com/acl-org/acl-style-files/blob/master/acl_latex.tex) for an example.
|
||||
|
||||
Please follow the paper formatting guidelines general to *ACL
|
||||
conferences:
|
||||
|
||||
- [Paper formatting guidelines](https://acl-org.github.io/ACLPUB/formatting.html)
|
||||
|
||||
Authors may not modify these style files or use templates designed for
|
||||
other conferences.
|
||||
|
||||
## Instructions for publications chairs
|
||||
|
||||
To adapt the style files for your conference, please fork this repository and
|
||||
make necessary changes. Minimally, you'll need to update the name of
|
||||
the conference and rename the files.
|
||||
|
||||
If you make improvements to the templates that should be propagated to
|
||||
future conferences, please submit a pull request. Thank you in
|
||||
advance!
|
||||
|
||||
In older versions of the templates, authors were asked to fill in the
|
||||
START submission ID so that it would be stamped at the top of each
|
||||
page of the anonymized version. This is no longer needed, because it
|
||||
is now possible to do this stamping automatically within
|
||||
START. Currently, the way to do this is for the program chair to email
|
||||
support@softconf.com and request it.
|
||||
|
||||
## Instructions for making changes to style files
|
||||
|
||||
- merge pull request in github, or push to github
|
||||
- git pull from github to a local repository
|
||||
- then, git push from your local repository to overleaf project
|
||||
- Overleaf project is https://www.overleaf.com/project/5f64f1fb97c4c50001b60549
|
||||
- Overleaf git url is https://git.overleaf.com/5f64f1fb97c4c50001b60549
|
||||
- then, click "Submit" and then "Submit as Template" in overleaf in order to ask overleaf to update the overleaf template from the overleaf project
|
||||
@@ -0,0 +1,312 @@
|
||||
% This is the LaTex style file for *ACL.
|
||||
% The official sources can be found at
|
||||
%
|
||||
% https://github.com/acl-org/acl-style-files/
|
||||
%
|
||||
% This package is activated by adding
|
||||
%
|
||||
% \usepackage{acl}
|
||||
%
|
||||
% to your LaTeX file. When submitting your paper for review, add the "review" option:
|
||||
%
|
||||
% \usepackage[review]{acl}
|
||||
|
||||
\newif\ifacl@finalcopy
|
||||
\newif\ifacl@anonymize
|
||||
\newif\ifacl@linenumbers
|
||||
\newif\ifacl@pagenumbers
|
||||
\DeclareOption{final}{\acl@finalcopytrue\acl@anonymizefalse\acl@linenumbersfalse\acl@pagenumbersfalse}
|
||||
\DeclareOption{review}{\acl@finalcopyfalse\acl@anonymizetrue\acl@linenumberstrue\acl@pagenumberstrue}
|
||||
\DeclareOption{preprint}{\acl@finalcopytrue\acl@anonymizefalse\acl@linenumbersfalse\acl@pagenumberstrue}
|
||||
\ExecuteOptions{final} % final copy is the default
|
||||
|
||||
% include hyperref, unless user specifies nohyperref option like this:
|
||||
% \usepackage[nohyperref]{acl}
|
||||
\newif\ifacl@hyperref
|
||||
\DeclareOption{hyperref}{\acl@hyperreftrue}
|
||||
\DeclareOption{nohyperref}{\acl@hyperreffalse}
|
||||
\ExecuteOptions{hyperref} % default is to use hyperref
|
||||
\ProcessOptions\relax
|
||||
|
||||
\typeout{Conference Style for ACL}
|
||||
|
||||
\usepackage{xcolor}
|
||||
|
||||
\ifacl@linenumbers
|
||||
% Add draft line numbering via the lineno package
|
||||
% https://texblog.org/2012/02/08/adding-line-numbers-to-documents/
|
||||
\usepackage[switch,mathlines]{lineno}
|
||||
|
||||
% Line numbers in gray Helvetica 8pt
|
||||
\font\aclhv = phvb at 8pt
|
||||
\renewcommand\linenumberfont{\aclhv\color{lightgray}}
|
||||
|
||||
% Zero-fill line numbers
|
||||
% NUMBER with left flushed zeros \fillzeros[<WIDTH>]<NUMBER>
|
||||
\newcount\cv@tmpc@ \newcount\cv@tmpc
|
||||
\def\fillzeros[#1]#2{\cv@tmpc@=#2\relax\ifnum\cv@tmpc@<0\cv@tmpc@=-\cv@tmpc@\fi
|
||||
\cv@tmpc=1 %
|
||||
\loop\ifnum\cv@tmpc@<10 \else \divide\cv@tmpc@ by 10 \advance\cv@tmpc by 1 \fi
|
||||
\ifnum\cv@tmpc@=10\relax\cv@tmpc@=11\relax\fi \ifnum\cv@tmpc@>10 \repeat
|
||||
\ifnum#2<0\advance\cv@tmpc1\relax-\fi
|
||||
\loop\ifnum\cv@tmpc<#1\relax0\advance\cv@tmpc1\relax\fi \ifnum\cv@tmpc<#1 \repeat
|
||||
\cv@tmpc@=#2\relax\ifnum\cv@tmpc@<0\cv@tmpc@=-\cv@tmpc@\fi \relax\the\cv@tmpc@}%
|
||||
\renewcommand\thelinenumber{\fillzeros[3]{\arabic{linenumber}}}
|
||||
\AtBeginDocument{\linenumbers}
|
||||
|
||||
\setlength{\linenumbersep}{1.6cm}
|
||||
|
||||
% Bug: An equation with $$ ... $$ isn't numbered, nor is the previous line.
|
||||
|
||||
% Patch amsmath commands so that the previous line and the equation itself
|
||||
% are numbered. Bug: multline has an extra line number.
|
||||
% https://tex.stackexchange.com/questions/461186/how-to-use-lineno-with-amsmath-align
|
||||
\usepackage{etoolbox} %% <- for \pretocmd, \apptocmd and \patchcmd
|
||||
|
||||
\newcommand*\linenomathpatch[1]{%
|
||||
\expandafter\pretocmd\csname #1\endcsname {\linenomath}{}{}%
|
||||
\expandafter\pretocmd\csname #1*\endcsname {\linenomath}{}{}%
|
||||
\expandafter\apptocmd\csname end#1\endcsname {\endlinenomath}{}{}%
|
||||
\expandafter\apptocmd\csname end#1*\endcsname {\endlinenomath}{}{}%
|
||||
}
|
||||
\newcommand*\linenomathpatchAMS[1]{%
|
||||
\expandafter\pretocmd\csname #1\endcsname {\linenomathAMS}{}{}%
|
||||
\expandafter\pretocmd\csname #1*\endcsname {\linenomathAMS}{}{}%
|
||||
\expandafter\apptocmd\csname end#1\endcsname {\endlinenomath}{}{}%
|
||||
\expandafter\apptocmd\csname end#1*\endcsname {\endlinenomath}{}{}%
|
||||
}
|
||||
|
||||
%% Definition of \linenomathAMS depends on whether the mathlines option is provided
|
||||
\expandafter\ifx\linenomath\linenomathWithnumbers
|
||||
\let\linenomathAMS\linenomathWithnumbers
|
||||
%% The following line gets rid of an extra line numbers at the bottom:
|
||||
\patchcmd\linenomathAMS{\advance\postdisplaypenalty\linenopenalty}{}{}{}
|
||||
\else
|
||||
\let\linenomathAMS\linenomathNonumbers
|
||||
\fi
|
||||
|
||||
\AtBeginDocument{%
|
||||
\linenomathpatch{equation}%
|
||||
\linenomathpatchAMS{gather}%
|
||||
\linenomathpatchAMS{multline}%
|
||||
\linenomathpatchAMS{align}%
|
||||
\linenomathpatchAMS{alignat}%
|
||||
\linenomathpatchAMS{flalign}%
|
||||
}
|
||||
\else
|
||||
% Hack to ignore these commands, which review mode puts into the .aux file.
|
||||
\newcommand{\@LN@col}[1]{}
|
||||
\newcommand{\@LN}[2]{}
|
||||
\newcommand{\nolinenumbers}{}
|
||||
\fi
|
||||
|
||||
\PassOptionsToPackage{a4paper,margin=2.5cm,heightrounded=true}{geometry}
|
||||
\RequirePackage{geometry}
|
||||
|
||||
\setlength\columnsep{0.6cm}
|
||||
\newlength\titlebox
|
||||
\setlength\titlebox{11\baselineskip}
|
||||
% \titlebox should be a multiple of \baselineskip so that
|
||||
% column height remaining fits an exact number of lines of text
|
||||
|
||||
\flushbottom \twocolumn \sloppy
|
||||
|
||||
% We're never going to need a table of contents, so just flush it to
|
||||
% save space --- suggested by drstrip@sandia-2
|
||||
\def\addcontentsline#1#2#3{}
|
||||
|
||||
\ifacl@pagenumbers
|
||||
\pagenumbering{arabic}
|
||||
\else
|
||||
\thispagestyle{empty}
|
||||
\pagestyle{empty}
|
||||
\fi
|
||||
|
||||
%% Title and Authors %%
|
||||
|
||||
\let\Thanks\thanks % \Thanks and \thanks used to be different, but keep this for backwards compatibility.
|
||||
|
||||
\newcommand\outauthor{%
|
||||
\begin{tabular}[t]{c}
|
||||
\ifacl@anonymize
|
||||
\bfseries Anonymous ACL submission
|
||||
\else
|
||||
\bfseries\@author
|
||||
\fi
|
||||
\end{tabular}}
|
||||
|
||||
% Mostly taken from deproc.
|
||||
\AtBeginDocument{
|
||||
\def\maketitle{\par
|
||||
\begingroup
|
||||
\def\thefootnote{\fnsymbol{footnote}}
|
||||
\twocolumn[\@maketitle]
|
||||
\@thanks
|
||||
\endgroup
|
||||
\setcounter{footnote}{0}
|
||||
\let\maketitle\relax
|
||||
\let\@maketitle\relax
|
||||
\gdef\@thanks{}\gdef\@author{}\gdef\@title{}\let\thanks\relax}
|
||||
\def\@maketitle{\vbox to \titlebox{\hsize\textwidth
|
||||
\linewidth\hsize \vskip 0.125in minus 0.125in \centering
|
||||
{\Large\bfseries \@title \par} \vskip 0.2in plus 1fil minus 0.1in
|
||||
{\def\and{\unskip\enspace{\rmfamily and}\enspace}%
|
||||
\def\And{\end{tabular}\hss \egroup \hskip 1in plus 2fil
|
||||
\hbox to 0pt\bgroup\hss \begin{tabular}[t]{c}\bfseries}%
|
||||
\def\AND{\end{tabular}\hss\egroup \hfil\hfil\egroup
|
||||
\vskip 0.25in plus 1fil minus 0.125in
|
||||
\hbox to \linewidth\bgroup\large \hfil\hfil
|
||||
\hbox to 0pt\bgroup\hss \begin{tabular}[t]{c}\bfseries}
|
||||
\hbox to \linewidth\bgroup\large \hfil\hfil
|
||||
\hbox to 0pt\bgroup\hss
|
||||
\outauthor
|
||||
\hss\egroup
|
||||
\hfil\hfil\egroup}
|
||||
\vskip 0.3in plus 2fil minus 0.1in
|
||||
}}
|
||||
}
|
||||
|
||||
% margins and font size for abstract
|
||||
\renewenvironment{abstract}%
|
||||
{\begin{center}\large\textbf{\abstractname}\end{center}%
|
||||
\begin{list}{}%
|
||||
{\setlength{\rightmargin}{0.6cm}%
|
||||
\setlength{\leftmargin}{0.6cm}}%
|
||||
\item[]\ignorespaces%
|
||||
\@setsize\normalsize{12pt}\xpt\@xpt
|
||||
}%
|
||||
{\unskip\end{list}}
|
||||
|
||||
% Resizing figure and table captions - SL
|
||||
% Support for interacting with the caption, subfigure, and subcaption packages - SL
|
||||
\RequirePackage{caption}
|
||||
\DeclareCaptionFont{10pt}{\fontsize{10pt}{12pt}\selectfont}
|
||||
\captionsetup{font=10pt}
|
||||
|
||||
\RequirePackage{natbib}
|
||||
% for citation commands in the .tex, authors can use:
|
||||
% \citep, \citet, and \citeyearpar for compatibility with natbib, or
|
||||
% \cite, \newcite, and \shortcite for compatibility with older ACL .sty files
|
||||
\renewcommand\cite{\citep} % to get "(Author Year)" with natbib
|
||||
\newcommand\shortcite{\citeyearpar}% to get "(Year)" with natbib
|
||||
\newcommand\newcite{\citet} % to get "Author (Year)" with natbib
|
||||
\newcommand{\citeposs}[1]{\citeauthor{#1}'s (\citeyear{#1})} % to get "Author's (Year)"
|
||||
|
||||
\bibliographystyle{acl_natbib}
|
||||
|
||||
% Bibliography
|
||||
|
||||
% Don't put a label in the bibliography at all. Just use the unlabeled format
|
||||
% instead.
|
||||
\def\thebibliography#1{\vskip\parskip%
|
||||
\vskip\baselineskip%
|
||||
\def\baselinestretch{1}%
|
||||
\ifx\@currsize\normalsize\@normalsize\else\@currsize\fi%
|
||||
\vskip-\parskip%
|
||||
\vskip-\baselineskip%
|
||||
\section*{References\@mkboth
|
||||
{References}{References}}\list
|
||||
{}{\setlength{\labelwidth}{0pt}\setlength{\leftmargin}{\parindent}
|
||||
\setlength{\itemindent}{-\parindent}}
|
||||
\def\newblock{\hskip .11em plus .33em minus -.07em}
|
||||
\sloppy\clubpenalty4000\widowpenalty4000
|
||||
\sfcode`\.=1000\relax}
|
||||
\let\endthebibliography=\endlist
|
||||
|
||||
|
||||
% Allow for a bibliography of sources of attested examples
|
||||
\def\thesourcebibliography#1{\vskip\parskip%
|
||||
\vskip\baselineskip%
|
||||
\def\baselinestretch{1}%
|
||||
\ifx\@currsize\normalsize\@normalsize\else\@currsize\fi%
|
||||
\vskip-\parskip%
|
||||
\vskip-\baselineskip%
|
||||
\section*{Sources of Attested Examples\@mkboth
|
||||
{Sources of Attested Examples}{Sources of Attested Examples}}\list
|
||||
{}{\setlength{\labelwidth}{0pt}\setlength{\leftmargin}{\parindent}
|
||||
\setlength{\itemindent}{-\parindent}}
|
||||
\def\newblock{\hskip .11em plus .33em minus -.07em}
|
||||
\sloppy\clubpenalty4000\widowpenalty4000
|
||||
\sfcode`\.=1000\relax}
|
||||
\let\endthesourcebibliography=\endlist
|
||||
|
||||
% sections with less space
|
||||
\def\section{\@startsection {section}{1}{\z@}{-2.0ex plus
|
||||
-0.5ex minus -.2ex}{1.5ex plus 0.3ex minus .2ex}{\large\bfseries\raggedright}}
|
||||
\def\subsection{\@startsection{subsection}{2}{\z@}{-1.8ex plus
|
||||
-0.5ex minus -.2ex}{0.8ex plus .2ex}{\normalsize\bfseries\raggedright}}
|
||||
%% changed by KO to - values to get the initial parindent right
|
||||
\def\subsubsection{\@startsection{subsubsection}{3}{\z@}{-1.5ex plus
|
||||
-0.5ex minus -.2ex}{0.5ex plus .2ex}{\normalsize\bfseries\raggedright}}
|
||||
\def\paragraph{\@startsection{paragraph}{4}{\z@}{1.5ex plus
|
||||
0.5ex minus .2ex}{-1em}{\normalsize\bfseries}}
|
||||
\def\subparagraph{\@startsection{subparagraph}{5}{\parindent}{1.5ex plus
|
||||
0.5ex minus .2ex}{-1em}{\normalsize\bfseries}}
|
||||
|
||||
% Footnotes
|
||||
\footnotesep 6.65pt %
|
||||
\skip\footins 9pt plus 4pt minus 2pt
|
||||
\def\footnoterule{\kern-3pt \hrule width 5pc \kern 2.6pt }
|
||||
\setcounter{footnote}{0}
|
||||
|
||||
% Lists and paragraphs
|
||||
\parindent 1em
|
||||
\topsep 4pt plus 1pt minus 2pt
|
||||
\partopsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep 2pt plus 1pt minus 0.5pt
|
||||
\parsep 2pt plus 1pt minus 0.5pt
|
||||
|
||||
\leftmargin 2em \leftmargini\leftmargin \leftmarginii 2em
|
||||
\leftmarginiii 1.5em \leftmarginiv 1.0em \leftmarginv .5em \leftmarginvi .5em
|
||||
\labelwidth\leftmargini\advance\labelwidth-\labelsep \labelsep 5pt
|
||||
|
||||
\def\@listi{\leftmargin\leftmargini}
|
||||
\def\@listii{\leftmargin\leftmarginii
|
||||
\labelwidth\leftmarginii\advance\labelwidth-\labelsep
|
||||
\topsep 2pt plus 1pt minus 0.5pt
|
||||
\parsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep \parsep}
|
||||
\def\@listiii{\leftmargin\leftmarginiii
|
||||
\labelwidth\leftmarginiii\advance\labelwidth-\labelsep
|
||||
\topsep 1pt plus 0.5pt minus 0.5pt
|
||||
\parsep \z@ \partopsep 0.5pt plus 0pt minus 0.5pt
|
||||
\itemsep \topsep}
|
||||
\def\@listiv{\leftmargin\leftmarginiv
|
||||
\labelwidth\leftmarginiv\advance\labelwidth-\labelsep}
|
||||
\def\@listv{\leftmargin\leftmarginv
|
||||
\labelwidth\leftmarginv\advance\labelwidth-\labelsep}
|
||||
\def\@listvi{\leftmargin\leftmarginvi
|
||||
\labelwidth\leftmarginvi\advance\labelwidth-\labelsep}
|
||||
|
||||
\abovedisplayskip 7pt plus2pt minus5pt%
|
||||
\belowdisplayskip \abovedisplayskip
|
||||
\abovedisplayshortskip 0pt plus3pt%
|
||||
\belowdisplayshortskip 4pt plus3pt minus3pt%
|
||||
|
||||
% Less leading in most fonts (due to the narrow columns)
|
||||
% The choices were between 1-pt and 1.5-pt leading
|
||||
\def\@normalsize{\@setsize\normalsize{11pt}\xpt\@xpt}
|
||||
\def\small{\@setsize\small{10pt}\ixpt\@ixpt}
|
||||
\def\footnotesize{\@setsize\footnotesize{10pt}\ixpt\@ixpt}
|
||||
\def\scriptsize{\@setsize\scriptsize{8pt}\viipt\@viipt}
|
||||
\def\tiny{\@setsize\tiny{7pt}\vipt\@vipt}
|
||||
\def\large{\@setsize\large{14pt}\xiipt\@xiipt}
|
||||
\def\Large{\@setsize\Large{16pt}\xivpt\@xivpt}
|
||||
\def\LARGE{\@setsize\LARGE{20pt}\xviipt\@xviipt}
|
||||
\def\huge{\@setsize\huge{23pt}\xxpt\@xxpt}
|
||||
\def\Huge{\@setsize\Huge{28pt}\xxvpt\@xxvpt}
|
||||
|
||||
% The hyperref manual (section 9) says hyperref should be loaded after natbib
|
||||
\ifacl@hyperref
|
||||
\PassOptionsToPackage{breaklinks}{hyperref}
|
||||
\RequirePackage{hyperref}
|
||||
% make links dark blue
|
||||
\definecolor{darkblue}{rgb}{0, 0, 0.5}
|
||||
\hypersetup{colorlinks=true, citecolor=darkblue, linkcolor=darkblue, urlcolor=darkblue}
|
||||
\else
|
||||
% This definition is used if the hyperref package is not loaded.
|
||||
% It provides a backup, no-op definiton of \href.
|
||||
% This is necessary because \href command is used in the acl_natbib.bst file.
|
||||
\def\href#1#2{{#2}}
|
||||
\usepackage{url}
|
||||
\fi
|
||||
@@ -0,0 +1,377 @@
|
||||
\documentclass[11pt]{article}
|
||||
|
||||
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
|
||||
% Change to "preprint" to generate a non-anonymous version with page numbers.
|
||||
\usepackage[review]{acl}
|
||||
|
||||
% Standard package includes
|
||||
\usepackage{times}
|
||||
\usepackage{latexsym}
|
||||
|
||||
% For proper rendering and hyphenation of words containing Latin characters (including in bib files)
|
||||
\usepackage[T1]{fontenc}
|
||||
% For Vietnamese characters
|
||||
% \usepackage[T5]{fontenc}
|
||||
% See https://www.latex-project.org/help/documentation/encguide.pdf for other character sets
|
||||
|
||||
% This assumes your files are encoded as UTF8
|
||||
\usepackage[utf8]{inputenc}
|
||||
|
||||
% This is not strictly necessary, and may be commented out,
|
||||
% but it will improve the layout of the manuscript,
|
||||
% and will typically save some space.
|
||||
\usepackage{microtype}
|
||||
|
||||
% This is also not strictly necessary, and may be commented out.
|
||||
% However, it will improve the aesthetics of text in
|
||||
% the typewriter font.
|
||||
\usepackage{inconsolata}
|
||||
|
||||
%Including images in your LaTeX document requires adding
|
||||
%additional package(s)
|
||||
\usepackage{graphicx}
|
||||
|
||||
% If the title and author information does not fit in the area allocated, uncomment the following
|
||||
%
|
||||
%\setlength\titlebox{<dim>}
|
||||
%
|
||||
% and set <dim> to something 5cm or larger.
|
||||
|
||||
\title{Instructions for *ACL Proceedings}
|
||||
|
||||
% Author information can be set in various styles:
|
||||
% For several authors from the same institution:
|
||||
% \author{Author 1 \and ... \and Author n \\
|
||||
% Address line \\ ... \\ Address line}
|
||||
% if the names do not fit well on one line use
|
||||
% Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
|
||||
% For authors from different institutions:
|
||||
% \author{Author 1 \\ Address line \\ ... \\ Address line
|
||||
% \And ... \And
|
||||
% Author n \\ Address line \\ ... \\ Address line}
|
||||
% To start a separate ``row'' of authors use \AND, as in
|
||||
% \author{Author 1 \\ Address line \\ ... \\ Address line
|
||||
% \AND
|
||||
% Author 2 \\ Address line \\ ... \\ Address line \And
|
||||
% Author 3 \\ Address line \\ ... \\ Address line}
|
||||
|
||||
\author{First Author \\
|
||||
Affiliation / Address line 1 \\
|
||||
Affiliation / Address line 2 \\
|
||||
Affiliation / Address line 3 \\
|
||||
\texttt{email@domain} \\\And
|
||||
Second Author \\
|
||||
Affiliation / Address line 1 \\
|
||||
Affiliation / Address line 2 \\
|
||||
Affiliation / Address line 3 \\
|
||||
\texttt{email@domain} \\}
|
||||
|
||||
%\author{
|
||||
% \textbf{First Author\textsuperscript{1}},
|
||||
% \textbf{Second Author\textsuperscript{1,2}},
|
||||
% \textbf{Third T. Author\textsuperscript{1}},
|
||||
% \textbf{Fourth Author\textsuperscript{1}},
|
||||
%\\
|
||||
% \textbf{Fifth Author\textsuperscript{1,2}},
|
||||
% \textbf{Sixth Author\textsuperscript{1}},
|
||||
% \textbf{Seventh Author\textsuperscript{1}},
|
||||
% \textbf{Eighth Author \textsuperscript{1,2,3,4}},
|
||||
%\\
|
||||
% \textbf{Ninth Author\textsuperscript{1}},
|
||||
% \textbf{Tenth Author\textsuperscript{1}},
|
||||
% \textbf{Eleventh E. Author\textsuperscript{1,2,3,4,5}},
|
||||
% \textbf{Twelfth Author\textsuperscript{1}},
|
||||
%\\
|
||||
% \textbf{Thirteenth Author\textsuperscript{3}},
|
||||
% \textbf{Fourteenth F. Author\textsuperscript{2,4}},
|
||||
% \textbf{Fifteenth Author\textsuperscript{1}},
|
||||
% \textbf{Sixteenth Author\textsuperscript{1}},
|
||||
%\\
|
||||
% \textbf{Seventeenth S. Author\textsuperscript{4,5}},
|
||||
% \textbf{Eighteenth Author\textsuperscript{3,4}},
|
||||
% \textbf{Nineteenth N. Author\textsuperscript{2,5}},
|
||||
% \textbf{Twentieth Author\textsuperscript{1}}
|
||||
%\\
|
||||
%\\
|
||||
% \textsuperscript{1}Affiliation 1,
|
||||
% \textsuperscript{2}Affiliation 2,
|
||||
% \textsuperscript{3}Affiliation 3,
|
||||
% \textsuperscript{4}Affiliation 4,
|
||||
% \textsuperscript{5}Affiliation 5
|
||||
%\\
|
||||
% \small{
|
||||
% \textbf{Correspondence:} \href{mailto:email@domain}{email@domain}
|
||||
% }
|
||||
%}
|
||||
|
||||
\begin{document}
|
||||
\maketitle
|
||||
\begin{abstract}
|
||||
This document is a supplement to the general instructions for *ACL authors. It contains instructions for using the \LaTeX{} style files for ACL conferences.
|
||||
The document itself conforms to its own specifications, and is therefore an example of what your manuscript should look like.
|
||||
These instructions should be used both for papers submitted for review and for final versions of accepted papers.
|
||||
\end{abstract}
|
||||
|
||||
\section{Introduction}
|
||||
|
||||
These instructions are for authors submitting papers to *ACL conferences using \LaTeX. They are not self-contained. All authors must follow the general instructions for *ACL proceedings,\footnote{\url{http://acl-org.github.io/ACLPUB/formatting.html}} and this document contains additional instructions for the \LaTeX{} style files.
|
||||
|
||||
The templates include the \LaTeX{} source of this document (\texttt{acl\_latex.tex}),
|
||||
the \LaTeX{} style file used to format it (\texttt{acl.sty}),
|
||||
an ACL bibliography style (\texttt{acl\_natbib.bst}),
|
||||
an example bibliography (\texttt{custom.bib}),
|
||||
and the bibliography for the ACL Anthology (\texttt{anthology.bib}).
|
||||
|
||||
\section{Engines}
|
||||
|
||||
To produce a PDF file, pdf\LaTeX{} is strongly recommended (over original \LaTeX{} plus dvips+ps2pdf or dvipdf).
|
||||
The style file \texttt{acl.sty} can also be used with
|
||||
lua\LaTeX{} and
|
||||
Xe\LaTeX{}, which are especially suitable for text in non-Latin scripts.
|
||||
The file \texttt{acl\_lualatex.tex} in this repository provides
|
||||
an example of how to use \texttt{acl.sty} with either
|
||||
lua\LaTeX{} or
|
||||
Xe\LaTeX{}.
|
||||
|
||||
\section{Preamble}
|
||||
|
||||
The first line of the file must be
|
||||
\begin{quote}
|
||||
\begin{verbatim}
|
||||
\documentclass[11pt]{article}
|
||||
\end{verbatim}
|
||||
\end{quote}
|
||||
|
||||
To load the style file in the review version:
|
||||
\begin{quote}
|
||||
\begin{verbatim}
|
||||
\usepackage[review]{acl}
|
||||
\end{verbatim}
|
||||
\end{quote}
|
||||
For the final version, omit the \verb|review| option:
|
||||
\begin{quote}
|
||||
\begin{verbatim}
|
||||
\usepackage{acl}
|
||||
\end{verbatim}
|
||||
\end{quote}
|
||||
|
||||
To use Times Roman, put the following in the preamble:
|
||||
\begin{quote}
|
||||
\begin{verbatim}
|
||||
\usepackage{times}
|
||||
\end{verbatim}
|
||||
\end{quote}
|
||||
(Alternatives like txfonts or newtx are also acceptable.)
|
||||
|
||||
Please see the \LaTeX{} source of this document for comments on other packages that may be useful.
|
||||
|
||||
Set the title and author using \verb|\title| and \verb|\author|. Within the author list, format multiple authors using \verb|\and| and \verb|\And| and \verb|\AND|; please see the \LaTeX{} source for examples.
|
||||
|
||||
By default, the box containing the title and author names is set to the minimum of 5 cm. If you need more space, include the following in the preamble:
|
||||
\begin{quote}
|
||||
\begin{verbatim}
|
||||
\setlength\titlebox{<dim>}
|
||||
\end{verbatim}
|
||||
\end{quote}
|
||||
where \verb|<dim>| is replaced with a length. Do not set this length smaller than 5 cm.
|
||||
|
||||
\section{Document Body}
|
||||
|
||||
\subsection{Footnotes}
|
||||
|
||||
Footnotes are inserted with the \verb|\footnote| command.\footnote{This is a footnote.}
|
||||
|
||||
\subsection{Tables and figures}
|
||||
|
||||
See Table~\ref{tab:accents} for an example of a table and its caption.
|
||||
\textbf{Do not override the default caption sizes.}
|
||||
|
||||
\begin{table}
|
||||
\centering
|
||||
\begin{tabular}{lc}
|
||||
\hline
|
||||
\textbf{Command} & \textbf{Output} \\
|
||||
\hline
|
||||
\verb|{\"a}| & {\"a} \\
|
||||
\verb|{\^e}| & {\^e} \\
|
||||
\verb|{\`i}| & {\`i} \\
|
||||
\verb|{\.I}| & {\.I} \\
|
||||
\verb|{\o}| & {\o} \\
|
||||
\verb|{\'u}| & {\'u} \\
|
||||
\verb|{\aa}| & {\aa} \\\hline
|
||||
\end{tabular}
|
||||
\begin{tabular}{lc}
|
||||
\hline
|
||||
\textbf{Command} & \textbf{Output} \\
|
||||
\hline
|
||||
\verb|{\c c}| & {\c c} \\
|
||||
\verb|{\u g}| & {\u g} \\
|
||||
\verb|{\l}| & {\l} \\
|
||||
\verb|{\~n}| & {\~n} \\
|
||||
\verb|{\H o}| & {\H o} \\
|
||||
\verb|{\v r}| & {\v r} \\
|
||||
\verb|{\ss}| & {\ss} \\
|
||||
\hline
|
||||
\end{tabular}
|
||||
\caption{Example commands for accented characters, to be used in, \emph{e.g.}, Bib\TeX{} entries.}
|
||||
\label{tab:accents}
|
||||
\end{table}
|
||||
|
||||
As much as possible, fonts in figures should conform
|
||||
to the document fonts. See Figure~\ref{fig:experiments} for an example of a figure and its caption.
|
||||
|
||||
Using the \verb|graphicx| package graphics files can be included within figure
|
||||
environment at an appropriate point within the text.
|
||||
The \verb|graphicx| package supports various optional arguments to control the
|
||||
appearance of the figure.
|
||||
You must include it explicitly in the \LaTeX{} preamble (after the
|
||||
\verb|\documentclass| declaration and before \verb|\begin{document}|) using
|
||||
\verb|\usepackage{graphicx}|.
|
||||
|
||||
\begin{figure}[t]
|
||||
\includegraphics[width=\columnwidth]{example-image-golden}
|
||||
\caption{A figure with a caption that runs for more than one line.
|
||||
Example image is usually available through the \texttt{mwe} package
|
||||
without even mentioning it in the preamble.}
|
||||
\label{fig:experiments}
|
||||
\end{figure}
|
||||
|
||||
\begin{figure*}[t]
|
||||
\includegraphics[width=0.48\linewidth]{example-image-a} \hfill
|
||||
\includegraphics[width=0.48\linewidth]{example-image-b}
|
||||
\caption {A minimal working example to demonstrate how to place
|
||||
two images side-by-side.}
|
||||
\end{figure*}
|
||||
|
||||
\subsection{Hyperlinks}
|
||||
|
||||
Users of older versions of \LaTeX{} may encounter the following error during compilation:
|
||||
\begin{quote}
|
||||
\verb|\pdfendlink| ended up in different nesting level than \verb|\pdfstartlink|.
|
||||
\end{quote}
|
||||
This happens when pdf\LaTeX{} is used and a citation splits across a page boundary. The best way to fix this is to upgrade \LaTeX{} to 2018-12-01 or later.
|
||||
|
||||
\subsection{Citations}
|
||||
|
||||
\begin{table*}
|
||||
\centering
|
||||
\begin{tabular}{lll}
|
||||
\hline
|
||||
\textbf{Output} & \textbf{natbib command} & \textbf{ACL only command} \\
|
||||
\hline
|
||||
\citep{Gusfield:97} & \verb|\citep| & \\
|
||||
\citealp{Gusfield:97} & \verb|\citealp| & \\
|
||||
\citet{Gusfield:97} & \verb|\citet| & \\
|
||||
\citeyearpar{Gusfield:97} & \verb|\citeyearpar| & \\
|
||||
\citeposs{Gusfield:97} & & \verb|\citeposs| \\
|
||||
\hline
|
||||
\end{tabular}
|
||||
\caption{\label{citation-guide}
|
||||
Citation commands supported by the style file.
|
||||
The style is based on the natbib package and supports all natbib citation commands.
|
||||
It also supports commands defined in previous ACL style files for compatibility.
|
||||
}
|
||||
\end{table*}
|
||||
|
||||
Table~\ref{citation-guide} shows the syntax supported by the style files.
|
||||
We encourage you to use the natbib styles.
|
||||
You can use the command \verb|\citet| (cite in text) to get ``author (year)'' citations, like this citation to a paper by \citet{Gusfield:97}.
|
||||
You can use the command \verb|\citep| (cite in parentheses) to get ``(author, year)'' citations \citep{Gusfield:97}.
|
||||
You can use the command \verb|\citealp| (alternative cite without parentheses) to get ``author, year'' citations, which is useful for using citations within parentheses (e.g. \citealp{Gusfield:97}).
|
||||
|
||||
A possessive citation can be made with the command \verb|\citeposs|.
|
||||
This is not a standard natbib command, so it is generally not compatible
|
||||
with other style files.
|
||||
|
||||
\subsection{References}
|
||||
|
||||
\nocite{Ando2005,andrew2007scalable,rasooli-tetrault-2015}
|
||||
|
||||
The \LaTeX{} and Bib\TeX{} style files provided roughly follow the American Psychological Association format.
|
||||
If your own bib file is named \texttt{custom.bib}, then placing the following before any appendices in your \LaTeX{} file will generate the references section for you:
|
||||
\begin{quote}
|
||||
\begin{verbatim}
|
||||
\bibliography{custom}
|
||||
\end{verbatim}
|
||||
\end{quote}
|
||||
|
||||
You can obtain the complete ACL Anthology as a Bib\TeX{} file from \url{https://aclweb.org/anthology/anthology.bib.gz}.
|
||||
To include both the Anthology and your own .bib file, use the following instead of the above.
|
||||
\begin{quote}
|
||||
\begin{verbatim}
|
||||
\bibliography{anthology,custom}
|
||||
\end{verbatim}
|
||||
\end{quote}
|
||||
|
||||
Please see Section~\ref{sec:bibtex} for information on preparing Bib\TeX{} files.
|
||||
|
||||
\subsection{Equations}
|
||||
|
||||
An example equation is shown below:
|
||||
\begin{equation}
|
||||
\label{eq:example}
|
||||
A = \pi r^2
|
||||
\end{equation}
|
||||
|
||||
Labels for equation numbers, sections, subsections, figures and tables
|
||||
are all defined with the \verb|\label{label}| command and cross references
|
||||
to them are made with the \verb|\ref{label}| command.
|
||||
|
||||
This an example cross-reference to Equation~\ref{eq:example}.
|
||||
|
||||
\subsection{Appendices}
|
||||
|
||||
Use \verb|\appendix| before any appendix section to switch the section numbering over to letters. See Appendix~\ref{sec:appendix} for an example.
|
||||
|
||||
\section{Bib\TeX{} Files}
|
||||
\label{sec:bibtex}
|
||||
|
||||
Unicode cannot be used in Bib\TeX{} entries, and some ways of typing special characters can disrupt Bib\TeX's alphabetization. The recommended way of typing special characters is shown in Table~\ref{tab:accents}.
|
||||
|
||||
Please ensure that Bib\TeX{} records contain DOIs or URLs when possible, and for all the ACL materials that you reference.
|
||||
Use the \verb|doi| field for DOIs and the \verb|url| field for URLs.
|
||||
If a Bib\TeX{} entry has a URL or DOI field, the paper title in the references section will appear as a hyperlink to the paper, using the hyperref \LaTeX{} package.
|
||||
|
||||
\section*{Limitations}
|
||||
|
||||
This document does not cover the content requirements for ACL or any
|
||||
other specific venue. Check the author instructions for
|
||||
information on
|
||||
maximum page lengths, the required ``Limitations'' section,
|
||||
and so on.
|
||||
|
||||
\section*{Acknowledgments}
|
||||
|
||||
This document has been adapted
|
||||
by Steven Bethard, Ryan Cotterell and Rui Yan
|
||||
from the instructions for earlier ACL and NAACL proceedings, including those for
|
||||
ACL 2019 by Douwe Kiela and Ivan Vuli\'{c},
|
||||
NAACL 2019 by Stephanie Lukin and Alla Roskovskaya,
|
||||
ACL 2018 by Shay Cohen, Kevin Gimpel, and Wei Lu,
|
||||
NAACL 2018 by Margaret Mitchell and Stephanie Lukin,
|
||||
Bib\TeX{} suggestions for (NA)ACL 2017/2018 from Jason Eisner,
|
||||
ACL 2017 by Dan Gildea and Min-Yen Kan,
|
||||
NAACL 2017 by Margaret Mitchell,
|
||||
ACL 2012 by Maggie Li and Michael White,
|
||||
ACL 2010 by Jing-Shin Chang and Philipp Koehn,
|
||||
ACL 2008 by Johanna D. Moore, Simone Teufel, James Allan, and Sadaoki Furui,
|
||||
ACL 2005 by Hwee Tou Ng and Kemal Oflazer,
|
||||
ACL 2002 by Eugene Charniak and Dekang Lin,
|
||||
and earlier ACL and EACL formats written by several people, including
|
||||
John Chen, Henry S. Thompson and Donald Walker.
|
||||
Additional elements were taken from the formatting instructions of the \emph{International Joint Conference on Artificial Intelligence} and the \emph{Conference on Computer Vision and Pattern Recognition}.
|
||||
|
||||
% Bibliography entries for the entire Anthology, followed by custom entries
|
||||
%\bibliography{custom,anthology-overleaf-1,anthology-overleaf-2}
|
||||
|
||||
% Custom bibliography entries only
|
||||
\bibliography{custom}
|
||||
|
||||
\appendix
|
||||
|
||||
\section{Example Appendix}
|
||||
\label{sec:appendix}
|
||||
|
||||
This is an appendix.
|
||||
|
||||
\end{document}
|
||||
@@ -0,0 +1,101 @@
|
||||
% This file compiles with both LuaLaTeX and XeLaTeX
|
||||
\documentclass[11pt]{article}
|
||||
|
||||
% Change "review" to "final" to generate the final (sometimes called camera-ready) version.
|
||||
% Change to "preprint" to generate a non-anonymous version with page numbers.
|
||||
\usepackage[review]{acl}
|
||||
|
||||
% This is not strictly necessary, and may be commented out,
|
||||
% but it will improve the layout of the manuscript,
|
||||
% and will typically save some space.
|
||||
\usepackage{microtype}
|
||||
|
||||
% If the title and author information does not fit in the area allocated, uncomment the following
|
||||
%
|
||||
%\setlength\titlebox{<dim>}
|
||||
%
|
||||
% and set <dim> to something 5cm or larger.
|
||||
|
||||
% These font selection commands work with
|
||||
% LuaLaTeX and XeLaTeX, but not pdfLaTeX.
|
||||
\usepackage[english,bidi=default]{babel} % English as the main language.
|
||||
\babelfont{rm}{TeXGyreTermesX} % similar to Times
|
||||
%%% include whatever languages you need below this line
|
||||
\babelprovide[import]{hindi}
|
||||
\babelfont[*devanagari]{rm}{Lohit Devanagari}
|
||||
\babelprovide[import]{arabic}
|
||||
\babelfont[*arabic]{rm}{Noto Sans Arabic}
|
||||
|
||||
|
||||
%\usepackage{polyglossia}
|
||||
%\setdefaultlanguage{english}
|
||||
%\setotherlanguages{arabic,russian,thai,hindi,kannada}
|
||||
|
||||
%%%%%
|
||||
|
||||
|
||||
\title{LuaLaTeX and XeLaTeX Template for *ACL Style Files}
|
||||
|
||||
% Author information can be set in various styles:
|
||||
% For several authors from the same institution:
|
||||
% \author{Author 1 \and ... \and Author n \\
|
||||
% Address line \\ ... \\ Address line}
|
||||
% if the names do not fit well on one line use
|
||||
% Author 1 \\ {\bf Author 2} \\ ... \\ {\bf Author n} \\
|
||||
% For authors from different institutions:
|
||||
% \author{Author 1 \\ Address line \\ ... \\ Address line
|
||||
% \And ... \And
|
||||
% Author n \\ Address line \\ ... \\ Address line}
|
||||
% To start a separate ``row'' of authors use \AND, as in
|
||||
% \author{Author 1 \\ Address line \\ ... \\ Address line
|
||||
% \AND
|
||||
% Author 2 \\ Address line \\ ... \\ Address line \And
|
||||
% Author 3 \\ Address line \\ ... \\ Address line}
|
||||
|
||||
\author{First Author \\
|
||||
Affiliation / Address line 1 \\
|
||||
Affiliation / Address line 2 \\
|
||||
Affiliation / Address line 3 \\
|
||||
\texttt{email@domain} \\\And
|
||||
Second Author \\
|
||||
Affiliation / Address line 1 \\
|
||||
Affiliation / Address line 2 \\
|
||||
Affiliation / Address line 3 \\
|
||||
\texttt{email@domain} \\}
|
||||
|
||||
\begin{document}
|
||||
|
||||
\maketitle
|
||||
\begin{abstract}
|
||||
This document provides an example showing how
|
||||
to use the *ACL style files with either
|
||||
LuaLaTeX or XeLaTeX.
|
||||
\end{abstract}
|
||||
|
||||
|
||||
\section{Introduction}
|
||||
|
||||
Please see the general instructions
|
||||
in the file \verb|acl_latex.tex|.
|
||||
|
||||
Here are some examples of text in various languages.
|
||||
|
||||
Hindi: \foreignlanguage{hindi}{मानव अधिकारों की सार्वभौम घोषणा}
|
||||
|
||||
Arabic: \foreignlanguage{arabic}{الإعلان العالمي لحقوق الإنسان}
|
||||
|
||||
Here is an example citation:
|
||||
\citet{Gusfield:97} argues that...
|
||||
|
||||
|
||||
% Entries for the entire Anthology, followed by custom entries
|
||||
\bibliography{custom}
|
||||
|
||||
\appendix
|
||||
|
||||
\section{Example Appendix}
|
||||
\label{sec:appendix}
|
||||
|
||||
This is an appendix.
|
||||
|
||||
\end{document}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,26 @@
|
||||
For citing papers in the ACL Anthology, we provide a single consolidated
|
||||
BibTeX file containing all of its papers. The bibkeys in these papers are
|
||||
designed to be semantic in nature: {names}-{year}-{words}, where
|
||||
- `names` is the concatenated last names of the authors when there is just
|
||||
one or two authors, or `lastname-etal` for 3+
|
||||
- `year` is the four-digit year
|
||||
- `words` is the first significant word in the title, or more, if necessary,
|
||||
to preserve uniqueness
|
||||
|
||||
For example, https://aclanthology.org/N04-1035 can be cited as \cite{galley-etal-2004-whats}.
|
||||
|
||||
The consolidated file can be downloaded from here:
|
||||
- https://aclanthology.org/anthology.bib
|
||||
|
||||
Unfortunately, as of 2024 or so, this file is now larger than 50 MB, which is Overleaf's
|
||||
bib file size limit. Consequently, the Anthology shards the file automatically into
|
||||
49 MB shards.
|
||||
|
||||
There are currently (2025) two files:
|
||||
- https://aclanthology.org/anthology-1.bib
|
||||
- https://aclanthology.org/anthology-2.bib
|
||||
|
||||
You can download these directly from Overleaf from New File -> From External URL,
|
||||
and then adding them to the \bibliography line in acl_latex.tex:
|
||||
|
||||
\bibliography{custom,anthology-1,anthology-2}
|
||||
@@ -0,0 +1,70 @@
|
||||
% Use this file for citations not found in the ACL Anthology (contained in "anthology.bib").
|
||||
|
||||
@book{Aho:72,
|
||||
author = {Alfred V. Aho and Jeffrey D. Ullman},
|
||||
title = {The Theory of Parsing, Translation and Compiling},
|
||||
year = "1972",
|
||||
volume = "1",
|
||||
publisher = {Prentice-Hall},
|
||||
address = {Englewood Cliffs, NJ}
|
||||
}
|
||||
|
||||
@book{APA:83,
|
||||
author = {{American Psychological Association}},
|
||||
title = {Publications Manual},
|
||||
year = "1983",
|
||||
publisher = {American Psychological Association},
|
||||
address = {Washington, DC}
|
||||
}
|
||||
|
||||
@article{Chandra:81,
|
||||
author = {Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer},
|
||||
year = "1981",
|
||||
title = {Alternation},
|
||||
journal = {Journal of the Association for Computing Machinery},
|
||||
volume = "28",
|
||||
number = "1",
|
||||
pages = "114--133",
|
||||
doi = "10.1145/322234.322243",
|
||||
}
|
||||
|
||||
@inproceedings{andrew2007scalable,
|
||||
title={Scalable training of {L1}-regularized log-linear models},
|
||||
author={Andrew, Galen and Gao, Jianfeng},
|
||||
booktitle={Proceedings of the 24th International Conference on Machine Learning},
|
||||
pages={33--40},
|
||||
year={2007},
|
||||
}
|
||||
|
||||
@book{Gusfield:97,
|
||||
author = {Dan Gusfield},
|
||||
title = {Algorithms on Strings, Trees and Sequences},
|
||||
year = "1997",
|
||||
publisher = {Cambridge University Press},
|
||||
address = {Cambridge, UK}
|
||||
}
|
||||
|
||||
@article{rasooli-tetrault-2015,
|
||||
author = {Mohammad Sadegh Rasooli and Joel R. Tetreault},
|
||||
title = {Yara Parser: {A} Fast and Accurate Dependency Parser},
|
||||
journal = {Computing Research Repository},
|
||||
volume = {arXiv:1503.06733},
|
||||
year = {2015},
|
||||
url = {http://arxiv.org/abs/1503.06733},
|
||||
note = {version 2}
|
||||
}
|
||||
|
||||
@article{Ando2005,
|
||||
Acmid = {1194905},
|
||||
Author = {Ando, Rie Kubota and Zhang, Tong},
|
||||
Issn = {1532-4435},
|
||||
Issue_Date = {12/1/2005},
|
||||
Journal = {Journal of Machine Learning Research},
|
||||
Month = dec,
|
||||
Numpages = {37},
|
||||
Pages = {1817--1853},
|
||||
Publisher = {JMLR.org},
|
||||
Title = {A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data},
|
||||
Volume = {6},
|
||||
Year = {2005}
|
||||
}
|
||||
@@ -0,0 +1,326 @@
|
||||
# Instructions for *ACL Proceedings
|
||||
|
||||
The following instructions are for authors of papers submitted for review to ACL conferences (hereafter, "review version") or paper accepted for publication in its proceedings (hereafter, "final version").
|
||||
All authors are required to adhere to these specifications.
|
||||
|
||||
## Style Files
|
||||
|
||||
*ACL provides style files for LaTeX and Microsoft Word that meet these requirements. They can be found at:
|
||||
|
||||
> https://acl-org.github.io/ACLPUB/
|
||||
|
||||
We strongly recommend the use of these style files, which have been appropriately tailored for the *ACL proceedings.
|
||||
|
||||
## Paper Length
|
||||
|
||||
The conference accepts submissions of long papers and short papers.
|
||||
Review versions of long papers may have up to eight (8) pages of content plus unlimited pages for references.
|
||||
Upon acceptance, final versions of long papers will be given one additional page -- up to nine (9) pages of content plus unlimited pages for acknowledgements and references -- so that reviewers' comments can be taken into account.
|
||||
Review versions of short papers may have up to four (4) pages of content, plus unlimited pages for references.
|
||||
Final versions of short papers may have up to five (5) pages, plus unlimited pages for acknowledgements and references.
|
||||
For both long and short papers, all figures and tables that are part of the main text must fit within these page limits.
|
||||
|
||||
The conference encourages submission of appendices and supplementary material, which are not required to fit within these page limits. However, review versions of papers must be self-contained: it is optional for reviewers to look at appendices or supplementary material. Please see [Appendices](#Appendices) and [Supplementary](#Supplementary Material) for more information.
|
||||
|
||||
Review versions should not refer, for further detail, to documents, code or data resources that are not available to the reviewers.
|
||||
|
||||
Papers that do not conform to these requirements may be rejected without review.
|
||||
|
||||
Workshop chairs may have different rules for allowed length and whether appendices or supplementary materials are welcome.
|
||||
As always, the respective call for papers is the authoritative source.
|
||||
|
||||
## Anonymity
|
||||
|
||||
As reviewing will be double-blind, review versions must not include any identifying information about the authors (such as names, affiliations, or URLs).
|
||||
Self-references that reveal the author's identity, e.g.,
|
||||
|
||||
> We previously showed (Gusfield, 1997)...
|
||||
|
||||
must be avoided, and anonymous citations, e.g.,
|
||||
|
||||
> We previously showed (Anonymous, 1997)...
|
||||
|
||||
should also be avoided. Instead, use citations such as
|
||||
|
||||
> Gusfield (1997) previously showed...
|
||||
|
||||
Review versions must not include acknowledgements.
|
||||
|
||||
**Papers that do not conform to these requirements may be rejected without review.**
|
||||
|
||||
Any preliminary non-archival versions of submitted papers should be listed in the submission form but not in the review version of the paper.
|
||||
Reviewers are generally aware that authors may present preliminary versions of their work in other venues, but will not be provided the list of previous presentations from the submission form.
|
||||
|
||||
Once a paper has been accepted to the conference, the final version should include the author's names and affiliations, and is allowed to use self-references.
|
||||
|
||||
## Multiple Submission
|
||||
|
||||
Papers that have been or will be submitted to other meetings or publications must indicate this at submission time in the START submission form, and must be withdrawn from the other venues if accepted by *ACL.
|
||||
Authors of papers accepted for presentation at *ACL must notify the program chairs by the deadline for final versions ("camera-ready deadline") whether the paper will be presented.
|
||||
We will not accept for publication or presentation any papers that overlap significantly in content or results with papers that will be (or have been) published elsewhere.
|
||||
|
||||
Authors submitting more than one paper to *ACL must ensure that submissions do not overlap significantly (>25%) with each other in content or results.
|
||||
|
||||
## Formatting Instructions
|
||||
|
||||
### File Format
|
||||
|
||||
Papers must be in Adobe Portable Document Format (PDF).
|
||||
Please make sure that your PDF file embeds all necessary fonts (especially for tree diagrams, symbols, and Asian languages).
|
||||
When you print or create the PDF file, there is usually an option in your printer setup to include none, all or just non-standard fonts.
|
||||
Please make sure that you select the option of including *all* the fonts.
|
||||
**Before sending it, test your PDF by printing it from a computer different from the one where it was created.**
|
||||
|
||||
Some word processors may generate very large PDF files, where each page is rendered as an image.
|
||||
Such images may reproduce poorly.
|
||||
In this case, try alternative ways to obtain the PDF.
|
||||
|
||||
All papers must use **A4 paper format** (21 cm x 29.7 cm).
|
||||
Papers must not be submitted with any other paper size.
|
||||
|
||||
If you cannot meet the above requirements, please contact the publication chairs as soon as possible.
|
||||
|
||||
### Layout
|
||||
|
||||
All text except for page numbers must fit within the margins.
|
||||
|
||||
Review versions should have page numbers, centered in the bottom margin, but **pages should not be numbered in the final version.**
|
||||
|
||||
Manuscripts must be set in two columns.
|
||||
Exceptions to the two-column format include the title, authors' names and complete addresses, which must be centered at the top of the first page, and any full-width figures or tables.
|
||||
|
||||
The exact dimensions for a page on A4 paper are:
|
||||
|
||||
* Left margin: 2.5 cm
|
||||
* Right margin: 2.5 cm
|
||||
* Top margin: 2.5 cm
|
||||
* Bottom margin: 2.5 cm
|
||||
* Column width: 7.7 cm
|
||||
* Column height: 24.7 cm
|
||||
* Gap between columns: 0.6 cm
|
||||
|
||||
In the review version, a ruler (line numbers in the left and right margins of the article) should be printed, so that reviewers may comment on particular lines in the paper.
|
||||
The ruler should not change the appearance of any other content on the page.
|
||||
The final version should not contain a ruler.
|
||||
|
||||
### Fonts
|
||||
|
||||
All text (except non-Latin scripts and mathematical formulas) should be set in **Times Roman**.
|
||||
If Times Roman is unavailable, you may use **Times New Roman** or **Computer Modern Roman.**
|
||||
|
||||
The following table specifies what font sizes and styles must be used for each type of text in the manuscript.
|
||||
|
||||
| Type of Text | Font Size | Style |
|
||||
| --------------------- | --------- | ----- |
|
||||
| paper title | 15 pt | bold |
|
||||
| author names | 12 pt | bold |
|
||||
| author affiliation | 12 pt | |
|
||||
| the word ``Abstract'' | 12 pt | bold |
|
||||
| section titles | 12 pt | bold |
|
||||
| subsection titles | 11 pt | bold |
|
||||
| document text | 11 pt | |
|
||||
| captions | 10 pt | |
|
||||
| abstract text | 10 pt | |
|
||||
| bibliography | 10 pt | |
|
||||
| footnotes | 9 pt | |
|
||||
|
||||
### Title and Authors
|
||||
|
||||
Center the title, author's name(s) and affiliation(s) across both columns.
|
||||
|
||||
Place the title centered at the top of the first page, in 15-point bold.
|
||||
Long titles should be typed on two lines without a blank line intervening.
|
||||
Put the title 2.5 cm from the top of the page.
|
||||
Write the title in [title case](https://apastyle.apa.org/style-grammar-guidelines/capitalization/title-case); do not write the title in all capital letters, except for acronyms (e.g., "BLEU") or proper nouns ("English") that are normally uppercased or capitalized.
|
||||
|
||||
Place the author name(s) and affiliation(s) under the title.
|
||||
Write authors' full names; do not abbreviate given names to initials, unless they are normally written as initials ("Margaret Mitchell", not "M. Mitchell").
|
||||
Do not format surnames in all capitals ("Mitchell", not "MITCHELL").
|
||||
|
||||
Do not use footnotes for affiliations.
|
||||
The affiliation should contain the author's complete address, and if possible, an electronic mail address.
|
||||
|
||||
The title, author names and addresses should be completely identical to those entered to the paper submission website in order to maintain the consistency of author information among all publications of the conference.
|
||||
If they are different, the publication chairs may resolve the difference without consulting with you; so it is in your own interest to double-check that the information is consistent.
|
||||
|
||||
Start the body of the first page 7.5 cm from the top of the page.
|
||||
**Even in the review version of the paper, you should maintain space for names and addresses so that they will fit in the final version.**
|
||||
|
||||
### Abstract
|
||||
|
||||
Type the abstract at the beginning of the first column.
|
||||
Center the word **Abstract** in 12 point bold above the body of the abstract.
|
||||
The width of the abstract should be smaller than the
|
||||
normal column width by 0.6 cm on each side.
|
||||
The abstract text should be 10 point roman, single-spaced.
|
||||
|
||||
The abstract should be a concise summary of the general thesis and conclusions of the paper.
|
||||
It should be no longer than 200 words.
|
||||
|
||||
### Text
|
||||
|
||||
Begin typing the main body of the text immediately after the abstract, continuing in two columns.
|
||||
The text should be 11 point roman, single-spaced.
|
||||
|
||||
Indent 0.4 cm when starting a new paragraph, except for the first paragraph in a section.
|
||||
|
||||
### Sections
|
||||
|
||||
Use numbered sections (Arabic numerals) to facilitate cross references.
|
||||
Number subsections with the section number and the subsection number separated by a dot, in Arabic numerals, e.g.,
|
||||
|
||||
> 1 Introduction
|
||||
|
||||
or
|
||||
|
||||
> 6.1 File Format
|
||||
|
||||
### Footnotes
|
||||
Put footnotes at the bottom of the page and use 9 point font.
|
||||
They may be numbered or referred to by asterisks or other symbols.
|
||||
Footnotes should be separated from the text by a line.
|
||||
|
||||
### Figures and tables
|
||||
|
||||
Place figures and tables in the paper near where they are first discussed, rather than at the end, if possible.
|
||||
Wide figures/tables may run across both columns.
|
||||
|
||||
To accommodate people who are color-blind (as well as those printing with black-and-white printers), grayscale readability is strongly encouraged.
|
||||
Color is not forbidden, but authors should ensure that tables and figures do not rely solely on color to convey critical distinctions.
|
||||
|
||||
**Captions:**
|
||||
Provide a caption for every figure/table; number each one sequentially in the form:
|
||||
|
||||
> Figure 1: Caption of the Figure.
|
||||
|
||||
and
|
||||
|
||||
> Table 1: Caption of the Table.
|
||||
|
||||
Captions should be placed below figures/tables, in 10 point roman type.
|
||||
Captions that are one line are centered.
|
||||
Captions longer than one line are left-aligned.
|
||||
|
||||
### Hyperlinks
|
||||
|
||||
Within-document and external hyperlinks should be dark blue (hex #000099), not underlined or boxed.
|
||||
|
||||
### Non-English Text
|
||||
|
||||
Text in languages other than English should be accompanied by translations into English, and text in scripts other than Latin should \emph{also} be accompanied by transliterations into Latin script, since not all readers can recognize non-Latin characters easily.
|
||||
|
||||
For example, παράδειγμα *paradeigma* ‘example’ is a Greek word, and this is a Greek sentence:
|
||||
|
||||
> Αυτό είναι ένα παράδειγμα.
|
||||
> auto einai ena paradeigma.
|
||||
> ‘This is an example.’
|
||||
|
||||
### Citations
|
||||
|
||||
Citations within the text appear in parentheses (Gusfield, 1997), or, if the author's name appears in the text itself: Gusfield (1997).
|
||||
Append lowercase letters to the year in cases of ambiguities.
|
||||
Cite papers with two authors using both authors' names (Aho and Ullman, 1972), but cite papers with more than two authors by the first author's name and ``et al.'' (Chandra et al., 1981).
|
||||
Collapse multiple citations into a single pair of parentheses (Gusfield, 1997; Aho and Ullman, 1972).
|
||||
|
||||
Refrain from using full citations as sentence constituents.
|
||||
Instead of
|
||||
|
||||
> (Gusfield, 1997) showed that ...
|
||||
> In (Gusfield, 1997), ...''
|
||||
|
||||
write
|
||||
|
||||
> Gusfield (1997) showed that ...
|
||||
> In Gusfield (1997), ...
|
||||
|
||||
Submissions should accurately reference prior and related work, including code and data.
|
||||
If a piece of prior work appeared in multiple venues, the version that appeared in a refereed, archival venue should be referenced.
|
||||
If multiple versions of a piece of prior work exist, the one used by the authors should be referenced.
|
||||
|
||||
### Acknowledgments
|
||||
|
||||
The acknowledgments should go immediately before the references.
|
||||
Do not number the acknowledgments section.
|
||||
Do not include this section in the review version.
|
||||
|
||||
### References
|
||||
|
||||
Gather the full set of references together under the unnumbered section heading **References**.
|
||||
Place the References section before any Appendices.
|
||||
Arrange the references alphabetically by first author, rather than by order of occurrence in the text.
|
||||
|
||||
Provide as complete a citation as possible, using a consistent format, such as the [one for Computational Linguistics](http://cljournal.org/style_guide_refs.html) or the one in the [Publication Manual of the American Psychological Association](https://apastyle.apa.org/products/publication-manual-7th-edition).
|
||||
Use full names for authors, not just initials.
|
||||
Authors should not rely on automated citation indices to provide accurate references for prior and related work.
|
||||
|
||||
As part of our work to make ACL materials more widely used and cited outside of our discipline, ACL has registered as a CrossRef member, as a registrant of Digital Object Identifiers (DOIs), the standard for registering permanent URNs for referencing scholarly materials.
|
||||
|
||||
All references are required to contain DOIs of all cited works when possible, or, as a second resort, links to ACL Anthology pages.
|
||||
Appropriate records should be found for most materials in the current [ACL Anthology](https://aclweb.org/anthology/).
|
||||
|
||||
Example article in a journal:
|
||||
|
||||
> Rie Kubota Ando and Tong Zhang. 2005. [A framework for learning predictive structures from multiple tasks and unlabeled data](https://www.jmlr.org/papers/v6/ando05a.html). *Journal of Machine Learning Research*, 6:1817–1853.
|
||||
|
||||
Example paper in non-ACL proceedings, with DOI:
|
||||
|
||||
> Galen Andrew and Jianfeng Gao. 2007. [Scalable training of L1-regularized log-linear models](https://doi.org/10.1145/1273496.1273501). In *Proceedings of the 24th International Conference on Machine Learning*, pages 33–40.
|
||||
|
||||
Example ACL Anthology paper with DOI:
|
||||
|
||||
> James Goodman, Andreas Vlachos, and Jason Naradowsky. 2016. [Noise reduction and targeted exploration in imitation learning for Abstract Meaning Representation parsing](http://dx.doi.org/10.18653/v1/P16-1001). In *Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 1–45711, Berlin, Germany. Association for Computational Linguistics.
|
||||
|
||||
Example ACL Anthology paper without DOI:
|
||||
|
||||
> Benjamin Börschinger and Mark Johnson. 2011. [A particle filter algorithm for Bayesian word segmentation](https://www.aclweb.org/anthology/U11-1004/). In *Proceedings of the Australasian Language Technology Association Workshop 2011*, pages 10–44718, Canberra, Australia.
|
||||
|
||||
Example arXiv paper:
|
||||
|
||||
> Mohammad Sadegh Rasooli and Joel R. Tetreault. 2015. [Yara parser: A fast and accurate dependency parser](http://arxiv.org/abs/1503.06733). *Computing Research Repository*, arXiv:1503.06733. Version 2.
|
||||
|
||||
## Appendices
|
||||
|
||||
Appendices are material that can be read, and include lemmas, formulas, proofs, and tables that are not critical to the reading and understanding of the paper.
|
||||
Letter them in sequence and provide an informative title:
|
||||
|
||||
> Appendix A. Title of Appendix
|
||||
|
||||
The appendices come after the references.
|
||||
|
||||
Review versions of appendices must follow the same anonymity guidelines as the main paper.
|
||||
|
||||
## Supplementary Material
|
||||
|
||||
Submissions may include non-readable supplementary material used in the work and described in the paper.
|
||||
Any accompanying software and/or data should include licenses and documentation of research review as appropriate.
|
||||
Supplementary material may report preprocessing decisions, model parameters, and other details necessary for the replication of the experiments reported in the paper.
|
||||
Seemingly small preprocessing decisions can sometimes make a large difference in performance, so it is crucial to record such decisions to precisely characterize state-of-the-art methods.
|
||||
|
||||
Nonetheless, supplementary material should be supplementary (rather than central) to the paper.
|
||||
**Submissions that misuse the supplementary material may be rejected without review.**
|
||||
Supplementary material may include explanations or details of proofs or derivations that do not fit into the paper, lists of features or feature templates, sample inputs and outputs for a system, pseudo-code or source code, and data.
|
||||
(Source code and data should be separate uploads, rather than part of the paper).
|
||||
|
||||
The paper should not rely on the supplementary material: while the paper may refer to and cite the supplementary material and the supplementary material will be available to the reviewers, they will not be asked to review the supplementary material.
|
||||
|
||||
Review versions of supplementary material must follow the same anonymity guidelines as the main paper.
|
||||
|
||||
## Credits
|
||||
|
||||
This document has been adapted from the instructions for earlier ACL and NAACL proceedings, including those for
|
||||
ACL 2020 by Steven Bethard, Ryan Cotterell and Rui Yan,
|
||||
ACL 2019 by Douwe Kiela and Ivan Ivan Vulić,
|
||||
NAACL 2019 by Stephanie Lukin and Alla Roskovskaya,
|
||||
ACL 2018 by Shay Cohen, Kevin Gimpel, and Wei Lu,
|
||||
NAACL 2018 by Margaret Mitchell and Stephanie Lukin,
|
||||
BibTeX suggestions for (NA)ACL 2017/2018 from Jason Eisner,
|
||||
ACL 2017 by Dan Gildea and Min-Yen Kan,
|
||||
NAACL 2017 by Margaret Mitchell,
|
||||
ACL 2012 by Maggie Li and Michael White,
|
||||
ACL 2010 by Jing-Shin Chang and Philipp Koehn,
|
||||
ACL 2008 by Johanna D. Moore, Simone Teufel, James Allan, and Sadaoki Furui,
|
||||
ACL 2005 by Hwee Tou Ng and Kemal Oflazer,
|
||||
ACL 2002 by Eugene Charniak and Dekang Lin,
|
||||
and earlier ACL and EACL formats written by several people, including
|
||||
John Chen, Henry S. Thompson and Donald Walker.
|
||||
Additional elements were taken from the formatting instructions of the *International Joint Conference on Artificial Intelligence* and the *Conference on Computer Vision and Pattern Recognition*.
|
||||
@@ -0,0 +1,3 @@
|
||||
# Template
|
||||
|
||||
Template and style files for CoLM 2025
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
@inproceedings{Vaswani+2017,
|
||||
author = {Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, \L ukasz and Polosukhin, Illia},
|
||||
booktitle = {Advances in Neural Information Processing Systems},
|
||||
pages = {},
|
||||
publisher = {Curran Associates, Inc.},
|
||||
title = {Attention is All you Need},
|
||||
url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf},
|
||||
volume = {30},
|
||||
year = {2017}
|
||||
}
|
||||
|
||||
+1440
File diff suppressed because it is too large
Load Diff
BIN
Binary file not shown.
+218
@@ -0,0 +1,218 @@
|
||||
%%%% COLM Macros (LaTex)
|
||||
%%%% Adapted by Yoav Artzi and Sasha Rush from Hugo Larochelle's adaptation for ICLR, which has been adaptated from the NIPS stylefile Macros
|
||||
%%%% Style File
|
||||
%%%% Dec 12, 1990 Rev Aug 14, 1991; Sept, 1995; April, 1997; April, 1999; October 2014
|
||||
|
||||
% This file can be used with Latex2e whether running in main mode, or
|
||||
% 2.09 compatibility mode.
|
||||
%
|
||||
% If using main mode, you need to include the commands
|
||||
% \documentclass{article}
|
||||
% \usepackage{colm14submit_e}
|
||||
%
|
||||
|
||||
% Define options
|
||||
\newif\ifcolmsubmission
|
||||
\newif\ifcolmpreprint
|
||||
\newif\ifcolmfinal
|
||||
|
||||
% Set submission as default
|
||||
\colmsubmissiontrue
|
||||
\colmpreprintfalse
|
||||
\colmfinalfalse
|
||||
|
||||
% Define option handling
|
||||
\DeclareOption{submission}{\colmsubmissiontrue\colmpreprintfalse\colmfinalfalse}
|
||||
\DeclareOption{preprint}{\colmsubmissionfalse\colmpreprinttrue\colmfinalfalse}
|
||||
\DeclareOption{final}{\colmsubmissionfalse\colmpreprintfalse\colmfinaltrue}
|
||||
\ProcessOptions\relax
|
||||
|
||||
|
||||
% Palatino font
|
||||
\RequirePackage{tgpagella} % text only
|
||||
\RequirePackage{mathpazo} % math & text
|
||||
\RequirePackage{inconsolata} % for tt font
|
||||
|
||||
% Change the overall width of the page. If these parameters are
|
||||
% changed, they will require corresponding changes in the
|
||||
% maketitle section.
|
||||
%
|
||||
\usepackage{eso-pic} % used by \AddToShipoutPicture
|
||||
\RequirePackage{fancyhdr}
|
||||
\RequirePackage{natbib}
|
||||
|
||||
% modification to natbib citations
|
||||
\setcitestyle{authoryear,round,citesep={;},aysep={,},yysep={;}}
|
||||
|
||||
\renewcommand{\topfraction}{0.95} % let figure take up nearly whole page
|
||||
\renewcommand{\textfraction}{0.05} % let figure take up nearly whole page
|
||||
|
||||
|
||||
% Specify the dimensions of each page
|
||||
|
||||
\setlength{\paperheight}{11in}
|
||||
\setlength{\paperwidth}{8.5in}
|
||||
|
||||
|
||||
\oddsidemargin .5in % Note \oddsidemargin = \evensidemargin
|
||||
\evensidemargin .5in
|
||||
\marginparwidth 0.07 true in
|
||||
%\marginparwidth 0.75 true in
|
||||
%\topmargin 0 true pt % Nominal distance from top of page to top of
|
||||
%\topmargin 0.125in
|
||||
\topmargin -0.625in
|
||||
\addtolength{\headsep}{0.25in}
|
||||
\textheight 9.0 true in % Height of text (including footnotes & figures)
|
||||
\textwidth 5.5 true in % Width of text line.
|
||||
\widowpenalty=10000
|
||||
\clubpenalty=10000
|
||||
|
||||
% \thispagestyle{empty} \pagestyle{empty}
|
||||
\flushbottom \sloppy
|
||||
|
||||
% We're never going to need a table of contents, so just flush it to
|
||||
% save space --- suggested by drstrip@sandia-2
|
||||
\def\addcontentsline#1#2#3{}
|
||||
|
||||
% Title stuff, taken from deproc.
|
||||
\def\maketitle{\par
|
||||
\begingroup
|
||||
\def\thefootnote{\fnsymbol{footnote}}
|
||||
\def\@makefnmark{\hbox to 0pt{$^{\@thefnmark}$\hss}} % for perfect author
|
||||
% name centering
|
||||
% The footnote-mark was overlapping the footnote-text,
|
||||
% added the following to fix this problem (MK)
|
||||
\long\def\@makefntext##1{\parindent 1em\noindent
|
||||
\hbox to1.8em{\hss $\m@th ^{\@thefnmark}$}##1}
|
||||
\@maketitle \@thanks
|
||||
\endgroup
|
||||
\setcounter{footnote}{0}
|
||||
\let\maketitle\relax \let\@maketitle\relax
|
||||
\gdef\@thanks{}\gdef\@author{}\gdef\@title{}\let\thanks\relax}
|
||||
|
||||
% The toptitlebar has been raised to top-justify the first page
|
||||
|
||||
\usepackage{fancyhdr}
|
||||
\pagestyle{fancy}
|
||||
\renewcommand{\headrulewidth}{1.5pt}
|
||||
\fancyhead{}
|
||||
|
||||
% Title (includes both anonymized and non-anonymized versions)
|
||||
\def\@maketitle{\vbox{\hsize\textwidth
|
||||
%\linewidth\hsize \vskip 0.1in \toptitlebar \centering
|
||||
{\Large\bf \@title\par}
|
||||
%\bottomtitlebar % \vskip 0.1in % minus
|
||||
\ifcolmfinal
|
||||
\lhead{Published as a conference paper at COLM 2025}
|
||||
\def\And{\end{tabular}\hfil\linebreak[0]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\def\AND{\end{tabular}\hfil\linebreak[4]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\@author\end{tabular}%
|
||||
\else\ifcolmpreprint
|
||||
\lhead{Preprint. Under review.}
|
||||
\def\And{\end{tabular}\hfil\linebreak[0]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\def\AND{\end{tabular}\hfil\linebreak[4]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\@author\end{tabular}%
|
||||
\else
|
||||
\lhead{Under review as a conference paper at COLM 2025}
|
||||
\def\And{\end{tabular}\hfil\linebreak[0]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\def\AND{\end{tabular}\hfil\linebreak[4]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}Anonymous authors\\Paper under double-blind review\end{tabular}%
|
||||
\fi\fi
|
||||
\vskip 0.3in minus 0.1in}}
|
||||
|
||||
\renewenvironment{abstract}{\vskip.075in\centerline{\large\bf
|
||||
Abstract}\vspace{0.5ex}\begin{quote}}{\par\end{quote}\vskip 1ex}
|
||||
|
||||
% Less leading in most fonts (due to the narrow columns)
|
||||
% The choices were between 1-pt and 1.5-pt leading
|
||||
%\def\@normalsize{\@setsize\normalsize{11pt}\xpt\@xpt} % got rid of @ (MK)
|
||||
\def\normalsize{\@setsize\normalsize{11pt}\xpt\@xpt}
|
||||
\def\small{\@setsize\small{10pt}\ixpt\@ixpt}
|
||||
\def\footnotesize{\@setsize\footnotesize{10pt}\ixpt\@ixpt}
|
||||
\def\scriptsize{\@setsize\scriptsize{8pt}\viipt\@viipt}
|
||||
\def\tiny{\@setsize\tiny{7pt}\vipt\@vipt}
|
||||
\def\large{\@setsize\large{14pt}\xiipt\@xiipt}
|
||||
\def\Large{\@setsize\Large{16pt}\xivpt\@xivpt}
|
||||
\def\LARGE{\@setsize\LARGE{20pt}\xviipt\@xviipt}
|
||||
\def\huge{\@setsize\huge{23pt}\xxpt\@xxpt}
|
||||
\def\Huge{\@setsize\Huge{28pt}\xxvpt\@xxvpt}
|
||||
|
||||
|
||||
|
||||
% sections with less space
|
||||
\def\section{\@startsection {section}{1}{\z@}{-2.0ex plus
|
||||
-0.5ex minus -.2ex}{1.5ex plus 0.3ex
|
||||
minus0.2ex}{\large\bf\raggedright}}
|
||||
|
||||
\def\subsection{\@startsection{subsection}{2}{\z@}{-1.8ex plus
|
||||
-0.5ex minus -.2ex}{0.8ex plus .2ex}{\normalsize\bf\raggedright}}
|
||||
\def\subsubsection{\@startsection{subsubsection}{3}{\z@}{-1.5ex
|
||||
plus -0.5ex minus -.2ex}{0.5ex plus
|
||||
.2ex}{\normalsize\bf\itshape\raggedright}}
|
||||
\def\paragraph{\@startsection{paragraph}{4}{\z@}{1.5ex plus
|
||||
0.5ex minus .2ex}{-1em}{\normalsize\bf}}
|
||||
\def\subparagraph{\@startsection{subparagraph}{5}{\z@}{1.5ex plus
|
||||
0.5ex minus .2ex}{-1em}{\normalsize\it}}
|
||||
\def\subsubsubsection{\vskip
|
||||
5pt{\noindent\normalsize\raggedright}}
|
||||
|
||||
|
||||
% Footnotes
|
||||
\footnotesep 6.65pt %
|
||||
\skip\footins 9pt plus 4pt minus 2pt
|
||||
\def\footnoterule{\kern-3pt \hrule width 12pc \kern 2.6pt }
|
||||
\setcounter{footnote}{0}
|
||||
|
||||
% Lists and paragraphs
|
||||
\parindent 0pt
|
||||
\topsep 4pt plus 1pt minus 2pt
|
||||
\partopsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep 2pt plus 1pt minus 0.5pt
|
||||
\parsep 2pt plus 1pt minus 0.5pt
|
||||
\parskip .5pc
|
||||
|
||||
|
||||
%\leftmargin2em
|
||||
\leftmargin3pc
|
||||
\leftmargini\leftmargin \leftmarginii 2em
|
||||
\leftmarginiii 1.5em \leftmarginiv 1.0em \leftmarginv .5em
|
||||
|
||||
%\labelsep \labelsep 5pt
|
||||
|
||||
\def\@listi{\leftmargin\leftmargini}
|
||||
\def\@listii{\leftmargin\leftmarginii
|
||||
\labelwidth\leftmarginii\advance\labelwidth-\labelsep
|
||||
\topsep 2pt plus 1pt minus 0.5pt
|
||||
\parsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep \parsep}
|
||||
\def\@listiii{\leftmargin\leftmarginiii
|
||||
\labelwidth\leftmarginiii\advance\labelwidth-\labelsep
|
||||
\topsep 1pt plus 0.5pt minus 0.5pt
|
||||
\parsep \z@ \partopsep 0.5pt plus 0pt minus 0.5pt
|
||||
\itemsep \topsep}
|
||||
\def\@listiv{\leftmargin\leftmarginiv
|
||||
\labelwidth\leftmarginiv\advance\labelwidth-\labelsep}
|
||||
\def\@listv{\leftmargin\leftmarginv
|
||||
\labelwidth\leftmarginv\advance\labelwidth-\labelsep}
|
||||
\def\@listvi{\leftmargin\leftmarginvi
|
||||
\labelwidth\leftmarginvi\advance\labelwidth-\labelsep}
|
||||
|
||||
\abovedisplayskip 7pt plus2pt minus5pt%
|
||||
\belowdisplayskip \abovedisplayskip
|
||||
\abovedisplayshortskip 0pt plus3pt%
|
||||
\belowdisplayshortskip 4pt plus3pt minus3pt%
|
||||
|
||||
|
||||
\def\toptitlebar{\hrule height4pt\vskip .25in\vskip-\parskip}
|
||||
|
||||
\def\bottomtitlebar{\vskip .29in\vskip-\parskip\hrule height1pt\vskip
|
||||
.09in} %
|
||||
%Reduced second vskip to compensate for adding the strut in \@author
|
||||
|
||||
|
||||
+305
@@ -0,0 +1,305 @@
|
||||
|
||||
\documentclass{article} % For LaTeX2e
|
||||
\usepackage[submission]{colm2025_conference}
|
||||
|
||||
\usepackage{microtype}
|
||||
\usepackage{hyperref}
|
||||
\usepackage{url}
|
||||
\usepackage{booktabs}
|
||||
|
||||
\usepackage{lineno}
|
||||
|
||||
\definecolor{darkblue}{rgb}{0, 0, 0.5}
|
||||
\hypersetup{colorlinks=true, citecolor=darkblue, linkcolor=darkblue, urlcolor=darkblue}
|
||||
|
||||
|
||||
\title{Formatting Instructions for COLM 2025 \\ Conference Submissions}
|
||||
|
||||
% Authors must not appear in the submitted version. They should be hidden
|
||||
% as long as the \colmfinalcopy macro remains commented out below.
|
||||
% Non-anonymous submissions will be rejected without review.
|
||||
|
||||
\author{Antiquus S.~Hippocampus, Natalia Cerebro \& Amelie P. Amygdale \thanks{ Use footnote for providing further information
|
||||
about author (webpage, alternative address)---\emph{not} for acknowledging
|
||||
funding agencies. Funding acknowledgements go at the end of the paper.} \\
|
||||
Department of Computer Science\\
|
||||
Cranberry-Lemon University\\
|
||||
Pittsburgh, PA 15213, USA \\
|
||||
\texttt{\{hippo,brain,jen\}@cs.cranberry-lemon.edu} \\
|
||||
\And
|
||||
Ji Q. Ren \& Yevgeny LeNet \\
|
||||
Department of Computational Neuroscience \\
|
||||
University of the Witwatersrand \\
|
||||
Joburg, South Africa \\
|
||||
\texttt{\{robot,net\}@wits.ac.za} \\
|
||||
\AND
|
||||
Coauthor \\
|
||||
Affiliation \\
|
||||
Address \\
|
||||
\texttt{email}
|
||||
}
|
||||
|
||||
% The \author macro works with any number of authors. There are two commands
|
||||
% used to separate the names and addresses of multiple authors: \And and \AND.
|
||||
%
|
||||
% Using \And between authors leaves it to \LaTeX{} to determine where to break
|
||||
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
|
||||
% puts 3 of 4 authors names on the first line, and the last on the second
|
||||
% line, try using \AND instead of \And before the third author name.
|
||||
|
||||
\newcommand{\fix}{\marginpar{FIX}}
|
||||
\newcommand{\new}{\marginpar{NEW}}
|
||||
|
||||
\begin{document}
|
||||
|
||||
\ifcolmsubmission
|
||||
\linenumbers
|
||||
\fi
|
||||
|
||||
\maketitle
|
||||
|
||||
\begin{abstract}
|
||||
The abstract paragraph should be indented 1/2~inch (3~picas) on both left and
|
||||
right-hand margins. Use 10~point type, with a vertical spacing of 11~points.
|
||||
The word \textit{Abstract} must be centered and in point size 12. Two
|
||||
line spaces precede the abstract. The abstract must be limited to one
|
||||
paragraph.
|
||||
\end{abstract}
|
||||
|
||||
\section{Submission of conference papers to COLM 2025}
|
||||
|
||||
COLM requires electronic submissions, processed by
|
||||
\url{https://openreview.net/}. See COLM's website for more instructions.
|
||||
The format for the submissions is a variant of the NeurIPS and ICLR formats.
|
||||
Please read carefully the instructions below, and follow them
|
||||
faithfully.
|
||||
|
||||
|
||||
\subsection{Style}
|
||||
|
||||
Papers to be submitted to COLM 2025 must be prepared according to the
|
||||
instructions presented here.
|
||||
|
||||
%% Please note that we have introduced automatic line number generation
|
||||
%% into the style file for \LaTeXe. This is to help reviewers
|
||||
%% refer to specific lines of the paper when they make their comments. Please do
|
||||
%% NOT refer to these line numbers in your paper as they will be removed from the
|
||||
%% style file for the final version of accepted papers.
|
||||
|
||||
Authors are required to use the COLM \LaTeX{} style files obtainable at the
|
||||
COLM website. Please make sure you use the current files and
|
||||
not previous versions. Tweaking the style files may be grounds for rejection.
|
||||
|
||||
\subsubsection{Copy Options}
|
||||
|
||||
If your paper is ultimately accepted, the option {\tt
|
||||
{\textbackslash}final} should be set for the {\tt {\textbackslash}usepackage[submission]\{colm2025\_conference\}} command for the camera ready version. The {\tt submission} options is the default, and is to be used for all submissions during the review process. It also turns on the line numbers. If you wish to submit a preprint, the option {\tt preprint} should be used.
|
||||
|
||||
|
||||
|
||||
\subsection{Retrieval of style files}
|
||||
|
||||
The style files for COLM and other conference information are available online at:
|
||||
\begin{center}
|
||||
\url{http://www.colmweb.org/}
|
||||
\end{center}
|
||||
The file \verb+colm2025_conference.pdf+ contains these
|
||||
instructions and illustrates the
|
||||
various formatting requirements your COLM paper must satisfy.
|
||||
Submissions must be made using \LaTeX{} and the style files
|
||||
\verb+colm2025_conference.sty+ and \verb+colm2025_conference.bst+ (to be used with \LaTeX{}2e). The file
|
||||
\verb+colm2025_conference.tex+ may be used as a ``shell'' for writing your paper. All you
|
||||
have to do is replace the author, title, abstract, and text of the paper with
|
||||
your own.
|
||||
|
||||
The formatting instructions contained in these style files are summarized in
|
||||
sections \ref{gen_inst}, \ref{headings}, and \ref{others} below.
|
||||
|
||||
\section{General formatting instructions}
|
||||
\label{gen_inst}
|
||||
|
||||
The text must be confined within a rectangle 5.5~inches (33~picas) wide and
|
||||
9~inches (54~picas) long. The left margin is 1.5~inch (9~picas).
|
||||
Use 10~point type with a vertical spacing of 11~points. Palatino is the
|
||||
preferred typeface throughout, and is mandatory for the main text. Paragraphs are separated by 1/2~line space, with no indentation.
|
||||
|
||||
Paper title is 17~point and left-aligned.
|
||||
All pages should start at 1~inch (6~picas) from the top of the page.
|
||||
|
||||
Please verify that any custom header information you may add does not override the style defined in this document. This has been known to occur especially when submissions are converted to a new template from a previous one (i.e., for re-submission to a different venue).
|
||||
|
||||
Authors' names are
|
||||
set in boldface, and each name is placed above its corresponding
|
||||
address. The lead author's name is to be listed first, and
|
||||
the co-authors' names are set to follow. Authors sharing the
|
||||
same address can be on the same line.
|
||||
|
||||
Please pay special attention to the instructions in section \ref{others}
|
||||
regarding figures, tables, acknowledgements, and references.
|
||||
|
||||
|
||||
There will be a strict upper limit of 9 pages for the main text of the initial submission, with unlimited additional pages for citations.
|
||||
|
||||
We strongly recommend following arXiv's guidelines for making your paper friendly for HTML conversion: \url{https://info.arxiv.org/help/submit_latex_best_practices.html}.
|
||||
|
||||
|
||||
\section{Headings: first level}
|
||||
\label{headings}
|
||||
|
||||
First level headings are in lower case (except for first word and proper nouns), bold face,
|
||||
flush left and in point size 12. One line space before the first level
|
||||
heading and 1/2~line space after the first level heading.
|
||||
|
||||
\subsection{Headings: second level}
|
||||
|
||||
Second level headings are in lower case (except for first word and proper nouns), bold face,
|
||||
flush left and in point size 10. One line space before the second level
|
||||
heading and 1/2~line space after the second level heading.
|
||||
|
||||
\subsubsection{Headings: third level}
|
||||
|
||||
Third level headings are in lower case (except for first word and proper nouns), bold face, italics,
|
||||
flush left and in point size 10. One line space before the third level
|
||||
heading and 1/2~line space after the third level heading.
|
||||
|
||||
\section{Citations, figures, tables, references}\label{others}
|
||||
|
||||
These instructions apply to everyone, regardless of the formatter being used.
|
||||
|
||||
\subsection{Citations within the text}
|
||||
|
||||
Citations within the text should be based on the \texttt{natbib} package
|
||||
and include the authors' last names and year (with the ``et~al.'' construct
|
||||
for more than two authors). When the authors or the publication are
|
||||
included in the sentence, the citation should not be in parenthesis using \verb|\citet{}| (as
|
||||
in ``See \citet{Vaswani+2017} for more information.''). Otherwise, the citation
|
||||
should be in parenthesis using \verb|\citep{}| (as in ``Transformers are a key tool
|
||||
for developing language models~\citep{Vaswani+2017}.'').
|
||||
|
||||
The corresponding references are to be listed in alphabetical order of
|
||||
authors, in the \textsc{References} section. As to the format of the
|
||||
references themselves, any style is acceptable as long as it is used
|
||||
consistently.
|
||||
|
||||
\subsection{Footnotes}
|
||||
|
||||
Indicate footnotes with a number\footnote{Sample of the first footnote} in the
|
||||
text. Place the footnotes at the bottom of the page on which they appear.
|
||||
Precede the footnote with a horizontal rule of 2~inches
|
||||
(12~picas).\footnote{Sample of the second footnote}
|
||||
|
||||
\subsection{Figures}
|
||||
|
||||
All artwork must be neat, clean, and legible. Lines should be dark
|
||||
enough for purposes of reproduction; art work should not be
|
||||
hand-drawn. Any text within the figure must be readable. We ask to not use font sizes below {\tt small}. We strongly recommend to use vector representations (e.g., pdf or svg) for all diagrams.
|
||||
We strongly recommend positioning all figures at the top or bottom of the page.
|
||||
|
||||
The figure number and caption always appear below the figure. Place one line space before the figure caption, and one line space after the figure. The figure caption is lower case (except for first word and proper nouns); figures are numbered consecutively.
|
||||
Make sure the figure caption does not get separated from the figure.
|
||||
Leave sufficient space to avoid splitting the figure and figure caption.
|
||||
|
||||
You may use color figures.
|
||||
However, it is best for the
|
||||
figure captions and the paper body to make sense if the paper is printed
|
||||
either in black/white or in color.
|
||||
\begin{figure}[t]
|
||||
\begin{center}
|
||||
%\framebox[4.0in]{$\;$}
|
||||
\fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
|
||||
\end{center}
|
||||
\caption{Sample figure caption.}
|
||||
\end{figure}
|
||||
|
||||
\subsection{Tables}
|
||||
|
||||
All tables must be centered, neat, clean and legible. Do not use hand-drawn tables. The table number and title always appear below the table. See Table~\ref{sample-table}. Please do not use font sizes below {\tt small} in tables. We recommend using {\tt booktabs} or a similar package to style tables.
|
||||
We strongly recommend positioning all tables at the top or bottom of the page.
|
||||
|
||||
Place one line space before the table title, one line space after the table title, and one line space after the table. The table title must be lowercase (except for first word and proper nouns); tables are numbered consecutively.
|
||||
|
||||
\begin{table}[t]
|
||||
\begin{center}
|
||||
\begin{tabular}{ll}
|
||||
\toprule
|
||||
\multicolumn{1}{c}{\bf PART} &\multicolumn{1}{c}{\bf DESCRIPTION} \\
|
||||
\midrule
|
||||
Dendrite &Input terminal \\
|
||||
Axon &Output terminal \\
|
||||
Soma &Cell body (contains cell nucleus) \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\end{center}
|
||||
\caption{Sample table title}\label{sample-table}
|
||||
\end{table}
|
||||
|
||||
|
||||
|
||||
|
||||
\section{Final instructions}
|
||||
Do not change any aspects of the formatting parameters in the style files.
|
||||
In particular, do not modify the width or length of the rectangle the text
|
||||
should fit into, and do not change font sizes (except perhaps in the
|
||||
\textsc{References} section; see below). Please note that pages should be
|
||||
numbered.
|
||||
|
||||
\section{Preparing PostScript or PDF files}
|
||||
|
||||
Please prepare PostScript or PDF files with paper size ``US Letter'', and
|
||||
not, for example, ``A4''. The -t
|
||||
letter option on dvips will produce US Letter files.
|
||||
|
||||
Consider directly generating PDF files using \verb+pdflatex+
|
||||
(especially if you are a MiKTeX user).
|
||||
PDF figures must be substituted for EPS figures, however.
|
||||
|
||||
Otherwise, please generate your PostScript and PDF files with the following commands:
|
||||
\begin{verbatim}
|
||||
dvips mypaper.dvi -t letter -Ppdf -G0 -o mypaper.ps
|
||||
ps2pdf mypaper.ps mypaper.pdf
|
||||
\end{verbatim}
|
||||
|
||||
\subsection{Margins in LaTeX}
|
||||
|
||||
Most of the margin problems come from figures positioned by hand using
|
||||
\verb+\special+ or other commands. We suggest using the command
|
||||
\verb+\includegraphics+
|
||||
from the graphicx package. Always specify the figure width as a multiple of
|
||||
the line width as in the example below using .eps graphics
|
||||
\begin{verbatim}
|
||||
\usepackage[dvips]{graphicx} ...
|
||||
\includegraphics[width=0.8\linewidth]{myfile.eps}
|
||||
\end{verbatim}
|
||||
or % Apr 2009 addition
|
||||
\begin{verbatim}
|
||||
\usepackage[pdftex]{graphicx} ...
|
||||
\includegraphics[width=0.8\linewidth]{myfile.pdf}
|
||||
\end{verbatim}
|
||||
for .pdf graphics.
|
||||
See section~4.4 in the graphics bundle documentation (\url{http://www.ctan.org/tex-archive/macros/latex/required/graphics/grfguide.ps})
|
||||
|
||||
A number of width problems arise when LaTeX cannot properly hyphenate a
|
||||
line. Please give LaTeX hyphenation hints using the \verb+\-+ command.
|
||||
|
||||
\section*{Author Contributions}
|
||||
If you'd like to, you may include a section for author contributions as is done
|
||||
in many journals. This is optional and at the discretion of the authors.
|
||||
|
||||
\section*{Acknowledgments}
|
||||
Use unnumbered first level headings for the acknowledgments. All
|
||||
acknowledgments, including those to funding agencies, go at the end of the paper.
|
||||
|
||||
\section*{Ethics Statement}
|
||||
Authors can add an optional ethics statement to the paper.
|
||||
For papers that touch on ethical issues, this section will be evaluated as part of the review process. The ethics statement should come at the end of the paper. It does not count toward the page limit, but should not be more than 1 page.
|
||||
|
||||
|
||||
|
||||
\bibliography{colm2025_conference}
|
||||
\bibliographystyle{colm2025_conference}
|
||||
|
||||
\appendix
|
||||
\section{Appendix}
|
||||
You may include other additional sections here.
|
||||
|
||||
\end{document}
|
||||
@@ -0,0 +1,485 @@
|
||||
% fancyhdr.sty version 3.2
|
||||
% Fancy headers and footers for LaTeX.
|
||||
% Piet van Oostrum,
|
||||
% Dept of Computer and Information Sciences, University of Utrecht,
|
||||
% Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands
|
||||
% Telephone: +31 30 2532180. Email: piet@cs.uu.nl
|
||||
% ========================================================================
|
||||
% LICENCE:
|
||||
% This file may be distributed under the terms of the LaTeX Project Public
|
||||
% License, as described in lppl.txt in the base LaTeX distribution.
|
||||
% Either version 1 or, at your option, any later version.
|
||||
% ========================================================================
|
||||
% MODIFICATION HISTORY:
|
||||
% Sep 16, 1994
|
||||
% version 1.4: Correction for use with \reversemargin
|
||||
% Sep 29, 1994:
|
||||
% version 1.5: Added the \iftopfloat, \ifbotfloat and \iffloatpage commands
|
||||
% Oct 4, 1994:
|
||||
% version 1.6: Reset single spacing in headers/footers for use with
|
||||
% setspace.sty or doublespace.sty
|
||||
% Oct 4, 1994:
|
||||
% version 1.7: changed \let\@mkboth\markboth to
|
||||
% \def\@mkboth{\protect\markboth} to make it more robust
|
||||
% Dec 5, 1994:
|
||||
% version 1.8: corrections for amsbook/amsart: define \@chapapp and (more
|
||||
% importantly) use the \chapter/sectionmark definitions from ps@headings if
|
||||
% they exist (which should be true for all standard classes).
|
||||
% May 31, 1995:
|
||||
% version 1.9: The proposed \renewcommand{\headrulewidth}{\iffloatpage...
|
||||
% construction in the doc did not work properly with the fancyplain style.
|
||||
% June 1, 1995:
|
||||
% version 1.91: The definition of \@mkboth wasn't restored on subsequent
|
||||
% \pagestyle{fancy}'s.
|
||||
% June 1, 1995:
|
||||
% version 1.92: The sequence \pagestyle{fancyplain} \pagestyle{plain}
|
||||
% \pagestyle{fancy} would erroneously select the plain version.
|
||||
% June 1, 1995:
|
||||
% version 1.93: \fancypagestyle command added.
|
||||
% Dec 11, 1995:
|
||||
% version 1.94: suggested by Conrad Hughes <chughes@maths.tcd.ie>
|
||||
% CJCH, Dec 11, 1995: added \footruleskip to allow control over footrule
|
||||
% position (old hardcoded value of .3\normalbaselineskip is far too high
|
||||
% when used with very small footer fonts).
|
||||
% Jan 31, 1996:
|
||||
% version 1.95: call \@normalsize in the reset code if that is defined,
|
||||
% otherwise \normalsize.
|
||||
% this is to solve a problem with ucthesis.cls, as this doesn't
|
||||
% define \@currsize. Unfortunately for latex209 calling \normalsize doesn't
|
||||
% work as this is optimized to do very little, so there \@normalsize should
|
||||
% be called. Hopefully this code works for all versions of LaTeX known to
|
||||
% mankind.
|
||||
% April 25, 1996:
|
||||
% version 1.96: initialize \headwidth to a magic (negative) value to catch
|
||||
% most common cases that people change it before calling \pagestyle{fancy}.
|
||||
% Note it can't be initialized when reading in this file, because
|
||||
% \textwidth could be changed afterwards. This is quite probable.
|
||||
% We also switch to \MakeUppercase rather than \uppercase and introduce a
|
||||
% \nouppercase command for use in headers. and footers.
|
||||
% May 3, 1996:
|
||||
% version 1.97: Two changes:
|
||||
% 1. Undo the change in version 1.8 (using the pagestyle{headings} defaults
|
||||
% for the chapter and section marks. The current version of amsbook and
|
||||
% amsart classes don't seem to need them anymore. Moreover the standard
|
||||
% latex classes don't use \markboth if twoside isn't selected, and this is
|
||||
% confusing as \leftmark doesn't work as expected.
|
||||
% 2. include a call to \ps@empty in ps@@fancy. This is to solve a problem
|
||||
% in the amsbook and amsart classes, that make global changes to \topskip,
|
||||
% which are reset in \ps@empty. Hopefully this doesn't break other things.
|
||||
% May 7, 1996:
|
||||
% version 1.98:
|
||||
% Added % after the line \def\nouppercase
|
||||
% May 7, 1996:
|
||||
% version 1.99: This is the alpha version of fancyhdr 2.0
|
||||
% Introduced the new commands \fancyhead, \fancyfoot, and \fancyhf.
|
||||
% Changed \headrulewidth, \footrulewidth, \footruleskip to
|
||||
% macros rather than length parameters, In this way they can be
|
||||
% conditionalized and they don't consume length registers. There is no need
|
||||
% to have them as length registers unless you want to do calculations with
|
||||
% them, which is unlikely. Note that this may make some uses of them
|
||||
% incompatible (i.e. if you have a file that uses \setlength or \xxxx=)
|
||||
% May 10, 1996:
|
||||
% version 1.99a:
|
||||
% Added a few more % signs
|
||||
% May 10, 1996:
|
||||
% version 1.99b:
|
||||
% Changed the syntax of \f@nfor to be resistent to catcode changes of :=
|
||||
% Removed the [1] from the defs of \lhead etc. because the parameter is
|
||||
% consumed by the \@[xy]lhead etc. macros.
|
||||
% June 24, 1997:
|
||||
% version 1.99c:
|
||||
% corrected \nouppercase to also include the protected form of \MakeUppercase
|
||||
% \global added to manipulation of \headwidth.
|
||||
% \iffootnote command added.
|
||||
% Some comments added about \@fancyhead and \@fancyfoot.
|
||||
% Aug 24, 1998
|
||||
% version 1.99d
|
||||
% Changed the default \ps@empty to \ps@@empty in order to allow
|
||||
% \fancypagestyle{empty} redefinition.
|
||||
% Oct 11, 2000
|
||||
% version 2.0
|
||||
% Added LPPL license clause.
|
||||
%
|
||||
% A check for \headheight is added. An errormessage is given (once) if the
|
||||
% header is too large. Empty headers don't generate the error even if
|
||||
% \headheight is very small or even 0pt.
|
||||
% Warning added for the use of 'E' option when twoside option is not used.
|
||||
% In this case the 'E' fields will never be used.
|
||||
%
|
||||
% Mar 10, 2002
|
||||
% version 2.1beta
|
||||
% New command: \fancyhfoffset[place]{length}
|
||||
% defines offsets to be applied to the header/footer to let it stick into
|
||||
% the margins (if length > 0).
|
||||
% place is like in fancyhead, except that only E,O,L,R can be used.
|
||||
% This replaces the old calculation based on \headwidth and the marginpar
|
||||
% area.
|
||||
% \headwidth will be dynamically calculated in the headers/footers when
|
||||
% this is used.
|
||||
%
|
||||
% Mar 26, 2002
|
||||
% version 2.1beta2
|
||||
% \fancyhfoffset now also takes h,f as possible letters in the argument to
|
||||
% allow the header and footer widths to be different.
|
||||
% New commands \fancyheadoffset and \fancyfootoffset added comparable to
|
||||
% \fancyhead and \fancyfoot.
|
||||
% Errormessages and warnings have been made more informative.
|
||||
%
|
||||
% Dec 9, 2002
|
||||
% version 2.1
|
||||
% The defaults for \footrulewidth, \plainheadrulewidth and
|
||||
% \plainfootrulewidth are changed from \z@skip to 0pt. In this way when
|
||||
% someone inadvertantly uses \setlength to change any of these, the value
|
||||
% of \z@skip will not be changed, rather an errormessage will be given.
|
||||
|
||||
% March 3, 2004
|
||||
% Release of version 3.0
|
||||
|
||||
% Oct 7, 2004
|
||||
% version 3.1
|
||||
% Added '\endlinechar=13' to \fancy@reset to prevent problems with
|
||||
% includegraphics in header when verbatiminput is active.
|
||||
|
||||
% March 22, 2005
|
||||
% version 3.2
|
||||
% reset \everypar (the real one) in \fancy@reset because spanish.ldf does
|
||||
% strange things with \everypar between << and >>.
|
||||
|
||||
\def\ifancy@mpty#1{\def\temp@a{#1}\ifx\temp@a\@empty}
|
||||
|
||||
\def\fancy@def#1#2{\ifancy@mpty{#2}\fancy@gbl\def#1{\leavevmode}\else
|
||||
\fancy@gbl\def#1{#2\strut}\fi}
|
||||
|
||||
\let\fancy@gbl\global
|
||||
|
||||
\def\@fancyerrmsg#1{%
|
||||
\ifx\PackageError\undefined
|
||||
\errmessage{#1}\else
|
||||
\PackageError{Fancyhdr}{#1}{}\fi}
|
||||
\def\@fancywarning#1{%
|
||||
\ifx\PackageWarning\undefined
|
||||
\errmessage{#1}\else
|
||||
\PackageWarning{Fancyhdr}{#1}{}\fi}
|
||||
|
||||
% Usage: \@forc \var{charstring}{command to be executed for each char}
|
||||
% This is similar to LaTeX's \@tfor, but expands the charstring.
|
||||
|
||||
\def\@forc#1#2#3{\expandafter\f@rc\expandafter#1\expandafter{#2}{#3}}
|
||||
\def\f@rc#1#2#3{\def\temp@ty{#2}\ifx\@empty\temp@ty\else
|
||||
\f@@rc#1#2\f@@rc{#3}\fi}
|
||||
\def\f@@rc#1#2#3\f@@rc#4{\def#1{#2}#4\f@rc#1{#3}{#4}}
|
||||
|
||||
% Usage: \f@nfor\name:=list\do{body}
|
||||
% Like LaTeX's \@for but an empty list is treated as a list with an empty
|
||||
% element
|
||||
|
||||
\newcommand{\f@nfor}[3]{\edef\@fortmp{#2}%
|
||||
\expandafter\@forloop#2,\@nil,\@nil\@@#1{#3}}
|
||||
|
||||
% Usage: \def@ult \cs{defaults}{argument}
|
||||
% sets \cs to the characters from defaults appearing in argument
|
||||
% or defaults if it would be empty. All characters are lowercased.
|
||||
|
||||
\newcommand\def@ult[3]{%
|
||||
\edef\temp@a{\lowercase{\edef\noexpand\temp@a{#3}}}\temp@a
|
||||
\def#1{}%
|
||||
\@forc\tmpf@ra{#2}%
|
||||
{\expandafter\if@in\tmpf@ra\temp@a{\edef#1{#1\tmpf@ra}}{}}%
|
||||
\ifx\@empty#1\def#1{#2}\fi}
|
||||
%
|
||||
% \if@in <char><set><truecase><falsecase>
|
||||
%
|
||||
\newcommand{\if@in}[4]{%
|
||||
\edef\temp@a{#2}\def\temp@b##1#1##2\temp@b{\def\temp@b{##1}}%
|
||||
\expandafter\temp@b#2#1\temp@b\ifx\temp@a\temp@b #4\else #3\fi}
|
||||
|
||||
\newcommand{\fancyhead}{\@ifnextchar[{\f@ncyhf\fancyhead h}%
|
||||
{\f@ncyhf\fancyhead h[]}}
|
||||
\newcommand{\fancyfoot}{\@ifnextchar[{\f@ncyhf\fancyfoot f}%
|
||||
{\f@ncyhf\fancyfoot f[]}}
|
||||
\newcommand{\fancyhf}{\@ifnextchar[{\f@ncyhf\fancyhf{}}%
|
||||
{\f@ncyhf\fancyhf{}[]}}
|
||||
|
||||
% New commands for offsets added
|
||||
|
||||
\newcommand{\fancyheadoffset}{\@ifnextchar[{\f@ncyhfoffs\fancyheadoffset h}%
|
||||
{\f@ncyhfoffs\fancyheadoffset h[]}}
|
||||
\newcommand{\fancyfootoffset}{\@ifnextchar[{\f@ncyhfoffs\fancyfootoffset f}%
|
||||
{\f@ncyhfoffs\fancyfootoffset f[]}}
|
||||
\newcommand{\fancyhfoffset}{\@ifnextchar[{\f@ncyhfoffs\fancyhfoffset{}}%
|
||||
{\f@ncyhfoffs\fancyhfoffset{}[]}}
|
||||
|
||||
% The header and footer fields are stored in command sequences with
|
||||
% names of the form: \f@ncy<x><y><z> with <x> for [eo], <y> from [lcr]
|
||||
% and <z> from [hf].
|
||||
|
||||
\def\f@ncyhf#1#2[#3]#4{%
|
||||
\def\temp@c{}%
|
||||
\@forc\tmpf@ra{#3}%
|
||||
{\expandafter\if@in\tmpf@ra{eolcrhf,EOLCRHF}%
|
||||
{}{\edef\temp@c{\temp@c\tmpf@ra}}}%
|
||||
\ifx\@empty\temp@c\else
|
||||
\@fancyerrmsg{Illegal char `\temp@c' in \string#1 argument:
|
||||
[#3]}%
|
||||
\fi
|
||||
\f@nfor\temp@c{#3}%
|
||||
{\def@ult\f@@@eo{eo}\temp@c
|
||||
\if@twoside\else
|
||||
\if\f@@@eo e\@fancywarning
|
||||
{\string#1's `E' option without twoside option is useless}\fi\fi
|
||||
\def@ult\f@@@lcr{lcr}\temp@c
|
||||
\def@ult\f@@@hf{hf}{#2\temp@c}%
|
||||
\@forc\f@@eo\f@@@eo
|
||||
{\@forc\f@@lcr\f@@@lcr
|
||||
{\@forc\f@@hf\f@@@hf
|
||||
{\expandafter\fancy@def\csname
|
||||
f@ncy\f@@eo\f@@lcr\f@@hf\endcsname
|
||||
{#4}}}}}}
|
||||
|
||||
\def\f@ncyhfoffs#1#2[#3]#4{%
|
||||
\def\temp@c{}%
|
||||
\@forc\tmpf@ra{#3}%
|
||||
{\expandafter\if@in\tmpf@ra{eolrhf,EOLRHF}%
|
||||
{}{\edef\temp@c{\temp@c\tmpf@ra}}}%
|
||||
\ifx\@empty\temp@c\else
|
||||
\@fancyerrmsg{Illegal char `\temp@c' in \string#1 argument:
|
||||
[#3]}%
|
||||
\fi
|
||||
\f@nfor\temp@c{#3}%
|
||||
{\def@ult\f@@@eo{eo}\temp@c
|
||||
\if@twoside\else
|
||||
\if\f@@@eo e\@fancywarning
|
||||
{\string#1's `E' option without twoside option is useless}\fi\fi
|
||||
\def@ult\f@@@lcr{lr}\temp@c
|
||||
\def@ult\f@@@hf{hf}{#2\temp@c}%
|
||||
\@forc\f@@eo\f@@@eo
|
||||
{\@forc\f@@lcr\f@@@lcr
|
||||
{\@forc\f@@hf\f@@@hf
|
||||
{\expandafter\setlength\csname
|
||||
f@ncyO@\f@@eo\f@@lcr\f@@hf\endcsname
|
||||
{#4}}}}}%
|
||||
\fancy@setoffs}
|
||||
|
||||
% Fancyheadings version 1 commands. These are more or less deprecated,
|
||||
% but they continue to work.
|
||||
|
||||
\newcommand{\lhead}{\@ifnextchar[{\@xlhead}{\@ylhead}}
|
||||
\def\@xlhead[#1]#2{\fancy@def\f@ncyelh{#1}\fancy@def\f@ncyolh{#2}}
|
||||
\def\@ylhead#1{\fancy@def\f@ncyelh{#1}\fancy@def\f@ncyolh{#1}}
|
||||
|
||||
\newcommand{\chead}{\@ifnextchar[{\@xchead}{\@ychead}}
|
||||
\def\@xchead[#1]#2{\fancy@def\f@ncyech{#1}\fancy@def\f@ncyoch{#2}}
|
||||
\def\@ychead#1{\fancy@def\f@ncyech{#1}\fancy@def\f@ncyoch{#1}}
|
||||
|
||||
\newcommand{\rhead}{\@ifnextchar[{\@xrhead}{\@yrhead}}
|
||||
\def\@xrhead[#1]#2{\fancy@def\f@ncyerh{#1}\fancy@def\f@ncyorh{#2}}
|
||||
\def\@yrhead#1{\fancy@def\f@ncyerh{#1}\fancy@def\f@ncyorh{#1}}
|
||||
|
||||
\newcommand{\lfoot}{\@ifnextchar[{\@xlfoot}{\@ylfoot}}
|
||||
\def\@xlfoot[#1]#2{\fancy@def\f@ncyelf{#1}\fancy@def\f@ncyolf{#2}}
|
||||
\def\@ylfoot#1{\fancy@def\f@ncyelf{#1}\fancy@def\f@ncyolf{#1}}
|
||||
|
||||
\newcommand{\cfoot}{\@ifnextchar[{\@xcfoot}{\@ycfoot}}
|
||||
\def\@xcfoot[#1]#2{\fancy@def\f@ncyecf{#1}\fancy@def\f@ncyocf{#2}}
|
||||
\def\@ycfoot#1{\fancy@def\f@ncyecf{#1}\fancy@def\f@ncyocf{#1}}
|
||||
|
||||
\newcommand{\rfoot}{\@ifnextchar[{\@xrfoot}{\@yrfoot}}
|
||||
\def\@xrfoot[#1]#2{\fancy@def\f@ncyerf{#1}\fancy@def\f@ncyorf{#2}}
|
||||
\def\@yrfoot#1{\fancy@def\f@ncyerf{#1}\fancy@def\f@ncyorf{#1}}
|
||||
|
||||
\newlength{\fancy@headwidth}
|
||||
\let\headwidth\fancy@headwidth
|
||||
\newlength{\f@ncyO@elh}
|
||||
\newlength{\f@ncyO@erh}
|
||||
\newlength{\f@ncyO@olh}
|
||||
\newlength{\f@ncyO@orh}
|
||||
\newlength{\f@ncyO@elf}
|
||||
\newlength{\f@ncyO@erf}
|
||||
\newlength{\f@ncyO@olf}
|
||||
\newlength{\f@ncyO@orf}
|
||||
\newcommand{\headrulewidth}{0.4pt}
|
||||
\newcommand{\footrulewidth}{0pt}
|
||||
\newcommand{\footruleskip}{.3\normalbaselineskip}
|
||||
|
||||
% Fancyplain stuff shouldn't be used anymore (rather
|
||||
% \fancypagestyle{plain} should be used), but it must be present for
|
||||
% compatibility reasons.
|
||||
|
||||
\newcommand{\plainheadrulewidth}{0pt}
|
||||
\newcommand{\plainfootrulewidth}{0pt}
|
||||
\newif\if@fancyplain \@fancyplainfalse
|
||||
\def\fancyplain#1#2{\if@fancyplain#1\else#2\fi}
|
||||
|
||||
\headwidth=-123456789sp %magic constant
|
||||
|
||||
% Command to reset various things in the headers:
|
||||
% a.o. single spacing (taken from setspace.sty)
|
||||
% and the catcode of ^^M (so that epsf files in the header work if a
|
||||
% verbatim crosses a page boundary)
|
||||
% It also defines a \nouppercase command that disables \uppercase and
|
||||
% \Makeuppercase. It can only be used in the headers and footers.
|
||||
\let\fnch@everypar\everypar% save real \everypar because of spanish.ldf
|
||||
\def\fancy@reset{\fnch@everypar{}\restorecr\endlinechar=13
|
||||
\def\baselinestretch{1}%
|
||||
\def\nouppercase##1{{\let\uppercase\relax\let\MakeUppercase\relax
|
||||
\expandafter\let\csname MakeUppercase \endcsname\relax##1}}%
|
||||
\ifx\undefined\@newbaseline% NFSS not present; 2.09 or 2e
|
||||
\ifx\@normalsize\undefined \normalsize % for ucthesis.cls
|
||||
\else \@normalsize \fi
|
||||
\else% NFSS (2.09) present
|
||||
\@newbaseline%
|
||||
\fi}
|
||||
|
||||
% Initialization of the head and foot text.
|
||||
|
||||
% The default values still contain \fancyplain for compatibility.
|
||||
\fancyhf{} % clear all
|
||||
% lefthead empty on ``plain'' pages, \rightmark on even, \leftmark on odd pages
|
||||
% evenhead empty on ``plain'' pages, \leftmark on even, \rightmark on odd pages
|
||||
\if@twoside
|
||||
\fancyhead[el,or]{\fancyplain{}{\sl\rightmark}}
|
||||
\fancyhead[er,ol]{\fancyplain{}{\sl\leftmark}}
|
||||
\else
|
||||
\fancyhead[l]{\fancyplain{}{\sl\rightmark}}
|
||||
\fancyhead[r]{\fancyplain{}{\sl\leftmark}}
|
||||
\fi
|
||||
\fancyfoot[c]{\rm\thepage} % page number
|
||||
|
||||
% Use box 0 as a temp box and dimen 0 as temp dimen.
|
||||
% This can be done, because this code will always
|
||||
% be used inside another box, and therefore the changes are local.
|
||||
|
||||
\def\@fancyvbox#1#2{\setbox0\vbox{#2}\ifdim\ht0>#1\@fancywarning
|
||||
{\string#1 is too small (\the#1): ^^J Make it at least \the\ht0.^^J
|
||||
We now make it that large for the rest of the document.^^J
|
||||
This may cause the page layout to be inconsistent, however\@gobble}%
|
||||
\dimen0=#1\global\setlength{#1}{\ht0}\ht0=\dimen0\fi
|
||||
\box0}
|
||||
|
||||
% Put together a header or footer given the left, center and
|
||||
% right text, fillers at left and right and a rule.
|
||||
% The \lap commands put the text into an hbox of zero size,
|
||||
% so overlapping text does not generate an errormessage.
|
||||
% These macros have 5 parameters:
|
||||
% 1. LEFTSIDE BEARING % This determines at which side the header will stick
|
||||
% out. When \fancyhfoffset is used this calculates \headwidth, otherwise
|
||||
% it is \hss or \relax (after expansion).
|
||||
% 2. \f@ncyolh, \f@ncyelh, \f@ncyolf or \f@ncyelf. This is the left component.
|
||||
% 3. \f@ncyoch, \f@ncyech, \f@ncyocf or \f@ncyecf. This is the middle comp.
|
||||
% 4. \f@ncyorh, \f@ncyerh, \f@ncyorf or \f@ncyerf. This is the right component.
|
||||
% 5. RIGHTSIDE BEARING. This is always \relax or \hss (after expansion).
|
||||
|
||||
\def\@fancyhead#1#2#3#4#5{#1\hbox to\headwidth{\fancy@reset
|
||||
\@fancyvbox\headheight{\hbox
|
||||
{\rlap{\parbox[b]{\headwidth}{\raggedright#2}}\hfill
|
||||
\parbox[b]{\headwidth}{\centering#3}\hfill
|
||||
\llap{\parbox[b]{\headwidth}{\raggedleft#4}}}\headrule}}#5}
|
||||
|
||||
\def\@fancyfoot#1#2#3#4#5{#1\hbox to\headwidth{\fancy@reset
|
||||
\@fancyvbox\footskip{\footrule
|
||||
\hbox{\rlap{\parbox[t]{\headwidth}{\raggedright#2}}\hfill
|
||||
\parbox[t]{\headwidth}{\centering#3}\hfill
|
||||
\llap{\parbox[t]{\headwidth}{\raggedleft#4}}}}}#5}
|
||||
|
||||
\def\headrule{{\if@fancyplain\let\headrulewidth\plainheadrulewidth\fi
|
||||
\hrule\@height\headrulewidth\@width\headwidth \vskip-\headrulewidth}}
|
||||
|
||||
\def\footrule{{\if@fancyplain\let\footrulewidth\plainfootrulewidth\fi
|
||||
\vskip-\footruleskip\vskip-\footrulewidth
|
||||
\hrule\@width\headwidth\@height\footrulewidth\vskip\footruleskip}}
|
||||
|
||||
\def\ps@fancy{%
|
||||
\@ifundefined{@chapapp}{\let\@chapapp\chaptername}{}%for amsbook
|
||||
%
|
||||
% Define \MakeUppercase for old LaTeXen.
|
||||
% Note: we used \def rather than \let, so that \let\uppercase\relax (from
|
||||
% the version 1 documentation) will still work.
|
||||
%
|
||||
\@ifundefined{MakeUppercase}{\def\MakeUppercase{\uppercase}}{}%
|
||||
\@ifundefined{chapter}{\def\sectionmark##1{\markboth
|
||||
{\MakeUppercase{\ifnum \c@secnumdepth>\z@
|
||||
\thesection\hskip 1em\relax \fi ##1}}{}}%
|
||||
\def\subsectionmark##1{\markright {\ifnum \c@secnumdepth >\@ne
|
||||
\thesubsection\hskip 1em\relax \fi ##1}}}%
|
||||
{\def\chaptermark##1{\markboth {\MakeUppercase{\ifnum \c@secnumdepth>\m@ne
|
||||
\@chapapp\ \thechapter. \ \fi ##1}}{}}%
|
||||
\def\sectionmark##1{\markright{\MakeUppercase{\ifnum \c@secnumdepth >\z@
|
||||
\thesection. \ \fi ##1}}}}%
|
||||
%\csname ps@headings\endcsname % use \ps@headings defaults if they exist
|
||||
\ps@@fancy
|
||||
\gdef\ps@fancy{\@fancyplainfalse\ps@@fancy}%
|
||||
% Initialize \headwidth if the user didn't
|
||||
%
|
||||
\ifdim\headwidth<0sp
|
||||
%
|
||||
% This catches the case that \headwidth hasn't been initialized and the
|
||||
% case that the user added something to \headwidth in the expectation that
|
||||
% it was initialized to \textwidth. We compensate this now. This loses if
|
||||
% the user intended to multiply it by a factor. But that case is more
|
||||
% likely done by saying something like \headwidth=1.2\textwidth.
|
||||
% The doc says you have to change \headwidth after the first call to
|
||||
% \pagestyle{fancy}. This code is just to catch the most common cases were
|
||||
% that requirement is violated.
|
||||
%
|
||||
\global\advance\headwidth123456789sp\global\advance\headwidth\textwidth
|
||||
\fi}
|
||||
\def\ps@fancyplain{\ps@fancy \let\ps@plain\ps@plain@fancy}
|
||||
\def\ps@plain@fancy{\@fancyplaintrue\ps@@fancy}
|
||||
\let\ps@@empty\ps@empty
|
||||
\def\ps@@fancy{%
|
||||
\ps@@empty % This is for amsbook/amsart, which do strange things with \topskip
|
||||
\def\@mkboth{\protect\markboth}%
|
||||
\def\@oddhead{\@fancyhead\fancy@Oolh\f@ncyolh\f@ncyoch\f@ncyorh\fancy@Oorh}%
|
||||
\def\@oddfoot{\@fancyfoot\fancy@Oolf\f@ncyolf\f@ncyocf\f@ncyorf\fancy@Oorf}%
|
||||
\def\@evenhead{\@fancyhead\fancy@Oelh\f@ncyelh\f@ncyech\f@ncyerh\fancy@Oerh}%
|
||||
\def\@evenfoot{\@fancyfoot\fancy@Oelf\f@ncyelf\f@ncyecf\f@ncyerf\fancy@Oerf}%
|
||||
}
|
||||
% Default definitions for compatibility mode:
|
||||
% These cause the header/footer to take the defined \headwidth as width
|
||||
% And to shift in the direction of the marginpar area
|
||||
|
||||
\def\fancy@Oolh{\if@reversemargin\hss\else\relax\fi}
|
||||
\def\fancy@Oorh{\if@reversemargin\relax\else\hss\fi}
|
||||
\let\fancy@Oelh\fancy@Oorh
|
||||
\let\fancy@Oerh\fancy@Oolh
|
||||
|
||||
\let\fancy@Oolf\fancy@Oolh
|
||||
\let\fancy@Oorf\fancy@Oorh
|
||||
\let\fancy@Oelf\fancy@Oelh
|
||||
\let\fancy@Oerf\fancy@Oerh
|
||||
|
||||
% New definitions for the use of \fancyhfoffset
|
||||
% These calculate the \headwidth from \textwidth and the specified offsets.
|
||||
|
||||
\def\fancy@offsolh{\headwidth=\textwidth\advance\headwidth\f@ncyO@olh
|
||||
\advance\headwidth\f@ncyO@orh\hskip-\f@ncyO@olh}
|
||||
\def\fancy@offselh{\headwidth=\textwidth\advance\headwidth\f@ncyO@elh
|
||||
\advance\headwidth\f@ncyO@erh\hskip-\f@ncyO@elh}
|
||||
|
||||
\def\fancy@offsolf{\headwidth=\textwidth\advance\headwidth\f@ncyO@olf
|
||||
\advance\headwidth\f@ncyO@orf\hskip-\f@ncyO@olf}
|
||||
\def\fancy@offself{\headwidth=\textwidth\advance\headwidth\f@ncyO@elf
|
||||
\advance\headwidth\f@ncyO@erf\hskip-\f@ncyO@elf}
|
||||
|
||||
\def\fancy@setoffs{%
|
||||
% Just in case \let\headwidth\textwidth was used
|
||||
\fancy@gbl\let\headwidth\fancy@headwidth
|
||||
\fancy@gbl\let\fancy@Oolh\fancy@offsolh
|
||||
\fancy@gbl\let\fancy@Oelh\fancy@offselh
|
||||
\fancy@gbl\let\fancy@Oorh\hss
|
||||
\fancy@gbl\let\fancy@Oerh\hss
|
||||
\fancy@gbl\let\fancy@Oolf\fancy@offsolf
|
||||
\fancy@gbl\let\fancy@Oelf\fancy@offself
|
||||
\fancy@gbl\let\fancy@Oorf\hss
|
||||
\fancy@gbl\let\fancy@Oerf\hss}
|
||||
|
||||
\newif\iffootnote
|
||||
\let\latex@makecol\@makecol
|
||||
\def\@makecol{\ifvoid\footins\footnotetrue\else\footnotefalse\fi
|
||||
\let\topfloat\@toplist\let\botfloat\@botlist\latex@makecol}
|
||||
\def\iftopfloat#1#2{\ifx\topfloat\empty #2\else #1\fi}
|
||||
\def\ifbotfloat#1#2{\ifx\botfloat\empty #2\else #1\fi}
|
||||
\def\iffloatpage#1#2{\if@fcolmade #1\else #2\fi}
|
||||
|
||||
\newcommand{\fancypagestyle}[2]{%
|
||||
\@namedef{ps@#1}{\let\fancy@gbl\relax#2\relax\ps@fancy}}
|
||||
@@ -0,0 +1,508 @@
|
||||
%%%%% NEW MATH DEFINITIONS %%%%%
|
||||
|
||||
\usepackage{amsmath,amsfonts,bm}
|
||||
|
||||
% Mark sections of captions for referring to divisions of figures
|
||||
\newcommand{\figleft}{{\em (Left)}}
|
||||
\newcommand{\figcenter}{{\em (Center)}}
|
||||
\newcommand{\figright}{{\em (Right)}}
|
||||
\newcommand{\figtop}{{\em (Top)}}
|
||||
\newcommand{\figbottom}{{\em (Bottom)}}
|
||||
\newcommand{\captiona}{{\em (a)}}
|
||||
\newcommand{\captionb}{{\em (b)}}
|
||||
\newcommand{\captionc}{{\em (c)}}
|
||||
\newcommand{\captiond}{{\em (d)}}
|
||||
|
||||
% Highlight a newly defined term
|
||||
\newcommand{\newterm}[1]{{\bf #1}}
|
||||
|
||||
|
||||
% Figure reference, lower-case.
|
||||
\def\figref#1{figure~\ref{#1}}
|
||||
% Figure reference, capital. For start of sentence
|
||||
\def\Figref#1{Figure~\ref{#1}}
|
||||
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
|
||||
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
|
||||
% Section reference, lower-case.
|
||||
\def\secref#1{section~\ref{#1}}
|
||||
% Section reference, capital.
|
||||
\def\Secref#1{Section~\ref{#1}}
|
||||
% Reference to two sections.
|
||||
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
|
||||
% Reference to three sections.
|
||||
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
|
||||
% Reference to an equation, lower-case.
|
||||
\def\eqref#1{equation~\ref{#1}}
|
||||
% Reference to an equation, upper case
|
||||
\def\Eqref#1{Equation~\ref{#1}}
|
||||
% A raw reference to an equation---avoid using if possible
|
||||
\def\plaineqref#1{\ref{#1}}
|
||||
% Reference to a chapter, lower-case.
|
||||
\def\chapref#1{chapter~\ref{#1}}
|
||||
% Reference to an equation, upper case.
|
||||
\def\Chapref#1{Chapter~\ref{#1}}
|
||||
% Reference to a range of chapters
|
||||
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
|
||||
% Reference to an algorithm, lower-case.
|
||||
\def\algref#1{algorithm~\ref{#1}}
|
||||
% Reference to an algorithm, upper case.
|
||||
\def\Algref#1{Algorithm~\ref{#1}}
|
||||
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
|
||||
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
|
||||
% Reference to a part, lower case
|
||||
\def\partref#1{part~\ref{#1}}
|
||||
% Reference to a part, upper case
|
||||
\def\Partref#1{Part~\ref{#1}}
|
||||
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}
|
||||
|
||||
\def\ceil#1{\lceil #1 \rceil}
|
||||
\def\floor#1{\lfloor #1 \rfloor}
|
||||
\def\1{\bm{1}}
|
||||
\newcommand{\train}{\mathcal{D}}
|
||||
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
|
||||
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}
|
||||
|
||||
\def\eps{{\epsilon}}
|
||||
|
||||
|
||||
% Random variables
|
||||
\def\reta{{\textnormal{$\eta$}}}
|
||||
\def\ra{{\textnormal{a}}}
|
||||
\def\rb{{\textnormal{b}}}
|
||||
\def\rc{{\textnormal{c}}}
|
||||
\def\rd{{\textnormal{d}}}
|
||||
\def\re{{\textnormal{e}}}
|
||||
\def\rf{{\textnormal{f}}}
|
||||
\def\rg{{\textnormal{g}}}
|
||||
\def\rh{{\textnormal{h}}}
|
||||
\def\ri{{\textnormal{i}}}
|
||||
\def\rj{{\textnormal{j}}}
|
||||
\def\rk{{\textnormal{k}}}
|
||||
\def\rl{{\textnormal{l}}}
|
||||
% rm is already a command, just don't name any random variables m
|
||||
\def\rn{{\textnormal{n}}}
|
||||
\def\ro{{\textnormal{o}}}
|
||||
\def\rp{{\textnormal{p}}}
|
||||
\def\rq{{\textnormal{q}}}
|
||||
\def\rr{{\textnormal{r}}}
|
||||
\def\rs{{\textnormal{s}}}
|
||||
\def\rt{{\textnormal{t}}}
|
||||
\def\ru{{\textnormal{u}}}
|
||||
\def\rv{{\textnormal{v}}}
|
||||
\def\rw{{\textnormal{w}}}
|
||||
\def\rx{{\textnormal{x}}}
|
||||
\def\ry{{\textnormal{y}}}
|
||||
\def\rz{{\textnormal{z}}}
|
||||
|
||||
% Random vectors
|
||||
\def\rvepsilon{{\mathbf{\epsilon}}}
|
||||
\def\rvtheta{{\mathbf{\theta}}}
|
||||
\def\rva{{\mathbf{a}}}
|
||||
\def\rvb{{\mathbf{b}}}
|
||||
\def\rvc{{\mathbf{c}}}
|
||||
\def\rvd{{\mathbf{d}}}
|
||||
\def\rve{{\mathbf{e}}}
|
||||
\def\rvf{{\mathbf{f}}}
|
||||
\def\rvg{{\mathbf{g}}}
|
||||
\def\rvh{{\mathbf{h}}}
|
||||
\def\rvu{{\mathbf{i}}}
|
||||
\def\rvj{{\mathbf{j}}}
|
||||
\def\rvk{{\mathbf{k}}}
|
||||
\def\rvl{{\mathbf{l}}}
|
||||
\def\rvm{{\mathbf{m}}}
|
||||
\def\rvn{{\mathbf{n}}}
|
||||
\def\rvo{{\mathbf{o}}}
|
||||
\def\rvp{{\mathbf{p}}}
|
||||
\def\rvq{{\mathbf{q}}}
|
||||
\def\rvr{{\mathbf{r}}}
|
||||
\def\rvs{{\mathbf{s}}}
|
||||
\def\rvt{{\mathbf{t}}}
|
||||
\def\rvu{{\mathbf{u}}}
|
||||
\def\rvv{{\mathbf{v}}}
|
||||
\def\rvw{{\mathbf{w}}}
|
||||
\def\rvx{{\mathbf{x}}}
|
||||
\def\rvy{{\mathbf{y}}}
|
||||
\def\rvz{{\mathbf{z}}}
|
||||
|
||||
% Elements of random vectors
|
||||
\def\erva{{\textnormal{a}}}
|
||||
\def\ervb{{\textnormal{b}}}
|
||||
\def\ervc{{\textnormal{c}}}
|
||||
\def\ervd{{\textnormal{d}}}
|
||||
\def\erve{{\textnormal{e}}}
|
||||
\def\ervf{{\textnormal{f}}}
|
||||
\def\ervg{{\textnormal{g}}}
|
||||
\def\ervh{{\textnormal{h}}}
|
||||
\def\ervi{{\textnormal{i}}}
|
||||
\def\ervj{{\textnormal{j}}}
|
||||
\def\ervk{{\textnormal{k}}}
|
||||
\def\ervl{{\textnormal{l}}}
|
||||
\def\ervm{{\textnormal{m}}}
|
||||
\def\ervn{{\textnormal{n}}}
|
||||
\def\ervo{{\textnormal{o}}}
|
||||
\def\ervp{{\textnormal{p}}}
|
||||
\def\ervq{{\textnormal{q}}}
|
||||
\def\ervr{{\textnormal{r}}}
|
||||
\def\ervs{{\textnormal{s}}}
|
||||
\def\ervt{{\textnormal{t}}}
|
||||
\def\ervu{{\textnormal{u}}}
|
||||
\def\ervv{{\textnormal{v}}}
|
||||
\def\ervw{{\textnormal{w}}}
|
||||
\def\ervx{{\textnormal{x}}}
|
||||
\def\ervy{{\textnormal{y}}}
|
||||
\def\ervz{{\textnormal{z}}}
|
||||
|
||||
% Random matrices
|
||||
\def\rmA{{\mathbf{A}}}
|
||||
\def\rmB{{\mathbf{B}}}
|
||||
\def\rmC{{\mathbf{C}}}
|
||||
\def\rmD{{\mathbf{D}}}
|
||||
\def\rmE{{\mathbf{E}}}
|
||||
\def\rmF{{\mathbf{F}}}
|
||||
\def\rmG{{\mathbf{G}}}
|
||||
\def\rmH{{\mathbf{H}}}
|
||||
\def\rmI{{\mathbf{I}}}
|
||||
\def\rmJ{{\mathbf{J}}}
|
||||
\def\rmK{{\mathbf{K}}}
|
||||
\def\rmL{{\mathbf{L}}}
|
||||
\def\rmM{{\mathbf{M}}}
|
||||
\def\rmN{{\mathbf{N}}}
|
||||
\def\rmO{{\mathbf{O}}}
|
||||
\def\rmP{{\mathbf{P}}}
|
||||
\def\rmQ{{\mathbf{Q}}}
|
||||
\def\rmR{{\mathbf{R}}}
|
||||
\def\rmS{{\mathbf{S}}}
|
||||
\def\rmT{{\mathbf{T}}}
|
||||
\def\rmU{{\mathbf{U}}}
|
||||
\def\rmV{{\mathbf{V}}}
|
||||
\def\rmW{{\mathbf{W}}}
|
||||
\def\rmX{{\mathbf{X}}}
|
||||
\def\rmY{{\mathbf{Y}}}
|
||||
\def\rmZ{{\mathbf{Z}}}
|
||||
|
||||
% Elements of random matrices
|
||||
\def\ermA{{\textnormal{A}}}
|
||||
\def\ermB{{\textnormal{B}}}
|
||||
\def\ermC{{\textnormal{C}}}
|
||||
\def\ermD{{\textnormal{D}}}
|
||||
\def\ermE{{\textnormal{E}}}
|
||||
\def\ermF{{\textnormal{F}}}
|
||||
\def\ermG{{\textnormal{G}}}
|
||||
\def\ermH{{\textnormal{H}}}
|
||||
\def\ermI{{\textnormal{I}}}
|
||||
\def\ermJ{{\textnormal{J}}}
|
||||
\def\ermK{{\textnormal{K}}}
|
||||
\def\ermL{{\textnormal{L}}}
|
||||
\def\ermM{{\textnormal{M}}}
|
||||
\def\ermN{{\textnormal{N}}}
|
||||
\def\ermO{{\textnormal{O}}}
|
||||
\def\ermP{{\textnormal{P}}}
|
||||
\def\ermQ{{\textnormal{Q}}}
|
||||
\def\ermR{{\textnormal{R}}}
|
||||
\def\ermS{{\textnormal{S}}}
|
||||
\def\ermT{{\textnormal{T}}}
|
||||
\def\ermU{{\textnormal{U}}}
|
||||
\def\ermV{{\textnormal{V}}}
|
||||
\def\ermW{{\textnormal{W}}}
|
||||
\def\ermX{{\textnormal{X}}}
|
||||
\def\ermY{{\textnormal{Y}}}
|
||||
\def\ermZ{{\textnormal{Z}}}
|
||||
|
||||
% Vectors
|
||||
\def\vzero{{\bm{0}}}
|
||||
\def\vone{{\bm{1}}}
|
||||
\def\vmu{{\bm{\mu}}}
|
||||
\def\vtheta{{\bm{\theta}}}
|
||||
\def\va{{\bm{a}}}
|
||||
\def\vb{{\bm{b}}}
|
||||
\def\vc{{\bm{c}}}
|
||||
\def\vd{{\bm{d}}}
|
||||
\def\ve{{\bm{e}}}
|
||||
\def\vf{{\bm{f}}}
|
||||
\def\vg{{\bm{g}}}
|
||||
\def\vh{{\bm{h}}}
|
||||
\def\vi{{\bm{i}}}
|
||||
\def\vj{{\bm{j}}}
|
||||
\def\vk{{\bm{k}}}
|
||||
\def\vl{{\bm{l}}}
|
||||
\def\vm{{\bm{m}}}
|
||||
\def\vn{{\bm{n}}}
|
||||
\def\vo{{\bm{o}}}
|
||||
\def\vp{{\bm{p}}}
|
||||
\def\vq{{\bm{q}}}
|
||||
\def\vr{{\bm{r}}}
|
||||
\def\vs{{\bm{s}}}
|
||||
\def\vt{{\bm{t}}}
|
||||
\def\vu{{\bm{u}}}
|
||||
\def\vv{{\bm{v}}}
|
||||
\def\vw{{\bm{w}}}
|
||||
\def\vx{{\bm{x}}}
|
||||
\def\vy{{\bm{y}}}
|
||||
\def\vz{{\bm{z}}}
|
||||
|
||||
% Elements of vectors
|
||||
\def\evalpha{{\alpha}}
|
||||
\def\evbeta{{\beta}}
|
||||
\def\evepsilon{{\epsilon}}
|
||||
\def\evlambda{{\lambda}}
|
||||
\def\evomega{{\omega}}
|
||||
\def\evmu{{\mu}}
|
||||
\def\evpsi{{\psi}}
|
||||
\def\evsigma{{\sigma}}
|
||||
\def\evtheta{{\theta}}
|
||||
\def\eva{{a}}
|
||||
\def\evb{{b}}
|
||||
\def\evc{{c}}
|
||||
\def\evd{{d}}
|
||||
\def\eve{{e}}
|
||||
\def\evf{{f}}
|
||||
\def\evg{{g}}
|
||||
\def\evh{{h}}
|
||||
\def\evi{{i}}
|
||||
\def\evj{{j}}
|
||||
\def\evk{{k}}
|
||||
\def\evl{{l}}
|
||||
\def\evm{{m}}
|
||||
\def\evn{{n}}
|
||||
\def\evo{{o}}
|
||||
\def\evp{{p}}
|
||||
\def\evq{{q}}
|
||||
\def\evr{{r}}
|
||||
\def\evs{{s}}
|
||||
\def\evt{{t}}
|
||||
\def\evu{{u}}
|
||||
\def\evv{{v}}
|
||||
\def\evw{{w}}
|
||||
\def\evx{{x}}
|
||||
\def\evy{{y}}
|
||||
\def\evz{{z}}
|
||||
|
||||
% Matrix
|
||||
\def\mA{{\bm{A}}}
|
||||
\def\mB{{\bm{B}}}
|
||||
\def\mC{{\bm{C}}}
|
||||
\def\mD{{\bm{D}}}
|
||||
\def\mE{{\bm{E}}}
|
||||
\def\mF{{\bm{F}}}
|
||||
\def\mG{{\bm{G}}}
|
||||
\def\mH{{\bm{H}}}
|
||||
\def\mI{{\bm{I}}}
|
||||
\def\mJ{{\bm{J}}}
|
||||
\def\mK{{\bm{K}}}
|
||||
\def\mL{{\bm{L}}}
|
||||
\def\mM{{\bm{M}}}
|
||||
\def\mN{{\bm{N}}}
|
||||
\def\mO{{\bm{O}}}
|
||||
\def\mP{{\bm{P}}}
|
||||
\def\mQ{{\bm{Q}}}
|
||||
\def\mR{{\bm{R}}}
|
||||
\def\mS{{\bm{S}}}
|
||||
\def\mT{{\bm{T}}}
|
||||
\def\mU{{\bm{U}}}
|
||||
\def\mV{{\bm{V}}}
|
||||
\def\mW{{\bm{W}}}
|
||||
\def\mX{{\bm{X}}}
|
||||
\def\mY{{\bm{Y}}}
|
||||
\def\mZ{{\bm{Z}}}
|
||||
\def\mBeta{{\bm{\beta}}}
|
||||
\def\mPhi{{\bm{\Phi}}}
|
||||
\def\mLambda{{\bm{\Lambda}}}
|
||||
\def\mSigma{{\bm{\Sigma}}}
|
||||
|
||||
% Tensor
|
||||
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
|
||||
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
|
||||
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
|
||||
\def\tA{{\tens{A}}}
|
||||
\def\tB{{\tens{B}}}
|
||||
\def\tC{{\tens{C}}}
|
||||
\def\tD{{\tens{D}}}
|
||||
\def\tE{{\tens{E}}}
|
||||
\def\tF{{\tens{F}}}
|
||||
\def\tG{{\tens{G}}}
|
||||
\def\tH{{\tens{H}}}
|
||||
\def\tI{{\tens{I}}}
|
||||
\def\tJ{{\tens{J}}}
|
||||
\def\tK{{\tens{K}}}
|
||||
\def\tL{{\tens{L}}}
|
||||
\def\tM{{\tens{M}}}
|
||||
\def\tN{{\tens{N}}}
|
||||
\def\tO{{\tens{O}}}
|
||||
\def\tP{{\tens{P}}}
|
||||
\def\tQ{{\tens{Q}}}
|
||||
\def\tR{{\tens{R}}}
|
||||
\def\tS{{\tens{S}}}
|
||||
\def\tT{{\tens{T}}}
|
||||
\def\tU{{\tens{U}}}
|
||||
\def\tV{{\tens{V}}}
|
||||
\def\tW{{\tens{W}}}
|
||||
\def\tX{{\tens{X}}}
|
||||
\def\tY{{\tens{Y}}}
|
||||
\def\tZ{{\tens{Z}}}
|
||||
|
||||
|
||||
% Graph
|
||||
\def\gA{{\mathcal{A}}}
|
||||
\def\gB{{\mathcal{B}}}
|
||||
\def\gC{{\mathcal{C}}}
|
||||
\def\gD{{\mathcal{D}}}
|
||||
\def\gE{{\mathcal{E}}}
|
||||
\def\gF{{\mathcal{F}}}
|
||||
\def\gG{{\mathcal{G}}}
|
||||
\def\gH{{\mathcal{H}}}
|
||||
\def\gI{{\mathcal{I}}}
|
||||
\def\gJ{{\mathcal{J}}}
|
||||
\def\gK{{\mathcal{K}}}
|
||||
\def\gL{{\mathcal{L}}}
|
||||
\def\gM{{\mathcal{M}}}
|
||||
\def\gN{{\mathcal{N}}}
|
||||
\def\gO{{\mathcal{O}}}
|
||||
\def\gP{{\mathcal{P}}}
|
||||
\def\gQ{{\mathcal{Q}}}
|
||||
\def\gR{{\mathcal{R}}}
|
||||
\def\gS{{\mathcal{S}}}
|
||||
\def\gT{{\mathcal{T}}}
|
||||
\def\gU{{\mathcal{U}}}
|
||||
\def\gV{{\mathcal{V}}}
|
||||
\def\gW{{\mathcal{W}}}
|
||||
\def\gX{{\mathcal{X}}}
|
||||
\def\gY{{\mathcal{Y}}}
|
||||
\def\gZ{{\mathcal{Z}}}
|
||||
|
||||
% Sets
|
||||
\def\sA{{\mathbb{A}}}
|
||||
\def\sB{{\mathbb{B}}}
|
||||
\def\sC{{\mathbb{C}}}
|
||||
\def\sD{{\mathbb{D}}}
|
||||
% Don't use a set called E, because this would be the same as our symbol
|
||||
% for expectation.
|
||||
\def\sF{{\mathbb{F}}}
|
||||
\def\sG{{\mathbb{G}}}
|
||||
\def\sH{{\mathbb{H}}}
|
||||
\def\sI{{\mathbb{I}}}
|
||||
\def\sJ{{\mathbb{J}}}
|
||||
\def\sK{{\mathbb{K}}}
|
||||
\def\sL{{\mathbb{L}}}
|
||||
\def\sM{{\mathbb{M}}}
|
||||
\def\sN{{\mathbb{N}}}
|
||||
\def\sO{{\mathbb{O}}}
|
||||
\def\sP{{\mathbb{P}}}
|
||||
\def\sQ{{\mathbb{Q}}}
|
||||
\def\sR{{\mathbb{R}}}
|
||||
\def\sS{{\mathbb{S}}}
|
||||
\def\sT{{\mathbb{T}}}
|
||||
\def\sU{{\mathbb{U}}}
|
||||
\def\sV{{\mathbb{V}}}
|
||||
\def\sW{{\mathbb{W}}}
|
||||
\def\sX{{\mathbb{X}}}
|
||||
\def\sY{{\mathbb{Y}}}
|
||||
\def\sZ{{\mathbb{Z}}}
|
||||
|
||||
% Entries of a matrix
|
||||
\def\emLambda{{\Lambda}}
|
||||
\def\emA{{A}}
|
||||
\def\emB{{B}}
|
||||
\def\emC{{C}}
|
||||
\def\emD{{D}}
|
||||
\def\emE{{E}}
|
||||
\def\emF{{F}}
|
||||
\def\emG{{G}}
|
||||
\def\emH{{H}}
|
||||
\def\emI{{I}}
|
||||
\def\emJ{{J}}
|
||||
\def\emK{{K}}
|
||||
\def\emL{{L}}
|
||||
\def\emM{{M}}
|
||||
\def\emN{{N}}
|
||||
\def\emO{{O}}
|
||||
\def\emP{{P}}
|
||||
\def\emQ{{Q}}
|
||||
\def\emR{{R}}
|
||||
\def\emS{{S}}
|
||||
\def\emT{{T}}
|
||||
\def\emU{{U}}
|
||||
\def\emV{{V}}
|
||||
\def\emW{{W}}
|
||||
\def\emX{{X}}
|
||||
\def\emY{{Y}}
|
||||
\def\emZ{{Z}}
|
||||
\def\emSigma{{\Sigma}}
|
||||
|
||||
% entries of a tensor
|
||||
% Same font as tensor, without \bm wrapper
|
||||
\newcommand{\etens}[1]{\mathsfit{#1}}
|
||||
\def\etLambda{{\etens{\Lambda}}}
|
||||
\def\etA{{\etens{A}}}
|
||||
\def\etB{{\etens{B}}}
|
||||
\def\etC{{\etens{C}}}
|
||||
\def\etD{{\etens{D}}}
|
||||
\def\etE{{\etens{E}}}
|
||||
\def\etF{{\etens{F}}}
|
||||
\def\etG{{\etens{G}}}
|
||||
\def\etH{{\etens{H}}}
|
||||
\def\etI{{\etens{I}}}
|
||||
\def\etJ{{\etens{J}}}
|
||||
\def\etK{{\etens{K}}}
|
||||
\def\etL{{\etens{L}}}
|
||||
\def\etM{{\etens{M}}}
|
||||
\def\etN{{\etens{N}}}
|
||||
\def\etO{{\etens{O}}}
|
||||
\def\etP{{\etens{P}}}
|
||||
\def\etQ{{\etens{Q}}}
|
||||
\def\etR{{\etens{R}}}
|
||||
\def\etS{{\etens{S}}}
|
||||
\def\etT{{\etens{T}}}
|
||||
\def\etU{{\etens{U}}}
|
||||
\def\etV{{\etens{V}}}
|
||||
\def\etW{{\etens{W}}}
|
||||
\def\etX{{\etens{X}}}
|
||||
\def\etY{{\etens{Y}}}
|
||||
\def\etZ{{\etens{Z}}}
|
||||
|
||||
% The true underlying data generating distribution
|
||||
\newcommand{\pdata}{p_{\rm{data}}}
|
||||
% The empirical distribution defined by the training set
|
||||
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
|
||||
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
|
||||
% The model distribution
|
||||
\newcommand{\pmodel}{p_{\rm{model}}}
|
||||
\newcommand{\Pmodel}{P_{\rm{model}}}
|
||||
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
|
||||
% Stochastic autoencoder distributions
|
||||
\newcommand{\pencode}{p_{\rm{encoder}}}
|
||||
\newcommand{\pdecode}{p_{\rm{decoder}}}
|
||||
\newcommand{\precons}{p_{\rm{reconstruct}}}
|
||||
|
||||
\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution
|
||||
|
||||
\newcommand{\E}{\mathbb{E}}
|
||||
\newcommand{\Ls}{\mathcal{L}}
|
||||
\newcommand{\R}{\mathbb{R}}
|
||||
\newcommand{\emp}{\tilde{p}}
|
||||
\newcommand{\lr}{\alpha}
|
||||
\newcommand{\reg}{\lambda}
|
||||
\newcommand{\rect}{\mathrm{rectifier}}
|
||||
\newcommand{\softmax}{\mathrm{softmax}}
|
||||
\newcommand{\sigmoid}{\sigma}
|
||||
\newcommand{\softplus}{\zeta}
|
||||
\newcommand{\KL}{D_{\mathrm{KL}}}
|
||||
\newcommand{\Var}{\mathrm{Var}}
|
||||
\newcommand{\standarderror}{\mathrm{SE}}
|
||||
\newcommand{\Cov}{\mathrm{Cov}}
|
||||
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
|
||||
% But then they seem to use $L^2$ for vectors throughout the site, and so does
|
||||
% wikipedia.
|
||||
\newcommand{\normlzero}{L^0}
|
||||
\newcommand{\normlone}{L^1}
|
||||
\newcommand{\normltwo}{L^2}
|
||||
\newcommand{\normlp}{L^p}
|
||||
\newcommand{\normmax}{L^\infty}
|
||||
|
||||
\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.
|
||||
|
||||
\DeclareMathOperator*{\argmax}{arg\,max}
|
||||
\DeclareMathOperator*{\argmin}{arg\,min}
|
||||
|
||||
\DeclareMathOperator{\sign}{sign}
|
||||
\DeclareMathOperator{\Tr}{Tr}
|
||||
\let\ab\allowbreak
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,485 @@
|
||||
% fancyhdr.sty version 3.2
|
||||
% Fancy headers and footers for LaTeX.
|
||||
% Piet van Oostrum,
|
||||
% Dept of Computer and Information Sciences, University of Utrecht,
|
||||
% Padualaan 14, P.O. Box 80.089, 3508 TB Utrecht, The Netherlands
|
||||
% Telephone: +31 30 2532180. Email: piet@cs.uu.nl
|
||||
% ========================================================================
|
||||
% LICENCE:
|
||||
% This file may be distributed under the terms of the LaTeX Project Public
|
||||
% License, as described in lppl.txt in the base LaTeX distribution.
|
||||
% Either version 1 or, at your option, any later version.
|
||||
% ========================================================================
|
||||
% MODIFICATION HISTORY:
|
||||
% Sep 16, 1994
|
||||
% version 1.4: Correction for use with \reversemargin
|
||||
% Sep 29, 1994:
|
||||
% version 1.5: Added the \iftopfloat, \ifbotfloat and \iffloatpage commands
|
||||
% Oct 4, 1994:
|
||||
% version 1.6: Reset single spacing in headers/footers for use with
|
||||
% setspace.sty or doublespace.sty
|
||||
% Oct 4, 1994:
|
||||
% version 1.7: changed \let\@mkboth\markboth to
|
||||
% \def\@mkboth{\protect\markboth} to make it more robust
|
||||
% Dec 5, 1994:
|
||||
% version 1.8: corrections for amsbook/amsart: define \@chapapp and (more
|
||||
% importantly) use the \chapter/sectionmark definitions from ps@headings if
|
||||
% they exist (which should be true for all standard classes).
|
||||
% May 31, 1995:
|
||||
% version 1.9: The proposed \renewcommand{\headrulewidth}{\iffloatpage...
|
||||
% construction in the doc did not work properly with the fancyplain style.
|
||||
% June 1, 1995:
|
||||
% version 1.91: The definition of \@mkboth wasn't restored on subsequent
|
||||
% \pagestyle{fancy}'s.
|
||||
% June 1, 1995:
|
||||
% version 1.92: The sequence \pagestyle{fancyplain} \pagestyle{plain}
|
||||
% \pagestyle{fancy} would erroneously select the plain version.
|
||||
% June 1, 1995:
|
||||
% version 1.93: \fancypagestyle command added.
|
||||
% Dec 11, 1995:
|
||||
% version 1.94: suggested by Conrad Hughes <chughes@maths.tcd.ie>
|
||||
% CJCH, Dec 11, 1995: added \footruleskip to allow control over footrule
|
||||
% position (old hardcoded value of .3\normalbaselineskip is far too high
|
||||
% when used with very small footer fonts).
|
||||
% Jan 31, 1996:
|
||||
% version 1.95: call \@normalsize in the reset code if that is defined,
|
||||
% otherwise \normalsize.
|
||||
% this is to solve a problem with ucthesis.cls, as this doesn't
|
||||
% define \@currsize. Unfortunately for latex209 calling \normalsize doesn't
|
||||
% work as this is optimized to do very little, so there \@normalsize should
|
||||
% be called. Hopefully this code works for all versions of LaTeX known to
|
||||
% mankind.
|
||||
% April 25, 1996:
|
||||
% version 1.96: initialize \headwidth to a magic (negative) value to catch
|
||||
% most common cases that people change it before calling \pagestyle{fancy}.
|
||||
% Note it can't be initialized when reading in this file, because
|
||||
% \textwidth could be changed afterwards. This is quite probable.
|
||||
% We also switch to \MakeUppercase rather than \uppercase and introduce a
|
||||
% \nouppercase command for use in headers. and footers.
|
||||
% May 3, 1996:
|
||||
% version 1.97: Two changes:
|
||||
% 1. Undo the change in version 1.8 (using the pagestyle{headings} defaults
|
||||
% for the chapter and section marks. The current version of amsbook and
|
||||
% amsart classes don't seem to need them anymore. Moreover the standard
|
||||
% latex classes don't use \markboth if twoside isn't selected, and this is
|
||||
% confusing as \leftmark doesn't work as expected.
|
||||
% 2. include a call to \ps@empty in ps@@fancy. This is to solve a problem
|
||||
% in the amsbook and amsart classes, that make global changes to \topskip,
|
||||
% which are reset in \ps@empty. Hopefully this doesn't break other things.
|
||||
% May 7, 1996:
|
||||
% version 1.98:
|
||||
% Added % after the line \def\nouppercase
|
||||
% May 7, 1996:
|
||||
% version 1.99: This is the alpha version of fancyhdr 2.0
|
||||
% Introduced the new commands \fancyhead, \fancyfoot, and \fancyhf.
|
||||
% Changed \headrulewidth, \footrulewidth, \footruleskip to
|
||||
% macros rather than length parameters, In this way they can be
|
||||
% conditionalized and they don't consume length registers. There is no need
|
||||
% to have them as length registers unless you want to do calculations with
|
||||
% them, which is unlikely. Note that this may make some uses of them
|
||||
% incompatible (i.e. if you have a file that uses \setlength or \xxxx=)
|
||||
% May 10, 1996:
|
||||
% version 1.99a:
|
||||
% Added a few more % signs
|
||||
% May 10, 1996:
|
||||
% version 1.99b:
|
||||
% Changed the syntax of \f@nfor to be resistent to catcode changes of :=
|
||||
% Removed the [1] from the defs of \lhead etc. because the parameter is
|
||||
% consumed by the \@[xy]lhead etc. macros.
|
||||
% June 24, 1997:
|
||||
% version 1.99c:
|
||||
% corrected \nouppercase to also include the protected form of \MakeUppercase
|
||||
% \global added to manipulation of \headwidth.
|
||||
% \iffootnote command added.
|
||||
% Some comments added about \@fancyhead and \@fancyfoot.
|
||||
% Aug 24, 1998
|
||||
% version 1.99d
|
||||
% Changed the default \ps@empty to \ps@@empty in order to allow
|
||||
% \fancypagestyle{empty} redefinition.
|
||||
% Oct 11, 2000
|
||||
% version 2.0
|
||||
% Added LPPL license clause.
|
||||
%
|
||||
% A check for \headheight is added. An errormessage is given (once) if the
|
||||
% header is too large. Empty headers don't generate the error even if
|
||||
% \headheight is very small or even 0pt.
|
||||
% Warning added for the use of 'E' option when twoside option is not used.
|
||||
% In this case the 'E' fields will never be used.
|
||||
%
|
||||
% Mar 10, 2002
|
||||
% version 2.1beta
|
||||
% New command: \fancyhfoffset[place]{length}
|
||||
% defines offsets to be applied to the header/footer to let it stick into
|
||||
% the margins (if length > 0).
|
||||
% place is like in fancyhead, except that only E,O,L,R can be used.
|
||||
% This replaces the old calculation based on \headwidth and the marginpar
|
||||
% area.
|
||||
% \headwidth will be dynamically calculated in the headers/footers when
|
||||
% this is used.
|
||||
%
|
||||
% Mar 26, 2002
|
||||
% version 2.1beta2
|
||||
% \fancyhfoffset now also takes h,f as possible letters in the argument to
|
||||
% allow the header and footer widths to be different.
|
||||
% New commands \fancyheadoffset and \fancyfootoffset added comparable to
|
||||
% \fancyhead and \fancyfoot.
|
||||
% Errormessages and warnings have been made more informative.
|
||||
%
|
||||
% Dec 9, 2002
|
||||
% version 2.1
|
||||
% The defaults for \footrulewidth, \plainheadrulewidth and
|
||||
% \plainfootrulewidth are changed from \z@skip to 0pt. In this way when
|
||||
% someone inadvertantly uses \setlength to change any of these, the value
|
||||
% of \z@skip will not be changed, rather an errormessage will be given.
|
||||
|
||||
% March 3, 2004
|
||||
% Release of version 3.0
|
||||
|
||||
% Oct 7, 2004
|
||||
% version 3.1
|
||||
% Added '\endlinechar=13' to \fancy@reset to prevent problems with
|
||||
% includegraphics in header when verbatiminput is active.
|
||||
|
||||
% March 22, 2005
|
||||
% version 3.2
|
||||
% reset \everypar (the real one) in \fancy@reset because spanish.ldf does
|
||||
% strange things with \everypar between << and >>.
|
||||
|
||||
\def\ifancy@mpty#1{\def\temp@a{#1}\ifx\temp@a\@empty}
|
||||
|
||||
\def\fancy@def#1#2{\ifancy@mpty{#2}\fancy@gbl\def#1{\leavevmode}\else
|
||||
\fancy@gbl\def#1{#2\strut}\fi}
|
||||
|
||||
\let\fancy@gbl\global
|
||||
|
||||
\def\@fancyerrmsg#1{%
|
||||
\ifx\PackageError\undefined
|
||||
\errmessage{#1}\else
|
||||
\PackageError{Fancyhdr}{#1}{}\fi}
|
||||
\def\@fancywarning#1{%
|
||||
\ifx\PackageWarning\undefined
|
||||
\errmessage{#1}\else
|
||||
\PackageWarning{Fancyhdr}{#1}{}\fi}
|
||||
|
||||
% Usage: \@forc \var{charstring}{command to be executed for each char}
|
||||
% This is similar to LaTeX's \@tfor, but expands the charstring.
|
||||
|
||||
\def\@forc#1#2#3{\expandafter\f@rc\expandafter#1\expandafter{#2}{#3}}
|
||||
\def\f@rc#1#2#3{\def\temp@ty{#2}\ifx\@empty\temp@ty\else
|
||||
\f@@rc#1#2\f@@rc{#3}\fi}
|
||||
\def\f@@rc#1#2#3\f@@rc#4{\def#1{#2}#4\f@rc#1{#3}{#4}}
|
||||
|
||||
% Usage: \f@nfor\name:=list\do{body}
|
||||
% Like LaTeX's \@for but an empty list is treated as a list with an empty
|
||||
% element
|
||||
|
||||
\newcommand{\f@nfor}[3]{\edef\@fortmp{#2}%
|
||||
\expandafter\@forloop#2,\@nil,\@nil\@@#1{#3}}
|
||||
|
||||
% Usage: \def@ult \cs{defaults}{argument}
|
||||
% sets \cs to the characters from defaults appearing in argument
|
||||
% or defaults if it would be empty. All characters are lowercased.
|
||||
|
||||
\newcommand\def@ult[3]{%
|
||||
\edef\temp@a{\lowercase{\edef\noexpand\temp@a{#3}}}\temp@a
|
||||
\def#1{}%
|
||||
\@forc\tmpf@ra{#2}%
|
||||
{\expandafter\if@in\tmpf@ra\temp@a{\edef#1{#1\tmpf@ra}}{}}%
|
||||
\ifx\@empty#1\def#1{#2}\fi}
|
||||
%
|
||||
% \if@in <char><set><truecase><falsecase>
|
||||
%
|
||||
\newcommand{\if@in}[4]{%
|
||||
\edef\temp@a{#2}\def\temp@b##1#1##2\temp@b{\def\temp@b{##1}}%
|
||||
\expandafter\temp@b#2#1\temp@b\ifx\temp@a\temp@b #4\else #3\fi}
|
||||
|
||||
\newcommand{\fancyhead}{\@ifnextchar[{\f@ncyhf\fancyhead h}%
|
||||
{\f@ncyhf\fancyhead h[]}}
|
||||
\newcommand{\fancyfoot}{\@ifnextchar[{\f@ncyhf\fancyfoot f}%
|
||||
{\f@ncyhf\fancyfoot f[]}}
|
||||
\newcommand{\fancyhf}{\@ifnextchar[{\f@ncyhf\fancyhf{}}%
|
||||
{\f@ncyhf\fancyhf{}[]}}
|
||||
|
||||
% New commands for offsets added
|
||||
|
||||
\newcommand{\fancyheadoffset}{\@ifnextchar[{\f@ncyhfoffs\fancyheadoffset h}%
|
||||
{\f@ncyhfoffs\fancyheadoffset h[]}}
|
||||
\newcommand{\fancyfootoffset}{\@ifnextchar[{\f@ncyhfoffs\fancyfootoffset f}%
|
||||
{\f@ncyhfoffs\fancyfootoffset f[]}}
|
||||
\newcommand{\fancyhfoffset}{\@ifnextchar[{\f@ncyhfoffs\fancyhfoffset{}}%
|
||||
{\f@ncyhfoffs\fancyhfoffset{}[]}}
|
||||
|
||||
% The header and footer fields are stored in command sequences with
|
||||
% names of the form: \f@ncy<x><y><z> with <x> for [eo], <y> from [lcr]
|
||||
% and <z> from [hf].
|
||||
|
||||
\def\f@ncyhf#1#2[#3]#4{%
|
||||
\def\temp@c{}%
|
||||
\@forc\tmpf@ra{#3}%
|
||||
{\expandafter\if@in\tmpf@ra{eolcrhf,EOLCRHF}%
|
||||
{}{\edef\temp@c{\temp@c\tmpf@ra}}}%
|
||||
\ifx\@empty\temp@c\else
|
||||
\@fancyerrmsg{Illegal char `\temp@c' in \string#1 argument:
|
||||
[#3]}%
|
||||
\fi
|
||||
\f@nfor\temp@c{#3}%
|
||||
{\def@ult\f@@@eo{eo}\temp@c
|
||||
\if@twoside\else
|
||||
\if\f@@@eo e\@fancywarning
|
||||
{\string#1's `E' option without twoside option is useless}\fi\fi
|
||||
\def@ult\f@@@lcr{lcr}\temp@c
|
||||
\def@ult\f@@@hf{hf}{#2\temp@c}%
|
||||
\@forc\f@@eo\f@@@eo
|
||||
{\@forc\f@@lcr\f@@@lcr
|
||||
{\@forc\f@@hf\f@@@hf
|
||||
{\expandafter\fancy@def\csname
|
||||
f@ncy\f@@eo\f@@lcr\f@@hf\endcsname
|
||||
{#4}}}}}}
|
||||
|
||||
\def\f@ncyhfoffs#1#2[#3]#4{%
|
||||
\def\temp@c{}%
|
||||
\@forc\tmpf@ra{#3}%
|
||||
{\expandafter\if@in\tmpf@ra{eolrhf,EOLRHF}%
|
||||
{}{\edef\temp@c{\temp@c\tmpf@ra}}}%
|
||||
\ifx\@empty\temp@c\else
|
||||
\@fancyerrmsg{Illegal char `\temp@c' in \string#1 argument:
|
||||
[#3]}%
|
||||
\fi
|
||||
\f@nfor\temp@c{#3}%
|
||||
{\def@ult\f@@@eo{eo}\temp@c
|
||||
\if@twoside\else
|
||||
\if\f@@@eo e\@fancywarning
|
||||
{\string#1's `E' option without twoside option is useless}\fi\fi
|
||||
\def@ult\f@@@lcr{lr}\temp@c
|
||||
\def@ult\f@@@hf{hf}{#2\temp@c}%
|
||||
\@forc\f@@eo\f@@@eo
|
||||
{\@forc\f@@lcr\f@@@lcr
|
||||
{\@forc\f@@hf\f@@@hf
|
||||
{\expandafter\setlength\csname
|
||||
f@ncyO@\f@@eo\f@@lcr\f@@hf\endcsname
|
||||
{#4}}}}}%
|
||||
\fancy@setoffs}
|
||||
|
||||
% Fancyheadings version 1 commands. These are more or less deprecated,
|
||||
% but they continue to work.
|
||||
|
||||
\newcommand{\lhead}{\@ifnextchar[{\@xlhead}{\@ylhead}}
|
||||
\def\@xlhead[#1]#2{\fancy@def\f@ncyelh{#1}\fancy@def\f@ncyolh{#2}}
|
||||
\def\@ylhead#1{\fancy@def\f@ncyelh{#1}\fancy@def\f@ncyolh{#1}}
|
||||
|
||||
\newcommand{\chead}{\@ifnextchar[{\@xchead}{\@ychead}}
|
||||
\def\@xchead[#1]#2{\fancy@def\f@ncyech{#1}\fancy@def\f@ncyoch{#2}}
|
||||
\def\@ychead#1{\fancy@def\f@ncyech{#1}\fancy@def\f@ncyoch{#1}}
|
||||
|
||||
\newcommand{\rhead}{\@ifnextchar[{\@xrhead}{\@yrhead}}
|
||||
\def\@xrhead[#1]#2{\fancy@def\f@ncyerh{#1}\fancy@def\f@ncyorh{#2}}
|
||||
\def\@yrhead#1{\fancy@def\f@ncyerh{#1}\fancy@def\f@ncyorh{#1}}
|
||||
|
||||
\newcommand{\lfoot}{\@ifnextchar[{\@xlfoot}{\@ylfoot}}
|
||||
\def\@xlfoot[#1]#2{\fancy@def\f@ncyelf{#1}\fancy@def\f@ncyolf{#2}}
|
||||
\def\@ylfoot#1{\fancy@def\f@ncyelf{#1}\fancy@def\f@ncyolf{#1}}
|
||||
|
||||
\newcommand{\cfoot}{\@ifnextchar[{\@xcfoot}{\@ycfoot}}
|
||||
\def\@xcfoot[#1]#2{\fancy@def\f@ncyecf{#1}\fancy@def\f@ncyocf{#2}}
|
||||
\def\@ycfoot#1{\fancy@def\f@ncyecf{#1}\fancy@def\f@ncyocf{#1}}
|
||||
|
||||
\newcommand{\rfoot}{\@ifnextchar[{\@xrfoot}{\@yrfoot}}
|
||||
\def\@xrfoot[#1]#2{\fancy@def\f@ncyerf{#1}\fancy@def\f@ncyorf{#2}}
|
||||
\def\@yrfoot#1{\fancy@def\f@ncyerf{#1}\fancy@def\f@ncyorf{#1}}
|
||||
|
||||
\newlength{\fancy@headwidth}
|
||||
\let\headwidth\fancy@headwidth
|
||||
\newlength{\f@ncyO@elh}
|
||||
\newlength{\f@ncyO@erh}
|
||||
\newlength{\f@ncyO@olh}
|
||||
\newlength{\f@ncyO@orh}
|
||||
\newlength{\f@ncyO@elf}
|
||||
\newlength{\f@ncyO@erf}
|
||||
\newlength{\f@ncyO@olf}
|
||||
\newlength{\f@ncyO@orf}
|
||||
\newcommand{\headrulewidth}{0.4pt}
|
||||
\newcommand{\footrulewidth}{0pt}
|
||||
\newcommand{\footruleskip}{.3\normalbaselineskip}
|
||||
|
||||
% Fancyplain stuff shouldn't be used anymore (rather
|
||||
% \fancypagestyle{plain} should be used), but it must be present for
|
||||
% compatibility reasons.
|
||||
|
||||
\newcommand{\plainheadrulewidth}{0pt}
|
||||
\newcommand{\plainfootrulewidth}{0pt}
|
||||
\newif\if@fancyplain \@fancyplainfalse
|
||||
\def\fancyplain#1#2{\if@fancyplain#1\else#2\fi}
|
||||
|
||||
\headwidth=-123456789sp %magic constant
|
||||
|
||||
% Command to reset various things in the headers:
|
||||
% a.o. single spacing (taken from setspace.sty)
|
||||
% and the catcode of ^^M (so that epsf files in the header work if a
|
||||
% verbatim crosses a page boundary)
|
||||
% It also defines a \nouppercase command that disables \uppercase and
|
||||
% \Makeuppercase. It can only be used in the headers and footers.
|
||||
\let\fnch@everypar\everypar% save real \everypar because of spanish.ldf
|
||||
\def\fancy@reset{\fnch@everypar{}\restorecr\endlinechar=13
|
||||
\def\baselinestretch{1}%
|
||||
\def\nouppercase##1{{\let\uppercase\relax\let\MakeUppercase\relax
|
||||
\expandafter\let\csname MakeUppercase \endcsname\relax##1}}%
|
||||
\ifx\undefined\@newbaseline% NFSS not present; 2.09 or 2e
|
||||
\ifx\@normalsize\undefined \normalsize % for ucthesis.cls
|
||||
\else \@normalsize \fi
|
||||
\else% NFSS (2.09) present
|
||||
\@newbaseline%
|
||||
\fi}
|
||||
|
||||
% Initialization of the head and foot text.
|
||||
|
||||
% The default values still contain \fancyplain for compatibility.
|
||||
\fancyhf{} % clear all
|
||||
% lefthead empty on ``plain'' pages, \rightmark on even, \leftmark on odd pages
|
||||
% evenhead empty on ``plain'' pages, \leftmark on even, \rightmark on odd pages
|
||||
\if@twoside
|
||||
\fancyhead[el,or]{\fancyplain{}{\sl\rightmark}}
|
||||
\fancyhead[er,ol]{\fancyplain{}{\sl\leftmark}}
|
||||
\else
|
||||
\fancyhead[l]{\fancyplain{}{\sl\rightmark}}
|
||||
\fancyhead[r]{\fancyplain{}{\sl\leftmark}}
|
||||
\fi
|
||||
\fancyfoot[c]{\rm\thepage} % page number
|
||||
|
||||
% Use box 0 as a temp box and dimen 0 as temp dimen.
|
||||
% This can be done, because this code will always
|
||||
% be used inside another box, and therefore the changes are local.
|
||||
|
||||
\def\@fancyvbox#1#2{\setbox0\vbox{#2}\ifdim\ht0>#1\@fancywarning
|
||||
{\string#1 is too small (\the#1): ^^J Make it at least \the\ht0.^^J
|
||||
We now make it that large for the rest of the document.^^J
|
||||
This may cause the page layout to be inconsistent, however\@gobble}%
|
||||
\dimen0=#1\global\setlength{#1}{\ht0}\ht0=\dimen0\fi
|
||||
\box0}
|
||||
|
||||
% Put together a header or footer given the left, center and
|
||||
% right text, fillers at left and right and a rule.
|
||||
% The \lap commands put the text into an hbox of zero size,
|
||||
% so overlapping text does not generate an errormessage.
|
||||
% These macros have 5 parameters:
|
||||
% 1. LEFTSIDE BEARING % This determines at which side the header will stick
|
||||
% out. When \fancyhfoffset is used this calculates \headwidth, otherwise
|
||||
% it is \hss or \relax (after expansion).
|
||||
% 2. \f@ncyolh, \f@ncyelh, \f@ncyolf or \f@ncyelf. This is the left component.
|
||||
% 3. \f@ncyoch, \f@ncyech, \f@ncyocf or \f@ncyecf. This is the middle comp.
|
||||
% 4. \f@ncyorh, \f@ncyerh, \f@ncyorf or \f@ncyerf. This is the right component.
|
||||
% 5. RIGHTSIDE BEARING. This is always \relax or \hss (after expansion).
|
||||
|
||||
\def\@fancyhead#1#2#3#4#5{#1\hbox to\headwidth{\fancy@reset
|
||||
\@fancyvbox\headheight{\hbox
|
||||
{\rlap{\parbox[b]{\headwidth}{\raggedright#2}}\hfill
|
||||
\parbox[b]{\headwidth}{\centering#3}\hfill
|
||||
\llap{\parbox[b]{\headwidth}{\raggedleft#4}}}\headrule}}#5}
|
||||
|
||||
\def\@fancyfoot#1#2#3#4#5{#1\hbox to\headwidth{\fancy@reset
|
||||
\@fancyvbox\footskip{\footrule
|
||||
\hbox{\rlap{\parbox[t]{\headwidth}{\raggedright#2}}\hfill
|
||||
\parbox[t]{\headwidth}{\centering#3}\hfill
|
||||
\llap{\parbox[t]{\headwidth}{\raggedleft#4}}}}}#5}
|
||||
|
||||
\def\headrule{{\if@fancyplain\let\headrulewidth\plainheadrulewidth\fi
|
||||
\hrule\@height\headrulewidth\@width\headwidth \vskip-\headrulewidth}}
|
||||
|
||||
\def\footrule{{\if@fancyplain\let\footrulewidth\plainfootrulewidth\fi
|
||||
\vskip-\footruleskip\vskip-\footrulewidth
|
||||
\hrule\@width\headwidth\@height\footrulewidth\vskip\footruleskip}}
|
||||
|
||||
\def\ps@fancy{%
|
||||
\@ifundefined{@chapapp}{\let\@chapapp\chaptername}{}%for amsbook
|
||||
%
|
||||
% Define \MakeUppercase for old LaTeXen.
|
||||
% Note: we used \def rather than \let, so that \let\uppercase\relax (from
|
||||
% the version 1 documentation) will still work.
|
||||
%
|
||||
\@ifundefined{MakeUppercase}{\def\MakeUppercase{\uppercase}}{}%
|
||||
\@ifundefined{chapter}{\def\sectionmark##1{\markboth
|
||||
{\MakeUppercase{\ifnum \c@secnumdepth>\z@
|
||||
\thesection\hskip 1em\relax \fi ##1}}{}}%
|
||||
\def\subsectionmark##1{\markright {\ifnum \c@secnumdepth >\@ne
|
||||
\thesubsection\hskip 1em\relax \fi ##1}}}%
|
||||
{\def\chaptermark##1{\markboth {\MakeUppercase{\ifnum \c@secnumdepth>\m@ne
|
||||
\@chapapp\ \thechapter. \ \fi ##1}}{}}%
|
||||
\def\sectionmark##1{\markright{\MakeUppercase{\ifnum \c@secnumdepth >\z@
|
||||
\thesection. \ \fi ##1}}}}%
|
||||
%\csname ps@headings\endcsname % use \ps@headings defaults if they exist
|
||||
\ps@@fancy
|
||||
\gdef\ps@fancy{\@fancyplainfalse\ps@@fancy}%
|
||||
% Initialize \headwidth if the user didn't
|
||||
%
|
||||
\ifdim\headwidth<0sp
|
||||
%
|
||||
% This catches the case that \headwidth hasn't been initialized and the
|
||||
% case that the user added something to \headwidth in the expectation that
|
||||
% it was initialized to \textwidth. We compensate this now. This loses if
|
||||
% the user intended to multiply it by a factor. But that case is more
|
||||
% likely done by saying something like \headwidth=1.2\textwidth.
|
||||
% The doc says you have to change \headwidth after the first call to
|
||||
% \pagestyle{fancy}. This code is just to catch the most common cases were
|
||||
% that requirement is violated.
|
||||
%
|
||||
\global\advance\headwidth123456789sp\global\advance\headwidth\textwidth
|
||||
\fi}
|
||||
\def\ps@fancyplain{\ps@fancy \let\ps@plain\ps@plain@fancy}
|
||||
\def\ps@plain@fancy{\@fancyplaintrue\ps@@fancy}
|
||||
\let\ps@@empty\ps@empty
|
||||
\def\ps@@fancy{%
|
||||
\ps@@empty % This is for amsbook/amsart, which do strange things with \topskip
|
||||
\def\@mkboth{\protect\markboth}%
|
||||
\def\@oddhead{\@fancyhead\fancy@Oolh\f@ncyolh\f@ncyoch\f@ncyorh\fancy@Oorh}%
|
||||
\def\@oddfoot{\@fancyfoot\fancy@Oolf\f@ncyolf\f@ncyocf\f@ncyorf\fancy@Oorf}%
|
||||
\def\@evenhead{\@fancyhead\fancy@Oelh\f@ncyelh\f@ncyech\f@ncyerh\fancy@Oerh}%
|
||||
\def\@evenfoot{\@fancyfoot\fancy@Oelf\f@ncyelf\f@ncyecf\f@ncyerf\fancy@Oerf}%
|
||||
}
|
||||
% Default definitions for compatibility mode:
|
||||
% These cause the header/footer to take the defined \headwidth as width
|
||||
% And to shift in the direction of the marginpar area
|
||||
|
||||
\def\fancy@Oolh{\if@reversemargin\hss\else\relax\fi}
|
||||
\def\fancy@Oorh{\if@reversemargin\relax\else\hss\fi}
|
||||
\let\fancy@Oelh\fancy@Oorh
|
||||
\let\fancy@Oerh\fancy@Oolh
|
||||
|
||||
\let\fancy@Oolf\fancy@Oolh
|
||||
\let\fancy@Oorf\fancy@Oorh
|
||||
\let\fancy@Oelf\fancy@Oelh
|
||||
\let\fancy@Oerf\fancy@Oerh
|
||||
|
||||
% New definitions for the use of \fancyhfoffset
|
||||
% These calculate the \headwidth from \textwidth and the specified offsets.
|
||||
|
||||
\def\fancy@offsolh{\headwidth=\textwidth\advance\headwidth\f@ncyO@olh
|
||||
\advance\headwidth\f@ncyO@orh\hskip-\f@ncyO@olh}
|
||||
\def\fancy@offselh{\headwidth=\textwidth\advance\headwidth\f@ncyO@elh
|
||||
\advance\headwidth\f@ncyO@erh\hskip-\f@ncyO@elh}
|
||||
|
||||
\def\fancy@offsolf{\headwidth=\textwidth\advance\headwidth\f@ncyO@olf
|
||||
\advance\headwidth\f@ncyO@orf\hskip-\f@ncyO@olf}
|
||||
\def\fancy@offself{\headwidth=\textwidth\advance\headwidth\f@ncyO@elf
|
||||
\advance\headwidth\f@ncyO@erf\hskip-\f@ncyO@elf}
|
||||
|
||||
\def\fancy@setoffs{%
|
||||
% Just in case \let\headwidth\textwidth was used
|
||||
\fancy@gbl\let\headwidth\fancy@headwidth
|
||||
\fancy@gbl\let\fancy@Oolh\fancy@offsolh
|
||||
\fancy@gbl\let\fancy@Oelh\fancy@offselh
|
||||
\fancy@gbl\let\fancy@Oorh\hss
|
||||
\fancy@gbl\let\fancy@Oerh\hss
|
||||
\fancy@gbl\let\fancy@Oolf\fancy@offsolf
|
||||
\fancy@gbl\let\fancy@Oelf\fancy@offself
|
||||
\fancy@gbl\let\fancy@Oorf\hss
|
||||
\fancy@gbl\let\fancy@Oerf\hss}
|
||||
|
||||
\newif\iffootnote
|
||||
\let\latex@makecol\@makecol
|
||||
\def\@makecol{\ifvoid\footins\footnotetrue\else\footnotefalse\fi
|
||||
\let\topfloat\@toplist\let\botfloat\@botlist\latex@makecol}
|
||||
\def\iftopfloat#1#2{\ifx\topfloat\empty #2\else #1\fi}
|
||||
\def\ifbotfloat#1#2{\ifx\botfloat\empty #2\else #1\fi}
|
||||
\def\iffloatpage#1#2{\if@fcolmade #1\else #2\fi}
|
||||
|
||||
\newcommand{\fancypagestyle}[2]{%
|
||||
\@namedef{ps@#1}{\let\fancy@gbl\relax#2\relax\ps@fancy}}
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
@incollection{Bengio+chapter2007,
|
||||
author = {Bengio, Yoshua and LeCun, Yann},
|
||||
booktitle = {Large Scale Kernel Machines},
|
||||
publisher = {MIT Press},
|
||||
title = {Scaling Learning Algorithms Towards {AI}},
|
||||
year = {2007}
|
||||
}
|
||||
|
||||
@article{Hinton06,
|
||||
author = {Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye},
|
||||
journal = {Neural Computation},
|
||||
pages = {1527--1554},
|
||||
title = {A Fast Learning Algorithm for Deep Belief Nets},
|
||||
volume = {18},
|
||||
year = {2006}
|
||||
}
|
||||
|
||||
@book{goodfellow2016deep,
|
||||
title={Deep learning},
|
||||
author={Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron and Bengio, Yoshua},
|
||||
volume={1},
|
||||
year={2016},
|
||||
publisher={MIT Press}
|
||||
}
|
||||
+1440
File diff suppressed because it is too large
Load Diff
BIN
Binary file not shown.
+246
@@ -0,0 +1,246 @@
|
||||
%%%% ICLR Macros (LaTex)
|
||||
%%%% Adapted by Hugo Larochelle from the NIPS stylefile Macros
|
||||
%%%% Style File
|
||||
%%%% Dec 12, 1990 Rev Aug 14, 1991; Sept, 1995; April, 1997; April, 1999; October 2014
|
||||
|
||||
% This file can be used with Latex2e whether running in main mode, or
|
||||
% 2.09 compatibility mode.
|
||||
%
|
||||
% If using main mode, you need to include the commands
|
||||
% \documentclass{article}
|
||||
% \usepackage{iclr14submit_e,times}
|
||||
%
|
||||
|
||||
% Change the overall width of the page. If these parameters are
|
||||
% changed, they will require corresponding changes in the
|
||||
% maketitle section.
|
||||
%
|
||||
\usepackage{eso-pic} % used by \AddToShipoutPicture
|
||||
\RequirePackage{fancyhdr}
|
||||
\RequirePackage{natbib}
|
||||
|
||||
% modification to natbib citations
|
||||
\setcitestyle{authoryear,round,citesep={;},aysep={,},yysep={;}}
|
||||
|
||||
\renewcommand{\topfraction}{0.95} % let figure take up nearly whole page
|
||||
\renewcommand{\textfraction}{0.05} % let figure take up nearly whole page
|
||||
|
||||
% Define iclrfinal, set to true if iclrfinalcopy is defined
|
||||
\newif\ificlrfinal
|
||||
\iclrfinalfalse
|
||||
\def\iclrfinalcopy{\iclrfinaltrue}
|
||||
\font\iclrtenhv = phvb at 8pt
|
||||
|
||||
% Specify the dimensions of each page
|
||||
|
||||
\setlength{\paperheight}{11in}
|
||||
\setlength{\paperwidth}{8.5in}
|
||||
|
||||
|
||||
\oddsidemargin .5in % Note \oddsidemargin = \evensidemargin
|
||||
\evensidemargin .5in
|
||||
\marginparwidth 0.07 true in
|
||||
%\marginparwidth 0.75 true in
|
||||
%\topmargin 0 true pt % Nominal distance from top of page to top of
|
||||
%\topmargin 0.125in
|
||||
\topmargin -0.625in
|
||||
\addtolength{\headsep}{0.25in}
|
||||
\textheight 9.0 true in % Height of text (including footnotes & figures)
|
||||
\textwidth 5.5 true in % Width of text line.
|
||||
\widowpenalty=10000
|
||||
\clubpenalty=10000
|
||||
|
||||
% \thispagestyle{empty} \pagestyle{empty}
|
||||
\flushbottom \sloppy
|
||||
|
||||
% We're never going to need a table of contents, so just flush it to
|
||||
% save space --- suggested by drstrip@sandia-2
|
||||
\def\addcontentsline#1#2#3{}
|
||||
|
||||
% Title stuff, taken from deproc.
|
||||
\def\maketitle{\par
|
||||
\begingroup
|
||||
\def\thefootnote{\fnsymbol{footnote}}
|
||||
\def\@makefnmark{\hbox to 0pt{$^{\@thefnmark}$\hss}} % for perfect author
|
||||
% name centering
|
||||
% The footnote-mark was overlapping the footnote-text,
|
||||
% added the following to fix this problem (MK)
|
||||
\long\def\@makefntext##1{\parindent 1em\noindent
|
||||
\hbox to1.8em{\hss $\m@th ^{\@thefnmark}$}##1}
|
||||
\@maketitle \@thanks
|
||||
\endgroup
|
||||
\setcounter{footnote}{0}
|
||||
\let\maketitle\relax \let\@maketitle\relax
|
||||
\gdef\@thanks{}\gdef\@author{}\gdef\@title{}\let\thanks\relax}
|
||||
|
||||
% The toptitlebar has been raised to top-justify the first page
|
||||
|
||||
\usepackage{fancyhdr}
|
||||
\pagestyle{fancy}
|
||||
\fancyhead{}
|
||||
|
||||
% Title (includes both anonimized and non-anonimized versions)
|
||||
\def\@maketitle{\vbox{\hsize\textwidth
|
||||
%\linewidth\hsize \vskip 0.1in \toptitlebar \centering
|
||||
{\LARGE\sc \@title\par}
|
||||
%\bottomtitlebar % \vskip 0.1in % minus
|
||||
\ificlrfinal
|
||||
\lhead{Published as a conference paper at ICLR 2026}
|
||||
\def\And{\end{tabular}\hfil\linebreak[0]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\def\AND{\end{tabular}\hfil\linebreak[4]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\@author\end{tabular}%
|
||||
\else
|
||||
\lhead{Under review as a conference paper at ICLR 2026}
|
||||
\def\And{\end{tabular}\hfil\linebreak[0]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\def\AND{\end{tabular}\hfil\linebreak[4]\hfil
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}\ignorespaces}%
|
||||
\begin{tabular}[t]{l}\bf\rule{\z@}{24pt}Anonymous authors\\Paper under double-blind review\end{tabular}%
|
||||
\fi
|
||||
\vskip 0.3in minus 0.1in}}
|
||||
|
||||
\renewenvironment{abstract}{\vskip.075in\centerline{\large\sc
|
||||
Abstract}\vspace{0.5ex}\begin{quote}}{\par\end{quote}\vskip 1ex}
|
||||
|
||||
% sections with less space
|
||||
\def\section{\@startsection {section}{1}{\z@}{-2.0ex plus
|
||||
-0.5ex minus -.2ex}{1.5ex plus 0.3ex
|
||||
minus0.2ex}{\large\sc\raggedright}}
|
||||
|
||||
\def\subsection{\@startsection{subsection}{2}{\z@}{-1.8ex plus
|
||||
-0.5ex minus -.2ex}{0.8ex plus .2ex}{\normalsize\sc\raggedright}}
|
||||
\def\subsubsection{\@startsection{subsubsection}{3}{\z@}{-1.5ex
|
||||
plus -0.5ex minus -.2ex}{0.5ex plus
|
||||
.2ex}{\normalsize\sc\raggedright}}
|
||||
\def\paragraph{\@startsection{paragraph}{4}{\z@}{1.5ex plus
|
||||
0.5ex minus .2ex}{-1em}{\normalsize\bf}}
|
||||
\def\subparagraph{\@startsection{subparagraph}{5}{\z@}{1.5ex plus
|
||||
0.5ex minus .2ex}{-1em}{\normalsize\sc}}
|
||||
\def\subsubsubsection{\vskip
|
||||
5pt{\noindent\normalsize\rm\raggedright}}
|
||||
|
||||
|
||||
% Footnotes
|
||||
\footnotesep 6.65pt %
|
||||
\skip\footins 9pt plus 4pt minus 2pt
|
||||
\def\footnoterule{\kern-3pt \hrule width 12pc \kern 2.6pt }
|
||||
\setcounter{footnote}{0}
|
||||
|
||||
% Lists and paragraphs
|
||||
\parindent 0pt
|
||||
\topsep 4pt plus 1pt minus 2pt
|
||||
\partopsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep 2pt plus 1pt minus 0.5pt
|
||||
\parsep 2pt plus 1pt minus 0.5pt
|
||||
\parskip .5pc
|
||||
|
||||
|
||||
%\leftmargin2em
|
||||
\leftmargin3pc
|
||||
\leftmargini\leftmargin \leftmarginii 2em
|
||||
\leftmarginiii 1.5em \leftmarginiv 1.0em \leftmarginv .5em
|
||||
|
||||
%\labelsep \labelsep 5pt
|
||||
|
||||
\def\@listi{\leftmargin\leftmargini}
|
||||
\def\@listii{\leftmargin\leftmarginii
|
||||
\labelwidth\leftmarginii\advance\labelwidth-\labelsep
|
||||
\topsep 2pt plus 1pt minus 0.5pt
|
||||
\parsep 1pt plus 0.5pt minus 0.5pt
|
||||
\itemsep \parsep}
|
||||
\def\@listiii{\leftmargin\leftmarginiii
|
||||
\labelwidth\leftmarginiii\advance\labelwidth-\labelsep
|
||||
\topsep 1pt plus 0.5pt minus 0.5pt
|
||||
\parsep \z@ \partopsep 0.5pt plus 0pt minus 0.5pt
|
||||
\itemsep \topsep}
|
||||
\def\@listiv{\leftmargin\leftmarginiv
|
||||
\labelwidth\leftmarginiv\advance\labelwidth-\labelsep}
|
||||
\def\@listv{\leftmargin\leftmarginv
|
||||
\labelwidth\leftmarginv\advance\labelwidth-\labelsep}
|
||||
\def\@listvi{\leftmargin\leftmarginvi
|
||||
\labelwidth\leftmarginvi\advance\labelwidth-\labelsep}
|
||||
|
||||
\abovedisplayskip 7pt plus2pt minus5pt%
|
||||
\belowdisplayskip \abovedisplayskip
|
||||
\abovedisplayshortskip 0pt plus3pt%
|
||||
\belowdisplayshortskip 4pt plus3pt minus3pt%
|
||||
|
||||
% Less leading in most fonts (due to the narrow columns)
|
||||
% The choices were between 1-pt and 1.5-pt leading
|
||||
%\def\@normalsize{\@setsize\normalsize{11pt}\xpt\@xpt} % got rid of @ (MK)
|
||||
\def\normalsize{\@setsize\normalsize{11pt}\xpt\@xpt}
|
||||
\def\small{\@setsize\small{10pt}\ixpt\@ixpt}
|
||||
\def\footnotesize{\@setsize\footnotesize{10pt}\ixpt\@ixpt}
|
||||
\def\scriptsize{\@setsize\scriptsize{8pt}\viipt\@viipt}
|
||||
\def\tiny{\@setsize\tiny{7pt}\vipt\@vipt}
|
||||
\def\large{\@setsize\large{14pt}\xiipt\@xiipt}
|
||||
\def\Large{\@setsize\Large{16pt}\xivpt\@xivpt}
|
||||
\def\LARGE{\@setsize\LARGE{20pt}\xviipt\@xviipt}
|
||||
\def\huge{\@setsize\huge{23pt}\xxpt\@xxpt}
|
||||
\def\Huge{\@setsize\Huge{28pt}\xxvpt\@xxvpt}
|
||||
|
||||
\def\toptitlebar{\hrule height4pt\vskip .25in\vskip-\parskip}
|
||||
|
||||
\def\bottomtitlebar{\vskip .29in\vskip-\parskip\hrule height1pt\vskip
|
||||
.09in} %
|
||||
%Reduced second vskip to compensate for adding the strut in \@author
|
||||
|
||||
|
||||
|
||||
%% % Vertical Ruler
|
||||
%% % This code is, largely, from the CVPR 2010 conference style file
|
||||
%% % ----- define vruler
|
||||
\makeatletter
|
||||
\newbox\iclrrulerbox
|
||||
\newcount\iclrrulercount
|
||||
\newdimen\iclrruleroffset
|
||||
\newdimen\cv@lineheight
|
||||
\newdimen\cv@boxheight
|
||||
\newbox\cv@tmpbox
|
||||
\newcount\cv@refno
|
||||
\newcount\cv@tot
|
||||
% NUMBER with left flushed zeros \fillzeros[<WIDTH>]<NUMBER>
|
||||
\newcount\cv@tmpc@ \newcount\cv@tmpc
|
||||
\def\fillzeros[#1]#2{\cv@tmpc@=#2\relax\ifnum\cv@tmpc@<0\cv@tmpc@=-\cv@tmpc@\fi
|
||||
\cv@tmpc=1 %
|
||||
\loop\ifnum\cv@tmpc@<10 \else \divide\cv@tmpc@ by 10 \advance\cv@tmpc by 1 \fi
|
||||
\ifnum\cv@tmpc@=10\relax\cv@tmpc@=11\relax\fi \ifnum\cv@tmpc@>10 \repeat
|
||||
\ifnum#2<0\advance\cv@tmpc1\relax-\fi
|
||||
\loop\ifnum\cv@tmpc<#1\relax0\advance\cv@tmpc1\relax\fi \ifnum\cv@tmpc<#1 \repeat
|
||||
\cv@tmpc@=#2\relax\ifnum\cv@tmpc@<0\cv@tmpc@=-\cv@tmpc@\fi \relax\the\cv@tmpc@}%
|
||||
% \makevruler[<SCALE>][<INITIAL_COUNT>][<STEP>][<DIGITS>][<HEIGHT>]
|
||||
\def\makevruler[#1][#2][#3][#4][#5]{\begingroup\offinterlineskip
|
||||
\textheight=#5\vbadness=10000\vfuzz=120ex\overfullrule=0pt%
|
||||
\global\setbox\iclrrulerbox=\vbox to \textheight{%
|
||||
{\parskip=0pt\hfuzz=150em\cv@boxheight=\textheight
|
||||
\cv@lineheight=#1\global\iclrrulercount=#2%
|
||||
\cv@tot\cv@boxheight\divide\cv@tot\cv@lineheight\advance\cv@tot2%
|
||||
\cv@refno1\vskip-\cv@lineheight\vskip1ex%
|
||||
\loop\setbox\cv@tmpbox=\hbox to0cm{{\iclrtenhv\hfil\fillzeros[#4]\iclrrulercount}}%
|
||||
\ht\cv@tmpbox\cv@lineheight\dp\cv@tmpbox0pt\box\cv@tmpbox\break
|
||||
\advance\cv@refno1\global\advance\iclrrulercount#3\relax
|
||||
\ifnum\cv@refno<\cv@tot\repeat}}\endgroup}%
|
||||
\makeatother
|
||||
% ----- end of vruler
|
||||
|
||||
% \makevruler[<SCALE>][<INITIAL_COUNT>][<STEP>][<DIGITS>][<HEIGHT>]
|
||||
\def\iclrruler#1{\makevruler[12pt][#1][1][3][0.993\textheight]\usebox{\iclrrulerbox}}
|
||||
\AddToShipoutPicture{%
|
||||
\ificlrfinal\else
|
||||
\iclrruleroffset=\textheight
|
||||
\advance\iclrruleroffset by -3.7pt
|
||||
\color[rgb]{.7,.7,.7}
|
||||
\AtTextUpperLeft{%
|
||||
\put(\LenToUnit{-35pt},\LenToUnit{-\iclrruleroffset}){%left ruler
|
||||
\iclrruler{\iclrrulercount}}
|
||||
}
|
||||
\fi
|
||||
}
|
||||
% %% To add a vertical bar on the side
|
||||
% \AddToShipoutPicture{
|
||||
% \AtTextLowerLeft{
|
||||
% \hspace*{-1.8cm}
|
||||
% \colorbox[rgb]{0.7,0.7,0.7}{\small \parbox[b][\textheight]{0.1cm}{}}}
|
||||
% }
|
||||
+414
@@ -0,0 +1,414 @@
|
||||
|
||||
\documentclass{article} % For LaTeX2e
|
||||
\usepackage{iclr2026_conference,times}
|
||||
|
||||
% Optional math commands from https://github.com/goodfeli/dlbook_notation.
|
||||
\input{math_commands.tex}
|
||||
|
||||
\usepackage{hyperref}
|
||||
\usepackage{url}
|
||||
|
||||
|
||||
\title{Formatting Instructions for ICLR 2026 \\ Conference Submissions}
|
||||
|
||||
% Authors must not appear in the submitted version. They should be hidden
|
||||
% as long as the \iclrfinalcopy macro remains commented out below.
|
||||
% Non-anonymous submissions will be rejected without review.
|
||||
|
||||
\author{Antiquus S.~Hippocampus, Natalia Cerebro \& Amelie P. Amygdale \thanks{ Use footnote for providing further information
|
||||
about author (webpage, alternative address)---\emph{not} for acknowledging
|
||||
funding agencies. Funding acknowledgements go at the end of the paper.} \\
|
||||
Department of Computer Science\\
|
||||
Cranberry-Lemon University\\
|
||||
Pittsburgh, PA 15213, USA \\
|
||||
\texttt{\{hippo,brain,jen\}@cs.cranberry-lemon.edu} \\
|
||||
\And
|
||||
Ji Q. Ren \& Yevgeny LeNet \\
|
||||
Department of Computational Neuroscience \\
|
||||
University of the Witwatersrand \\
|
||||
Joburg, South Africa \\
|
||||
\texttt{\{robot,net\}@wits.ac.za} \\
|
||||
\AND
|
||||
Coauthor \\
|
||||
Affiliation \\
|
||||
Address \\
|
||||
\texttt{email}
|
||||
}
|
||||
|
||||
% The \author macro works with any number of authors. There are two commands
|
||||
% used to separate the names and addresses of multiple authors: \And and \AND.
|
||||
%
|
||||
% Using \And between authors leaves it to \LaTeX{} to determine where to break
|
||||
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
|
||||
% puts 3 of 4 authors names on the first line, and the last on the second
|
||||
% line, try using \AND instead of \And before the third author name.
|
||||
|
||||
\newcommand{\fix}{\marginpar{FIX}}
|
||||
\newcommand{\new}{\marginpar{NEW}}
|
||||
|
||||
%\iclrfinalcopy % Uncomment for camera-ready version, but NOT for submission.
|
||||
\begin{document}
|
||||
|
||||
|
||||
\maketitle
|
||||
|
||||
\begin{abstract}
|
||||
The abstract paragraph should be indented 1/2~inch (3~picas) on both left and
|
||||
right-hand margins. Use 10~point type, with a vertical spacing of 11~points.
|
||||
The word \textsc{Abstract} must be centered, in small caps, and in point size 12. Two
|
||||
line spaces precede the abstract. The abstract must be limited to one
|
||||
paragraph.
|
||||
\end{abstract}
|
||||
|
||||
\section{Submission of conference papers to ICLR 2026}
|
||||
|
||||
ICLR requires electronic submissions, processed by
|
||||
\url{https://openreview.net/}. See ICLR's website for more instructions.
|
||||
|
||||
If your paper is ultimately accepted, the statement {\tt
|
||||
{\textbackslash}iclrfinalcopy} should be inserted to adjust the
|
||||
format to the camera ready requirements.
|
||||
|
||||
The format for the submissions is a variant of the NeurIPS format.
|
||||
Please read carefully the instructions below, and follow them
|
||||
faithfully.
|
||||
|
||||
\subsection{Style}
|
||||
|
||||
Papers to be submitted to ICLR 2026 must be prepared according to the
|
||||
instructions presented here.
|
||||
|
||||
%% Please note that we have introduced automatic line number generation
|
||||
%% into the style file for \LaTeXe. This is to help reviewers
|
||||
%% refer to specific lines of the paper when they make their comments. Please do
|
||||
%% NOT refer to these line numbers in your paper as they will be removed from the
|
||||
%% style file for the final version of accepted papers.
|
||||
|
||||
Authors are required to use the ICLR \LaTeX{} style files obtainable at the
|
||||
ICLR website. Please make sure you use the current files and
|
||||
not previous versions. Tweaking the style files may be grounds for rejection.
|
||||
|
||||
\subsection{Retrieval of style files}
|
||||
|
||||
The style files for ICLR and other conference information are available online at:
|
||||
\begin{center}
|
||||
\url{http://www.iclr.cc/}
|
||||
\end{center}
|
||||
The file \verb+iclr2026_conference.pdf+ contains these
|
||||
instructions and illustrates the
|
||||
various formatting requirements your ICLR paper must satisfy.
|
||||
Submissions must be made using \LaTeX{} and the style files
|
||||
\verb+iclr2026_conference.sty+ and \verb+iclr2026_conference.bst+ (to be used with \LaTeX{}2e). The file
|
||||
\verb+iclr2026_conference.tex+ may be used as a ``shell'' for writing your paper. All you
|
||||
have to do is replace the author, title, abstract, and text of the paper with
|
||||
your own.
|
||||
|
||||
The formatting instructions contained in these style files are summarized in
|
||||
sections \ref{gen_inst}, \ref{headings}, and \ref{others} below.
|
||||
|
||||
\section{General formatting instructions}
|
||||
\label{gen_inst}
|
||||
|
||||
The text must be confined within a rectangle 5.5~inches (33~picas) wide and
|
||||
9~inches (54~picas) long. The left margin is 1.5~inch (9~picas).
|
||||
Use 10~point type with a vertical spacing of 11~points. Times New Roman is the
|
||||
preferred typeface throughout. Paragraphs are separated by 1/2~line space,
|
||||
with no indentation.
|
||||
|
||||
Paper title is 17~point, in small caps and left-aligned.
|
||||
All pages should start at 1~inch (6~picas) from the top of the page.
|
||||
|
||||
Authors' names are
|
||||
set in boldface, and each name is placed above its corresponding
|
||||
address. The lead author's name is to be listed first, and
|
||||
the co-authors' names are set to follow. Authors sharing the
|
||||
same address can be on the same line.
|
||||
|
||||
Please pay special attention to the instructions in section \ref{others}
|
||||
regarding figures, tables, acknowledgments, and references.
|
||||
|
||||
|
||||
There will be a strict upper limit of \textbf{9 pages} for the main text of the initial submission, with unlimited additional pages for citations. This limit will be expanded to \textbf{10 pages} for rebuttal/camera ready.
|
||||
|
||||
\section{Headings: first level}
|
||||
\label{headings}
|
||||
|
||||
First level headings are in small caps,
|
||||
flush left and in point size 12. One line space before the first level
|
||||
heading and 1/2~line space after the first level heading.
|
||||
|
||||
\subsection{Headings: second level}
|
||||
|
||||
Second level headings are in small caps,
|
||||
flush left and in point size 10. One line space before the second level
|
||||
heading and 1/2~line space after the second level heading.
|
||||
|
||||
\subsubsection{Headings: third level}
|
||||
|
||||
Third level headings are in small caps,
|
||||
flush left and in point size 10. One line space before the third level
|
||||
heading and 1/2~line space after the third level heading.
|
||||
|
||||
\section{Citations, figures, tables, references}
|
||||
\label{others}
|
||||
|
||||
These instructions apply to everyone, regardless of the formatter being used.
|
||||
|
||||
\subsection{Citations within the text}
|
||||
|
||||
Citations within the text should be based on the \texttt{natbib} package
|
||||
and include the authors' last names and year (with the ``et~al.'' construct
|
||||
for more than two authors). When the authors or the publication are
|
||||
included in the sentence, the citation should not be in parenthesis using \verb|\citet{}| (as
|
||||
in ``See \citet{Hinton06} for more information.''). Otherwise, the citation
|
||||
should be in parenthesis using \verb|\citep{}| (as in ``Deep learning shows promise to make progress
|
||||
towards AI~\citep{Bengio+chapter2007}.'').
|
||||
|
||||
The corresponding references are to be listed in alphabetical order of
|
||||
authors, in the \textsc{References} section. As to the format of the
|
||||
references themselves, any style is acceptable as long as it is used
|
||||
consistently.
|
||||
|
||||
\subsection{Footnotes}
|
||||
|
||||
Indicate footnotes with a number\footnote{Sample of the first footnote} in the
|
||||
text. Place the footnotes at the bottom of the page on which they appear.
|
||||
Precede the footnote with a horizontal rule of 2~inches
|
||||
(12~picas).\footnote{Sample of the second footnote}
|
||||
|
||||
\subsection{Figures}
|
||||
|
||||
All artwork must be neat, clean, and legible. Lines should be dark
|
||||
enough for purposes of reproduction; art work should not be
|
||||
hand-drawn. The figure number and caption always appear after the
|
||||
figure. Place one line space before the figure caption, and one line
|
||||
space after the figure. The figure caption is lower case (except for
|
||||
first word and proper nouns); figures are numbered consecutively.
|
||||
|
||||
Make sure the figure caption does not get separated from the figure.
|
||||
Leave sufficient space to avoid splitting the figure and figure caption.
|
||||
|
||||
You may use color figures.
|
||||
However, it is best for the
|
||||
figure captions and the paper body to make sense if the paper is printed
|
||||
either in black/white or in color.
|
||||
\begin{figure}[h]
|
||||
\begin{center}
|
||||
%\framebox[4.0in]{$\;$}
|
||||
\fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
|
||||
\end{center}
|
||||
\caption{Sample figure caption.}
|
||||
\end{figure}
|
||||
|
||||
\subsection{Tables}
|
||||
|
||||
All tables must be centered, neat, clean and legible. Do not use hand-drawn
|
||||
tables. The table number and title always appear before the table. See
|
||||
Table~\ref{sample-table}.
|
||||
|
||||
Place one line space before the table title, one line space after the table
|
||||
title, and one line space after the table. The table title must be lower case
|
||||
(except for first word and proper nouns); tables are numbered consecutively.
|
||||
|
||||
\begin{table}[t]
|
||||
\caption{Sample table title}
|
||||
\label{sample-table}
|
||||
\begin{center}
|
||||
\begin{tabular}{ll}
|
||||
\multicolumn{1}{c}{\bf PART} &\multicolumn{1}{c}{\bf DESCRIPTION}
|
||||
\\ \hline \\
|
||||
Dendrite &Input terminal \\
|
||||
Axon &Output terminal \\
|
||||
Soma &Cell body (contains cell nucleus) \\
|
||||
\end{tabular}
|
||||
\end{center}
|
||||
\end{table}
|
||||
|
||||
\section{Default Notation}
|
||||
|
||||
In an attempt to encourage standardized notation, we have included the
|
||||
notation file from the textbook, \textit{Deep Learning}
|
||||
\cite{goodfellow2016deep} available at
|
||||
\url{https://github.com/goodfeli/dlbook_notation/}. Use of this style
|
||||
is not required and can be disabled by commenting out
|
||||
\texttt{math\_commands.tex}.
|
||||
|
||||
|
||||
\centerline{\bf Numbers and Arrays}
|
||||
\bgroup
|
||||
\def\arraystretch{1.5}
|
||||
\begin{tabular}{p{1in}p{3.25in}}
|
||||
$\displaystyle a$ & A scalar (integer or real)\\
|
||||
$\displaystyle \va$ & A vector\\
|
||||
$\displaystyle \mA$ & A matrix\\
|
||||
$\displaystyle \tA$ & A tensor\\
|
||||
$\displaystyle \mI_n$ & Identity matrix with $n$ rows and $n$ columns\\
|
||||
$\displaystyle \mI$ & Identity matrix with dimensionality implied by context\\
|
||||
$\displaystyle \ve^{(i)}$ & Standard basis vector $[0,\dots,0,1,0,\dots,0]$ with a 1 at position $i$\\
|
||||
$\displaystyle \text{diag}(\va)$ & A square, diagonal matrix with diagonal entries given by $\va$\\
|
||||
$\displaystyle \ra$ & A scalar random variable\\
|
||||
$\displaystyle \rva$ & A vector-valued random variable\\
|
||||
$\displaystyle \rmA$ & A matrix-valued random variable\\
|
||||
\end{tabular}
|
||||
\egroup
|
||||
\vspace{0.25cm}
|
||||
|
||||
\centerline{\bf Sets and Graphs}
|
||||
\bgroup
|
||||
\def\arraystretch{1.5}
|
||||
|
||||
\begin{tabular}{p{1.25in}p{3.25in}}
|
||||
$\displaystyle \sA$ & A set\\
|
||||
$\displaystyle \R$ & The set of real numbers \\
|
||||
$\displaystyle \{0, 1\}$ & The set containing 0 and 1 \\
|
||||
$\displaystyle \{0, 1, \dots, n \}$ & The set of all integers between $0$ and $n$\\
|
||||
$\displaystyle [a, b]$ & The real interval including $a$ and $b$\\
|
||||
$\displaystyle (a, b]$ & The real interval excluding $a$ but including $b$\\
|
||||
$\displaystyle \sA \backslash \sB$ & Set subtraction, i.e., the set containing the elements of $\sA$ that are not in $\sB$\\
|
||||
$\displaystyle \gG$ & A graph\\
|
||||
$\displaystyle \parents_\gG(\ervx_i)$ & The parents of $\ervx_i$ in $\gG$
|
||||
\end{tabular}
|
||||
\vspace{0.25cm}
|
||||
|
||||
|
||||
\centerline{\bf Indexing}
|
||||
\bgroup
|
||||
\def\arraystretch{1.5}
|
||||
|
||||
\begin{tabular}{p{1.25in}p{3.25in}}
|
||||
$\displaystyle \eva_i$ & Element $i$ of vector $\va$, with indexing starting at 1 \\
|
||||
$\displaystyle \eva_{-i}$ & All elements of vector $\va$ except for element $i$ \\
|
||||
$\displaystyle \emA_{i,j}$ & Element $i, j$ of matrix $\mA$ \\
|
||||
$\displaystyle \mA_{i, :}$ & Row $i$ of matrix $\mA$ \\
|
||||
$\displaystyle \mA_{:, i}$ & Column $i$ of matrix $\mA$ \\
|
||||
$\displaystyle \etA_{i, j, k}$ & Element $(i, j, k)$ of a 3-D tensor $\tA$\\
|
||||
$\displaystyle \tA_{:, :, i}$ & 2-D slice of a 3-D tensor\\
|
||||
$\displaystyle \erva_i$ & Element $i$ of the random vector $\rva$ \\
|
||||
\end{tabular}
|
||||
\egroup
|
||||
\vspace{0.25cm}
|
||||
|
||||
|
||||
\centerline{\bf Calculus}
|
||||
\bgroup
|
||||
\def\arraystretch{1.5}
|
||||
\begin{tabular}{p{1.25in}p{3.25in}}
|
||||
% NOTE: the [2ex] on the next line adds extra height to that row of the table.
|
||||
% Without that command, the fraction on the first line is too tall and collides
|
||||
% with the fraction on the second line.
|
||||
$\displaystyle\frac{d y} {d x}$ & Derivative of $y$ with respect to $x$\\ [2ex]
|
||||
$\displaystyle \frac{\partial y} {\partial x} $ & Partial derivative of $y$ with respect to $x$ \\
|
||||
$\displaystyle \nabla_\vx y $ & Gradient of $y$ with respect to $\vx$ \\
|
||||
$\displaystyle \nabla_\mX y $ & Matrix derivatives of $y$ with respect to $\mX$ \\
|
||||
$\displaystyle \nabla_\tX y $ & Tensor containing derivatives of $y$ with respect to $\tX$ \\
|
||||
$\displaystyle \frac{\partial f}{\partial \vx} $ & Jacobian matrix $\mJ \in \R^{m\times n}$ of $f: \R^n \rightarrow \R^m$\\
|
||||
$\displaystyle \nabla_\vx^2 f(\vx)\text{ or }\mH( f)(\vx)$ & The Hessian matrix of $f$ at input point $\vx$\\
|
||||
$\displaystyle \int f(\vx) d\vx $ & Definite integral over the entire domain of $\vx$ \\
|
||||
$\displaystyle \int_\sS f(\vx) d\vx$ & Definite integral with respect to $\vx$ over the set $\sS$ \\
|
||||
\end{tabular}
|
||||
\egroup
|
||||
\vspace{0.25cm}
|
||||
|
||||
\centerline{\bf Probability and Information Theory}
|
||||
\bgroup
|
||||
\def\arraystretch{1.5}
|
||||
\begin{tabular}{p{1.25in}p{3.25in}}
|
||||
$\displaystyle P(\ra)$ & A probability distribution over a discrete variable\\
|
||||
$\displaystyle p(\ra)$ & A probability distribution over a continuous variable, or over
|
||||
a variable whose type has not been specified\\
|
||||
$\displaystyle \ra \sim P$ & Random variable $\ra$ has distribution $P$\\% so thing on left of \sim should always be a random variable, with name beginning with \r
|
||||
$\displaystyle \E_{\rx\sim P} [ f(x) ]\text{ or } \E f(x)$ & Expectation of $f(x)$ with respect to $P(\rx)$ \\
|
||||
$\displaystyle \Var(f(x)) $ & Variance of $f(x)$ under $P(\rx)$ \\
|
||||
$\displaystyle \Cov(f(x),g(x)) $ & Covariance of $f(x)$ and $g(x)$ under $P(\rx)$\\
|
||||
$\displaystyle H(\rx) $ & Shannon entropy of the random variable $\rx$\\
|
||||
$\displaystyle \KL ( P \Vert Q ) $ & Kullback-Leibler divergence of P and Q \\
|
||||
$\displaystyle \mathcal{N} ( \vx ; \vmu , \mSigma)$ & Gaussian distribution %
|
||||
over $\vx$ with mean $\vmu$ and covariance $\mSigma$ \\
|
||||
\end{tabular}
|
||||
\egroup
|
||||
\vspace{0.25cm}
|
||||
|
||||
\centerline{\bf Functions}
|
||||
\bgroup
|
||||
\def\arraystretch{1.5}
|
||||
\begin{tabular}{p{1.25in}p{3.25in}}
|
||||
$\displaystyle f: \sA \rightarrow \sB$ & The function $f$ with domain $\sA$ and range $\sB$\\
|
||||
$\displaystyle f \circ g $ & Composition of the functions $f$ and $g$ \\
|
||||
$\displaystyle f(\vx ; \vtheta) $ & A function of $\vx$ parametrized by $\vtheta$.
|
||||
(Sometimes we write $f(\vx)$ and omit the argument $\vtheta$ to lighten notation) \\
|
||||
$\displaystyle \log x$ & Natural logarithm of $x$ \\
|
||||
$\displaystyle \sigma(x)$ & Logistic sigmoid, $\displaystyle \frac{1} {1 + \exp(-x)}$ \\
|
||||
$\displaystyle \zeta(x)$ & Softplus, $\log(1 + \exp(x))$ \\
|
||||
$\displaystyle || \vx ||_p $ & $\normlp$ norm of $\vx$ \\
|
||||
$\displaystyle || \vx || $ & $\normltwo$ norm of $\vx$ \\
|
||||
$\displaystyle x^+$ & Positive part of $x$, i.e., $\max(0,x)$\\
|
||||
$\displaystyle \1_\mathrm{condition}$ & is 1 if the condition is true, 0 otherwise\\
|
||||
\end{tabular}
|
||||
\egroup
|
||||
\vspace{0.25cm}
|
||||
|
||||
|
||||
|
||||
\section{Final instructions}
|
||||
Do not change any aspects of the formatting parameters in the style files.
|
||||
In particular, do not modify the width or length of the rectangle the text
|
||||
should fit into, and do not change font sizes (except perhaps in the
|
||||
\textsc{References} section; see below). Please note that pages should be
|
||||
numbered.
|
||||
|
||||
\section{Preparing PostScript or PDF files}
|
||||
|
||||
Please prepare PostScript or PDF files with paper size ``US Letter'', and
|
||||
not, for example, ``A4''. The -t
|
||||
letter option on dvips will produce US Letter files.
|
||||
|
||||
Consider directly generating PDF files using \verb+pdflatex+
|
||||
(especially if you are a MiKTeX user).
|
||||
PDF figures must be substituted for EPS figures, however.
|
||||
|
||||
Otherwise, please generate your PostScript and PDF files with the following commands:
|
||||
\begin{verbatim}
|
||||
dvips mypaper.dvi -t letter -Ppdf -G0 -o mypaper.ps
|
||||
ps2pdf mypaper.ps mypaper.pdf
|
||||
\end{verbatim}
|
||||
|
||||
\subsection{Margins in LaTeX}
|
||||
|
||||
Most of the margin problems come from figures positioned by hand using
|
||||
\verb+\special+ or other commands. We suggest using the command
|
||||
\verb+\includegraphics+
|
||||
from the graphicx package. Always specify the figure width as a multiple of
|
||||
the line width as in the example below using .eps graphics
|
||||
\begin{verbatim}
|
||||
\usepackage[dvips]{graphicx} ...
|
||||
\includegraphics[width=0.8\linewidth]{myfile.eps}
|
||||
\end{verbatim}
|
||||
or % Apr 2009 addition
|
||||
\begin{verbatim}
|
||||
\usepackage[pdftex]{graphicx} ...
|
||||
\includegraphics[width=0.8\linewidth]{myfile.pdf}
|
||||
\end{verbatim}
|
||||
for .pdf graphics.
|
||||
See section~4.4 in the graphics bundle documentation (\url{http://www.ctan.org/tex-archive/macros/latex/required/graphics/grfguide.ps})
|
||||
|
||||
A number of width problems arise when LaTeX cannot properly hyphenate a
|
||||
line. Please give LaTeX hyphenation hints using the \verb+\-+ command.
|
||||
|
||||
\subsubsection*{Author Contributions}
|
||||
If you'd like to, you may include a section for author contributions as is done
|
||||
in many journals. This is optional and at the discretion of the authors.
|
||||
|
||||
\subsubsection*{Acknowledgments}
|
||||
Use unnumbered third level headings for the acknowledgments. All
|
||||
acknowledgments, including those to funding agencies, go at the end of the paper.
|
||||
|
||||
|
||||
\bibliography{iclr2026_conference}
|
||||
\bibliographystyle{iclr2026_conference}
|
||||
|
||||
\appendix
|
||||
\section{Appendix}
|
||||
You may include other additional sections here.
|
||||
|
||||
|
||||
\end{document}
|
||||
@@ -0,0 +1,508 @@
|
||||
%%%%% NEW MATH DEFINITIONS %%%%%
|
||||
|
||||
\usepackage{amsmath,amsfonts,bm}
|
||||
|
||||
% Mark sections of captions for referring to divisions of figures
|
||||
\newcommand{\figleft}{{\em (Left)}}
|
||||
\newcommand{\figcenter}{{\em (Center)}}
|
||||
\newcommand{\figright}{{\em (Right)}}
|
||||
\newcommand{\figtop}{{\em (Top)}}
|
||||
\newcommand{\figbottom}{{\em (Bottom)}}
|
||||
\newcommand{\captiona}{{\em (a)}}
|
||||
\newcommand{\captionb}{{\em (b)}}
|
||||
\newcommand{\captionc}{{\em (c)}}
|
||||
\newcommand{\captiond}{{\em (d)}}
|
||||
|
||||
% Highlight a newly defined term
|
||||
\newcommand{\newterm}[1]{{\bf #1}}
|
||||
|
||||
|
||||
% Figure reference, lower-case.
|
||||
\def\figref#1{figure~\ref{#1}}
|
||||
% Figure reference, capital. For start of sentence
|
||||
\def\Figref#1{Figure~\ref{#1}}
|
||||
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
|
||||
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
|
||||
% Section reference, lower-case.
|
||||
\def\secref#1{section~\ref{#1}}
|
||||
% Section reference, capital.
|
||||
\def\Secref#1{Section~\ref{#1}}
|
||||
% Reference to two sections.
|
||||
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
|
||||
% Reference to three sections.
|
||||
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
|
||||
% Reference to an equation, lower-case.
|
||||
\def\eqref#1{equation~\ref{#1}}
|
||||
% Reference to an equation, upper case
|
||||
\def\Eqref#1{Equation~\ref{#1}}
|
||||
% A raw reference to an equation---avoid using if possible
|
||||
\def\plaineqref#1{\ref{#1}}
|
||||
% Reference to a chapter, lower-case.
|
||||
\def\chapref#1{chapter~\ref{#1}}
|
||||
% Reference to an equation, upper case.
|
||||
\def\Chapref#1{Chapter~\ref{#1}}
|
||||
% Reference to a range of chapters
|
||||
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
|
||||
% Reference to an algorithm, lower-case.
|
||||
\def\algref#1{algorithm~\ref{#1}}
|
||||
% Reference to an algorithm, upper case.
|
||||
\def\Algref#1{Algorithm~\ref{#1}}
|
||||
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
|
||||
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
|
||||
% Reference to a part, lower case
|
||||
\def\partref#1{part~\ref{#1}}
|
||||
% Reference to a part, upper case
|
||||
\def\Partref#1{Part~\ref{#1}}
|
||||
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}
|
||||
|
||||
\def\ceil#1{\lceil #1 \rceil}
|
||||
\def\floor#1{\lfloor #1 \rfloor}
|
||||
\def\1{\bm{1}}
|
||||
\newcommand{\train}{\mathcal{D}}
|
||||
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
|
||||
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}
|
||||
|
||||
\def\eps{{\epsilon}}
|
||||
|
||||
|
||||
% Random variables
|
||||
\def\reta{{\textnormal{$\eta$}}}
|
||||
\def\ra{{\textnormal{a}}}
|
||||
\def\rb{{\textnormal{b}}}
|
||||
\def\rc{{\textnormal{c}}}
|
||||
\def\rd{{\textnormal{d}}}
|
||||
\def\re{{\textnormal{e}}}
|
||||
\def\rf{{\textnormal{f}}}
|
||||
\def\rg{{\textnormal{g}}}
|
||||
\def\rh{{\textnormal{h}}}
|
||||
\def\ri{{\textnormal{i}}}
|
||||
\def\rj{{\textnormal{j}}}
|
||||
\def\rk{{\textnormal{k}}}
|
||||
\def\rl{{\textnormal{l}}}
|
||||
% rm is already a command, just don't name any random variables m
|
||||
\def\rn{{\textnormal{n}}}
|
||||
\def\ro{{\textnormal{o}}}
|
||||
\def\rp{{\textnormal{p}}}
|
||||
\def\rq{{\textnormal{q}}}
|
||||
\def\rr{{\textnormal{r}}}
|
||||
\def\rs{{\textnormal{s}}}
|
||||
\def\rt{{\textnormal{t}}}
|
||||
\def\ru{{\textnormal{u}}}
|
||||
\def\rv{{\textnormal{v}}}
|
||||
\def\rw{{\textnormal{w}}}
|
||||
\def\rx{{\textnormal{x}}}
|
||||
\def\ry{{\textnormal{y}}}
|
||||
\def\rz{{\textnormal{z}}}
|
||||
|
||||
% Random vectors
|
||||
\def\rvepsilon{{\mathbf{\epsilon}}}
|
||||
\def\rvtheta{{\mathbf{\theta}}}
|
||||
\def\rva{{\mathbf{a}}}
|
||||
\def\rvb{{\mathbf{b}}}
|
||||
\def\rvc{{\mathbf{c}}}
|
||||
\def\rvd{{\mathbf{d}}}
|
||||
\def\rve{{\mathbf{e}}}
|
||||
\def\rvf{{\mathbf{f}}}
|
||||
\def\rvg{{\mathbf{g}}}
|
||||
\def\rvh{{\mathbf{h}}}
|
||||
\def\rvu{{\mathbf{i}}}
|
||||
\def\rvj{{\mathbf{j}}}
|
||||
\def\rvk{{\mathbf{k}}}
|
||||
\def\rvl{{\mathbf{l}}}
|
||||
\def\rvm{{\mathbf{m}}}
|
||||
\def\rvn{{\mathbf{n}}}
|
||||
\def\rvo{{\mathbf{o}}}
|
||||
\def\rvp{{\mathbf{p}}}
|
||||
\def\rvq{{\mathbf{q}}}
|
||||
\def\rvr{{\mathbf{r}}}
|
||||
\def\rvs{{\mathbf{s}}}
|
||||
\def\rvt{{\mathbf{t}}}
|
||||
\def\rvu{{\mathbf{u}}}
|
||||
\def\rvv{{\mathbf{v}}}
|
||||
\def\rvw{{\mathbf{w}}}
|
||||
\def\rvx{{\mathbf{x}}}
|
||||
\def\rvy{{\mathbf{y}}}
|
||||
\def\rvz{{\mathbf{z}}}
|
||||
|
||||
% Elements of random vectors
|
||||
\def\erva{{\textnormal{a}}}
|
||||
\def\ervb{{\textnormal{b}}}
|
||||
\def\ervc{{\textnormal{c}}}
|
||||
\def\ervd{{\textnormal{d}}}
|
||||
\def\erve{{\textnormal{e}}}
|
||||
\def\ervf{{\textnormal{f}}}
|
||||
\def\ervg{{\textnormal{g}}}
|
||||
\def\ervh{{\textnormal{h}}}
|
||||
\def\ervi{{\textnormal{i}}}
|
||||
\def\ervj{{\textnormal{j}}}
|
||||
\def\ervk{{\textnormal{k}}}
|
||||
\def\ervl{{\textnormal{l}}}
|
||||
\def\ervm{{\textnormal{m}}}
|
||||
\def\ervn{{\textnormal{n}}}
|
||||
\def\ervo{{\textnormal{o}}}
|
||||
\def\ervp{{\textnormal{p}}}
|
||||
\def\ervq{{\textnormal{q}}}
|
||||
\def\ervr{{\textnormal{r}}}
|
||||
\def\ervs{{\textnormal{s}}}
|
||||
\def\ervt{{\textnormal{t}}}
|
||||
\def\ervu{{\textnormal{u}}}
|
||||
\def\ervv{{\textnormal{v}}}
|
||||
\def\ervw{{\textnormal{w}}}
|
||||
\def\ervx{{\textnormal{x}}}
|
||||
\def\ervy{{\textnormal{y}}}
|
||||
\def\ervz{{\textnormal{z}}}
|
||||
|
||||
% Random matrices
|
||||
\def\rmA{{\mathbf{A}}}
|
||||
\def\rmB{{\mathbf{B}}}
|
||||
\def\rmC{{\mathbf{C}}}
|
||||
\def\rmD{{\mathbf{D}}}
|
||||
\def\rmE{{\mathbf{E}}}
|
||||
\def\rmF{{\mathbf{F}}}
|
||||
\def\rmG{{\mathbf{G}}}
|
||||
\def\rmH{{\mathbf{H}}}
|
||||
\def\rmI{{\mathbf{I}}}
|
||||
\def\rmJ{{\mathbf{J}}}
|
||||
\def\rmK{{\mathbf{K}}}
|
||||
\def\rmL{{\mathbf{L}}}
|
||||
\def\rmM{{\mathbf{M}}}
|
||||
\def\rmN{{\mathbf{N}}}
|
||||
\def\rmO{{\mathbf{O}}}
|
||||
\def\rmP{{\mathbf{P}}}
|
||||
\def\rmQ{{\mathbf{Q}}}
|
||||
\def\rmR{{\mathbf{R}}}
|
||||
\def\rmS{{\mathbf{S}}}
|
||||
\def\rmT{{\mathbf{T}}}
|
||||
\def\rmU{{\mathbf{U}}}
|
||||
\def\rmV{{\mathbf{V}}}
|
||||
\def\rmW{{\mathbf{W}}}
|
||||
\def\rmX{{\mathbf{X}}}
|
||||
\def\rmY{{\mathbf{Y}}}
|
||||
\def\rmZ{{\mathbf{Z}}}
|
||||
|
||||
% Elements of random matrices
|
||||
\def\ermA{{\textnormal{A}}}
|
||||
\def\ermB{{\textnormal{B}}}
|
||||
\def\ermC{{\textnormal{C}}}
|
||||
\def\ermD{{\textnormal{D}}}
|
||||
\def\ermE{{\textnormal{E}}}
|
||||
\def\ermF{{\textnormal{F}}}
|
||||
\def\ermG{{\textnormal{G}}}
|
||||
\def\ermH{{\textnormal{H}}}
|
||||
\def\ermI{{\textnormal{I}}}
|
||||
\def\ermJ{{\textnormal{J}}}
|
||||
\def\ermK{{\textnormal{K}}}
|
||||
\def\ermL{{\textnormal{L}}}
|
||||
\def\ermM{{\textnormal{M}}}
|
||||
\def\ermN{{\textnormal{N}}}
|
||||
\def\ermO{{\textnormal{O}}}
|
||||
\def\ermP{{\textnormal{P}}}
|
||||
\def\ermQ{{\textnormal{Q}}}
|
||||
\def\ermR{{\textnormal{R}}}
|
||||
\def\ermS{{\textnormal{S}}}
|
||||
\def\ermT{{\textnormal{T}}}
|
||||
\def\ermU{{\textnormal{U}}}
|
||||
\def\ermV{{\textnormal{V}}}
|
||||
\def\ermW{{\textnormal{W}}}
|
||||
\def\ermX{{\textnormal{X}}}
|
||||
\def\ermY{{\textnormal{Y}}}
|
||||
\def\ermZ{{\textnormal{Z}}}
|
||||
|
||||
% Vectors
|
||||
\def\vzero{{\bm{0}}}
|
||||
\def\vone{{\bm{1}}}
|
||||
\def\vmu{{\bm{\mu}}}
|
||||
\def\vtheta{{\bm{\theta}}}
|
||||
\def\va{{\bm{a}}}
|
||||
\def\vb{{\bm{b}}}
|
||||
\def\vc{{\bm{c}}}
|
||||
\def\vd{{\bm{d}}}
|
||||
\def\ve{{\bm{e}}}
|
||||
\def\vf{{\bm{f}}}
|
||||
\def\vg{{\bm{g}}}
|
||||
\def\vh{{\bm{h}}}
|
||||
\def\vi{{\bm{i}}}
|
||||
\def\vj{{\bm{j}}}
|
||||
\def\vk{{\bm{k}}}
|
||||
\def\vl{{\bm{l}}}
|
||||
\def\vm{{\bm{m}}}
|
||||
\def\vn{{\bm{n}}}
|
||||
\def\vo{{\bm{o}}}
|
||||
\def\vp{{\bm{p}}}
|
||||
\def\vq{{\bm{q}}}
|
||||
\def\vr{{\bm{r}}}
|
||||
\def\vs{{\bm{s}}}
|
||||
\def\vt{{\bm{t}}}
|
||||
\def\vu{{\bm{u}}}
|
||||
\def\vv{{\bm{v}}}
|
||||
\def\vw{{\bm{w}}}
|
||||
\def\vx{{\bm{x}}}
|
||||
\def\vy{{\bm{y}}}
|
||||
\def\vz{{\bm{z}}}
|
||||
|
||||
% Elements of vectors
|
||||
\def\evalpha{{\alpha}}
|
||||
\def\evbeta{{\beta}}
|
||||
\def\evepsilon{{\epsilon}}
|
||||
\def\evlambda{{\lambda}}
|
||||
\def\evomega{{\omega}}
|
||||
\def\evmu{{\mu}}
|
||||
\def\evpsi{{\psi}}
|
||||
\def\evsigma{{\sigma}}
|
||||
\def\evtheta{{\theta}}
|
||||
\def\eva{{a}}
|
||||
\def\evb{{b}}
|
||||
\def\evc{{c}}
|
||||
\def\evd{{d}}
|
||||
\def\eve{{e}}
|
||||
\def\evf{{f}}
|
||||
\def\evg{{g}}
|
||||
\def\evh{{h}}
|
||||
\def\evi{{i}}
|
||||
\def\evj{{j}}
|
||||
\def\evk{{k}}
|
||||
\def\evl{{l}}
|
||||
\def\evm{{m}}
|
||||
\def\evn{{n}}
|
||||
\def\evo{{o}}
|
||||
\def\evp{{p}}
|
||||
\def\evq{{q}}
|
||||
\def\evr{{r}}
|
||||
\def\evs{{s}}
|
||||
\def\evt{{t}}
|
||||
\def\evu{{u}}
|
||||
\def\evv{{v}}
|
||||
\def\evw{{w}}
|
||||
\def\evx{{x}}
|
||||
\def\evy{{y}}
|
||||
\def\evz{{z}}
|
||||
|
||||
% Matrix
|
||||
\def\mA{{\bm{A}}}
|
||||
\def\mB{{\bm{B}}}
|
||||
\def\mC{{\bm{C}}}
|
||||
\def\mD{{\bm{D}}}
|
||||
\def\mE{{\bm{E}}}
|
||||
\def\mF{{\bm{F}}}
|
||||
\def\mG{{\bm{G}}}
|
||||
\def\mH{{\bm{H}}}
|
||||
\def\mI{{\bm{I}}}
|
||||
\def\mJ{{\bm{J}}}
|
||||
\def\mK{{\bm{K}}}
|
||||
\def\mL{{\bm{L}}}
|
||||
\def\mM{{\bm{M}}}
|
||||
\def\mN{{\bm{N}}}
|
||||
\def\mO{{\bm{O}}}
|
||||
\def\mP{{\bm{P}}}
|
||||
\def\mQ{{\bm{Q}}}
|
||||
\def\mR{{\bm{R}}}
|
||||
\def\mS{{\bm{S}}}
|
||||
\def\mT{{\bm{T}}}
|
||||
\def\mU{{\bm{U}}}
|
||||
\def\mV{{\bm{V}}}
|
||||
\def\mW{{\bm{W}}}
|
||||
\def\mX{{\bm{X}}}
|
||||
\def\mY{{\bm{Y}}}
|
||||
\def\mZ{{\bm{Z}}}
|
||||
\def\mBeta{{\bm{\beta}}}
|
||||
\def\mPhi{{\bm{\Phi}}}
|
||||
\def\mLambda{{\bm{\Lambda}}}
|
||||
\def\mSigma{{\bm{\Sigma}}}
|
||||
|
||||
% Tensor
|
||||
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
|
||||
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
|
||||
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
|
||||
\def\tA{{\tens{A}}}
|
||||
\def\tB{{\tens{B}}}
|
||||
\def\tC{{\tens{C}}}
|
||||
\def\tD{{\tens{D}}}
|
||||
\def\tE{{\tens{E}}}
|
||||
\def\tF{{\tens{F}}}
|
||||
\def\tG{{\tens{G}}}
|
||||
\def\tH{{\tens{H}}}
|
||||
\def\tI{{\tens{I}}}
|
||||
\def\tJ{{\tens{J}}}
|
||||
\def\tK{{\tens{K}}}
|
||||
\def\tL{{\tens{L}}}
|
||||
\def\tM{{\tens{M}}}
|
||||
\def\tN{{\tens{N}}}
|
||||
\def\tO{{\tens{O}}}
|
||||
\def\tP{{\tens{P}}}
|
||||
\def\tQ{{\tens{Q}}}
|
||||
\def\tR{{\tens{R}}}
|
||||
\def\tS{{\tens{S}}}
|
||||
\def\tT{{\tens{T}}}
|
||||
\def\tU{{\tens{U}}}
|
||||
\def\tV{{\tens{V}}}
|
||||
\def\tW{{\tens{W}}}
|
||||
\def\tX{{\tens{X}}}
|
||||
\def\tY{{\tens{Y}}}
|
||||
\def\tZ{{\tens{Z}}}
|
||||
|
||||
|
||||
% Graph
|
||||
\def\gA{{\mathcal{A}}}
|
||||
\def\gB{{\mathcal{B}}}
|
||||
\def\gC{{\mathcal{C}}}
|
||||
\def\gD{{\mathcal{D}}}
|
||||
\def\gE{{\mathcal{E}}}
|
||||
\def\gF{{\mathcal{F}}}
|
||||
\def\gG{{\mathcal{G}}}
|
||||
\def\gH{{\mathcal{H}}}
|
||||
\def\gI{{\mathcal{I}}}
|
||||
\def\gJ{{\mathcal{J}}}
|
||||
\def\gK{{\mathcal{K}}}
|
||||
\def\gL{{\mathcal{L}}}
|
||||
\def\gM{{\mathcal{M}}}
|
||||
\def\gN{{\mathcal{N}}}
|
||||
\def\gO{{\mathcal{O}}}
|
||||
\def\gP{{\mathcal{P}}}
|
||||
\def\gQ{{\mathcal{Q}}}
|
||||
\def\gR{{\mathcal{R}}}
|
||||
\def\gS{{\mathcal{S}}}
|
||||
\def\gT{{\mathcal{T}}}
|
||||
\def\gU{{\mathcal{U}}}
|
||||
\def\gV{{\mathcal{V}}}
|
||||
\def\gW{{\mathcal{W}}}
|
||||
\def\gX{{\mathcal{X}}}
|
||||
\def\gY{{\mathcal{Y}}}
|
||||
\def\gZ{{\mathcal{Z}}}
|
||||
|
||||
% Sets
|
||||
\def\sA{{\mathbb{A}}}
|
||||
\def\sB{{\mathbb{B}}}
|
||||
\def\sC{{\mathbb{C}}}
|
||||
\def\sD{{\mathbb{D}}}
|
||||
% Don't use a set called E, because this would be the same as our symbol
|
||||
% for expectation.
|
||||
\def\sF{{\mathbb{F}}}
|
||||
\def\sG{{\mathbb{G}}}
|
||||
\def\sH{{\mathbb{H}}}
|
||||
\def\sI{{\mathbb{I}}}
|
||||
\def\sJ{{\mathbb{J}}}
|
||||
\def\sK{{\mathbb{K}}}
|
||||
\def\sL{{\mathbb{L}}}
|
||||
\def\sM{{\mathbb{M}}}
|
||||
\def\sN{{\mathbb{N}}}
|
||||
\def\sO{{\mathbb{O}}}
|
||||
\def\sP{{\mathbb{P}}}
|
||||
\def\sQ{{\mathbb{Q}}}
|
||||
\def\sR{{\mathbb{R}}}
|
||||
\def\sS{{\mathbb{S}}}
|
||||
\def\sT{{\mathbb{T}}}
|
||||
\def\sU{{\mathbb{U}}}
|
||||
\def\sV{{\mathbb{V}}}
|
||||
\def\sW{{\mathbb{W}}}
|
||||
\def\sX{{\mathbb{X}}}
|
||||
\def\sY{{\mathbb{Y}}}
|
||||
\def\sZ{{\mathbb{Z}}}
|
||||
|
||||
% Entries of a matrix
|
||||
\def\emLambda{{\Lambda}}
|
||||
\def\emA{{A}}
|
||||
\def\emB{{B}}
|
||||
\def\emC{{C}}
|
||||
\def\emD{{D}}
|
||||
\def\emE{{E}}
|
||||
\def\emF{{F}}
|
||||
\def\emG{{G}}
|
||||
\def\emH{{H}}
|
||||
\def\emI{{I}}
|
||||
\def\emJ{{J}}
|
||||
\def\emK{{K}}
|
||||
\def\emL{{L}}
|
||||
\def\emM{{M}}
|
||||
\def\emN{{N}}
|
||||
\def\emO{{O}}
|
||||
\def\emP{{P}}
|
||||
\def\emQ{{Q}}
|
||||
\def\emR{{R}}
|
||||
\def\emS{{S}}
|
||||
\def\emT{{T}}
|
||||
\def\emU{{U}}
|
||||
\def\emV{{V}}
|
||||
\def\emW{{W}}
|
||||
\def\emX{{X}}
|
||||
\def\emY{{Y}}
|
||||
\def\emZ{{Z}}
|
||||
\def\emSigma{{\Sigma}}
|
||||
|
||||
% entries of a tensor
|
||||
% Same font as tensor, without \bm wrapper
|
||||
\newcommand{\etens}[1]{\mathsfit{#1}}
|
||||
\def\etLambda{{\etens{\Lambda}}}
|
||||
\def\etA{{\etens{A}}}
|
||||
\def\etB{{\etens{B}}}
|
||||
\def\etC{{\etens{C}}}
|
||||
\def\etD{{\etens{D}}}
|
||||
\def\etE{{\etens{E}}}
|
||||
\def\etF{{\etens{F}}}
|
||||
\def\etG{{\etens{G}}}
|
||||
\def\etH{{\etens{H}}}
|
||||
\def\etI{{\etens{I}}}
|
||||
\def\etJ{{\etens{J}}}
|
||||
\def\etK{{\etens{K}}}
|
||||
\def\etL{{\etens{L}}}
|
||||
\def\etM{{\etens{M}}}
|
||||
\def\etN{{\etens{N}}}
|
||||
\def\etO{{\etens{O}}}
|
||||
\def\etP{{\etens{P}}}
|
||||
\def\etQ{{\etens{Q}}}
|
||||
\def\etR{{\etens{R}}}
|
||||
\def\etS{{\etens{S}}}
|
||||
\def\etT{{\etens{T}}}
|
||||
\def\etU{{\etens{U}}}
|
||||
\def\etV{{\etens{V}}}
|
||||
\def\etW{{\etens{W}}}
|
||||
\def\etX{{\etens{X}}}
|
||||
\def\etY{{\etens{Y}}}
|
||||
\def\etZ{{\etens{Z}}}
|
||||
|
||||
% The true underlying data generating distribution
|
||||
\newcommand{\pdata}{p_{\rm{data}}}
|
||||
% The empirical distribution defined by the training set
|
||||
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
|
||||
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
|
||||
% The model distribution
|
||||
\newcommand{\pmodel}{p_{\rm{model}}}
|
||||
\newcommand{\Pmodel}{P_{\rm{model}}}
|
||||
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
|
||||
% Stochastic autoencoder distributions
|
||||
\newcommand{\pencode}{p_{\rm{encoder}}}
|
||||
\newcommand{\pdecode}{p_{\rm{decoder}}}
|
||||
\newcommand{\precons}{p_{\rm{reconstruct}}}
|
||||
|
||||
\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution
|
||||
|
||||
\newcommand{\E}{\mathbb{E}}
|
||||
\newcommand{\Ls}{\mathcal{L}}
|
||||
\newcommand{\R}{\mathbb{R}}
|
||||
\newcommand{\emp}{\tilde{p}}
|
||||
\newcommand{\lr}{\alpha}
|
||||
\newcommand{\reg}{\lambda}
|
||||
\newcommand{\rect}{\mathrm{rectifier}}
|
||||
\newcommand{\softmax}{\mathrm{softmax}}
|
||||
\newcommand{\sigmoid}{\sigma}
|
||||
\newcommand{\softplus}{\zeta}
|
||||
\newcommand{\KL}{D_{\mathrm{KL}}}
|
||||
\newcommand{\Var}{\mathrm{Var}}
|
||||
\newcommand{\standarderror}{\mathrm{SE}}
|
||||
\newcommand{\Cov}{\mathrm{Cov}}
|
||||
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
|
||||
% But then they seem to use $L^2$ for vectors throughout the site, and so does
|
||||
% wikipedia.
|
||||
\newcommand{\normlzero}{L^0}
|
||||
\newcommand{\normlone}{L^1}
|
||||
\newcommand{\normltwo}{L^2}
|
||||
\newcommand{\normlp}{L^p}
|
||||
\newcommand{\normmax}{L^\infty}
|
||||
|
||||
\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.
|
||||
|
||||
\DeclareMathOperator*{\argmax}{arg\,max}
|
||||
\DeclareMathOperator*{\argmin}{arg\,min}
|
||||
|
||||
\DeclareMathOperator{\sign}{sign}
|
||||
\DeclareMathOperator{\Tr}{Tr}
|
||||
\let\ab\allowbreak
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,79 @@
|
||||
% ALGORITHM STYLE -- Released 8 April 1996
|
||||
% for LaTeX-2e
|
||||
% Copyright -- 1994 Peter Williams
|
||||
% E-mail Peter.Williams@dsto.defence.gov.au
|
||||
\NeedsTeXFormat{LaTeX2e}
|
||||
\ProvidesPackage{algorithm}
|
||||
\typeout{Document Style `algorithm' - floating environment}
|
||||
|
||||
\RequirePackage{float}
|
||||
\RequirePackage{ifthen}
|
||||
\newcommand{\ALG@within}{nothing}
|
||||
\newboolean{ALG@within}
|
||||
\setboolean{ALG@within}{false}
|
||||
\newcommand{\ALG@floatstyle}{ruled}
|
||||
\newcommand{\ALG@name}{Algorithm}
|
||||
\newcommand{\listalgorithmname}{List of \ALG@name s}
|
||||
|
||||
% Declare Options
|
||||
% first appearance
|
||||
\DeclareOption{plain}{
|
||||
\renewcommand{\ALG@floatstyle}{plain}
|
||||
}
|
||||
\DeclareOption{ruled}{
|
||||
\renewcommand{\ALG@floatstyle}{ruled}
|
||||
}
|
||||
\DeclareOption{boxed}{
|
||||
\renewcommand{\ALG@floatstyle}{boxed}
|
||||
}
|
||||
% then numbering convention
|
||||
\DeclareOption{part}{
|
||||
\renewcommand{\ALG@within}{part}
|
||||
\setboolean{ALG@within}{true}
|
||||
}
|
||||
\DeclareOption{chapter}{
|
||||
\renewcommand{\ALG@within}{chapter}
|
||||
\setboolean{ALG@within}{true}
|
||||
}
|
||||
\DeclareOption{section}{
|
||||
\renewcommand{\ALG@within}{section}
|
||||
\setboolean{ALG@within}{true}
|
||||
}
|
||||
\DeclareOption{subsection}{
|
||||
\renewcommand{\ALG@within}{subsection}
|
||||
\setboolean{ALG@within}{true}
|
||||
}
|
||||
\DeclareOption{subsubsection}{
|
||||
\renewcommand{\ALG@within}{subsubsection}
|
||||
\setboolean{ALG@within}{true}
|
||||
}
|
||||
\DeclareOption{nothing}{
|
||||
\renewcommand{\ALG@within}{nothing}
|
||||
\setboolean{ALG@within}{true}
|
||||
}
|
||||
\DeclareOption*{\edef\ALG@name{\CurrentOption}}
|
||||
|
||||
% ALGORITHM
|
||||
%
|
||||
\ProcessOptions
|
||||
\floatstyle{\ALG@floatstyle}
|
||||
\ifthenelse{\boolean{ALG@within}}{
|
||||
\ifthenelse{\equal{\ALG@within}{part}}
|
||||
{\newfloat{algorithm}{htbp}{loa}[part]}{}
|
||||
\ifthenelse{\equal{\ALG@within}{chapter}}
|
||||
{\newfloat{algorithm}{htbp}{loa}[chapter]}{}
|
||||
\ifthenelse{\equal{\ALG@within}{section}}
|
||||
{\newfloat{algorithm}{htbp}{loa}[section]}{}
|
||||
\ifthenelse{\equal{\ALG@within}{subsection}}
|
||||
{\newfloat{algorithm}{htbp}{loa}[subsection]}{}
|
||||
\ifthenelse{\equal{\ALG@within}{subsubsection}}
|
||||
{\newfloat{algorithm}{htbp}{loa}[subsubsection]}{}
|
||||
\ifthenelse{\equal{\ALG@within}{nothing}}
|
||||
{\newfloat{algorithm}{htbp}{loa}}{}
|
||||
}{
|
||||
\newfloat{algorithm}{htbp}{loa}
|
||||
}
|
||||
\floatname{algorithm}{\ALG@name}
|
||||
|
||||
\newcommand{\listofalgorithms}{\listof{algorithm}{\listalgorithmname}}
|
||||
|
||||
@@ -0,0 +1,201 @@
|
||||
% ALGORITHMIC STYLE -- Released 8 APRIL 1996
|
||||
% for LaTeX version 2e
|
||||
% Copyright -- 1994 Peter Williams
|
||||
% E-mail PeterWilliams@dsto.defence.gov.au
|
||||
%
|
||||
% Modified by Alex Smola (08/2000)
|
||||
% E-mail Alex.Smola@anu.edu.au
|
||||
%
|
||||
\NeedsTeXFormat{LaTeX2e}
|
||||
\ProvidesPackage{algorithmic}
|
||||
\typeout{Document Style `algorithmic' - environment}
|
||||
%
|
||||
\RequirePackage{ifthen}
|
||||
\RequirePackage{calc}
|
||||
\newboolean{ALC@noend}
|
||||
\setboolean{ALC@noend}{false}
|
||||
\newcounter{ALC@line}
|
||||
\newcounter{ALC@rem}
|
||||
\newlength{\ALC@tlm}
|
||||
%
|
||||
\DeclareOption{noend}{\setboolean{ALC@noend}{true}}
|
||||
%
|
||||
\ProcessOptions
|
||||
%
|
||||
% ALGORITHMIC
|
||||
\newcommand{\algorithmicrequire}{\textbf{Require:}}
|
||||
\newcommand{\algorithmicensure}{\textbf{Ensure:}}
|
||||
\newcommand{\algorithmiccomment}[1]{\{#1\}}
|
||||
\newcommand{\algorithmicend}{\textbf{end}}
|
||||
\newcommand{\algorithmicif}{\textbf{if}}
|
||||
\newcommand{\algorithmicthen}{\textbf{then}}
|
||||
\newcommand{\algorithmicelse}{\textbf{else}}
|
||||
\newcommand{\algorithmicelsif}{\algorithmicelse\ \algorithmicif}
|
||||
\newcommand{\algorithmicendif}{\algorithmicend\ \algorithmicif}
|
||||
\newcommand{\algorithmicfor}{\textbf{for}}
|
||||
\newcommand{\algorithmicforall}{\textbf{for all}}
|
||||
\newcommand{\algorithmicdo}{\textbf{do}}
|
||||
\newcommand{\algorithmicendfor}{\algorithmicend\ \algorithmicfor}
|
||||
\newcommand{\algorithmicwhile}{\textbf{while}}
|
||||
\newcommand{\algorithmicendwhile}{\algorithmicend\ \algorithmicwhile}
|
||||
\newcommand{\algorithmicloop}{\textbf{loop}}
|
||||
\newcommand{\algorithmicendloop}{\algorithmicend\ \algorithmicloop}
|
||||
\newcommand{\algorithmicrepeat}{\textbf{repeat}}
|
||||
\newcommand{\algorithmicuntil}{\textbf{until}}
|
||||
|
||||
%changed by alex smola
|
||||
\newcommand{\algorithmicinput}{\textbf{input}}
|
||||
\newcommand{\algorithmicoutput}{\textbf{output}}
|
||||
\newcommand{\algorithmicset}{\textbf{set}}
|
||||
\newcommand{\algorithmictrue}{\textbf{true}}
|
||||
\newcommand{\algorithmicfalse}{\textbf{false}}
|
||||
\newcommand{\algorithmicand}{\textbf{and\ }}
|
||||
\newcommand{\algorithmicor}{\textbf{or\ }}
|
||||
\newcommand{\algorithmicfunction}{\textbf{function}}
|
||||
\newcommand{\algorithmicendfunction}{\algorithmicend\ \algorithmicfunction}
|
||||
\newcommand{\algorithmicmain}{\textbf{main}}
|
||||
\newcommand{\algorithmicendmain}{\algorithmicend\ \algorithmicmain}
|
||||
%end changed by alex smola
|
||||
|
||||
\def\ALC@item[#1]{%
|
||||
\if@noparitem \@donoparitem
|
||||
\else \if@inlabel \indent \par \fi
|
||||
\ifhmode \unskip\unskip \par \fi
|
||||
\if@newlist \if@nobreak \@nbitem \else
|
||||
\addpenalty\@beginparpenalty
|
||||
\addvspace\@topsep \addvspace{-\parskip}\fi
|
||||
\else \addpenalty\@itempenalty \addvspace\itemsep
|
||||
\fi
|
||||
\global\@inlabeltrue
|
||||
\fi
|
||||
\everypar{\global\@minipagefalse\global\@newlistfalse
|
||||
\if@inlabel\global\@inlabelfalse \hskip -\parindent \box\@labels
|
||||
\penalty\z@ \fi
|
||||
\everypar{}}\global\@nobreakfalse
|
||||
\if@noitemarg \@noitemargfalse \if@nmbrlist \refstepcounter{\@listctr}\fi \fi
|
||||
\sbox\@tempboxa{\makelabel{#1}}%
|
||||
\global\setbox\@labels
|
||||
\hbox{\unhbox\@labels \hskip \itemindent
|
||||
\hskip -\labelwidth \hskip -\ALC@tlm
|
||||
\ifdim \wd\@tempboxa >\labelwidth
|
||||
\box\@tempboxa
|
||||
\else \hbox to\labelwidth {\unhbox\@tempboxa}\fi
|
||||
\hskip \ALC@tlm}\ignorespaces}
|
||||
%
|
||||
\newenvironment{algorithmic}[1][0]{
|
||||
\let\@item\ALC@item
|
||||
\newcommand{\ALC@lno}{%
|
||||
\ifthenelse{\equal{\arabic{ALC@rem}}{0}}
|
||||
{{\footnotesize \arabic{ALC@line}:}}{}%
|
||||
}
|
||||
\let\@listii\@listi
|
||||
\let\@listiii\@listi
|
||||
\let\@listiv\@listi
|
||||
\let\@listv\@listi
|
||||
\let\@listvi\@listi
|
||||
\let\@listvii\@listi
|
||||
\newenvironment{ALC@g}{
|
||||
\begin{list}{\ALC@lno}{ \itemsep\z@ \itemindent\z@
|
||||
\listparindent\z@ \rightmargin\z@
|
||||
\topsep\z@ \partopsep\z@ \parskip\z@\parsep\z@
|
||||
\leftmargin 1em
|
||||
\addtolength{\ALC@tlm}{\leftmargin}
|
||||
}
|
||||
}
|
||||
{\end{list}}
|
||||
\newcommand{\ALC@it}{\addtocounter{ALC@line}{1}\addtocounter{ALC@rem}{1}\ifthenelse{\equal{\arabic{ALC@rem}}{#1}}{\setcounter{ALC@rem}{0}}{}\item}
|
||||
\newcommand{\ALC@com}[1]{\ifthenelse{\equal{##1}{default}}%
|
||||
{}{\ \algorithmiccomment{##1}}}
|
||||
\newcommand{\REQUIRE}{\item[\algorithmicrequire]}
|
||||
\newcommand{\ENSURE}{\item[\algorithmicensure]}
|
||||
\newcommand{\STATE}{\ALC@it}
|
||||
\newcommand{\COMMENT}[1]{\algorithmiccomment{##1}}
|
||||
%changes by alex smola
|
||||
\newcommand{\INPUT}{\item[\algorithmicinput]}
|
||||
\newcommand{\OUTPUT}{\item[\algorithmicoutput]}
|
||||
\newcommand{\SET}{\item[\algorithmicset]}
|
||||
% \newcommand{\TRUE}{\algorithmictrue}
|
||||
% \newcommand{\FALSE}{\algorithmicfalse}
|
||||
\newcommand{\AND}{\algorithmicand}
|
||||
\newcommand{\OR}{\algorithmicor}
|
||||
\newenvironment{ALC@func}{\begin{ALC@g}}{\end{ALC@g}}
|
||||
\newenvironment{ALC@main}{\begin{ALC@g}}{\end{ALC@g}}
|
||||
%end changes by alex smola
|
||||
\newenvironment{ALC@if}{\begin{ALC@g}}{\end{ALC@g}}
|
||||
\newenvironment{ALC@for}{\begin{ALC@g}}{\end{ALC@g}}
|
||||
\newenvironment{ALC@whl}{\begin{ALC@g}}{\end{ALC@g}}
|
||||
\newenvironment{ALC@loop}{\begin{ALC@g}}{\end{ALC@g}}
|
||||
\newenvironment{ALC@rpt}{\begin{ALC@g}}{\end{ALC@g}}
|
||||
\renewcommand{\\}{\@centercr}
|
||||
\newcommand{\IF}[2][default]{\ALC@it\algorithmicif\ ##2\ \algorithmicthen%
|
||||
\ALC@com{##1}\begin{ALC@if}}
|
||||
\newcommand{\SHORTIF}[2]{\ALC@it\algorithmicif\ ##1\
|
||||
\algorithmicthen\ {##2}}
|
||||
\newcommand{\ELSE}[1][default]{\end{ALC@if}\ALC@it\algorithmicelse%
|
||||
\ALC@com{##1}\begin{ALC@if}}
|
||||
\newcommand{\ELSIF}[2][default]%
|
||||
{\end{ALC@if}\ALC@it\algorithmicelsif\ ##2\ \algorithmicthen%
|
||||
\ALC@com{##1}\begin{ALC@if}}
|
||||
\newcommand{\FOR}[2][default]{\ALC@it\algorithmicfor\ ##2\ \algorithmicdo%
|
||||
\ALC@com{##1}\begin{ALC@for}}
|
||||
\newcommand{\FORALL}[2][default]{\ALC@it\algorithmicforall\ ##2\ %
|
||||
\algorithmicdo%
|
||||
\ALC@com{##1}\begin{ALC@for}}
|
||||
\newcommand{\SHORTFORALL}[2]{\ALC@it\algorithmicforall\ ##1\ %
|
||||
\algorithmicdo\ {##2}}
|
||||
\newcommand{\WHILE}[2][default]{\ALC@it\algorithmicwhile\ ##2\ %
|
||||
\algorithmicdo%
|
||||
\ALC@com{##1}\begin{ALC@whl}}
|
||||
\newcommand{\LOOP}[1][default]{\ALC@it\algorithmicloop%
|
||||
\ALC@com{##1}\begin{ALC@loop}}
|
||||
%changed by alex smola
|
||||
\newcommand{\FUNCTION}[2][default]{\ALC@it\algorithmicfunction\ ##2\ %
|
||||
\ALC@com{##1}\begin{ALC@func}}
|
||||
\newcommand{\MAIN}[2][default]{\ALC@it\algorithmicmain\ ##2\ %
|
||||
\ALC@com{##1}\begin{ALC@main}}
|
||||
%end changed by alex smola
|
||||
\newcommand{\REPEAT}[1][default]{\ALC@it\algorithmicrepeat%
|
||||
\ALC@com{##1}\begin{ALC@rpt}}
|
||||
\newcommand{\UNTIL}[1]{\end{ALC@rpt}\ALC@it\algorithmicuntil\ ##1}
|
||||
\ifthenelse{\boolean{ALC@noend}}{
|
||||
\newcommand{\ENDIF}{\end{ALC@if}}
|
||||
\newcommand{\ENDFOR}{\end{ALC@for}}
|
||||
\newcommand{\ENDWHILE}{\end{ALC@whl}}
|
||||
\newcommand{\ENDLOOP}{\end{ALC@loop}}
|
||||
\newcommand{\ENDFUNCTION}{\end{ALC@func}}
|
||||
\newcommand{\ENDMAIN}{\end{ALC@main}}
|
||||
}{
|
||||
\newcommand{\ENDIF}{\end{ALC@if}\ALC@it\algorithmicendif}
|
||||
\newcommand{\ENDFOR}{\end{ALC@for}\ALC@it\algorithmicendfor}
|
||||
\newcommand{\ENDWHILE}{\end{ALC@whl}\ALC@it\algorithmicendwhile}
|
||||
\newcommand{\ENDLOOP}{\end{ALC@loop}\ALC@it\algorithmicendloop}
|
||||
\newcommand{\ENDFUNCTION}{\end{ALC@func}\ALC@it\algorithmicendfunction}
|
||||
\newcommand{\ENDMAIN}{\end{ALC@main}\ALC@it\algorithmicendmain}
|
||||
}
|
||||
\renewcommand{\@toodeep}{}
|
||||
\begin{list}{\ALC@lno}{\setcounter{ALC@line}{0}\setcounter{ALC@rem}{0}%
|
||||
\itemsep\z@ \itemindent\z@ \listparindent\z@%
|
||||
\partopsep\z@ \parskip\z@ \parsep\z@%
|
||||
\labelsep 0.5em \topsep 0.2em%
|
||||
\ifthenelse{\equal{#1}{0}}
|
||||
{\labelwidth 0.5em }
|
||||
{\labelwidth 1.2em }
|
||||
\leftmargin\labelwidth \addtolength{\leftmargin}{\labelsep}
|
||||
\ALC@tlm\labelsep
|
||||
}
|
||||
}
|
||||
{\end{list}}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
@inproceedings{langley00,
|
||||
author = {P. Langley},
|
||||
title = {Crafting Papers on Machine Learning},
|
||||
year = {2000},
|
||||
pages = {1207--1216},
|
||||
editor = {Pat Langley},
|
||||
booktitle = {Proceedings of the 17th International Conference
|
||||
on Machine Learning (ICML 2000)},
|
||||
address = {Stanford, CA},
|
||||
publisher = {Morgan Kaufmann}
|
||||
}
|
||||
|
||||
@TechReport{mitchell80,
|
||||
author = "T. M. Mitchell",
|
||||
title = "The Need for Biases in Learning Generalizations",
|
||||
institution = "Computer Science Department, Rutgers University",
|
||||
year = "1980",
|
||||
address = "New Brunswick, MA",
|
||||
}
|
||||
|
||||
@phdthesis{kearns89,
|
||||
author = {M. J. Kearns},
|
||||
title = {Computational Complexity of Machine Learning},
|
||||
school = {Department of Computer Science, Harvard University},
|
||||
year = {1989}
|
||||
}
|
||||
|
||||
@Book{MachineLearningI,
|
||||
editor = "R. S. Michalski and J. G. Carbonell and T.
|
||||
M. Mitchell",
|
||||
title = "Machine Learning: An Artificial Intelligence
|
||||
Approach, Vol. I",
|
||||
publisher = "Tioga",
|
||||
year = "1983",
|
||||
address = "Palo Alto, CA"
|
||||
}
|
||||
|
||||
@Book{DudaHart2nd,
|
||||
author = "R. O. Duda and P. E. Hart and D. G. Stork",
|
||||
title = "Pattern Classification",
|
||||
publisher = "John Wiley and Sons",
|
||||
edition = "2nd",
|
||||
year = "2000"
|
||||
}
|
||||
|
||||
@misc{anonymous,
|
||||
title= {Suppressed for Anonymity},
|
||||
author= {Author, N. N.},
|
||||
year= {2021}
|
||||
}
|
||||
|
||||
@InCollection{Newell81,
|
||||
author = "A. Newell and P. S. Rosenbloom",
|
||||
title = "Mechanisms of Skill Acquisition and the Law of
|
||||
Practice",
|
||||
booktitle = "Cognitive Skills and Their Acquisition",
|
||||
pages = "1--51",
|
||||
publisher = "Lawrence Erlbaum Associates, Inc.",
|
||||
year = "1981",
|
||||
editor = "J. R. Anderson",
|
||||
chapter = "1",
|
||||
address = "Hillsdale, NJ"
|
||||
}
|
||||
|
||||
|
||||
@Article{Samuel59,
|
||||
author = "A. L. Samuel",
|
||||
title = "Some Studies in Machine Learning Using the Game of
|
||||
Checkers",
|
||||
journal = "IBM Journal of Research and Development",
|
||||
year = "1959",
|
||||
volume = "3",
|
||||
number = "3",
|
||||
pages = "211--229"
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,662 @@
|
||||
%%%%%%%% ICML 2026 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%
|
||||
|
||||
\documentclass{article}
|
||||
|
||||
% Recommended, but optional, packages for figures and better typesetting:
|
||||
\usepackage{microtype}
|
||||
\usepackage{graphicx}
|
||||
\usepackage{subcaption}
|
||||
\usepackage{booktabs} % for professional tables
|
||||
|
||||
% hyperref makes hyperlinks in the resulting PDF.
|
||||
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
|
||||
% please comment out the following usepackage line and replace
|
||||
% \usepackage{icml2026} with \usepackage[nohyperref]{icml2026} above.
|
||||
\usepackage{hyperref}
|
||||
|
||||
|
||||
% Attempt to make hyperref and algorithmic work together better:
|
||||
\newcommand{\theHalgorithm}{\arabic{algorithm}}
|
||||
|
||||
% Use the following line for the initial blind version submitted for review:
|
||||
\usepackage{icml2026}
|
||||
|
||||
% For preprint, use
|
||||
% \usepackage[preprint]{icml2026}
|
||||
|
||||
% If accepted, instead use the following line for the camera-ready submission:
|
||||
% \usepackage[accepted]{icml2026}
|
||||
|
||||
\usepackage{amsmath}
|
||||
\usepackage{amssymb}
|
||||
\usepackage{mathtools}
|
||||
\usepackage{amsthm}
|
||||
|
||||
|
||||
% if you use cleveref..
|
||||
\usepackage[capitalize,noabbrev]{cleveref}
|
||||
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
% THEOREMS
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
\theoremstyle{plain}
|
||||
\newtheorem{theorem}{Theorem}[section]
|
||||
\newtheorem{proposition}[theorem]{Proposition}
|
||||
\newtheorem{lemma}[theorem]{Lemma}
|
||||
\newtheorem{corollary}[theorem]{Corollary}
|
||||
\theoremstyle{definition}
|
||||
\newtheorem{definition}[theorem]{Definition}
|
||||
\newtheorem{assumption}[theorem]{Assumption}
|
||||
\theoremstyle{remark}
|
||||
\newtheorem{remark}[theorem]{Remark}
|
||||
|
||||
% Todonotes is useful during development; simply uncomment the next line
|
||||
% and comment out the line below the next line to turn off comments
|
||||
%\usepackage[disable,textsize=tiny]{todonotes}
|
||||
\usepackage[textsize=tiny]{todonotes}
|
||||
|
||||
% The \icmltitle you define below is probably too long as a header.
|
||||
% Therefore, a short form for the running title is supplied here:
|
||||
\icmltitlerunning{Submission and Formatting Instructions for ICML 2026}
|
||||
|
||||
\begin{document}
|
||||
|
||||
\twocolumn[
|
||||
\icmltitle{Submission and Formatting Instructions for \\
|
||||
International Conference on Machine Learning (ICML 2026)}
|
||||
|
||||
% It is OKAY to include author information, even for blind submissions: the
|
||||
% style file will automatically remove it for you unless you've provided
|
||||
% the [accepted] option to the icml2026 package.
|
||||
|
||||
% List of affiliations: The first argument should be a (short) identifier you
|
||||
% will use later to specify author affiliations Academic affiliations
|
||||
% should list Department, University, City, Region, Country Industry
|
||||
% affiliations should list Company, City, Region, Country
|
||||
|
||||
% You can specify symbols, otherwise they are numbered in order. Ideally, you
|
||||
% should not use this facility. Affiliations will be numbered in order of
|
||||
% appearance and this is the preferred way.
|
||||
\icmlsetsymbol{equal}{*}
|
||||
|
||||
\begin{icmlauthorlist}
|
||||
\icmlauthor{Firstname1 Lastname1}{equal,yyy}
|
||||
\icmlauthor{Firstname2 Lastname2}{equal,yyy,comp}
|
||||
\icmlauthor{Firstname3 Lastname3}{comp}
|
||||
\icmlauthor{Firstname4 Lastname4}{sch}
|
||||
\icmlauthor{Firstname5 Lastname5}{yyy}
|
||||
\icmlauthor{Firstname6 Lastname6}{sch,yyy,comp}
|
||||
\icmlauthor{Firstname7 Lastname7}{comp}
|
||||
%\icmlauthor{}{sch}
|
||||
\icmlauthor{Firstname8 Lastname8}{sch}
|
||||
\icmlauthor{Firstname8 Lastname8}{yyy,comp}
|
||||
%\icmlauthor{}{sch}
|
||||
%\icmlauthor{}{sch}
|
||||
\end{icmlauthorlist}
|
||||
|
||||
\icmlaffiliation{yyy}{Department of XXX, University of YYY, Location, Country}
|
||||
\icmlaffiliation{comp}{Company Name, Location, Country}
|
||||
\icmlaffiliation{sch}{School of ZZZ, Institute of WWW, Location, Country}
|
||||
|
||||
\icmlcorrespondingauthor{Firstname1 Lastname1}{first1.last1@xxx.edu}
|
||||
\icmlcorrespondingauthor{Firstname2 Lastname2}{first2.last2@www.uk}
|
||||
|
||||
% You may provide any keywords that you find helpful for describing your
|
||||
% paper; these are used to populate the "keywords" metadata in the PDF but
|
||||
% will not be shown in the document
|
||||
\icmlkeywords{Machine Learning, ICML}
|
||||
|
||||
\vskip 0.3in
|
||||
]
|
||||
|
||||
% this must go after the closing bracket ] following \twocolumn[ ...
|
||||
|
||||
% This command actually creates the footnote in the first column listing the
|
||||
% affiliations and the copyright notice. The command takes one argument, which
|
||||
% is text to display at the start of the footnote. The \icmlEqualContribution
|
||||
% command is standard text for equal contribution. Remove it (just {}) if you
|
||||
% do not need this facility.
|
||||
|
||||
% Use ONE of the following lines. DO NOT remove the command.
|
||||
% If you have no special notice, KEEP empty braces:
|
||||
\printAffiliationsAndNotice{} % no special notice (required even if empty)
|
||||
% Or, if applicable, use the standard equal contribution text:
|
||||
% \printAffiliationsAndNotice{\icmlEqualContribution}
|
||||
|
||||
\begin{abstract}
|
||||
This document provides a basic paper template and submission guidelines.
|
||||
Abstracts must be a single paragraph, ideally between 4--6 sentences long.
|
||||
Gross violations will trigger corrections at the camera-ready phase.
|
||||
\end{abstract}
|
||||
|
||||
\section{Electronic Submission}
|
||||
|
||||
Submission to ICML 2026 will be entirely electronic, via a web site
|
||||
(not email). Information about the submission process and \LaTeX\ templates
|
||||
are available on the conference web site at:
|
||||
\begin{center}
|
||||
\texttt{http://icml.cc/}
|
||||
\end{center}
|
||||
|
||||
The guidelines below will be enforced for initial submissions and
|
||||
camera-ready copies. Here is a brief summary:
|
||||
\begin{itemize}
|
||||
\item Submissions must be in PDF\@.
|
||||
\item If your paper has appendices, submit the appendix together with the
|
||||
main body and the references \textbf{as a single file}. Reviewers will not
|
||||
look for appendices as a separate PDF file. So if you submit such an extra
|
||||
file, reviewers will very likely miss it.
|
||||
\item Page limit: The main body of the paper has to be fitted to 8 pages,
|
||||
excluding references and appendices; the space for the latter two is not
|
||||
limited in pages, but the total file size may not exceed 10MB. For the
|
||||
final version of the paper, authors can add one extra page to the main
|
||||
body.
|
||||
\item \textbf{Do not include author information or acknowledgements} in your
|
||||
initial submission.
|
||||
\item Your paper should be in \textbf{10 point Times font}.
|
||||
\item Make sure your PDF file only uses Type-1 fonts.
|
||||
\item Place figure captions \emph{under} the figure (and omit titles from
|
||||
inside the graphic file itself). Place table captions \emph{over} the
|
||||
table.
|
||||
\item References must include page numbers whenever possible and be as
|
||||
complete as possible. Place multiple citations in chronological order.
|
||||
\item Do not alter the style template; in particular, do not compress the
|
||||
paper format by reducing the vertical spaces.
|
||||
\item Keep your abstract brief and self-contained, one paragraph and roughly
|
||||
4--6 sentences. Gross violations will require correction at the
|
||||
camera-ready phase. The title should have content words capitalized.
|
||||
\end{itemize}
|
||||
|
||||
\subsection{Submitting Papers}
|
||||
|
||||
\textbf{Anonymous Submission:} ICML uses double-blind review: no identifying
|
||||
author information may appear on the title page or in the paper
|
||||
itself. \cref{author info} gives further details.
|
||||
|
||||
\medskip
|
||||
|
||||
Authors must provide their manuscripts in \textbf{PDF} format.
|
||||
Furthermore, please make sure that files contain only embedded Type-1 fonts
|
||||
(e.g.,~using the program \texttt{pdffonts} in linux or using
|
||||
File/DocumentProperties/Fonts in Acrobat). Other fonts (like Type-3)
|
||||
might come from graphics files imported into the document.
|
||||
|
||||
Authors using \textbf{Word} must convert their document to PDF\@. Most
|
||||
of the latest versions of Word have the facility to do this
|
||||
automatically. Submissions will not be accepted in Word format or any
|
||||
format other than PDF\@. Really. We're not joking. Don't send Word.
|
||||
|
||||
Those who use \textbf{\LaTeX} should avoid including Type-3 fonts.
|
||||
Those using \texttt{latex} and \texttt{dvips} may need the following
|
||||
two commands:
|
||||
|
||||
{\footnotesize
|
||||
\begin{verbatim}
|
||||
dvips -Ppdf -tletter -G0 -o paper.ps paper.dvi
|
||||
ps2pdf paper.ps
|
||||
\end{verbatim}}
|
||||
It is a zero following the ``-G'', which tells dvips to use
|
||||
the config.pdf file. Newer \TeX\ distributions don't always need this
|
||||
option.
|
||||
|
||||
Using \texttt{pdflatex} rather than \texttt{latex}, often gives better
|
||||
results. This program avoids the Type-3 font problem, and supports more
|
||||
advanced features in the \texttt{microtype} package.
|
||||
|
||||
\textbf{Graphics files} should be a reasonable size, and included from
|
||||
an appropriate format. Use vector formats (.eps/.pdf) for plots,
|
||||
lossless bitmap formats (.png) for raster graphics with sharp lines, and
|
||||
jpeg for photo-like images.
|
||||
|
||||
The style file uses the \texttt{hyperref} package to make clickable
|
||||
links in documents. If this causes problems for you, add
|
||||
\texttt{nohyperref} as one of the options to the \texttt{icml2026}
|
||||
usepackage statement.
|
||||
|
||||
\subsection{Submitting Final Camera-Ready Copy}
|
||||
|
||||
The final versions of papers accepted for publication should follow the
|
||||
same format and naming convention as initial submissions, except that
|
||||
author information (names and affiliations) should be given. See
|
||||
\cref{final author} for formatting instructions.
|
||||
|
||||
The footnote, ``Preliminary work. Under review by the International
|
||||
Conference on Machine Learning (ICML). Do not distribute.'' must be
|
||||
modified to ``\textit{Proceedings of the
|
||||
$\mathit{43}^{rd}$ International Conference on Machine Learning},
|
||||
Seoul, South Korea, PMLR 306, 2026.
|
||||
Copyright 2026 by the author(s).''
|
||||
|
||||
For those using the \textbf{\LaTeX} style file, this change (and others) is
|
||||
handled automatically by simply changing
|
||||
$\mathtt{\backslash usepackage\{icml2026\}}$ to
|
||||
$$\mathtt{\backslash usepackage[accepted]\{icml2026\}}$$
|
||||
Authors using \textbf{Word} must edit the
|
||||
footnote on the first page of the document themselves.
|
||||
|
||||
Camera-ready copies should have the title of the paper as running head
|
||||
on each page except the first one. The running title consists of a
|
||||
single line centered above a horizontal rule which is $1$~point thick.
|
||||
The running head should be centered, bold and in $9$~point type. The
|
||||
rule should be $10$~points above the main text. For those using the
|
||||
\textbf{\LaTeX} style file, the original title is automatically set as running
|
||||
head using the \texttt{fancyhdr} package which is included in the ICML
|
||||
2026 style file package. In case that the original title exceeds the
|
||||
size restrictions, a shorter form can be supplied by using
|
||||
|
||||
\verb|\icmltitlerunning{...}|
|
||||
|
||||
just before $\mathtt{\backslash begin\{document\}}$.
|
||||
Authors using \textbf{Word} must edit the header of the document themselves.
|
||||
|
||||
\section{Format of the Paper}
|
||||
|
||||
All submissions must follow the specified format.
|
||||
|
||||
\subsection{Dimensions}
|
||||
|
||||
The text of the paper should be formatted in two columns, with an
|
||||
overall width of 6.75~inches, height of 9.0~inches, and 0.25~inches
|
||||
between the columns. The left margin should be 0.75~inches and the top
|
||||
margin 1.0~inch (2.54~cm). The right and bottom margins will depend on
|
||||
whether you print on US letter or A4 paper, but all final versions
|
||||
must be produced for US letter size.
|
||||
Do not write anything on the margins.
|
||||
|
||||
The paper body should be set in 10~point type with a vertical spacing
|
||||
of 11~points. Please use Times typeface throughout the text.
|
||||
|
||||
\subsection{Title}
|
||||
|
||||
The paper title should be set in 14~point bold type and centered
|
||||
between two horizontal rules that are 1~point thick, with 1.0~inch
|
||||
between the top rule and the top edge of the page. Capitalize the
|
||||
first letter of content words and put the rest of the title in lower
|
||||
case.
|
||||
You can use TeX math in the title (we suggest sparingly),
|
||||
but no custom macros, images, or other TeX commands.
|
||||
Please make sure that accents, special characters, etc., are entered using
|
||||
TeX commands and not using non-English characters.
|
||||
|
||||
\subsection{Author Information for Submission}
|
||||
\label{author info}
|
||||
|
||||
ICML uses double-blind review, so author information must not appear. If
|
||||
you are using \LaTeX\/ and the \texttt{icml2026.sty} file, use
|
||||
\verb+\icmlauthor{...}+ to specify authors and \verb+\icmlaffiliation{...}+
|
||||
to specify affiliations. (Read the TeX code used to produce this document for
|
||||
an example usage.) The author information will not be printed unless
|
||||
\texttt{accepted} is passed as an argument to the style file. Submissions that
|
||||
include the author information will not be reviewed.
|
||||
|
||||
\subsubsection{Self-Citations}
|
||||
|
||||
If you are citing published papers for which you are an author, refer
|
||||
to yourself in the third person. In particular, do not use phrases
|
||||
that reveal your identity (e.g., ``in previous work \cite{langley00}, we
|
||||
have shown \ldots'').
|
||||
|
||||
Do not anonymize citations in the reference section. The only exception are manuscripts that are
|
||||
not yet published (e.g., under submission). If you choose to refer to
|
||||
such unpublished manuscripts \cite{anonymous}, anonymized copies have
|
||||
to be submitted
|
||||
as Supplementary Material via OpenReview\@. However, keep in mind that an ICML
|
||||
paper should be self contained and should contain sufficient detail
|
||||
for the reviewers to evaluate the work. In particular, reviewers are
|
||||
not required to look at the Supplementary Material when writing their
|
||||
review (they are not required to look at more than the first $8$ pages of the submitted document).
|
||||
|
||||
\subsubsection{Camera-Ready Author Information}
|
||||
\label{final author}
|
||||
|
||||
If a paper is accepted, a final camera-ready copy must be prepared.
|
||||
%
|
||||
For camera-ready papers, author information should start 0.3~inches below the
|
||||
bottom rule surrounding the title. The authors' names should appear in 10~point
|
||||
bold type, in a row, separated by white space, and centered. Author names should
|
||||
not be broken across lines. Unbolded superscripted numbers, starting 1, should
|
||||
be used to refer to affiliations.
|
||||
|
||||
Affiliations should be numbered in the order of appearance. A single footnote
|
||||
block of text should be used to list all the affiliations. (Academic
|
||||
affiliations should list Department, University, City, State/Region, Country.
|
||||
Similarly for industrial affiliations.)
|
||||
|
||||
Each distinct affiliations should be listed once. If an author has multiple
|
||||
affiliations, multiple superscripts should be placed after the name, separated
|
||||
by thin spaces. If the authors would like to highlight equal contribution by
|
||||
multiple first authors, those authors should have an asterisk placed after their
|
||||
name in superscript, and the term ``\textsuperscript{*}Equal contribution"
|
||||
should be placed in the footnote block ahead of the list of affiliations. A
|
||||
list of corresponding authors and their emails (in the format Full Name
|
||||
\textless{}email@domain.com\textgreater{}) can follow the list of affiliations.
|
||||
Ideally only one or two names should be listed.
|
||||
|
||||
A sample file with author names is included in the ICML2026 style file
|
||||
package. Turn on the \texttt{[accepted]} option to the stylefile to
|
||||
see the names rendered. All of the guidelines above are implemented
|
||||
by the \LaTeX\ style file.
|
||||
|
||||
\subsection{Abstract}
|
||||
|
||||
The paper abstract should begin in the left column, 0.4~inches below the final
|
||||
address. The heading `Abstract' should be centered, bold, and in 11~point type.
|
||||
The abstract body should use 10~point type, with a vertical spacing of
|
||||
11~points, and should be indented 0.25~inches more than normal on left-hand and
|
||||
right-hand margins. Insert 0.4~inches of blank space after the body. Keep your
|
||||
abstract brief and self-contained, limiting it to one paragraph and roughly 4--6
|
||||
sentences. Gross violations will require correction at the camera-ready phase.
|
||||
|
||||
\subsection{Partitioning the Text}
|
||||
|
||||
You should organize your paper into sections and paragraphs to help readers
|
||||
place a structure on the material and understand its contributions.
|
||||
|
||||
\subsubsection{Sections and Subsections}
|
||||
|
||||
Section headings should be numbered, flush left, and set in 11~pt bold type
|
||||
with the content words capitalized. Leave 0.25~inches of space before the
|
||||
heading and 0.15~inches after the heading.
|
||||
|
||||
Similarly, subsection headings should be numbered, flush left, and set in 10~pt
|
||||
bold type with the content words capitalized. Leave
|
||||
0.2~inches of space before the heading and 0.13~inches afterward.
|
||||
|
||||
Finally, subsubsection headings should be numbered, flush left, and set in
|
||||
10~pt small caps with the content words capitalized. Leave
|
||||
0.18~inches of space before the heading and 0.1~inches after the heading.
|
||||
|
||||
Please use no more than three levels of headings.
|
||||
|
||||
\subsubsection{Paragraphs and Footnotes}
|
||||
|
||||
Within each section or subsection, you should further partition the paper into
|
||||
paragraphs. Do not indent the first line of a given paragraph, but insert a
|
||||
blank line between succeeding ones.
|
||||
|
||||
You can use footnotes\footnote{Footnotes should be complete sentences.}
|
||||
to provide readers with additional information about a topic without
|
||||
interrupting the flow of the paper. Indicate footnotes with a number in the
|
||||
text where the point is most relevant. Place the footnote in 9~point type at
|
||||
the bottom of the column in which it appears. Precede the first footnote in a
|
||||
column with a horizontal rule of 0.8~inches.\footnote{Multiple footnotes can
|
||||
appear in each column, in the same order as they appear in the text,
|
||||
but spread them across columns and pages if possible.}
|
||||
|
||||
\begin{figure}[ht]
|
||||
\vskip 0.2in
|
||||
\begin{center}
|
||||
\centerline{\includegraphics[width=\columnwidth]{icml_numpapers}}
|
||||
\caption{
|
||||
Historical locations and number of accepted papers for International
|
||||
Machine Learning Conferences (ICML 1993 -- ICML 2008) and International
|
||||
Workshops on Machine Learning (ML 1988 -- ML 1992). At the time this
|
||||
figure was produced, the number of accepted papers for ICML 2008 was
|
||||
unknown and instead estimated.
|
||||
}
|
||||
\label{icml-historical}
|
||||
\end{center}
|
||||
\end{figure}
|
||||
|
||||
\subsection{Figures}
|
||||
|
||||
You may want to include figures in the paper to illustrate your approach and
|
||||
results. Such artwork should be centered, legible, and separated from the text.
|
||||
Lines should be dark and at least 0.5~points thick for purposes of
|
||||
reproduction, and text should not appear on a gray background.
|
||||
|
||||
Label all distinct components of each figure. If the figure takes the form of a
|
||||
graph, then give a name for each axis and include a legend that briefly
|
||||
describes each curve. Do not include a title inside the figure; instead, the
|
||||
caption should serve this function.
|
||||
|
||||
Number figures sequentially, placing the figure number and caption \emph{after}
|
||||
the graphics, with at least 0.1~inches of space before the caption and
|
||||
0.1~inches after it, as in \cref{icml-historical}. The figure caption should be
|
||||
set in 9~point type and centered unless it runs two or more lines, in which
|
||||
case it should be flush left. You may float figures to the top or bottom of a
|
||||
column, and you may set wide figures across both columns (use the environment
|
||||
\texttt{figure*} in \LaTeX). Always place two-column figures at the top or
|
||||
bottom of the page.
|
||||
|
||||
\subsection{Algorithms}
|
||||
|
||||
If you are using \LaTeX, please use the ``algorithm'' and ``algorithmic''
|
||||
environments to format pseudocode. These require the corresponding stylefiles,
|
||||
algorithm.sty and algorithmic.sty, which are supplied with this package.
|
||||
\cref{alg:example} shows an example.
|
||||
|
||||
\begin{algorithm}[tb]
|
||||
\caption{Bubble Sort}
|
||||
\label{alg:example}
|
||||
\begin{algorithmic}
|
||||
\STATE {\bfseries Input:} data $x_i$, size $m$
|
||||
\REPEAT
|
||||
\STATE Initialize $noChange = true$.
|
||||
\FOR{$i=1$ {\bfseries to} $m-1$}
|
||||
\IF{$x_i > x_{i+1}$}
|
||||
\STATE Swap $x_i$ and $x_{i+1}$
|
||||
\STATE $noChange = false$
|
||||
\ENDIF
|
||||
\ENDFOR
|
||||
\UNTIL{$noChange$ is $true$}
|
||||
\end{algorithmic}
|
||||
\end{algorithm}
|
||||
|
||||
|
||||
\subsection{Tables}
|
||||
|
||||
You may also want to include tables that summarize material. Like figures,
|
||||
these should be centered, legible, and numbered consecutively. However, place
|
||||
the title \emph{above} the table with at least 0.1~inches of space before the
|
||||
title and the same after it, as in \cref{sample-table}. The table title should
|
||||
be set in 9~point type and centered unless it runs two or more lines, in which
|
||||
case it should be flush left.
|
||||
|
||||
% Note use of \abovespace and \belowspace to get reasonable spacing
|
||||
% above and below tabular lines.
|
||||
|
||||
\begin{table}[t]
|
||||
\caption{Classification accuracies for naive Bayes and flexible
|
||||
Bayes on various data sets.}
|
||||
\label{sample-table}
|
||||
\begin{center}
|
||||
\begin{small}
|
||||
\begin{sc}
|
||||
\begin{tabular}{lcccr}
|
||||
\toprule
|
||||
Data set & Naive & Flexible & Better? \\
|
||||
\midrule
|
||||
Breast & 95.9$\pm$ 0.2 & 96.7$\pm$ 0.2 & $\surd$ \\
|
||||
Cleveland & 83.3$\pm$ 0.6 & 80.0$\pm$ 0.6 & $\times$ \\
|
||||
Glass2 & 61.9$\pm$ 1.4 & 83.8$\pm$ 0.7 & $\surd$ \\
|
||||
Credit & 74.8$\pm$ 0.5 & 78.3$\pm$ 0.6 & \\
|
||||
Horse & 73.3$\pm$ 0.9 & 69.7$\pm$ 1.0 & $\times$ \\
|
||||
Meta & 67.1$\pm$ 0.6 & 76.5$\pm$ 0.5 & $\surd$ \\
|
||||
Pima & 75.1$\pm$ 0.6 & 73.9$\pm$ 0.5 & \\
|
||||
Vehicle & 44.9$\pm$ 0.6 & 61.5$\pm$ 0.4 & $\surd$ \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\end{sc}
|
||||
\end{small}
|
||||
\end{center}
|
||||
\vskip -0.1in
|
||||
\end{table}
|
||||
|
||||
Tables contain textual material, whereas figures contain graphical material.
|
||||
Specify the contents of each row and column in the table's topmost row. Again,
|
||||
you may float tables to a column's top or bottom, and set wide tables across
|
||||
both columns. Place two-column tables at the top or bottom of the page.
|
||||
|
||||
\subsection{Theorems and Such}
|
||||
The preferred way is to number definitions, propositions, lemmas, etc.
|
||||
consecutively, within sections, as shown below.
|
||||
\begin{definition}
|
||||
\label{def:inj}
|
||||
A function $f:X \to Y$ is injective if for any $x,y\in X$ different, $f(x)\ne
|
||||
f(y)$.
|
||||
\end{definition}
|
||||
Using \cref{def:inj} we immediate get the following result:
|
||||
\begin{proposition}
|
||||
If $f$ is injective mapping a set $X$ to another set $Y$,
|
||||
the cardinality of $Y$ is at least as large as that of $X$
|
||||
\end{proposition}
|
||||
\begin{proof}
|
||||
Left as an exercise to the reader.
|
||||
\end{proof}
|
||||
\cref{lem:usefullemma} stated next will prove to be useful.
|
||||
\begin{lemma}
|
||||
\label{lem:usefullemma}
|
||||
For any $f:X \to Y$ and $g:Y\to Z$ injective functions, $f \circ g$ is
|
||||
injective.
|
||||
\end{lemma}
|
||||
\begin{theorem}
|
||||
\label{thm:bigtheorem}
|
||||
If $f:X\to Y$ is bijective, the cardinality of $X$ and $Y$ are the same.
|
||||
\end{theorem}
|
||||
An easy corollary of \cref{thm:bigtheorem} is the following:
|
||||
\begin{corollary}
|
||||
If $f:X\to Y$ is bijective,
|
||||
the cardinality of $X$ is at least as large as that of $Y$.
|
||||
\end{corollary}
|
||||
\begin{assumption}
|
||||
The set $X$ is finite.
|
||||
\label{ass:xfinite}
|
||||
\end{assumption}
|
||||
\begin{remark}
|
||||
According to some, it is only the finite case (cf. \cref{ass:xfinite}) that
|
||||
is interesting.
|
||||
\end{remark}
|
||||
%restatable
|
||||
|
||||
\subsection{Citations and References}
|
||||
|
||||
Please use APA reference format regardless of your formatter or word processor.
|
||||
If you rely on the \LaTeX\/ bibliographic facility, use \texttt{natbib.sty} and
|
||||
\texttt{icml2026.bst} included in the style-file package to obtain this format.
|
||||
|
||||
Citations within the text should include the authors' last names and year. If
|
||||
the authors' names are included in the sentence, place only the year in
|
||||
parentheses, for example when referencing Arthur Samuel's pioneering work
|
||||
\yrcite{Samuel59}. Otherwise place the entire reference in parentheses with the
|
||||
authors and year separated by a comma \cite{Samuel59}. List multiple references
|
||||
separated by semicolons \cite{kearns89,Samuel59,mitchell80}. Use the `et~al.'
|
||||
construct only for citations with three or more authors or after listing all
|
||||
authors to a publication in an earlier reference \cite{MachineLearningI}.
|
||||
|
||||
Authors should cite their own work in the third person in the initial version
|
||||
of their paper submitted for blind review. Please refer to \cref{author info}
|
||||
for detailed instructions on how to cite your own papers.
|
||||
|
||||
Use an unnumbered first-level section heading for the references, and use a
|
||||
hanging indent style, with the first line of the reference flush against the
|
||||
left margin and subsequent lines indented by 10 points. The references at the
|
||||
end of this document give examples for journal articles \cite{Samuel59},
|
||||
conference publications \cite{langley00}, book chapters \cite{Newell81}, books
|
||||
\cite{DudaHart2nd}, edited volumes \cite{MachineLearningI}, technical reports
|
||||
\cite{mitchell80}, and dissertations \cite{kearns89}.
|
||||
|
||||
Alphabetize references by the surnames of the first authors, with single author
|
||||
entries preceding multiple author entries. Order references for the same
|
||||
authors by year of publication, with the earliest first. Make sure that each
|
||||
reference includes all relevant information (e.g., page numbers).
|
||||
|
||||
Please put some effort into making references complete, presentable, and
|
||||
consistent, e.g. use the actual current name of authors. If using bibtex,
|
||||
please protect capital letters of names and abbreviations in titles, for
|
||||
example, use \{B\}ayesian or \{L\}ipschitz in your .bib file.
|
||||
|
||||
\section*{Accessibility}
|
||||
|
||||
Authors are kindly asked to make their submissions as accessible as possible
|
||||
for everyone including people with disabilities and sensory or neurological
|
||||
differences. Tips of how to achieve this and what to pay attention to will be
|
||||
provided on the conference website \url{http://icml.cc/}.
|
||||
|
||||
\section*{Software and Data}
|
||||
|
||||
If a paper is accepted, we strongly encourage the publication of software and
|
||||
data with the camera-ready version of the paper whenever appropriate. This can
|
||||
be done by including a URL in the camera-ready copy. However, \textbf{do not}
|
||||
include URLs that reveal your institution or identity in your submission for
|
||||
review. Instead, provide an anonymous URL or upload the material as
|
||||
``Supplementary Material'' into the OpenReview reviewing system. Note that
|
||||
reviewers are not required to look at this material when writing their review.
|
||||
|
||||
% Acknowledgements should only appear in the accepted version.
|
||||
\section*{Acknowledgements}
|
||||
|
||||
\textbf{Do not} include acknowledgements in the initial version of the paper
|
||||
submitted for blind review.
|
||||
|
||||
If a paper is accepted, the final camera-ready version can (and usually should)
|
||||
include acknowledgements. Such acknowledgements should be placed at the end of
|
||||
the section, in an unnumbered section that does not count towards the paper
|
||||
page limit. Typically, this will include thanks to reviewers who gave useful
|
||||
comments, to colleagues who contributed to the ideas, and to funding agencies
|
||||
and corporate sponsors that provided financial support.
|
||||
|
||||
\section*{Impact Statement}
|
||||
|
||||
Authors are \textbf{required} to include a statement of the potential broader
|
||||
impact of their work, including its ethical aspects and future societal
|
||||
consequences. This statement should be in an unnumbered section at the end of
|
||||
the paper (co-located with Acknowledgements -- the two may appear in either
|
||||
order, but both must be before References), and does not count toward the paper
|
||||
page limit. In many cases, where the ethical impacts and expected societal
|
||||
implications are those that are well established when advancing the field of
|
||||
Machine Learning, substantial discussion is not required, and a simple
|
||||
statement such as the following will suffice:
|
||||
|
||||
``This paper presents work whose goal is to advance the field of Machine
|
||||
Learning. There are many potential societal consequences of our work, none
|
||||
which we feel must be specifically highlighted here.''
|
||||
|
||||
The above statement can be used verbatim in such cases, but we encourage
|
||||
authors to think about whether there is content which does warrant further
|
||||
discussion, as this statement will be apparent if the paper is later flagged
|
||||
for ethics review.
|
||||
|
||||
% In the unusual situation where you want a paper to appear in the
|
||||
% references without citing it in the main text, use \nocite
|
||||
\nocite{langley00}
|
||||
|
||||
\bibliography{example_paper}
|
||||
\bibliographystyle{icml2026}
|
||||
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
% APPENDIX
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
\newpage
|
||||
\appendix
|
||||
\onecolumn
|
||||
\section{You \emph{can} have an appendix here.}
|
||||
|
||||
You can have as much text here as you want. The main body must be at most $8$
|
||||
pages long. For the final version, one more page can be added. If you want, you
|
||||
can use an appendix like this one.
|
||||
|
||||
The $\mathtt{\backslash onecolumn}$ command above can be kept in place if you
|
||||
prefer a one-column appendix, or can be removed if you prefer a two-column
|
||||
appendix. Apart from this possible change, the style (font size, spacing,
|
||||
margins, page numbering, etc.) should be kept the same as the main body.
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
|
||||
\end{document}
|
||||
|
||||
% This document was modified from the file originally made available by
|
||||
% Pat Langley and Andrea Danyluk for ICML-2K. This version was created
|
||||
% by Iain Murray in 2018, and modified by Alexandre Bouchard in
|
||||
% 2019 and 2021 and by Csaba Szepesvari, Gang Niu and Sivan Sabato in 2022.
|
||||
% Modified again in 2023 and 2024 by Sivan Sabato and Jonathan Scarlett.
|
||||
% Previous contributors include Dan Roy, Lise Getoor and Tobias
|
||||
% Scheffer, which was slightly modified from the 2010 version by
|
||||
% Thorsten Joachims & Johannes Fuernkranz, slightly modified from the
|
||||
% 2009 version by Kiri Wagstaff and Sam Roweis's 2008 version, which is
|
||||
% slightly modified from Prasad Tadepalli's 2007 version which is a
|
||||
% lightly changed version of the previous year's version by Andrew
|
||||
% Moore, which was in turn edited from those of Kristian Kersting and
|
||||
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.
|
||||
@@ -0,0 +1,864 @@
|
||||
%%
|
||||
%% This is file `fancyhdr.sty',
|
||||
%% generated with the docstrip utility.
|
||||
%%
|
||||
%% The original source files were:
|
||||
%%
|
||||
%% fancyhdr.dtx (with options: `fancyhdr')
|
||||
%%
|
||||
%% This is a generated file.
|
||||
%%
|
||||
%% This file may be distributed and/or modified under the conditions of
|
||||
%% the LaTeX Project Public License, either version 1.3 of this license
|
||||
%% or (at your option) any later version. The latest version of this
|
||||
%% license is in:
|
||||
%%
|
||||
%% http://www.latex-project.org/lppl.txt
|
||||
%%
|
||||
%% and version 1.3 or later is part of all distributions of LaTeX version
|
||||
%% 2005/12/01 or later.
|
||||
%%
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
\NeedsTeXFormat{LaTeX2e}[2018-04-01]
|
||||
\ProvidesPackage{fancyhdr}%
|
||||
[2025/02/07 v5.2
|
||||
Extensive control of page headers and footers]%
|
||||
% Copyright (C) 1994-2025 by Pieter van Oostrum <pieter@vanoostrum.org>
|
||||
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
|
||||
\ifdefined\NewDocumentCommand\else\RequirePackage{xparse}\fi
|
||||
\newif\iff@nch@check
|
||||
\f@nch@checktrue
|
||||
\DeclareOption{nocheck}{%
|
||||
\f@nch@checkfalse
|
||||
}
|
||||
\let\f@nch@gbl\relax
|
||||
\newif\iff@nch@compatViii
|
||||
\DeclareOption{compatV3}{%
|
||||
\PackageWarningNoLine{fancyhdr}{The `compatV3' option is deprecated.\MessageBreak
|
||||
It will disappear in one of the following releases.\MessageBreak
|
||||
Please change your document to work\MessageBreak
|
||||
without this option}
|
||||
\let\f@nch@gbl\global
|
||||
\f@nch@compatViiitrue
|
||||
}
|
||||
\newif\iff@nch@twoside
|
||||
\f@nch@twosidefalse
|
||||
\DeclareOption{twoside}{%
|
||||
\if@twoside\else\f@nch@twosidetrue\fi
|
||||
}
|
||||
\newcommand\f@nch@def[2]{%
|
||||
\def\temp@a{#2}\ifx\temp@a\@empty\f@nch@gbl\def#1{}%
|
||||
\else\f@nch@gbl\def#1{#2\strut}\fi}
|
||||
\DeclareOption{myheadings}{%
|
||||
\@ifundefined{chapter}{%
|
||||
\def\ps@myheadings{\ps@f@nch@fancyproto \let\@mkboth\@gobbletwo
|
||||
\fancyhf{}
|
||||
\fancyhead[LE,RO]{\thepage}%
|
||||
\fancyhead[RE]{\slshape\leftmark}%
|
||||
\fancyhead[LO]{\slshape\rightmark}%
|
||||
\let\sectionmark\@gobble
|
||||
\let\subsectionmark\@gobble
|
||||
}%
|
||||
}%
|
||||
{\def\ps@myheadings{\ps@f@nch@fancyproto \let\@mkboth\@gobbletwo
|
||||
\fancyhf{}
|
||||
\fancyhead[LE,RO]{\thepage}%
|
||||
\fancyhead[RE]{\slshape\leftmark}%
|
||||
\fancyhead[LO]{\slshape\rightmark}%
|
||||
\let\chaptermark\@gobble
|
||||
\let\sectionmark\@gobble
|
||||
}%
|
||||
}%
|
||||
}
|
||||
\DeclareOption{headings}{%
|
||||
\@ifundefined{chapter}{%
|
||||
\if@twoside
|
||||
\def\ps@headings{\ps@f@nch@fancyproto \def\@mkboth{\protect\markboth}
|
||||
\fancyhf{}
|
||||
\fancyhead[LE,RO]{\thepage}%
|
||||
\fancyhead[RE]{\slshape\leftmark}%
|
||||
\fancyhead[LO]{\slshape\rightmark}%
|
||||
\def\sectionmark##1{%
|
||||
\markboth{\MakeUppercase{%
|
||||
\ifnum \c@secnumdepth >\z@ \thesection\quad \fi##1}}{}}%
|
||||
\def\subsectionmark##1{%
|
||||
\markright{%
|
||||
\ifnum \c@secnumdepth >\@ne \thesubsection\quad \fi##1}}%
|
||||
}%
|
||||
\else
|
||||
\def\ps@headings{\ps@f@nch@fancyproto \def\@mkboth{\protect\markboth}
|
||||
\fancyhf{}
|
||||
\fancyhead[LE,RO]{\thepage}%
|
||||
\fancyhead[RE]{\slshape\leftmark}%
|
||||
\fancyhead[LO]{\slshape\rightmark}%
|
||||
\def\sectionmark##1{%
|
||||
\markright {\MakeUppercase{%
|
||||
\ifnum \c@secnumdepth >\z@ \thesection\quad \fi##1}}}%
|
||||
\let\subsectionmark\@gobble % Not needed but inserted for safety
|
||||
}%
|
||||
\fi
|
||||
}{\if@twoside
|
||||
\def\ps@headings{\ps@f@nch@fancyproto \def\@mkboth{\protect\markboth}
|
||||
\fancyhf{}
|
||||
\fancyhead[LE,RO]{\thepage}%
|
||||
\fancyhead[RE]{\slshape\leftmark}%
|
||||
\fancyhead[LO]{\slshape\rightmark}%
|
||||
\def\chaptermark##1{%
|
||||
\markboth{\MakeUppercase{%
|
||||
\ifnum \c@secnumdepth >\m@ne \if@mainmatter
|
||||
\@chapapp\ \thechapter. \ \fi\fi##1}}{}}%
|
||||
\def\sectionmark##1{%
|
||||
\markright {\MakeUppercase{%
|
||||
\ifnum \c@secnumdepth >\z@ \thesection. \ \fi##1}}}%
|
||||
}%
|
||||
\else
|
||||
\def\ps@headings{\ps@f@nch@fancyproto \def\@mkboth{\protect\markboth}
|
||||
\fancyhf{}
|
||||
\fancyhead[LE,RO]{\thepage}%
|
||||
\fancyhead[RE]{\slshape\leftmark}%
|
||||
\fancyhead[LO]{\slshape\rightmark}%
|
||||
\def\chaptermark##1{%
|
||||
\markright{\MakeUppercase{%
|
||||
\ifnum \c@secnumdepth >\m@ne \if@mainmatter
|
||||
\@chapapp\ \thechapter. \ \fi\fi##1}}}%
|
||||
\let\sectionmark\@gobble % Not needed but inserted for safety
|
||||
}%
|
||||
\fi
|
||||
}%
|
||||
}
|
||||
\ProcessOptions*
|
||||
\newcommand{\f@nch@forc}[3]{\expandafter\f@nchf@rc\expandafter#1\expandafter{#2}{#3}}
|
||||
\newcommand{\f@nchf@rc}[3]{\def\temp@ty{#2}\ifx\@empty\temp@ty\else
|
||||
\f@nch@rc#1#2\f@nch@rc{#3}\fi}
|
||||
\long\def\f@nch@rc#1#2#3\f@nch@rc#4{\def#1{#2}#4\f@nchf@rc#1{#3}{#4}}
|
||||
\newcommand{\f@nch@for}[3]{\edef\@fortmp{#2}%
|
||||
\expandafter\@forloop#2,\@nil,\@nil\@@#1{#3}}
|
||||
\newcommand\f@nch@default[3]{%
|
||||
\edef\temp@a{\lowercase{\edef\noexpand\temp@a{#3}}}\temp@a \def#1{}%
|
||||
\f@nch@forc\tmpf@ra{#2}%
|
||||
{\expandafter\f@nch@ifin\tmpf@ra\temp@a{\edef#1{#1\tmpf@ra}}{}}%
|
||||
\ifx\@empty#1\def#1{#2}\fi}
|
||||
\newcommand{\f@nch@ifin}[4]{%
|
||||
\edef\temp@a{#2}\def\temp@b##1#1##2\temp@b{\def\temp@b{##1}}%
|
||||
\expandafter\temp@b#2#1\temp@b\ifx\temp@a\temp@b #4\else #3\fi}
|
||||
\newcommand{\fancyhead}[2][]{\f@nch@fancyhf\fancyhead h[#1]{#2}}%
|
||||
\newcommand{\fancyfoot}[2][]{\f@nch@fancyhf\fancyfoot f[#1]{#2}}%
|
||||
\newcommand{\fancyhf}[2][]{\f@nch@fancyhf\fancyhf {}[#1]{#2}}%
|
||||
\newcommand{\fancyheadoffset}[2][]{\f@nch@fancyhfoffs\fancyheadoffset h[#1]{#2}}%
|
||||
\newcommand{\fancyfootoffset}[2][]{\f@nch@fancyhfoffs\fancyfootoffset f[#1]{#2}}%
|
||||
\newcommand{\fancyhfoffset}[2][]{\f@nch@fancyhfoffs\fancyhfoffset {}[#1]{#2}}%
|
||||
\def\f@nch@fancyhf@Echeck#1{%
|
||||
\if@twoside\else
|
||||
\iff@nch@twoside\else
|
||||
\if\f@nch@@eo e%
|
||||
\PackageWarning{fancyhdr} {\string#1's `E' option without twoside option is useless.\MessageBreak
|
||||
Please consider using the `twoside' option}%
|
||||
\fi\fi\fi
|
||||
}
|
||||
\long\def\f@nch@fancyhf#1#2[#3]#4{%
|
||||
\def\temp@c{}%
|
||||
\f@nch@forc\tmpf@ra{#3}%
|
||||
{\expandafter\f@nch@ifin\tmpf@ra{eolcrhf,EOLCRHF}%
|
||||
{}{\edef\temp@c{\temp@c\tmpf@ra}}}%
|
||||
\ifx\@empty\temp@c\else \PackageError{fancyhdr}{Illegal char `\temp@c' in
|
||||
\string#1 argument: [#3]}{}%
|
||||
\fi \f@nch@for\temp@c{#3}%
|
||||
{\f@nch@default\f@nch@@eo{eo}\temp@c
|
||||
\f@nch@fancyhf@Echeck{#1}%
|
||||
\f@nch@default\f@nch@@lcr{lcr}\temp@c
|
||||
\f@nch@default\f@nch@@hf{hf}{#2\temp@c}%
|
||||
\f@nch@forc\f@nch@eo\f@nch@@eo
|
||||
{\f@nch@forc\f@nch@lcr\f@nch@@lcr
|
||||
{\f@nch@forc\f@nch@hf\f@nch@@hf
|
||||
{\expandafter\f@nch@def\csname
|
||||
f@nch@\f@nch@eo\f@nch@lcr\f@nch@hf\endcsname {#4}}}}}}
|
||||
\def\f@nch@fancyhfoffs#1#2[#3]#4{%
|
||||
\def\temp@c{}%
|
||||
\f@nch@forc\tmpf@ra{#3}%
|
||||
{\expandafter\f@nch@ifin\tmpf@ra{eolrhf,EOLRHF}%
|
||||
{}{\edef\temp@c{\temp@c\tmpf@ra}}}%
|
||||
\ifx\@empty\temp@c\else \PackageError{fancyhdr}{Illegal char `\temp@c' in
|
||||
\string#1 argument: [#3]}{}%
|
||||
\fi \f@nch@for\temp@c{#3}%
|
||||
{\f@nch@default\f@nch@@eo{eo}\temp@c
|
||||
\f@nch@fancyhf@Echeck{#1}%
|
||||
\f@nch@default\f@nch@@lcr{lr}\temp@c
|
||||
\f@nch@default\f@nch@@hf{hf}{#2\temp@c}%
|
||||
\f@nch@forc\f@nch@eo\f@nch@@eo
|
||||
{\f@nch@forc\f@nch@lcr\f@nch@@lcr
|
||||
{\f@nch@forc\f@nch@hf\f@nch@@hf
|
||||
{\expandafter\setlength\csname
|
||||
f@nch@offset@\f@nch@eo\f@nch@lcr\f@nch@hf\endcsname {#4}}}}}%
|
||||
\f@nch@setoffs}
|
||||
\NewDocumentCommand {\fancyheadwidth}{ s O{} O{} m }
|
||||
{\f@nch@fancyhfwidth{#1}\fancyheadwidth h[#2][#3]{#4}}%
|
||||
\NewDocumentCommand {\fancyfootwidth}{ s O{} O{} m }
|
||||
{\f@nch@fancyhfwidth{#1}\fancyfootwidth f[#2][#3]{#4}}%
|
||||
\NewDocumentCommand {\fancyhfwidth} { s O{} O{} m }
|
||||
{\f@nch@fancyhfwidth{#1}\fancyhfwidth {}[#2][#3]{#4}}%
|
||||
\def\f@nch@fancyhfwidth#1#2#3[#4][#5]#6{%
|
||||
\setlength\@tempdima{#6}%
|
||||
\def\temp@c{}%
|
||||
\f@nch@forc\tmpf@ra{#4}%
|
||||
{\expandafter\f@nch@ifin\tmpf@ra{eolcrhf,EOLCRHF}%
|
||||
{}{\edef\temp@c{\temp@c\tmpf@ra}}}%
|
||||
\ifx\@empty\temp@c\else \PackageError{fancyhdr}{Illegal char `\temp@c' in
|
||||
\string#2 argument: [#4]}{}%
|
||||
\fi
|
||||
\f@nch@for\temp@c{#4}%
|
||||
{\f@nch@default\f@nch@@eo{eo}\temp@c
|
||||
\f@nch@fancyhf@Echeck{#2}%
|
||||
\f@nch@default\f@nch@@lcr{lcr}\temp@c
|
||||
\f@nch@default\f@nch@@hf{hf}{#3\temp@c}%
|
||||
\f@nch@forc\f@nch@eo\f@nch@@eo
|
||||
{\f@nch@forc\f@nch@lcr\f@nch@@lcr
|
||||
{\f@nch@forc\f@nch@hf\f@nch@@hf
|
||||
{%
|
||||
\IfBooleanTF{#1}{%
|
||||
\expandafter\edef\csname
|
||||
f@nch@width@\f@nch@eo\f@nch@lcr\f@nch@hf\endcsname{\the\@tempdima}%
|
||||
}%
|
||||
{%
|
||||
\expandafter\def\csname
|
||||
f@nch@width@\f@nch@eo\f@nch@lcr\f@nch@hf\endcsname{#6}%
|
||||
}%
|
||||
\csname f@nchdrwdt@align@v@\f@nch@hf\endcsname
|
||||
\edef\f@nch@align@@h{\f@nch@lcr}%
|
||||
\def\temp@a{#5}%
|
||||
\ifx\temp@a\@empty \else \f@nchdrwdt@align#5\@nil{#2}\fi
|
||||
\expandafter\edef\csname
|
||||
f@nch@align@\f@nch@eo\f@nch@lcr\f@nch@hf\endcsname
|
||||
{\f@nch@align@@v\f@nch@align@@h}}}}}}
|
||||
\def\f@nch@width@elh{\headwidth}
|
||||
\def\f@nch@width@ech{\headwidth}
|
||||
\def\f@nch@width@erh{\headwidth}
|
||||
\def\f@nch@width@olh{\headwidth}
|
||||
\def\f@nch@width@och{\headwidth}
|
||||
\def\f@nch@width@orh{\headwidth}
|
||||
\def\f@nch@width@elf{\headwidth}
|
||||
\def\f@nch@width@ecf{\headwidth}
|
||||
\def\f@nch@width@erf{\headwidth}
|
||||
\def\f@nch@width@olf{\headwidth}
|
||||
\def\f@nch@width@ocf{\headwidth}
|
||||
\def\f@nch@width@orf{\headwidth}
|
||||
\def\f@nch@align@elh{bl}
|
||||
\def\f@nch@align@ech{bc}
|
||||
\def\f@nch@align@erh{br}
|
||||
\def\f@nch@align@olh{bl}
|
||||
\def\f@nch@align@och{bc}
|
||||
\def\f@nch@align@orh{br}
|
||||
\def\f@nch@align@elf{tl}
|
||||
\def\f@nch@align@ecf{tc}
|
||||
\def\f@nch@align@erf{tr}
|
||||
\def\f@nch@align@olf{tl}
|
||||
\def\f@nch@align@ocf{tc}
|
||||
\def\f@nch@align@orf{tr}
|
||||
\def\f@nchdrwdt@align@v@h{\def\f@nch@align@@v{b}}%
|
||||
\def\f@nchdrwdt@align@v@f{\def\f@nch@align@@v{t}}%
|
||||
\long\def\f@nchdrwdt@align#1#2\@nil#3{%
|
||||
\f@nch@ifin{#1}{TtcbB-}{%
|
||||
\f@nch@ifin{#1}{-}{}{\def\f@nch@align@@v{#1}}%
|
||||
\def\@tempa{#2}%
|
||||
\ifx\@tempa\@empty \else \def\f@nch@align@@h{#2}\fi
|
||||
}%
|
||||
{\def\f@nch@align@@h{#1}}%
|
||||
\expandafter\f@nch@ifin\expandafter{\f@nch@align@@h}{lcrj}{}%
|
||||
{\PackageError{fancyhdr}
|
||||
{\string#3: Illegal char `\f@nch@align@@h'\MessageBreak
|
||||
in alignment argument}{}}%
|
||||
}
|
||||
\newcommand{\lhead}[2][\f@nch@olh]%
|
||||
{\f@nch@def\f@nch@olh{#2}\f@nch@def\f@nch@elh{#1}}
|
||||
\newcommand{\chead}[2][\f@nch@och]%
|
||||
{\f@nch@def\f@nch@och{#2}\f@nch@def\f@nch@ech{#1}}
|
||||
\newcommand{\rhead}[2][\f@nch@orh]%
|
||||
{\f@nch@def\f@nch@orh{#2}\f@nch@def\f@nch@erh{#1}}
|
||||
\newcommand{\lfoot}[2][\f@nch@olf]%
|
||||
{\f@nch@def\f@nch@olf{#2}\f@nch@def\f@nch@elf{#1}}
|
||||
\newcommand{\cfoot}[2][\f@nch@ocf]%
|
||||
{\f@nch@def\f@nch@ocf{#2}\f@nch@def\f@nch@ecf{#1}}
|
||||
\newcommand{\rfoot}[2][\f@nch@orf]%
|
||||
{\f@nch@def\f@nch@orf{#2}\f@nch@def\f@nch@erf{#1}}
|
||||
\newlength{\f@nch@headwidth} \let\headwidth\f@nch@headwidth
|
||||
\newlength{\f@nch@offset@elh}
|
||||
\newlength{\f@nch@offset@erh}
|
||||
\newlength{\f@nch@offset@olh}
|
||||
\newlength{\f@nch@offset@orh}
|
||||
\newlength{\f@nch@offset@elf}
|
||||
\newlength{\f@nch@offset@erf}
|
||||
\newlength{\f@nch@offset@olf}
|
||||
\newlength{\f@nch@offset@orf}
|
||||
\newcommand{\headrulewidth}{0.4pt}
|
||||
\newcommand{\footrulewidth}{0pt}
|
||||
\@ifundefined{headruleskip}%
|
||||
{\newcommand{\headruleskip}{0pt}}{}
|
||||
\@ifundefined{footruleskip}%
|
||||
{\newcommand{\footruleskip}{.3\normalbaselineskip}}{}
|
||||
\newcommand{\plainheadrulewidth}{0pt}
|
||||
\newcommand{\plainfootrulewidth}{0pt}
|
||||
\newif\if@fancyplain \@fancyplainfalse
|
||||
\def\fancyplain#1#2{\if@fancyplain#1\else#2\fi}
|
||||
\headwidth=-123456789sp
|
||||
\let\f@nch@raggedleft\raggedleft
|
||||
\let\f@nch@raggedright\raggedright
|
||||
\let\f@nch@centering\centering
|
||||
\let\f@nch@everypar\everypar
|
||||
\ifdefined\ExplSyntaxOn
|
||||
\ExplSyntaxOn
|
||||
\providecommand\IfFormatAtLeastTF{\@ifl@t@r\fmtversion}
|
||||
\IfFormatAtLeastTF{2021-06-01}{
|
||||
\def\f@nch@saveclr@parhook #1{
|
||||
\expandafter\let\csname f@nch@__hook~#1\expandafter\endcsname
|
||||
\csname __hook~#1\endcsname
|
||||
\expandafter\let\csname f@nch@__hook_toplevel~#1\expandafter\endcsname
|
||||
\csname __hook_toplevel~#1\endcsname
|
||||
\expandafter\let\csname f@nch@__hook_next~#1\expandafter\endcsname
|
||||
\csname __hook_next~#1\endcsname
|
||||
\expandafter\let\csname f@nch@g__hook_#1_code_prop\expandafter\endcsname
|
||||
\csname g__hook_#1_code_prop\endcsname
|
||||
\RemoveFromHook{#1}[*]
|
||||
\ClearHookNext{#1}
|
||||
}
|
||||
\def\f@nch@restore@parhook #1{
|
||||
\global\expandafter\let\csname __hook~#1\expandafter\endcsname
|
||||
\csname f@nch@__hook~#1\endcsname
|
||||
\global\expandafter\let\csname __hook_toplevel~#1\expandafter\endcsname
|
||||
\csname f@nch@__hook_toplevel~#1\endcsname
|
||||
\global\expandafter\let\csname __hook_next~#1\expandafter\endcsname
|
||||
\csname f@nch@__hook_next~#1\endcsname
|
||||
\global\expandafter\let\csname g__hook_#1_code_prop\expandafter\endcsname
|
||||
\csname f@nch@g__hook_#1_code_prop\endcsname
|
||||
}
|
||||
\def\f@nch@resetpar{
|
||||
\f@nch@everypar{}
|
||||
\f@nch@saveclr@parhook{para/before}
|
||||
\f@nch@saveclr@parhook{para/begin}
|
||||
\f@nch@saveclr@parhook{para/end}
|
||||
\f@nch@saveclr@parhook{para/after}
|
||||
}
|
||||
\def\f@nch@restorepar{
|
||||
\f@nch@restore@parhook{para/before}
|
||||
\f@nch@restore@parhook{para/begin}
|
||||
\f@nch@restore@parhook{para/end}
|
||||
\f@nch@restore@parhook{para/after}
|
||||
}
|
||||
}{
|
||||
\def\f@nch@resetpar{
|
||||
\f@nch@everypar{}
|
||||
}
|
||||
\def\f@nch@restorepar{}
|
||||
}
|
||||
\ExplSyntaxOff
|
||||
\else
|
||||
\def\f@nch@resetpar{%
|
||||
\f@nch@everypar{}%
|
||||
}
|
||||
\def\f@nch@restorepar{}
|
||||
\fi
|
||||
\newcommand\f@nch@noUppercase[2][]{#2}
|
||||
\def\f@nch@reset{\f@nch@resetpar\restorecr\endlinechar=13
|
||||
\catcode`\\=0\catcode`\{=1\catcode`\}=2\catcode`\$=3\catcode`\&=4
|
||||
\catcode`\#=6\catcode`\^=7\catcode`\_=8\catcode`\ =10\catcode`\@=11
|
||||
\catcode`\:=11\catcode`\~=13\catcode`\%=14
|
||||
\catcode0=15 %NULL
|
||||
\catcode9=10 %TAB
|
||||
\let\\\@normalcr \let\raggedleft\f@nch@raggedleft
|
||||
\let\raggedright\f@nch@raggedright \let\centering\f@nch@centering
|
||||
\def\baselinestretch{1}%
|
||||
\hsize=\headwidth
|
||||
\def\nouppercase##1{{%
|
||||
\let\uppercase\relax\let\MakeUppercase\f@nch@noUppercase
|
||||
\expandafter\let\csname MakeUppercase \endcsname\relax
|
||||
\expandafter\def\csname MakeUppercase\space\space\space\endcsname
|
||||
[####1]####2{####2}%
|
||||
##1}}%
|
||||
\@ifundefined{@normalsize} {\normalsize} % for ucthesis.cls
|
||||
{\@normalsize}%
|
||||
}
|
||||
\newcommand*{\fancycenter}[1][1em]{%
|
||||
\@ifnextchar[{\f@nch@center{#1}}{\f@nch@center{#1}[3]}%
|
||||
}
|
||||
\def\f@nch@center#1[#2]#3#4#5{%
|
||||
\def\@tempa{#4}\ifx\@tempa\@empty
|
||||
\hbox to\linewidth{\color@begingroup{#3}\hfil {#5}\color@endgroup}%
|
||||
\else
|
||||
\setlength\@tempdima{#1}%
|
||||
\setlength{\@tempdimb}{#2\@tempdima}%
|
||||
\@tempdimc \@tempdimb \advance\@tempdimc -\@tempdima
|
||||
\setlength\@tempskipa{\@tempdimb \@plus 1fil \@minus \@tempdimc}%
|
||||
\@tempskipb\@tempskipa
|
||||
\def\@tempa{#3}\ifx\@tempa\@empty
|
||||
\addtolength\@tempskipa{\z@ \@minus \@tempdima}%
|
||||
\fi
|
||||
\def\@tempa{#5}\ifx\@tempa\@empty % empty right
|
||||
\addtolength\@tempskipb{\z@ \@minus \@tempdima}%
|
||||
\fi
|
||||
\settowidth{\@tempdimb}{#3}%
|
||||
\settowidth{\@tempdimc}{#5}%
|
||||
\ifdim\@tempdimb>\@tempdimc
|
||||
\advance\@tempdimb -\@tempdimc
|
||||
\addtolength\@tempskipb{\@tempdimb \@minus \@tempdimb}%
|
||||
\else
|
||||
\advance\@tempdimc -\@tempdimb
|
||||
\addtolength\@tempskipa{\@tempdimc \@minus \@tempdimc}%
|
||||
\fi
|
||||
\hbox to\linewidth{\color@begingroup{#3}\hskip \@tempskipa
|
||||
{#4}\hskip \@tempskipb {#5}\color@endgroup}%
|
||||
\fi
|
||||
}
|
||||
\newcommand{\f@nch@headinit}{}
|
||||
\newcommand{\fancyheadinit}[1]{%
|
||||
\def\f@nch@headinit{#1}%
|
||||
}
|
||||
\newcommand{\f@nch@footinit}{}
|
||||
\newcommand{\fancyfootinit}[1]{%
|
||||
\def\f@nch@footinit{#1}%
|
||||
}
|
||||
\newcommand{\fancyhfinit}[1]{%
|
||||
\def\f@nch@headinit{#1}%
|
||||
\def\f@nch@footinit{#1}%
|
||||
}
|
||||
\ifdefined\NewMirroredHookPair
|
||||
\NewMirroredHookPair{fancyhdr/before}{fancyhdr/after}
|
||||
\NewMirroredHookPair{fancyhdr/head/begin}{fancyhdr/head/end}
|
||||
\NewMirroredHookPair{fancyhdr/foot/begin}{fancyhdr/foot/end}
|
||||
\fi
|
||||
\newlength\f@nch@height
|
||||
\newlength\f@nch@footalignment
|
||||
\newif\iff@nch@footalign\f@nch@footalignfalse
|
||||
\newcommand{\fancyfootalign}[1]{%
|
||||
\def\temp@a{#1}%
|
||||
\ifx\temp@a\@empty
|
||||
\f@nch@footalignfalse
|
||||
\else
|
||||
\f@nch@footaligntrue
|
||||
\setlength\f@nch@footalignment{#1}%
|
||||
\fi
|
||||
}
|
||||
\newcommand\fancyhdrsettoheight[2]{%
|
||||
\expandafter\ifx\csname f@nch@#2\endcsname\fancyhdrsettoheight
|
||||
\else\PackageError{fancyhdr}{Unknown parameter #2 in \string\fancyhdrsettoheight}{}\fi
|
||||
\setbox\@tempboxa\hbox{{\f@nch@checkfalse\csname @#2\endcsname}}%
|
||||
\setlength{#1}\f@nch@height
|
||||
\setbox\@tempboxa\box\voidb@x
|
||||
}
|
||||
\let\f@nch@oddhead\fancyhdrsettoheight
|
||||
\let\f@nch@evenhead\fancyhdrsettoheight
|
||||
\let\f@nch@oddfoot\fancyhdrsettoheight
|
||||
\let\f@nch@evenfoot\fancyhdrsettoheight
|
||||
\newcommand\f@nch@vbox[2]{%
|
||||
\setbox0\vbox{#2}%
|
||||
\global\f@nch@height=\ht0
|
||||
\ifdim\ht0>#1\relax
|
||||
\iff@nch@check
|
||||
\dimen0=#1\advance\dimen0-\ht0
|
||||
\PackageWarning{fancyhdr}{%
|
||||
\string#1 is too small (\the#1): \MessageBreak
|
||||
Make it at least \the\ht0, for example:\MessageBreak
|
||||
\string\setlength{\string#1}{\the\ht0}%
|
||||
\iff@nch@compatViii .\MessageBreak
|
||||
We now make it that large for the rest of the document.\MessageBreak
|
||||
This may cause the page layout to be inconsistent, however
|
||||
\fi
|
||||
\ifx#1\headheight .\MessageBreak
|
||||
You might also make \topmargin smaller:\MessageBreak
|
||||
\string\addtolength{\string\topmargin}{\the\dimen0}%
|
||||
\fi
|
||||
\@gobble
|
||||
}%
|
||||
\iff@nch@compatViii
|
||||
\dimen0=#1\relax
|
||||
\global#1=\ht0\relax
|
||||
\ht0=\dimen0 %
|
||||
\else
|
||||
\ht0=#1\relax
|
||||
\fi
|
||||
\else
|
||||
\ht0=#1\relax
|
||||
\fi
|
||||
\fi
|
||||
\box0}
|
||||
\newcommand\f@nch@head[6]{%
|
||||
\f@nch@reset
|
||||
\ifdefined\UseHook\UseHook{fancyhdr/before}\UseHook{fancyhdr/head/begin}\fi
|
||||
\f@nch@headinit\relax
|
||||
#1%
|
||||
\hbox to\headwidth{%
|
||||
\f@nch@vbox\headheight{%
|
||||
\f@nch@hfbox{#2}{#3}{#4}{#6}{h}%
|
||||
\vskip\headruleskip\relax
|
||||
\headrule
|
||||
}%
|
||||
}%
|
||||
#5%
|
||||
\ifdefined\UseHook\UseHook{fancyhdr/head/end}\UseHook{fancyhdr/after}\fi
|
||||
\f@nch@restorepar
|
||||
}
|
||||
\newcommand\f@nch@foot[6]{%
|
||||
\f@nch@reset
|
||||
\ifdefined\UseHook\UseHook{fancyhdr/before}\UseHook{fancyhdr/foot/begin}\fi
|
||||
\f@nch@footinit\relax
|
||||
#1%
|
||||
\hbox to\headwidth{%
|
||||
\f@nch@vbox\footskip{%
|
||||
\setbox0=\vbox{\footrule}\unvbox0
|
||||
\vskip\footruleskip
|
||||
\f@nch@hfbox{#2}{#3}{#4}{#6}{f}%
|
||||
\iff@nch@footalign \vskip\f@nch@footalignment \fi
|
||||
}%
|
||||
}%
|
||||
#5%
|
||||
\ifdefined\UseHook\UseHook{fancyhdr/foot/end}\UseHook{fancyhdr/after}\fi
|
||||
\f@nch@restorepar
|
||||
}
|
||||
\newlength\f@nch@widthL
|
||||
\newlength\f@nch@widthC
|
||||
\newlength\f@nch@widthR
|
||||
\newcommand\f@nch@hfbox[5]{%
|
||||
\setlength\f@nch@widthL{\csname f@nch@width@#4l#5\endcsname}%
|
||||
\setlength\f@nch@widthC{\csname f@nch@width@#4c#5\endcsname}%
|
||||
\setlength\f@nch@widthR{\csname f@nch@width@#4r#5\endcsname}%
|
||||
\let\@tempa\f@nch@hfbox@center
|
||||
\ifdim \dimexpr \f@nch@widthL+\f@nch@widthC+\f@nch@widthR>\headwidth
|
||||
\else
|
||||
\ifdim \dimexpr \f@nch@widthL+0.5\f@nch@widthC>0.5\headwidth
|
||||
\let \@tempa\f@nch@hfbox@fit
|
||||
\fi
|
||||
\ifdim \dimexpr \f@nch@widthR+0.5\f@nch@widthC>0.5\headwidth
|
||||
\let \@tempa\f@nch@hfbox@fit
|
||||
\fi
|
||||
\fi
|
||||
\@tempa{#1}{#2}{#3}#4#5%
|
||||
}
|
||||
\newcommand\f@nch@hfbox@center[5]{%
|
||||
\hbox to \headwidth{%
|
||||
\rlap{\f@nch@parbox{#1}\f@nch@widthL{#4}l{#5}}%
|
||||
\hfill
|
||||
\f@nch@parbox{#2}\f@nch@widthC{#4}c{#5}%
|
||||
\hfill
|
||||
\llap{\f@nch@parbox{#3}\f@nch@widthR{#4}r{#5}}%
|
||||
}%
|
||||
}
|
||||
\newcommand\f@nch@hfbox@fit[5]{%
|
||||
\hbox to \headwidth{%
|
||||
\f@nch@parbox{#1}\f@nch@widthL{#4}l{#5}%
|
||||
\hfill
|
||||
\f@nch@parbox{#2}\f@nch@widthC{#4}c{#5}%
|
||||
\hfill
|
||||
\f@nch@parbox{#3}\f@nch@widthR{#4}r{#5}%
|
||||
}%
|
||||
}%
|
||||
\newcommand\f@nch@parbox[5]{%
|
||||
\expandafter\expandafter\expandafter\f@nch@parbox@align
|
||||
\csname f@nch@align@#3#4#5\endcsname
|
||||
\parbox[\f@nch@align@@v]{#2}%
|
||||
{%
|
||||
\f@nch@align@@pre
|
||||
\f@nch@align@@h\leavevmode\ignorespaces#1%
|
||||
\f@nch@align@@post
|
||||
}%
|
||||
}
|
||||
\newcommand\f@nch@parbox@align[2]{%
|
||||
\def\f@nch@align@@pre{}%
|
||||
\def\f@nch@align@@post{}%
|
||||
\csname f@nch@parbox@align@v#1\endcsname
|
||||
\csname f@nch@parbox@align@h#2\endcsname
|
||||
}
|
||||
\def\f@nch@parbox@align@vT{\def\f@nch@align@@v{t}\def\f@nch@align@@pre{\vspace{0pt}}}
|
||||
\def\f@nch@parbox@align@vt{\def\f@nch@align@@v{t}}
|
||||
\def\f@nch@parbox@align@vc{\def\f@nch@align@@v{c}}
|
||||
\def\f@nch@parbox@align@vb{\def\f@nch@align@@v{b}}
|
||||
\def\f@nch@parbox@align@vB{\def\f@nch@align@@v{b}\def\f@nch@align@@post{\vspace{0pt}}}
|
||||
\def\f@nch@parbox@align@hl{\def\f@nch@align@@h{\raggedright}}
|
||||
\def\f@nch@parbox@align@hc{\def\f@nch@align@@h{\centering}}
|
||||
\def\f@nch@parbox@align@hr{\def\f@nch@align@@h{\raggedleft}}
|
||||
\def\f@nch@parbox@align@hj{\def\f@nch@align@@h{}}
|
||||
\@ifundefined{@chapapp}{\let\@chapapp\chaptername}{}%
|
||||
\def\f@nch@initialise{%
|
||||
\@ifundefined{chapter}%
|
||||
{\def\sectionmark##1{\markboth{\MakeUppercase{\ifnum \c@secnumdepth>\z@
|
||||
\thesection\hskip 1em\relax
|
||||
\fi ##1}}{}}%
|
||||
\def\subsectionmark##1{\markright {\ifnum \c@secnumdepth >\@ne
|
||||
\thesubsection\hskip 1em\relax \fi ##1}}}%
|
||||
{\def\chaptermark##1{\markboth {\MakeUppercase{\ifnum
|
||||
\c@secnumdepth>\m@ne \@chapapp\ \thechapter. \ \fi ##1}}{}}%
|
||||
\def\sectionmark##1{\markright{\MakeUppercase{\ifnum \c@secnumdepth >\z@
|
||||
\thesection. \ \fi ##1}}}%
|
||||
}%
|
||||
\def\headrule{{\if@fancyplain\let\headrulewidth\plainheadrulewidth\fi
|
||||
\hrule\@height\headrulewidth\@width\headwidth
|
||||
\vskip-\headrulewidth}}%
|
||||
\def\footrule{{\if@fancyplain\let\footrulewidth\plainfootrulewidth\fi
|
||||
\hrule\@width\headwidth\@height\footrulewidth}}%
|
||||
\def\headrulewidth{0.4pt}%
|
||||
\def\footrulewidth{0pt}%
|
||||
\def\headruleskip{0pt}%
|
||||
\def\footruleskip{0.3\normalbaselineskip}%
|
||||
\fancyhf{}%
|
||||
\if@twoside
|
||||
\fancyhead[el,or]{\fancyplain{}{\slshape\rightmark}}%
|
||||
\fancyhead[er,ol]{\fancyplain{}{\slshape\leftmark}}%
|
||||
\else
|
||||
\fancyhead[l]{\fancyplain{}{\slshape\rightmark}}%
|
||||
\fancyhead[r]{\fancyplain{}{\slshape\leftmark}}%
|
||||
\fi
|
||||
\fancyfoot[c]{\rmfamily\thepage}% page number
|
||||
}
|
||||
\f@nch@initialise
|
||||
\def\ps@f@nch@fancyproto{%
|
||||
\ifdim\headwidth<0sp
|
||||
\global\advance\headwidth123456789sp\global\advance\headwidth\textwidth
|
||||
\fi
|
||||
\gdef\ps@f@nch@fancyproto{\@fancyplainfalse\ps@f@nch@fancycore}%
|
||||
\@fancyplainfalse\ps@f@nch@fancycore
|
||||
}%
|
||||
\@namedef{f@nch@ps@f@nch@fancyproto-is-fancyhdr}{}
|
||||
\def\ps@fancy{\ps@f@nch@fancyproto}
|
||||
\@namedef{f@nch@ps@fancy-is-fancyhdr}{}
|
||||
\def\ps@fancyplain{\ps@f@nch@fancyproto \let\ps@plain\ps@plain@fancy}
|
||||
\def\ps@plain@fancy{\@fancyplaintrue\ps@f@nch@fancycore}
|
||||
\let\f@nch@ps@empty\ps@empty
|
||||
\def\ps@f@nch@fancycore{%
|
||||
\f@nch@ps@empty
|
||||
\def\@mkboth{\protect\markboth}%
|
||||
\def\f@nch@oddhead{\f@nch@head\f@nch@Oolh\f@nch@olh\f@nch@och\f@nch@orh\f@nch@Oorh{o}}%
|
||||
\def\@oddhead{%
|
||||
\iff@nch@twoside
|
||||
\ifodd\c@page
|
||||
\f@nch@oddhead
|
||||
\else
|
||||
\@evenhead
|
||||
\fi
|
||||
\else
|
||||
\f@nch@oddhead
|
||||
\fi
|
||||
}
|
||||
\def\f@nch@oddfoot{\f@nch@foot\f@nch@Oolf\f@nch@olf\f@nch@ocf\f@nch@orf\f@nch@Oorf{o}}%
|
||||
\def\@oddfoot{%
|
||||
\iff@nch@twoside
|
||||
\ifodd\c@page
|
||||
\f@nch@oddfoot
|
||||
\else
|
||||
\@evenfoot
|
||||
\fi
|
||||
\else
|
||||
\f@nch@oddfoot
|
||||
\fi
|
||||
}
|
||||
\def\@evenhead{\f@nch@head\f@nch@Oelh\f@nch@elh\f@nch@ech\f@nch@erh\f@nch@Oerh{e}}%
|
||||
\def\@evenfoot{\f@nch@foot\f@nch@Oelf\f@nch@elf\f@nch@ecf\f@nch@erf\f@nch@Oerf{e}}%
|
||||
}
|
||||
\def\f@nch@Oolh{\if@reversemargin\hss\else\relax\fi}
|
||||
\def\f@nch@Oorh{\if@reversemargin\relax\else\hss\fi}
|
||||
\let\f@nch@Oelh\f@nch@Oorh
|
||||
\let\f@nch@Oerh\f@nch@Oolh
|
||||
\let\f@nch@Oolf\f@nch@Oolh
|
||||
\let\f@nch@Oorf\f@nch@Oorh
|
||||
\let\f@nch@Oelf\f@nch@Oelh
|
||||
\let\f@nch@Oerf\f@nch@Oerh
|
||||
\def\f@nch@offsolh{\headwidth=\textwidth\advance\headwidth\f@nch@offset@olh
|
||||
\advance\headwidth\f@nch@offset@orh\hskip-\f@nch@offset@olh}
|
||||
\def\f@nch@offselh{\headwidth=\textwidth\advance\headwidth\f@nch@offset@elh
|
||||
\advance\headwidth\f@nch@offset@erh\hskip-\f@nch@offset@elh}
|
||||
\def\f@nch@offsolf{\headwidth=\textwidth\advance\headwidth\f@nch@offset@olf
|
||||
\advance\headwidth\f@nch@offset@orf\hskip-\f@nch@offset@olf}
|
||||
\def\f@nch@offself{\headwidth=\textwidth\advance\headwidth\f@nch@offset@elf
|
||||
\advance\headwidth\f@nch@offset@erf\hskip-\f@nch@offset@elf}
|
||||
\def\f@nch@setoffs{%
|
||||
\f@nch@gbl\let\headwidth\f@nch@headwidth
|
||||
\f@nch@gbl\def\f@nch@Oolh{\f@nch@offsolh}%
|
||||
\f@nch@gbl\def\f@nch@Oelh{\f@nch@offselh}%
|
||||
\f@nch@gbl\def\f@nch@Oorh{\hss}%
|
||||
\f@nch@gbl\def\f@nch@Oerh{\hss}%
|
||||
\f@nch@gbl\def\f@nch@Oolf{\f@nch@offsolf}%
|
||||
\f@nch@gbl\def\f@nch@Oelf{\f@nch@offself}%
|
||||
\f@nch@gbl\def\f@nch@Oorf{\hss}%
|
||||
\f@nch@gbl\def\f@nch@Oerf{\hss}%
|
||||
}
|
||||
\newif\iff@nch@footnote
|
||||
\AtBeginDocument{%
|
||||
\let\latex@makecol\@makecol
|
||||
\def\@makecol{\ifvoid\footins\f@nch@footnotefalse\else\f@nch@footnotetrue\fi
|
||||
\let\f@nch@topfloat\@toplist\let\f@nch@botfloat\@botlist\latex@makecol}%
|
||||
}
|
||||
\newcommand\iftopfloat[2]{\ifx\f@nch@topfloat\@empty #2\else #1\fi}%
|
||||
\newcommand\ifbotfloat[2]{\ifx\f@nch@botfloat\@empty #2\else #1\fi}%
|
||||
\newcommand\iffloatpage[2]{\if@fcolmade #1\else #2\fi}%
|
||||
\newcommand\iffootnote[2]{\iff@nch@footnote #1\else #2\fi}%
|
||||
\ifx\@temptokenb\undefined \csname newtoks\endcsname\@temptokenb\fi
|
||||
\newif\iff@nch@pagestyle@star
|
||||
\newcommand\fancypagestyle{%
|
||||
\@ifstar{\f@nch@pagestyle@startrue\f@nch@pagestyle}%
|
||||
{\f@nch@pagestyle@starfalse\f@nch@pagestyle}%
|
||||
}
|
||||
\newcommand\f@nch@pagestyle[1]{%
|
||||
\@ifnextchar[{\f@nch@@pagestyle{#1}}{\f@nch@@pagestyle{#1}[f@nch@fancyproto]}%
|
||||
}
|
||||
\long\def\f@nch@@pagestyle#1[#2]#3{%
|
||||
\@ifundefined{ps@#2}{%
|
||||
\PackageError{fancyhdr}{\string\fancypagestyle: Unknown base page style `#2'}{}%
|
||||
}{%
|
||||
\@ifundefined{f@nch@ps@#2-is-fancyhdr}{%
|
||||
\PackageError{fancyhdr}{\string\fancypagestyle: Base page style `#2' is not fancyhdr-based}{}%
|
||||
}%
|
||||
{%
|
||||
\f@nch@pagestyle@setup
|
||||
\def\temp@b{\@namedef{ps@#1}}%
|
||||
\expandafter\temp@b\expandafter{\the\@temptokenb
|
||||
\let\f@nch@gbl\relax\@nameuse{ps@#2}#3\relax}%
|
||||
\@namedef{f@nch@ps@#1-is-fancyhdr}{}%
|
||||
}%
|
||||
}%
|
||||
}
|
||||
\newcommand\f@nch@pagestyle@setup{%
|
||||
\iff@nch@pagestyle@star
|
||||
\iff@nch@check\@temptokenb={\f@nch@checktrue}\else\@temptokenb={\f@nch@checkfalse}\fi
|
||||
\@tfor\temp@a:=
|
||||
\f@nch@olh\f@nch@och\f@nch@orh\f@nch@elh\f@nch@ech\f@nch@erh
|
||||
\f@nch@olf\f@nch@ocf\f@nch@orf\f@nch@elf\f@nch@ecf\f@nch@erf
|
||||
\f@nch@width@elh\f@nch@width@ech\f@nch@width@erh\f@nch@width@olh
|
||||
\f@nch@width@och\f@nch@width@orh\f@nch@width@elf\f@nch@width@ecf
|
||||
\f@nch@width@erf\f@nch@width@olf\f@nch@width@ocf\f@nch@width@orf
|
||||
\f@nch@align@elh\f@nch@align@ech\f@nch@align@erh\f@nch@align@olh
|
||||
\f@nch@align@och\f@nch@align@orh\f@nch@align@elf\f@nch@align@ecf
|
||||
\f@nch@align@erf\f@nch@align@olf\f@nch@align@ocf\f@nch@align@orf
|
||||
\f@nch@Oolh\f@nch@Oorh\f@nch@Oelh\f@nch@Oerh
|
||||
\f@nch@Oolf\f@nch@Oorf\f@nch@Oelf\f@nch@Oerf
|
||||
\f@nch@headinit\f@nch@footinit
|
||||
\headrule\headrulewidth\footrule\footrulewidth
|
||||
\do {%
|
||||
\toks@=\expandafter\expandafter\expandafter{\temp@a}%
|
||||
\toks@=\expandafter\expandafter\expandafter{%
|
||||
\expandafter\expandafter\expandafter\def
|
||||
\expandafter\expandafter\temp@a\expandafter{\the\toks@}}%
|
||||
\edef\temp@b{\@temptokenb={\the\@temptokenb\the\toks@}}%
|
||||
\temp@b
|
||||
}%
|
||||
\@tfor\temp@a:=
|
||||
\f@nch@offset@olh\f@nch@offset@orh\f@nch@offset@elh\f@nch@offset@erh
|
||||
\f@nch@offset@olf\f@nch@offset@orf\f@nch@offset@elf\f@nch@offset@erf
|
||||
\do {%
|
||||
\toks@=\expandafter\expandafter\expandafter{\expandafter\the\temp@a}%
|
||||
\toks@=\expandafter\expandafter\expandafter{%
|
||||
\expandafter\expandafter\expandafter\setlength
|
||||
\expandafter\expandafter\temp@a\expandafter{\the\toks@}}%
|
||||
\edef\temp@b{\@temptokenb={\the\@temptokenb\the\toks@}}%
|
||||
\temp@b
|
||||
}%
|
||||
\else
|
||||
\@temptokenb={}%
|
||||
\fi
|
||||
}
|
||||
\newcommand\fancypagestyleassign[2]{%
|
||||
\@ifundefined{ps@#2}{%
|
||||
\PackageError{fancyhdr}{\string\fancypagestyleassign: Unknown page style `#2'}{}%
|
||||
}{%
|
||||
\expandafter\let
|
||||
\csname ps@#1\expandafter\endcsname
|
||||
\csname ps@#2\endcsname
|
||||
\@ifundefined{f@nch@ps@#2-is-fancyhdr}{%
|
||||
\expandafter\let\csname f@nch@ps@#1-is-fancyhdr\endcsname\@undefined
|
||||
}{%
|
||||
\@namedef{f@nch@ps@#1-is-fancyhdr}{}%
|
||||
}%
|
||||
}%
|
||||
}
|
||||
\fancypagestyle*{fancydefault}{\f@nch@initialise}
|
||||
\def\f@nchdrbox@topstrut{\vrule height\ht\strutbox width\z@}
|
||||
\def\f@nchdrbox@botstrut{\vrule depth\dp\strutbox width\z@}
|
||||
\def\f@nchdrbox@nostrut{\noalign{\vspace{0pt}}\let\f@nchdrbox@@crstrut\f@nchdrbox@botstrut}
|
||||
\NewDocumentCommand{\fancyhdrbox}{ O{cl} o m }{%
|
||||
\begingroup
|
||||
\let\f@nchdrbox@@pre\f@nchdrbox@topstrut
|
||||
\let\f@nchdrbox@@postx\f@nchdrbox@botstrut
|
||||
\let\f@nchdrbox@@posty\relax
|
||||
\let\f@nchdrbox@@crstrut\strut
|
||||
\IfNoValueTF{#2}%
|
||||
{\let\f@nchdrbox@@halignto\@empty}%
|
||||
{\setlength\@tempdima{#2}%
|
||||
\def\f@nchdrbox@@halignto{to\@tempdima}}%
|
||||
\def\@tempa{#1}%
|
||||
\ifx\@tempa\@empty
|
||||
\f@nchdrbox@align cl\@nil{#3}%
|
||||
\else
|
||||
\f@nchdrbox@align #1\@nil{#3}%
|
||||
\fi
|
||||
\endgroup
|
||||
}
|
||||
\protected\def\f@nchdrbox@cr{%
|
||||
{\ifnum0=`}\fi\@ifstar\@f@nchdrbox@xcr\@f@nchdrbox@xcr}
|
||||
|
||||
\def\@f@nchdrbox@xcr{%
|
||||
\unskip\f@nchdrbox@@crstrut
|
||||
\@ifnextchar[\@f@nchdrbox@argc{\ifnum0=`{\fi}\cr}%
|
||||
}
|
||||
|
||||
\def\@f@nchdrbox@argc[#1]{%
|
||||
\ifnum0=`{\fi}%
|
||||
\ifdim #1>\z@
|
||||
\unskip\@f@nchdrbox@xargc{#1}%
|
||||
\else
|
||||
\@f@nchdrbox@yargc{#1}%
|
||||
\fi}
|
||||
|
||||
\def\@f@nchdrbox@xargc#1{\@tempdima #1\advance\@tempdima \dp \strutbox
|
||||
\vrule \@height\z@ \@depth\@tempdima \@width\z@ \cr}
|
||||
|
||||
\def\@f@nchdrbox@yargc#1{\cr\noalign{\setlength\@tempdima{#1}\vskip\@tempdima}}
|
||||
\def\f@nchdrbox@T{\let\f@nchdrbox@@pre\f@nchdrbox@nostrut
|
||||
\f@nchdrbox@t}
|
||||
\def\f@nchdrbox@t{\def\f@nchdrbox@@v{t}\def\f@nchdrbox@@h{l}}
|
||||
\def\f@nchdrbox@c{\def\f@nchdrbox@@v{c}\def\f@nchdrbox@@h{c}}
|
||||
\def\f@nchdrbox@b{\def\f@nchdrbox@@v{b}\def\f@nchdrbox@@h{l}}
|
||||
\def\f@nchdrbox@B{\let\f@nchdrbox@@postx\relax
|
||||
\def\f@nchdrbox@@posty{\vspace{0pt}}%
|
||||
\f@nchdrbox@b}
|
||||
\long\def\f@nchdrbox@align#1#2\@nil#3{%
|
||||
\f@nch@ifin{#1}{TtcbB}{%
|
||||
\@nameuse{f@nchdrbox@#1}%
|
||||
\def\@tempa{#2}%
|
||||
\ifx\@tempa\@empty\else \def\f@nchdrbox@@h{#2}\fi
|
||||
}%
|
||||
{\def\f@nchdrbox@@v{c}\def\f@nchdrbox@@h{#1}}%
|
||||
\expandafter\f@nch@ifin\expandafter{\f@nchdrbox@@h}{lcr}{}%
|
||||
{\PackageError{fancyhdr}{\string\fancyhdrbox: Illegal char `\f@nchdrbox@@h'\MessageBreak
|
||||
in alignment argument}{}}%
|
||||
\let\\\f@nchdrbox@cr
|
||||
\setbox0=\if \f@nchdrbox@@v t\vtop
|
||||
\else \vbox
|
||||
\fi
|
||||
{%
|
||||
\ialign \f@nchdrbox@@halignto
|
||||
\bgroup \relax
|
||||
{\if \f@nchdrbox@@h l\hskip 1sp\else \hfil \fi
|
||||
\ignorespaces ##\unskip
|
||||
\if\f@nchdrbox@@h r\else \hfil \fi
|
||||
}%
|
||||
\tabskip\z@skip \cr
|
||||
\f@nchdrbox@@pre
|
||||
#3\unskip \f@nchdrbox@@postx
|
||||
\crcr
|
||||
\egroup
|
||||
\f@nchdrbox@@posty
|
||||
}%
|
||||
\if\f@nchdrbox@@v c\@tempdima=\ht0\advance\@tempdima\dp0%
|
||||
\ht0=0.5\@tempdima\dp0=0.5\@tempdima\fi
|
||||
\leavevmode \box0
|
||||
}
|
||||
\@ifclassloaded{newlfm}
|
||||
{
|
||||
\let\ps@@empty\f@nch@ps@empty
|
||||
\AtBeginDocument{%
|
||||
\renewcommand{\@zfancyhead}[5]{\relax\hbox to\headwidth{\f@nch@reset
|
||||
\@zfancyvbox\headheight{\hbox
|
||||
{\rlap{\parbox[b]{\headwidth}{\raggedright\f@nch@olh}}\hfill
|
||||
\parbox[b]{\headwidth}{\centering\f@nch@olh}\hfill
|
||||
\llap{\parbox[b]{\headwidth}{\raggedleft\f@nch@orh}}}%
|
||||
\zheadrule}}\relax}%
|
||||
}
|
||||
}
|
||||
{}
|
||||
\endinput
|
||||
%%
|
||||
%% End of file `fancyhdr.sty'.
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user