# Gotcha Format

The on-disk contract for every `memories/*-gotchas.md` file. `node .agents/skills/cebreus-memories/gotchas.mjs sort` parses these files, so a block that ignores this format is a block the tooling has to repair.

## Block

```markdown
## 🚨 Gotcha Name

**Date Discovered:** 2026-08-21

Rule: The actionable constraint, one sentence, imperative.
```

The `## 🚨 ` marker is mandatory. It is the block boundary: a heading without it is treated as a repair case, and before the normalisation pass such headings were silently absorbed into the entry above them.

## The date is read, never recalled

Do NOT work out the date yourself. Asking for the timestamp of a conversation's first message is asking for something you cannot verify, which is why it used to come out as `UNKNOWN`. Run the tool and paste what it prints:

```bash
node .agents/skills/cebreus-memories/gotchas.mjs date
```

For a gotcha mined out of the backlog, pass the extract it came from, so a finding from March is not stamped with today:

```bash
node .agents/skills/cebreus-memories/gotchas.mjs date .temp/extracted_chats/<session>.txt
```

The command exits non-zero rather than emitting a date it cannot back with a source. That is the only case where the line is omitted. Never type a date you did not get from this command: a missing date is recoverable, a wrong one is not.

Entries filed before this rule carry no date. That is expected — do not backfill them, because the source is gone.

## Category to file

The filename is NOT derived from the category name. Use this table verbatim.

| Category            | Target file                         |
| ------------------- | ----------------------------------- |
| `Dependency`        | `memories/dependency-gotchas.md`    |
| `Build Pipeline`    | `memories/build-gotchas.md`         |
| `Logic/Correctness` | `memories/logic-gotchas.md`         |
| `Security`          | `memories/security-gotchas.md`      |
| `Architectural`     | `memories/architectural-gotchas.md` |
| `Workflow`          | `memories/workflow-gotchas.md`      |
| `Linting`           | `memories/linting-gotchas.md`       |

A workspace may add its own file when no category fits, e.g. `tools/upng-benchmark/memories/data-schema-gotchas.md`. The `-gotchas.md` suffix is what makes the file visible to the sort tool, so keep it.

Target path is `<scope-root>/memories/<file>`. Create the directory and the file if either is missing.

## Sort order

`node .agents/skills/cebreus-memories/gotchas.mjs sort [dir]` normalises headings and reorders blocks. Dated blocks lead, oldest first; undated blocks follow alphabetically. It prints the number of files it rewrote and exits non-zero when the target directory is missing. With no argument it walks the whole repo.

`node .agents/skills/cebreus-memories/gotchas.mjs audit [dir]` reports the state of the base without changing it: files and entry counts, entries sharing a title or rule text, empty files, and files missing from their scope's `AGENTS.md`.

## Index line

Every memory file must be referenced from the `AGENTS.md` of its own scope, under its Gotchas section.

Only the filename matters: `gotchas.mjs audit` checks that it appears, and nothing parses the sentence around it. Match the wording already used in that file instead of imposing one — the repository has two long-standing phrasings and rewriting the older one buys nothing. Both of these count as indexed:

```markdown
- Dependency constraints: read `memories/dependency-gotchas.md`.
- For logic/correctness-related constraints, read `memories/logic-gotchas.md`.
```

// Standalone by design: a skill runs wherever it is copied, so this script
// imports only the Node standard library and resolves every path against the
// working directory.
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import readline from 'node:readline';

const MEMORY_FILE_SUFFIX = '-gotchas.md';
const HEADING_PREFIX = '## ';
const GOTCHA_MARKER = '🚨';
const DATE_FIELD = '**Date Discovered:**';
const EXTRACT_DIRECTORY = path.join('.temp', 'extracted_chats');
const SKIPPED_DIRECTORIES = new Set(['node_modules', '.git', 'dist']);

// Entries have been filed under 🚨, 📦, 🎨, 🧹 and ⚛️, so a whitelist would rot.
const LEADING_PICTOGRAPHS = /^[\p{Extended_Pictographic}\uFE0F\u200D]+/v;
const LETTER_OR_DIGIT = /[\p{Letter}\p{Number}]/v;
const BLANK_LINE_RUN = /\n{3,}/gv;

const USAGE = 'Usage: date [extract] | mine | sort [dir] | audit [dir]';
const EXTRACT_START_FIELD = 'START:';

function abort(message) {
  throw new Error(`gotchas: ${message}`);
}

// Claude names its project directory after the repository path with separators
// flattened to dashes, Gemini after the repository folder alone.
function readTranscriptSources(repositoryRoot) {
  return [
    path.join(
      os.homedir(),
      '.claude/projects',
      repositoryRoot.replaceAll(path.sep, '-').replaceAll('.', '-'),
    ),
    path.join(
      os.homedir(),
      '.gemini/tmp',
      path.basename(repositoryRoot),
      'chats',
    ),
  ];
}

function toMessageText(content) {
  if (typeof content === 'string') {
    return content.trim();
  }
  if (!Array.isArray(content)) {
    return '';
  }
  return content
    .filter(
      (block) => block?.type === 'text' || typeof block?.text === 'string',
    )
    .map((block) => block.text ?? '')
    .join('')
    .trim();
}

function toChatEntry(record) {
  const role = record?.type;
  if (role !== 'user' && role !== 'assistant' && role !== 'gemini') {
    return null;
  }
  const text = toMessageText(record.message?.content ?? record.content);
  return text ? `ROLE: ${role}\n${text}` : null;
}

async function readChatEntries(filePath) {
  const entries = [];
  let start = 'UNKNOWN';
  const lines = readline.createInterface({
    crlfDelay: Infinity,
    input: fs.createReadStream(filePath),
  });

  for await (const line of lines) {
    let record;
    try {
      record = JSON.parse(line);
    } catch {
      continue;
    }
    if (start === 'UNKNOWN') {
      start = record.startTime || record.timestamp || start;
    }
    const entry = toChatEntry(record);
    if (entry) {
      entries.push(entry);
    }
  }

  return { entries, start };
}

function readTranscriptFiles(sources) {
  const files = [];

  for (const source of sources) {
    if (!fs.existsSync(source)) {
      continue;
    }
    for (const name of fs.readdirSync(source)) {
      if (name.endsWith('.jsonl')) {
        files.push(path.join(source, name));
      }
    }
  }

  return files.toSorted();
}

async function mine(repositoryRoot) {
  const sources = readTranscriptSources(repositoryRoot);
  const files = readTranscriptFiles(sources);
  if (files.length === 0) {
    abort(
      `no .jsonl transcripts found. Looked in: ${sources.join(', ')}. Run a CLI session in this repository first, or adjust readTranscriptSources.`,
    );
  }

  fs.mkdirSync(EXTRACT_DIRECTORY, { recursive: true });
  let written = 0;

  for (const filePath of files) {
    const { entries, start } = await readChatEntries(filePath);
    if (entries.length === 0) {
      continue;
    }
    fs.writeFileSync(
      path.join(EXTRACT_DIRECTORY, `${path.basename(filePath, '.jsonl')}.txt`),
      `START: ${start}\n\n${entries.join('\n\n---\n\n')}\n`,
      'utf8',
    );
    written += 1;
  }

  if (written === 0) {
    abort(
      `read ${files.length} transcript file(s) but none held usable messages. The transcript format may have changed.`,
    );
  }

  process.stdout.write(
    `mined ${written} transcript(s) into ${EXTRACT_DIRECTORY}\n`,
  );
}

// Not matched on the parent directory: a run from the repository root also
// walks `.agents/skills/cebreus-memories/`, whose only Markdown file is this skill.
function toIsoDate(value) {
  const parsed = new Date(value);
  return Number.isNaN(parsed.getTime())
    ? null
    : parsed.toISOString().slice(0, 10);
}

// The date a gotcha is filed is knowable; the timestamp of the first message in
// a conversation is not, which is why asking for it only ever produced UNKNOWN.
// Live capture uses today. Backlog capture reads the START line the mined
// extract already carries, so a finding from March is not stamped with today.
function readDiscoveryDate(extractFile) {
  if (extractFile === undefined) {
    return toIsoDate(Date.now());
  }
  if (!fs.existsSync(extractFile)) {
    abort(`extract does not exist: ${extractFile}`);
  }

  const firstLine = fs.readFileSync(extractFile, 'utf8').split('\n')[0] ?? '';
  if (!firstLine.startsWith(EXTRACT_START_FIELD)) {
    abort(
      `${extractFile} does not begin with a ${EXTRACT_START_FIELD} line, so its date cannot be trusted. Omit the date line instead of guessing.`,
    );
  }

  const date = toIsoDate(firstLine.slice(EXTRACT_START_FIELD.length).trim());
  if (date === null) {
    abort(
      `${extractFile} carries an unusable ${EXTRACT_START_FIELD} value. Omit the date line instead of guessing.`,
    );
  }
  return date;
}

function isMemoryFile(name) {
  return name.endsWith(MEMORY_FILE_SUFFIX);
}

function collectMemoryFiles(directory) {
  const files = [];

  for (const entry of fs.readdirSync(directory, { withFileTypes: true })) {
    const filePath = path.join(directory, entry.name);
    if (entry.isDirectory()) {
      if (!SKIPPED_DIRECTORIES.has(entry.name)) {
        files.push(...collectMemoryFiles(filePath));
      }
    } else if (isMemoryFile(entry.name)) {
      files.push(filePath);
    }
  }

  return files;
}

function readMemoryFiles(directory) {
  if (!fs.existsSync(directory)) {
    abort(`target does not exist: ${directory}`);
  }
  return collectMemoryFiles(directory).toSorted();
}

// String operations rather than a pattern: an optional marker before free text
// makes any regex here prone to backtracking.
function toGotchaTitle(headingLine) {
  const title = headingLine
    .slice(HEADING_PREFIX.length)
    .trim()
    .replace(LEADING_PICTOGRAPHS, '')
    .trim();

  return title.startsWith('[') && title.endsWith(']')
    ? title.slice(1, -1).trim()
    : title;
}

function toSortableDate(line) {
  const raw = line.split(DATE_FIELD)[1]?.trim();
  if (!raw || raw.includes('UNKNOWN')) {
    return Number.NaN;
  }
  return new Date(raw).getTime() || Number.NaN;
}

function toGotchaBlock(headingLine) {
  const title = toGotchaTitle(headingLine);
  return {
    date: Number.NaN,
    lines: [`${HEADING_PREFIX}${GOTCHA_MARKER} ${title}`],
    title,
  };
}

function appendLineToBlock(block, line) {
  block.lines.push(line);
  if (line.includes(DATE_FIELD)) {
    block.date = toSortableDate(line);
  }
}

function splitMemoryContent(content) {
  const blocks = [];
  const head = [];
  let block;

  for (const line of content.split('\n')) {
    if (line.startsWith(HEADING_PREFIX)) {
      block = toGotchaBlock(line);
      blocks.push(block);
    } else if (block) {
      appendLineToBlock(block, line);
    } else {
      head.push(line);
    }
  }

  return { blocks, head };
}

// Undated blocks sort alphabetically so the order stays stable across runs.
function compareBlocks(left, right) {
  const isLeftDated = !Number.isNaN(left.date);
  const isRightDated = !Number.isNaN(right.date);

  if (isLeftDated && isRightDated) {
    return left.date - right.date;
  }
  if (isLeftDated !== isRightDated) {
    return isLeftDated ? -1 : 1;
  }
  return left.title.toLowerCase().localeCompare(right.title.toLowerCase());
}

function toBodyFingerprint(content) {
  return content
    .split('\n')
    .filter((line) => !line.startsWith(HEADING_PREFIX) && line.trim() !== '')
    .map((line) => line.trim())
    .toSorted()
    .join('\n');
}

function toSortedMemoryContent(content) {
  const { blocks, head } = splitMemoryContent(content);
  if (blocks.length === 0) {
    return null;
  }

  const body = blocks
    .toSorted(compareBlocks)
    .map((block) => block.lines.join('\n').trim())
    .join('\n\n');

  return `${head.join('\n').trim()}\n\n${body}`
    .replaceAll(BLANK_LINE_RUN, '\n\n')
    .trim();
}

function sortMemoryFile(filePath) {
  const content = fs.readFileSync(filePath, 'utf8');
  const sorted = toSortedMemoryContent(content);
  if (sorted === null) {
    process.stderr.write(`gotchas: no gotcha blocks in ${filePath}, skipped\n`);
    return 0;
  }

  if (toBodyFingerprint(sorted) !== toBodyFingerprint(content)) {
    abort(
      `refusing to write ${filePath}: reordering changed its content, which is a bug in this tool, not in the file`,
    );
  }

  fs.writeFileSync(filePath, sorted, 'utf8');
  return 1;
}

function sort(directory) {
  let sorted = 0;
  for (const filePath of readMemoryFiles(directory)) {
    sorted += sortMemoryFile(filePath);
  }
  return sorted;
}

function toScopeRoot(filePath) {
  return path.dirname(path.dirname(filePath));
}

// Letters and digits only, so a rule reworded with different punctuation,
// casing or backticks still collides.
function toComparisonKey(text) {
  return [...text.toLowerCase()]
    .filter((character) => LETTER_OR_DIGIT.test(character))
    .join('');
}

function readEntries(filePath) {
  return splitMemoryContent(fs.readFileSync(filePath, 'utf8')).blocks.map(
    (block) => ({
      body: block.lines.slice(1).join(' ').trim(),
      title: block.lines[0].slice(HEADING_PREFIX.length).trim(),
    }),
  );
}

function isIndexedInScope(filePath) {
  const agentsFile = path.join(toScopeRoot(filePath), 'AGENTS.md');
  if (!fs.existsSync(agentsFile)) {
    return false;
  }
  return fs.readFileSync(agentsFile, 'utf8').includes(path.basename(filePath));
}

function addOccurrence(index, key, occurrence) {
  const seen = index.get(key) ?? [];
  seen.push(occurrence);
  index.set(key, seen);
}

function toDuplicateGroups(index) {
  return [...index.values()].filter((seen) => seen.length > 1);
}

function readAuditReport(directory) {
  const byTitle = new Map();
  const byBody = new Map();
  const rows = [];

  for (const filePath of readMemoryFiles(directory)) {
    const entries = readEntries(filePath);
    const shortPath = path.relative(directory, filePath);
    rows.push({
      entries: entries.length,
      indexed: isIndexedInScope(filePath),
      path: shortPath,
    });

    for (const entry of entries) {
      const where = `${shortPath} :: ${entry.title}`;
      addOccurrence(byTitle, toComparisonKey(entry.title), where);
      if (entry.body) {
        addOccurrence(byBody, toComparisonKey(entry.body), where);
      }
    }
  }

  return {
    duplicateBodies: toDuplicateGroups(byBody),
    duplicateTitles: toDuplicateGroups(byTitle),
    empty: rows.filter((row) => row.entries === 0),
    rows,
    unindexed: rows.filter((row) => !row.indexed),
  };
}

function toAuditLines(report) {
  const lines = [
    `files: ${report.rows.length}`,
    ...report.rows.map((row) => `  ${row.path} — ${row.entries} entries`),
  ];

  for (const [label, groups] of [
    ['identical titles', report.duplicateTitles],
    ['identical rule text', report.duplicateBodies],
  ]) {
    lines.push(`${label}: ${groups.length} group(s)`);
    for (const group of groups) {
      lines.push(...group.map((where) => `  ${where}`), '  --');
    }
  }

  lines.push(
    `empty files: ${report.empty.length}`,
    ...report.empty.map((row) => `  ${row.path}`),
    `not indexed in their scope AGENTS.md: ${report.unindexed.length}`,
    ...report.unindexed.map((row) => `  ${row.path}`),
  );

  return lines;
}

async function main() {
  const [command, target] = process.argv.slice(2);

  if (command === 'date') {
    process.stdout.write(`${readDiscoveryDate(target)}\n`);
    return;
  }
  if (command === 'mine') {
    await mine(path.resolve('.'));
    return;
  }
  if (command === 'sort') {
    process.stdout.write(`sorted ${sort(target ?? '.')} memory file(s)\n`);
    return;
  }
  if (command === 'audit') {
    const lines = toAuditLines(readAuditReport(target ?? '.'));
    process.stdout.write(`${lines.join('\n')}\n`);
    return;
  }
  abort(`unknown command: ${command ?? '(none)'}. ${USAGE}`);
}

try {
  await main();
} catch (error) {
  process.stderr.write(`${error instanceof Error ? error.message : error}\n`);
  process.exitCode = 1;
}