64 lines
2.4 KiB
Markdown
64 lines
2.4 KiB
Markdown
# 2026-03-09 - Daily Notes
|
|
|
|
## Projects Worked On
|
|
|
|
### ROM Library Organization
|
|
- **Goal**: Organize/categorize ~100K ROM files across multiple gaming systems
|
|
- **Library location**: R:\ drive (38.8 TB)
|
|
- **Quick scan completed**: 98,601 items, 1,701 GB total
|
|
|
|
**Breakdown by Manufacturer**:
|
|
- Computers (C64, Amiga, etc.): 47,327 items, 61.89 GB
|
|
- Arcade (MAME, Neo Geo): 12,951 items, 32.97 GB
|
|
- Atari: 12,399 items, 2.56 GB
|
|
- Nintendo: 12,017 items, 467.24 GB
|
|
- Sony (PSX, PS3, PSN): 3,106 items, 672.40 GB
|
|
- Sega: 2,747 items, 3.54 GB
|
|
- Microsoft: 1,661 items, 0.05 GB
|
|
|
|
**Top by count**: Commodore 64 (24,349), Atari (10,935), ArcadePC (8,750), MAME (8,651)
|
|
**Top by size**: PSN ISO Pack (672 GB), Nintendo 3DS (412 GB), TurboGrafx-CD (234 GB)
|
|
|
|
**Scripts created**:
|
|
- tools/rom-quick-scan.py - Quick inventory (completed)
|
|
- tools/rom-full-scan.py - Full scan with duplicate detection (running overnight)
|
|
|
|
**Output files**:
|
|
- rom-inventory/rom-inventory.json - Quick scan results
|
|
- rom-inventory/rom-full-*.json - Full scan results (pending)
|
|
|
|
**Phase 2**: Duplicate detection running overnight - MD5 hashing files under 50MB
|
|
|
|
## Issues Encountered
|
|
- Node gateway timeouts when running long scans (30s limit)
|
|
- Sub-agent reliability issues - spawning background processes more reliable
|
|
- Script escaping issues with f-strings in Python scripts
|
|
|
|
## Decisions Made
|
|
- Use folder counts for disc-based systems (PSX, PS2, etc.) instead of file counts
|
|
- Hash only files under 50MB for duplicate detection (speed vs coverage)
|
|
- Run full scan as background process, not sub-agent
|
|
|
|
## Overnight Scan Results (2026-03-10 00:02)
|
|
|
|
### Full Scan Completed
|
|
- **Total:** 107,069 items (6,103 GB / 6.1 TB)
|
|
- **Duplicates Found:** 4,790 duplicate sets (11,160 files)
|
|
|
|
**Worst Duplicate Offenders:**
|
|
- ArcadePC MegaPack: 100 copies of same binary
|
|
- M.A.M.E: 66 copies of big7.bin
|
|
- PopCap games: 48-36 copies of various .bin files
|
|
|
|
**Key Insight:** Many duplicates are internal game data files (not actual ROM duplicates) - .bin files used across multiple games in:
|
|
- ArcadePC MegaPack (100 copies of dialog data)
|
|
- Abandonware DOS (map tiles reused across levels)
|
|
- PopCap games (UI assets)
|
|
|
|
**Action Items:**
|
|
- Review duplicate report for actual ROM duplicates vs internal game data
|
|
- Consider excluding .bin files under 1MB from duplicate scanning (noise)
|
|
- Cross-reference with No-Intro DATs for missing ROMs
|
|
|
|
**Output File:** rom-inventory/rom-full-20260310-000245.json (1.6 MB)
|