rc-jav

Read-only duplicate scanner for JAV files across rclone remotes. Groups files by JAV ID (e.g. SSIS-001) and reports which copy to keep based on priority rules.

Priority rules

1. Video files inside configured VIP folders win first. Default VIP folder: ClearJAV.
2. If no VIP-folder video exists, Source always wins regardless of resolution/size.
3. .ts files rank below other video containers, even when the transport-stream copy is larger.
4. If no Source copy exists in the group, largest file size wins among the remaining Targets.
5. Suggestions only — script never deletes. Manual cleanup.

ID matching

Filename stem is matched against:

• Primary: ^([A-Za-z]+)-(\d+) — SSIS-001, MIDV-123, ABP-456
• Compound: ^(\w+(?:-\w+)+)-(\d+) — FC2-PPV-4894535, HEYZO-HD-1234
• Fallback: ^([A-Za-z0-9]+)-(\d+) — 1pondo-123, carib-456

IDs normalized to uppercase with leading zeros stripped from the number (so ssis-001 == SSIS-1 == SSIS-001). Anything after the ID ( - Actress [1080p]) is ignored for matching.

Part-suffix handling

Multi-part files (_1, _2, -1, -2, _A, _B, .1of4, (1), -pt1, -part1, -cd1, -disc1, trailing N) are normalized as {ID}#partN so they do not collide as false duplicates. Searching the base ID still finds all parts. Lettered _A / _B suffixes become part 1 / part 2.

Add more suffix shapes with repeatable --part-pattern regexes. The first capture group is the part number or one part letter and the pattern runs against the filename stem:

python rc-jav.py --scan --part-pattern '[-_ ]side[-_ ]?(A|B)$'
python rc-jav.py --part-pattern '_([CD])$' --save

Saved rules live in config.json as part_patterns. The extension Options page has the same custom part detector list for host-triggered searches, duplicate review, and cache rebuilds.

Files with no parseable ID are listed under "Skipped" at the end so you can spot misnamed files.

Rule checks

Focused rule tests cover ID extraction, multipart grouping safety, and duplicate KEEP ranking:

python -B -m unittest discover -s tests -v

Usage

python rc-jav.py \
  --source cq:personal-files/ClearJAV/ichika-matsumoto \
  --target cq:personal-files/JAV/TMP \

Flags:

• --source / -s REMOTE — priority remote path. Repeat for multiple.
• --target / -t REMOTE — non-priority remote path. Repeat for multiple.
• --format {console,txt,csv,json,all} — default console. Non-console formats write to --output-dir.
• --output-dir DIR — default ./reports.
• --no-color — disable ANSI colors but keep rich layout (tables, panels).
• --basic — plain text output, no rich tables/panels. Progress ticks every 25 files on stderr. Useful for piping or simple terminals.
• --rclone-bin PATH — path to rclone executable (default: rclone on PATH). Example: --rclone-bin C:\Programs\rclone\rclone.exe.
• --clearjav — shortcut: sets source = DEFAULT_SOURCE, target = DEFAULT_TARGET. Equivalent to --source cq:personal-files/ClearJAV --target cq:personal-files/JAV/TMP. Combine with --source/--target to override one side.

Examples:

# full library dupe scan, one flag
python rc-jav.py --clearjav

# same but only check one actress folder against TMP
python rc-jav.py --clearjav --source cq:personal-files/ClearJAV/ichika-matsumoto

Search mode

Check whether a JAV ID already exists in your library before downloading:

python rc-jav.py --search SSIS-001
python rc-jav.py --search SSIS-001 --search FC2-PPV-4894535

# wildcards (quote to avoid shell glob expansion)
python rc-jav.py --search "IPZZ-*"
python rc-jav.py --search "FC2-PPV-*"
python rc-jav.py --search "SSIS-???"     # exact 3-digit numeric

Wildcard syntax: * (any chars) and ? (one char), case-insensitive. Matches against normalized IDs in the index, including #partN suffixes automatically.

Range syntax: [N-M] inclusive both ends. Works inside any prefix.

python rc-jav.py --search "IPZZ-[820-860]"
python rc-jav.py --search "FC2-PPV-[4894500-4894600]"
python rc-jav.py --search "MIDV-[001-010]"   # zero-padding preserved

Quote in PowerShell/bash so [...] reaches Python literally. Reversed ranges (860-820) auto-swap.

With no --source / --target flags, only DEFAULT_TARGET (TMP) is scanned — the typical case for "do I already have this in my unsorted pile?". Pass --source cq:personal-files/ClearJAV to also check the priority library. Edit DEFAULT_SOURCE / DEFAULT_TARGET at the top of the script to change defaults. Remote scans are recursive.

Exit code: 0 if every query had at least one hit, 1 otherwise — useful for shell automation.

Name search (--name)

Substring search against filenames (case-insensitive). Find all files by actress, studio, tag, anything that appears in the filename.

python rc-jav.py --name Ichika
python rc-jav.py --name "Ichika Matsumoto"
python rc-jav.py --name Ichika --name Yui          # OR — files matching either
python rc-jav.py --name "Mat*"                     # glob wildcard
python rc-jav.py --search IPZZ-860 --name Ichika   # both — separate result blocks

• Multiple --name tokens = OR. Use one combined --name "foo bar" for AND/exact-substring.
• Matches against the filename stem only (not folder names).
• Auto-routes to cached mode because substring globs can't be server-side filtered on most backends. Pass -q to force quick anyway (slower).

Smart search mode (auto quick / cached)

The script auto-picks the right execution path per query and prints which one it chose:

Query shape	Picked mode	Reason
Single exact ID (IPZZ-860)	quick	live rclone --include, ~1–2s even on huge trees
Wildcard (IPZZ-*, SSIS-???)	cached	reliable normalized matching
Range (IPZZ-[820-860])	cached	avoids N rclone calls
Multiple --search flags	cached	warmup amortizes

Override:

• --quick / -q — force live rclone lookup (skips cache).
• --cache — force cache (builds it if cold).

Quick mode never reads or writes the cache. Cache mode honors --update and --no-cache as before.

Cache

Search mode caches each remote's file list in ./cache.json next to the script. Subsequent searches are near-instant.

• First run: scans + writes cache.
• Later runs: reads cache (banner shows CACHED 14m (154 files)).
• --update / -u: force re-scan + overwrite cache for the requested remotes.
• --no-cache: bypass cache (no read, no write).
• Stale warning when cache is older than 24h — still used, marked CACHED-STALE.
• Ctrl+C during a scan: rclone is terminated, cache for in-flight remote is NOT written.

Delete cache.json to reset everything.

Saving defaults (--save)

Persist --source, --target, --catalog, and/or --part-pattern to config.json so you don't have to type them every run.

# set default target
python rc-jav.py --target cq:personal-files/JAV/TMP --save

# set source + multiple targets at once
python rc-jav.py --source cq:personal-files/ClearJAV ^
                 --target cq:personal-files/JAV/TMP ^
                 --target cq:personal-files/JAV/SORTED ^
                 --save

# inspect
type config.json

Only the keys you explicitly pass are written — running --save --target X won't wipe a saved default_source. Delete config.json to reset to the hardcoded defaults at the top of rc-jav.py.

Scan-only (--scan)

Refresh the cache without running a search or dupe report — useful for Task Scheduler / cron pre-warming.

# default: refresh DEFAULT_TARGET (TMP)
python rc-jav.py --scan

# refresh both source and target
python rc-jav.py --scan --source cq:personal-files/ClearJAV --target cq:personal-files/JAV/TMP

# nightly via Task Scheduler
schtasks /Create /SC DAILY /ST 03:00 /TN "rc-jav nightly scan" ^
  /TR "python D:\DEV\Project\rclone-jav\rc-jav.py --scan --basic"

--scan always overwrites the cache for the remotes you list. Exit 0 = success, non-zero = rclone failure.

python rc-jav.py --search MIDV-999 ; if ($LASTEXITCODE -eq 0) { "have it" } else { "download" }

WinCatalog integration

WinCatalog's native .wcat format is proprietary, so the script reads its exports instead.

1. In WinCatalog: File → Export → choose CSV or XML.
2. Save into the wincatalog/ folder next to the script. All *.csv and *.xml files there are auto-loaded — drop in as many discs as you want.
3. Run as normal: python rc-jav.py --search IPZZ-860
4. Override or add extra paths with --catalog PATH (file or folder, repeatable).
5. To change the default folder, edit DEFAULT_CATALOG at the top of the script.

Re-export when your catalog changes; the script re-reads on every run (catalog data is not cached — it's already a local file).

Role of catalog hits:

• Search: shown as rows with source label Catalog. The disc/volume name is encoded into the path so you know which offline backup holds the file.
• Dupe mode: catalog entries appear in groups for awareness but are never marked KEEP or DELETE? — they're offline, can't be touched. A group is only flagged as a dupe when 2+ rclone copies exist.

CSV column auto-detection (case-insensitive, first match wins):

• Name: Name, File Name, Filename, Title
• Path: Path, Full Path, Location, Folder
• Size: Size, File Size, Bytes, Size (bytes)
• Disc: Disc, Disc Name, Disc Label, Volume, Source, Catalog, Media

XML: walks the tree, treats <File> / <f> nodes inside <Disc> / <Catalog> / <Volume> containers, with <Folder> nesting.

Requirements

• Python 3.9+
• pip install rich (used for progress bars + themed output)
• rclone on PATH with the relevant remotes configured.

UI

• Live per-file progress bar during scans (rclone size --json for total, then rclone lsf --files-only -R --format pst streamed).
• Banner panel showing run mode + per-remote cache status.
• Rich tables for search hits and duplicate groups.
• --no-color for plain output (CI, piping).

Roadmap

• Phase 1 (current): report duplicates + search.
• Phase 2: --apply mode that runs rclone delete on DELETE? candidates behind a confirmation gate.
• Phase 3: resolution-aware tiebreakers, move-to-review folder, scheduled runs.