admin
b9a24b3fb5
Adds benchmarks/host-fast-path.py and benchmarks/README.md.
The benchmark compares two paths for a cached single-ID search:
1. fast-path: in-process dict walk inside the native host
(handle_cached_search_fast in rcjav-host.py)
2. subprocess: shell out to `rc-jav.py --search ID --cache --format json`
Idle baseline against the live 7124-file cache (5 queries × 5 iter):
fast-path: median 0.46ms p95 0.61ms max 0.72ms
subprocess: median 919ms p95 1233ms max 1385ms
median speedup: 2000x
Decision: keep the fast path. The ~920ms subprocess cost is dominated
by Python interpreter startup + 1.3MB cache.json parse. That's
structural — it applies under idle Python too, not just when a scan
is running. The "Python actively scanning" condition from the original
roadmap doesn't change the verdict; it would only make the subprocess
path even slower while leaving the in-process path unaffected (the
fast path doesn't touch the scanning process).
The fast path is already correctly scoped — bails out for wildcards,
ranges, name searches, and --quick mode. Narrowing further would just
push more queries through the slow path with no upside.
Possible follow-up (not in scope here): memoize _load_host_cache with
mtime-based invalidation so the fast path doesn't reparse cache.json
on every call. Current per-call median (0.46ms) is already fast enough
that this is optional.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2.6 KiB
benchmarks/
Latency benchmarks for decisions in the rc-jav roadmap.
host-fast-path.py — Step 11 decision
The native-messaging host has an in-process fast path
(handle_cached_search_fast in rcjav-host.py) that answers simple
cached single-ID searches without shelling out to rc-jav.py. Step 11
asked: is the fast path actually pulling its weight, or could we
delete it / narrow it down?
Run
python benchmarks/host-fast-path.py [--queries Q1 Q2 ...] [--iterations N]
For the "Python actively scanning" condition, start rc-jav.py --scan
in a separate terminal first.
Findings (idle baseline, 2026-05-23, 7124-file cache)
=== aggregate (5 queries × 5 iterations, idle Python) ===
fast-path total:
n=25 min=0.43ms median=0.46ms mean=0.48ms p95=0.61ms max=0.72ms
subprocess total:
n=25 min=880.56ms median=919.45ms mean=965.55ms p95=1232.93ms max=1385.37ms
median speedup: 2000.1x
p95 speedup: 2036.0x
Decision: keep the fast path
The subprocess path costs ~920ms median per query even with the
interpreter doing nothing else — that's pure Python startup +
json.loads of the 1.3 MB cache.json. The fast path returns hits in
under 1ms. The 2000× speedup is structural (interpreter startup
overhead), not load-dependent, so it would apply equally under (a)
idle and (b) active-scan conditions.
The fast path is already correctly scoped — it bails out for
wildcards, ranges, name searches, and --quick mode (which forces a
live rclone hit). Narrowing further would just push more queries
through the slow path with no upside.
The "Python actively scanning" condition listed in the original roadmap was framed as the case where the fast path's value would be most obvious. The idle baseline already settles it; we don't need to gate the decision on the active-scan measurement, though running it remains a sanity check if cache.json grows substantially.
What this benchmark does NOT cover
- Latency from inside the browser extension (popup) to the host. Adds Brave's native-messaging protocol overhead on top of whichever path the host takes — but the relative difference between paths is preserved.
- Memory cost of the in-process cache load. The fast path loads
cache.json once per call today (no caching across calls inside the
host). A future optimization is to memoize
_load_host_cachewith mtime-based invalidation; left for follow-up if needed. - Cold-cache effects.
cache.jsonis large enough that the OS page cache matters; numbers above reflect a warm read. First call after a reboot may be slower for both paths but proportionally so.