admin
b9a24b3fb5
Adds benchmarks/host-fast-path.py and benchmarks/README.md.
The benchmark compares two paths for a cached single-ID search:
1. fast-path: in-process dict walk inside the native host
(handle_cached_search_fast in rcjav-host.py)
2. subprocess: shell out to `rc-jav.py --search ID --cache --format json`
Idle baseline against the live 7124-file cache (5 queries × 5 iter):
fast-path: median 0.46ms p95 0.61ms max 0.72ms
subprocess: median 919ms p95 1233ms max 1385ms
median speedup: 2000x
Decision: keep the fast path. The ~920ms subprocess cost is dominated
by Python interpreter startup + 1.3MB cache.json parse. That's
structural — it applies under idle Python too, not just when a scan
is running. The "Python actively scanning" condition from the original
roadmap doesn't change the verdict; it would only make the subprocess
path even slower while leaving the in-process path unaffected (the
fast path doesn't touch the scanning process).
The fast path is already correctly scoped — bails out for wildcards,
ranges, name searches, and --quick mode. Narrowing further would just
push more queries through the slow path with no upside.
Possible follow-up (not in scope here): memoize _load_host_cache with
mtime-based invalidation so the fast path doesn't reparse cache.json
on every call. Current per-call median (0.46ms) is already fast enough
that this is optional.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
67 lines
2.6 KiB
Markdown
67 lines
2.6 KiB
Markdown
# benchmarks/
|
||
|
||
Latency benchmarks for decisions in the rc-jav roadmap.
|
||
|
||
## host-fast-path.py — Step 11 decision
|
||
|
||
The native-messaging host has an in-process fast path
|
||
(`handle_cached_search_fast` in `rcjav-host.py`) that answers simple
|
||
cached single-ID searches without shelling out to `rc-jav.py`. Step 11
|
||
asked: is the fast path actually pulling its weight, or could we
|
||
delete it / narrow it down?
|
||
|
||
### Run
|
||
|
||
```
|
||
python benchmarks/host-fast-path.py [--queries Q1 Q2 ...] [--iterations N]
|
||
```
|
||
|
||
For the "Python actively scanning" condition, start `rc-jav.py --scan`
|
||
in a separate terminal first.
|
||
|
||
### Findings (idle baseline, 2026-05-23, 7124-file cache)
|
||
|
||
```
|
||
=== aggregate (5 queries × 5 iterations, idle Python) ===
|
||
fast-path total:
|
||
n=25 min=0.43ms median=0.46ms mean=0.48ms p95=0.61ms max=0.72ms
|
||
subprocess total:
|
||
n=25 min=880.56ms median=919.45ms mean=965.55ms p95=1232.93ms max=1385.37ms
|
||
median speedup: 2000.1x
|
||
p95 speedup: 2036.0x
|
||
```
|
||
|
||
### Decision: keep the fast path
|
||
|
||
The subprocess path costs ~920ms median per query even with the
|
||
interpreter doing nothing else — that's pure Python startup +
|
||
`json.loads` of the 1.3 MB cache.json. The fast path returns hits in
|
||
under 1ms. The 2000× speedup is structural (interpreter startup
|
||
overhead), not load-dependent, so it would apply equally under (a)
|
||
idle and (b) active-scan conditions.
|
||
|
||
The fast path is already correctly scoped — it bails out for
|
||
wildcards, ranges, name searches, and `--quick` mode (which forces a
|
||
live rclone hit). Narrowing further would just push more queries
|
||
through the slow path with no upside.
|
||
|
||
The "Python actively scanning" condition listed in the original
|
||
roadmap was framed as the case where the fast path's value would be
|
||
most obvious. The idle baseline already settles it; we don't need to
|
||
gate the decision on the active-scan measurement, though running it
|
||
remains a sanity check if cache.json grows substantially.
|
||
|
||
### What this benchmark does NOT cover
|
||
|
||
- Latency from inside the browser extension (popup) to the host. Adds
|
||
Brave's native-messaging protocol overhead on top of whichever path
|
||
the host takes — but the relative difference between paths is
|
||
preserved.
|
||
- Memory cost of the in-process cache load. The fast path loads
|
||
cache.json once per call today (no caching across calls inside the
|
||
host). A future optimization is to memoize `_load_host_cache` with
|
||
mtime-based invalidation; left for follow-up if needed.
|
||
- Cold-cache effects. `cache.json` is large enough that the OS page
|
||
cache matters; numbers above reflect a warm read. First call after
|
||
a reboot may be slower for both paths but proportionally so.
|