Step 11: benchmark host fast-path, decision = keep
Adds benchmarks/host-fast-path.py and benchmarks/README.md.
The benchmark compares two paths for a cached single-ID search:
1. fast-path: in-process dict walk inside the native host
(handle_cached_search_fast in rcjav-host.py)
2. subprocess: shell out to `rc-jav.py --search ID --cache --format json`
Idle baseline against the live 7124-file cache (5 queries × 5 iter):
fast-path: median 0.46ms p95 0.61ms max 0.72ms
subprocess: median 919ms p95 1233ms max 1385ms
median speedup: 2000x
Decision: keep the fast path. The ~920ms subprocess cost is dominated
by Python interpreter startup + 1.3MB cache.json parse. That's
structural — it applies under idle Python too, not just when a scan
is running. The "Python actively scanning" condition from the original
roadmap doesn't change the verdict; it would only make the subprocess
path even slower while leaving the in-process path unaffected (the
fast path doesn't touch the scanning process).
The fast path is already correctly scoped — bails out for wildcards,
ranges, name searches, and --quick mode. Narrowing further would just
push more queries through the slow path with no upside.
Possible follow-up (not in scope here): memoize _load_host_cache with
mtime-based invalidation so the fast path doesn't reparse cache.json
on every call. Current per-call median (0.46ms) is already fast enough
that this is optional.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,66 @@
|
||||
# benchmarks/
|
||||
|
||||
Latency benchmarks for decisions in the rc-jav roadmap.
|
||||
|
||||
## host-fast-path.py — Step 11 decision
|
||||
|
||||
The native-messaging host has an in-process fast path
|
||||
(`handle_cached_search_fast` in `rcjav-host.py`) that answers simple
|
||||
cached single-ID searches without shelling out to `rc-jav.py`. Step 11
|
||||
asked: is the fast path actually pulling its weight, or could we
|
||||
delete it / narrow it down?
|
||||
|
||||
### Run
|
||||
|
||||
```
|
||||
python benchmarks/host-fast-path.py [--queries Q1 Q2 ...] [--iterations N]
|
||||
```
|
||||
|
||||
For the "Python actively scanning" condition, start `rc-jav.py --scan`
|
||||
in a separate terminal first.
|
||||
|
||||
### Findings (idle baseline, 2026-05-23, 7124-file cache)
|
||||
|
||||
```
|
||||
=== aggregate (5 queries × 5 iterations, idle Python) ===
|
||||
fast-path total:
|
||||
n=25 min=0.43ms median=0.46ms mean=0.48ms p95=0.61ms max=0.72ms
|
||||
subprocess total:
|
||||
n=25 min=880.56ms median=919.45ms mean=965.55ms p95=1232.93ms max=1385.37ms
|
||||
median speedup: 2000.1x
|
||||
p95 speedup: 2036.0x
|
||||
```
|
||||
|
||||
### Decision: keep the fast path
|
||||
|
||||
The subprocess path costs ~920ms median per query even with the
|
||||
interpreter doing nothing else — that's pure Python startup +
|
||||
`json.loads` of the 1.3 MB cache.json. The fast path returns hits in
|
||||
under 1ms. The 2000× speedup is structural (interpreter startup
|
||||
overhead), not load-dependent, so it would apply equally under (a)
|
||||
idle and (b) active-scan conditions.
|
||||
|
||||
The fast path is already correctly scoped — it bails out for
|
||||
wildcards, ranges, name searches, and `--quick` mode (which forces a
|
||||
live rclone hit). Narrowing further would just push more queries
|
||||
through the slow path with no upside.
|
||||
|
||||
The "Python actively scanning" condition listed in the original
|
||||
roadmap was framed as the case where the fast path's value would be
|
||||
most obvious. The idle baseline already settles it; we don't need to
|
||||
gate the decision on the active-scan measurement, though running it
|
||||
remains a sanity check if cache.json grows substantially.
|
||||
|
||||
### What this benchmark does NOT cover
|
||||
|
||||
- Latency from inside the browser extension (popup) to the host. Adds
|
||||
Brave's native-messaging protocol overhead on top of whichever path
|
||||
the host takes — but the relative difference between paths is
|
||||
preserved.
|
||||
- Memory cost of the in-process cache load. The fast path loads
|
||||
cache.json once per call today (no caching across calls inside the
|
||||
host). A future optimization is to memoize `_load_host_cache` with
|
||||
mtime-based invalidation; left for follow-up if needed.
|
||||
- Cold-cache effects. `cache.json` is large enough that the OS page
|
||||
cache matters; numbers above reflect a warm read. First call after
|
||||
a reboot may be slower for both paths but proportionally so.
|
||||
Reference in New Issue
Block a user