Know when a task succeeds, stalls, or diverges.
Developer assistance, guided setup, compliance, and workflow completion.
"Tell me when installation succeeds or the user becomes blocked."PERCEPT
The perception runtime
Program whatdeserves attention.
Percept turns continuous multimodal streams into sparse, actionable events. Describe what matters. Percept operates the watch and tells your application when it happens.
LiveStream+Watch (Goal + Policy)stream<PerceptEvent>
Percept runtime · deterministic replay00:00.0
Standing watch
"Tell me when a package has been left at the door."
SILENT
Nothing deserves judgment yet.
Percept observes the stream without sending every frame to an expensive model.
{
"gate": "SILENT",
"reason": "no_material_change"
}0frames
0judge calls
0events
01 · The product and the problem
A frame can answer a question. A product has to operate a watch.
Models can perceive. Applications can act. Percept runs the continuous loop between them: deciding when evidence is useful, what moments belong together, when a judgment is needed, and whether one inspectable event should be emitted.
Judge only after evidence becomes useful.
Cheap attention signals and a bounded episode turn a burst of frame changes into one model call and one de-duplicated event.
1judge call
1logical event
boundedpending work
02 · The continuous model
From standing intent to one useful interruption.
The animation is illustrative — it explains the runtime through causality rather than decoration. A continuous stream is filtered, assembled into an event capsule, judged once, then routed to FIRE, ASK, or SILENT.
Plate 01
Continuous stream, one selected event.
Frames arrive without pause. Most pass through quietly. Attention selects a useful moment and seals it into one bounded episode.
01 · frames translate continuously02 · attention closes on one moment03 · evidence lifts into an episode04 · only one capsule reaches judgment
Runtime boundary
Percept operates the standing watch.
Streams and models remain replaceable inputs. Percept owns attention, episode construction, bounded judgment, context, decision semantics, replay, and event delivery. Your application still owns the action.
03 · Who it is for
A live stream. A sparse event. A real cost of missing it — or firing falsely.
The best early workloads are continuously observable, operationally meaningful, and already connected to an application that can use a reliable event.
Observe meaningful process change over time.
Inspections, manufacturing, field work, facilities, and security operations.
"Emit when the inspection step completes or deviates from plan."Give agents a reason to wake up.
Watch voice, vision, screen, and environmental streams without reacting continuously.
"Nudge me if I leave without my purse."Represent absence, persistence, and escalation.
Express concerns that are not reducible to one object detector or isolated frame.
"Alert when the person has not moved for the configured interval."Start the workflow at the meaningful moment.
Interviews, walkthroughs, support sessions, and real-time collaboration.
"Emit when the customer has completed the intake explanation."Standardize perception across applications.
One event contract, one replay surface, one evidence reference, and one runtime boundary.
watch(stream, goal) → WatchEvent04 · OSS · SDK
The smallest stable surface from a stream to an event.
Install the runtime, describe a watch in plain language, connect a source, and subscribe to typed events. The core is pure standard library — it runs offline on deterministic fakes with no API keys. The source is open.
pip install percept-vision
Plain-language watches
Author a WatchPlan in Python — or pass a bare string. The condition travels verbatim to the judge; it is never compiled to a rigid predicate.
Fail-closed judges
A requested judge is authorized, bound to a hash-pinned context, then dispatched — or the arm refuses. Never a silent fallback.
Vendor-neutral backends
Anthropic, Gemini, OpenRouter, and Deepgram sit behind one seam. Deterministic fakes run offline with no API keys.
Optimizer
Pre-registered evaluation plus a mechanical search over operator-declared safe knobs. The winner ships as an OptimizationLock.
import percept
# one WatchEvent per gate FIRE or ASK — silent frames yield nothing
async for event in percept.watch(stream, "the kettle boils"):
handle(event)
# or the batteries-included front door: watch a source, receive Incidents
async for incident in percept.run(
source="kitchen.mp4", # camera index, file, or RTSP
goals=["the milk is boiling over"],
fps=4.0,
):
notify(incident) # a durable Incident, deduped on redelivery05 · Playground
Try a simple watch. Then inspect the entire system.
The preview below is an illustration of the flow. The real Playground runs the actual OSS wheel against your media or a hosted sample — and a quiet run is an answer, not a failure.
Percept Playground · interactive preview
Ready to arm
Plain language → WatchPlan → armed watch
from percept.contracts.watchplan import WatchPlan
plan = WatchPlan(
# plain language — carried verbatim to the judge
condition=(
"The user successfully completes installation and starts Claude Code without an unresolved error."
),
)
# percept.watch(stream, plan) arms it and yields WatchEventsSILENT · subscriber waiting
Run the watch to produce an event.
The preview will expose arming, admission, episode assembly, one judge call, and one delivered event.
Preview: sample screen and video watches · Full: real runs on the provenance-stamped OSS wheel.Open full Playground →
06 · Benchmarks
Benchmark the runtime, not just the model.
A model can classify a clip and still fail in a continuous product. Percept's evaluation rail is pre-registered and offline: bars are declared before numbers exist, and silence is never a pass.
Illustrative curves only — no published numbers exist yet. These panels explain the measurement contract; published results will remain tied to locked configurations and reproducible artifacts.
Does the runtime remain bounded under overload?
Drive input faster than the judge. Pass only when pending depth and oldest age remain bounded, supersession is explicit, memory is flat, and stale completions cannot mutate the gate.
oldest pending agememory slopestale completionsdelivery isolation
Design partners
Bring us a stream and the event that matters.
We are working with teams that already have a continuous perception problem and a real cost of missing the event — or firing falsely.