Documentation

Sentinel Vision for Windows, version 0.1.0. Four programs that run detection and pose models on a webcam or a video and turn what they find into statuses.

Download and run

  1. Download SentinelVision-windows-x64.zip.
  2. Unzip it anywhere. Nothing is installed and nothing is written outside the folder.
  3. Double-click SentinelOffice.exe, SentinelShop.exe, SentinelHome.exe or SentinelExam.exe.

Each program starts on its sample clip. The clip has nothing drawn in it. The outlines and statuses are computed as it plays.

The programs are not code-signed, so Windows may ask before the first run. Choose "More info", then "Run anyway".

Keep the folder together. The programs read models, samples, zones and fonts at start and share the _internal folder. To use them on another PC, copy the whole folder.

It needs Windows 10 or 11, 64-bit. It runs on the CPU and uses a DirectX 12 graphics card when one is available.

The four programs

ProgramReportsModels it loads
SentinelOfficeEach seat as Working, Absent or Phone usage. Seats are learned from the first second.Objects
SentinelShopZones you draw: someone entering, someone staying past a time, more people than a limit.Objects
SentinelHomeZones by what is in them: a person, a vehicle or a pet. Pets are recognised and raise no alert.Objects
SentinelExamEach candidate's head direction, the desk it points at, phones, and a risk score from 0 to 100.Pose and objects

Keys

KeyDoes
WWebcam. Press again for the next camera.
OOpen a video file.
SBack to the sample clip.
MSwitch between the small model and the nano model.
[ ]Smaller or larger model input: 640, 960 or 1280 pixels.
RStart again. Seats are mapped afresh and timers go to zero.
ZDraw a zone (Shop and Home). Click each corner, press Enter, then pick the rule with a number key.
CClear all zones (Shop and Home).
PPause.
HShow or hide the key hints.
FFull screen.
XSave a screenshot to logs.
QQuit.

The readout and the log

The box in the bottom corner of the window shows what is running: the model, its input size, whether it is on the GPU or the CPU, the inference time for the last frame, the frame rate and the number of detections.

Every status change is written to a CSV file in logs, one file per session, with the clock time, the time into the stream, the source, the seat or zone, and what changed.

A program that cannot start writes the reason to sentinel.log in its folder.

Rules and thresholds

A status is one rule applied to what the models found. The rules are the same ones the demo clips on the site were made with.

Office

StatusRule
WorkingA person is detected in the seat and no phone is detected in their hands. It means the person is at the desk.
AbsentNo person in the seat for 0.75 seconds. Clears after 0.4 seconds back.
Phone usageA phone is detected inside the person's outline, away from where phones were lying during the first second, for 0.35 seconds.

Zones (Shop and Home)

RuleFires when
EntryA person, vehicle or pet, whichever the zone is set to, is inside for about a third of a second. The status stays up briefly after the zone empties.
DwellThe same tracked person has been inside for 5 seconds.
OccupancyMore people than the limit are inside for 0.6 seconds.

Zones you draw use the centre of each detected box. The zone files for the sample clips are in zones, and yours are saved there as custom_shop.json and custom_home.json.

Exam

  1. During the first second, each seated person becomes a seat with a desk zone around where their wrists rest and a baseline head direction.
  2. Head direction is the line from the midpoint of the ears to the nose. Yaw is how far that line has turned from the candidate's own baseline, in degrees.
  3. Past 38 degrees, the look has left the candidate's own paper. The target is the best-aligned desk within 30 degrees of the head direction and within seven shoulder-widths.
  4. A phone counts when it is inside the candidate's outline or desk zone for 0.35 seconds.
Risk scoreValueMeaning
Look hold0.35 sA look must last this long before it scores.
Look rate42 a secondAdded while the head points at another desk.
Decay7 a secondRemoved the rest of the time.
Phone85A phone at the desk lifts the score to at least this.
Review25Normal becomes Review.
Flagged60Review becomes Flagged.

Looking up, or away at nothing, scores nothing. A flag is a prompt for an invigilator to check the footage at that time. The program makes no finding about what the candidate was doing.

Model files

The models folder holds four ONNX files. The same files are in the repository.

FileSize in bytesSHA-256
yolo26n.onnx10,419,737a772bebdb74762b8567986ac206a5ecafb27e481c572cfa41435e3300acc4642
yolo26s.onnx38,769,1902a0b6ac6d7d441fbc71c574af44f8694f62fda33e940078c16037c105268e01a
yolo26n-pose.onnx12,534,179b690600113b28104f1041c7467842f43bf122b4df1037d2a598129dacc3e8287
yolo26s-pose.onnx42,267,783209b1d2b54efc281e2980d40e9eedd638999517802cba1fcf5bc59e35d7968e6

To check a file on Windows:

certutil -hashfile models\yolo26s.onnx SHA256

Each model takes one RGB image scaled to 0 to 1, at any width and height that are multiples of 32. The object models return 300 rows of box, confidence and class. The pose models return 300 rows of box, confidence and 17 keypoints. They are opset 17 exports of Ultralytics YOLO26 with its end-to-end head, so the rows are final detections.

Speed

Inference time per frame on a 2-core 2.1 GHz Xeon with no graphics card:

Model and inputOffice, Shop, HomeExam
Nano, 640 px40 to 57 ms94 ms
Nano, 1280 px165 to 173 ms380 ms
Small, 640 px119 to 158 ms256 ms
Small, 1280 px463 to 553 ms1,037 ms

Exam is slower because it runs the pose model and the object model on every frame.

The program chooses the small model at 1280 pixels for a video file on a graphics card, and 640 pixels on a CPU or for a webcam, where people are close to the camera. M, [ and ] change that while it runs.

When the PC cannot keep up with a file, frames are dropped so the clip stays at real speed. The rules run on clip time, so events fire at the same moments either way.

Source and build

The source is at github.com/forgehk/sentinel-vision: the programs, the rules, the tests and the workflow that builds each release.

That workflow runs on GitHub's Windows machines. It runs the tests, builds the folder, starts each of the four programs on its sample clip and stops unless each one reports the expected events. Only then is the zip published, with its SHA-256. The build log is public.

To build it yourself, install Python 3.10 or newer, download the source and double-click app\build_exe.bat. To run it without building:

cd app
run.bat exam

Limits

  • One person on a webcam cannot be flagged for looking at another desk, because there is no other desk. The yaw figure still moves and a phone still flags.
  • Head direction is measured in the picture from one camera. It reads a head turned sideways well, and a head tilted toward or away from the camera poorly. With the nose hidden, no direction is reported.
  • The gaze target is the best-aligned nearby desk. It is not a measurement of what the eyes are on.
  • Phones are small. The small model at 1280 pixels finds a phone held at a desk a few metres away. The nano model at 640 pixels usually does not.
  • Seats are learned from the first second. Press R to map them again after people have sat down.
  • The release check runs on a machine with no camera and no graphics card, so webcam input and GPU inference are not part of it.

Licence

Sentinel Vision is released under the GNU Affero General Public License v3.0. The YOLO26 models are from Ultralytics and carry the same licence. Closed-source commercial use of the models needs an Ultralytics Enterprise licence.