LightOnOCR-2 1B Base
9 configurations testedQuality criteria
Frozen browser-gates.json: paragraph token recall>=0.9/CER<=0.1, phrase recall>=0.8, required image-boxIoU>=0.6, EOS/non-repetition, per-page WebGPU<=60s/WASM<=120s and observed browser RSS<=16GiB. All three distinct image fixtures must pass; no sibling/device evidence transfer.
| Precision | Device | Result | |
|---|---|---|---|
Q4 | GPU · WebGPUAccelerated | Passed | Very slow |
Q4 | GPU · WebGPUAccelerated | Passed | Very slow |
Q4 | GPU · WebGPUAccelerated | Passed | Very slow |
Q4 | GPU · WebGPUAccelerated | Passed | Very slow |
Q4 | GPU · WebGPUAccelerated | Passed | Very slow |
Q4 | GPU · WebGPUAccelerated | Passed | Very slow |
Q4 | CPU · WASM | Passed | Very slow |
Q4 | CPU · WASM | Failed | — |
Q4 | CPU · WASM | Failed | — |
Test setup & sample details
- Quality gate
- Frozen browser-gates.json: paragraph token recall>=0.9/CER<=0.1, phrase recall>=0.8, required image-boxIoU>=0.6, EOS/non-repetition, per-page WebGPU<=60s/WASM<=120s and observed browser RSS<=16GiB. All three distinct image fixtures must pass; no sibling/device evidence transfer.
- Tested
- 2026-11-08T15:05:30.974Z
- Environment
- Isolated persistent Chrome157.0.8103.39 module worker per precision/device configuration on macOS Apple Silicon M3 Max96GiB. 1120px longest edge; original LightOn flat image-token expansion. Actual GPU dispatch tracing; CPU preprocessing/possible WASM shape fallback. Concurrent tracked mirror uploads recorded.
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- d722adb98a1a6b7cec6774ac3225cb93120e0936
- Fixture
{ "file": "model-samples/desktop/lightonocr2/fixtures/scientific-paragraph.png", "url": "https://assets.localai.tools/samples/v1/files/de0959e7c3fe59f4d706bb84d993cd1aae000a6b25bcef96f472d3a675871dfb/scientific-paragraph.png", "size": 26674, "sha1": "b4e9dd3ea6464bc8e1ff1f95134eccfc25622d52", "sha256": "de0959e7c3fe59f4d706bb84d993cd1aae000a6b25bcef96f472d3a675871dfb" }- Rerun when
- checkpoint/artifact revision; runtime version; adapter/preprocessing/quantization; fixture checksum or quality gates; browser/backend behavior
Per-case quality & failure details
[
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"complete": true,
"nonRepetitive": true,
"withinSpeedBudget": true,
"withinMemoryBudget": true,
"tokenRecall": 0.95,
"characterErrorRate": 0.006968641114982578,
"textQuality": true,
"passed": true
},
"runtime": {
"loadMs": 6871,
"inferenceMs": 5376.519999999553,
"totalMs": 12247.519999999553,
"observedBrowserRssBytes": 4834541568,
"pagesPerSecond": 0.18599391427914025,
"tokensPerSecond": 14.50752531377294,
"grade": "A",
"performanceMetric": {
"name": "pagesPerSecond",
"value": 0.18599391427914025
},
"device": "webgpu",
"precision": "q4",
"provider": "WebGPU"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"complete": true,
"nonRepetitive": true,
"withinSpeedBudget": true,
"withinMemoryBudget": true,
"phraseRecall": 0.8333333333333334,
"textQuality": true,
"passed": true
},
"runtime": {
"loadMs": 6871,
"inferenceMs": 24769.210000000894,
"totalMs": 31640.210000000894,
"observedBrowserRssBytes": 4879319040,
"pagesPerSecond": 0.04037270466034096,
"tokensPerSecond": 12.959638195969449,
"grade": "B",
"performanceMetric": {
"name": "pagesPerSecond",
"value": 0.04037270466034096
},
"device": "webgpu",
"precision": "q4",
"provider": "WebGPU"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"complete": true,
"nonRepetitive": true,
"withinSpeedBudget": true,
"withinMemoryBudget": true,
"phraseRecall": 1,
"textQuality": true,
"passed": true
},
"runtime": {
"loadMs": 6871,
"inferenceMs": 29176.455000001937,
"totalMs": 36047.45500000194,
"observedBrowserRssBytes": 4899995648,
"pagesPerSecond": 0.03427421186021172,
"tokensPerSecond": 1.713710593010586,
"grade": "B",
"performanceMetric": {
"name": "pagesPerSecond",
"value": 0.03427421186021172
},
"device": "webgpu",
"precision": "q4",
"provider": "WebGPU"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"complete": true,
"nonRepetitive": true,
"withinSpeedBudget": true,
"withinMemoryBudget": true,
"tokenRecall": 0.95,
"characterErrorRate": 0.006968641114982578,
"textQuality": true,
"passed": true
},
"runtime": {
"loadMs": 6871,
"inferenceMs": 5291.744999997318,
"totalMs": 12162.744999997318,
"observedBrowserRssBytes": 4924637184,
"pagesPerSecond": 0.1889735805486672,
"tokensPerSecond": 14.73993928279604,
"grade": "A",
"performanceMetric": {
"name": "pagesPerSecond",
"value": 0.1889735805486672
},
"device": "webgpu",
"precision": "q4",
"provider": "WebGPU"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"complete": true,
"nonRepetitive": true,
"withinSpeedBudget": true,
"withinMemoryBudget": true,
"tokenRecall": 0.95,
"characterErrorRate": 0.006968641114982578,
"textQuality": true,
"passed": true
},
"runtime": {
"loadMs": 6871,
"inferenceMs": 5245.5,
"totalMs": 12116.5,
"observedBrowserRssBytes": 4963876864,
"pagesPerSecond": 0.19063959584405682,
"tokensPerSecond": 14.869888475836431,
"grade": "A",
"performanceMetric": {
"name": "pagesPerSecond",
"value": 0.19063959584405682
},
"device": "webgpu",
"precision": "q4",
"provider": "WebGPU"
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"complete": true,
"nonRepetitive": true,
"withinSpeedBudget": true,
"withinMemoryBudget": true,
"tokenRecall": 0.95,
"characterErrorRate": 0.006968641114982578,
"textQuality": true,
"passed": true
},
"runtime": {
"loadMs": 6871,
"inferenceMs": 5270.469999998808,
"totalMs": 12141.469999998808,
"observedBrowserRssBytes": 4977049600,
"pagesPerSecond": 0.18973639922060578,
"tokensPerSecond": 14.799439139207252,
"grade": "A",
"performanceMetric": {
"name": "pagesPerSecond",
"value": 0.18973639922060578
},
"device": "webgpu",
"precision": "q4",
"provider": "WebGPU"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": {
"complete": true,
"nonRepetitive": true,
"withinSpeedBudget": true,
"withinMemoryBudget": true,
"actualNeuralPlacement": true,
"tokenRecall": 0.95,
"characterErrorRate": 0.006968641114982578,
"textQuality": true,
"passed": true
},
"runtime": {
"loadMs": 6495,
"inferenceMs": 31809.980000000447,
"totalMs": 38304.98000000045,
"observedBrowserRssBytes": 5826576384,
"pagesPerSecond": 0.031436674905170825,
"tokensPerSecond": 2.4520606426033247,
"grade": "C",
"performanceMetric": {
"name": "pagesPerSecond",
"value": 0.031436674905170825
},
"device": "wasm",
"precision": "q4",
"provider": "WASM CPU"
}
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"passed": false,
"error": "Error: page.evaluate: Error: failed to call OrtRun(). ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/common/safeint.h:17 static void SafeIntExceptionHandler<onnxruntime::OnnxRuntimeException>::SafeIntOnOverflow() Integer overflow\n\n at worker.onmessage (eval at evaluate (:291:30), <anonymous>:1:698)"
},
"runtime": {
"loadMs": 6495,
"inferenceMs": 265,
"observedBrowserRssBytes": 5182373888,
"grade": "F",
"device": "wasm",
"precision": "q4",
"provider": "WASM CPU"
},
"failure": "Error: page.evaluate: Error: failed to call OrtRun(). ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/common/safeint.h:17 static void SafeIntExceptionHandler<onnxruntime::OnnxRuntimeException>::SafeIntOnOverflow() Integer overflow\n\n at worker.onmessage (eval at evaluate (:291:30), <anonymous>:1:698)"
},
{
"precision": "q4",
"device": "wasm",
"status": "fail",
"quality": {
"passed": false,
"error": "Error: page.evaluate: Error: failed to call OrtRun(). ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/common/safeint.h:17 static void SafeIntExceptionHandler<onnxruntime::OnnxRuntimeException>::SafeIntOnOverflow() Integer overflow\n\n at worker.onmessage (eval at evaluate (:291:30), <anonymous>:1:698)"
},
"runtime": {
"loadMs": 6495,
"inferenceMs": 66,
"observedBrowserRssBytes": 5262950400,
"grade": "F",
"device": "wasm",
"precision": "q4",
"provider": "WASM CPU"
},
"failure": "Error: page.evaluate: Error: failed to call OrtRun(). ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/common/safeint.h:17 static void SafeIntExceptionHandler<onnxruntime::OnnxRuntimeException>::SafeIntOnOverflow() Integer overflow\n\n at worker.onmessage (eval at evaluate (:291:30), <anonymous>:1:698)"
}
]