Granite Docling 258M — Image to Text
10 configurations testedQuality criteria
Bounded 256-token page prefix: expected header token recall >= 0.9, >=2 coordinate groups, DocTags, and no degenerate repetition. Streaming equality audited separately for each advertised pair.
| Precision | Device | Result | |
|---|---|---|---|
Q4 | GPU · WebGPUAccelerated | Passed | Very slow |
FP32 | GPU · WebGPUAccelerated | Passed | Very slow |
Q4 | CPU · WASM | Passed | Very slow |
FP32 | CPU · WASM | Failed | — |
FP16 | CPU · WASM | Failed | — |
FP16 | GPU · WebGPU | Failed | — |
Q8 | CPU · WASM | Failed | — |
Q8 | GPU · WebGPU | Failed | — |
Q4F16 | CPU · WASM | Failed | — |
Q4F16 | GPU · WebGPU | Failed | — |
Test setup & sample details
- Quality gate
- Bounded 256-token page prefix: expected header token recall >= 0.9, >=2 coordinate groups, DocTags, and no degenerate repetition. Streaming equality audited separately for each advertised pair.
- Tested
- 2026-09-15T17:31:52.019Z
- Environment
- Not recorded
- Runtime
- Transformers.js 4.2.0
- Artifact revision
- e8602580df77443fc3421cf3bae0601da601e5c6
- Fixture
{ "file": "granite-docling-new-arxiv.png", "url": "https://f005.backblazeb2.com/file/hubters-webai-public/assets/benchmarks/granite-docling-new-arxiv.png", "width": 1190, "height": 1540, "preprocessing": "Pinned processor with image splitting; deterministic 256-token prefix benchmark.", "licenseUrl": "https://www.apache.org/licenses/LICENSE-2.0.txt", "size": 523477, "sha1": "fa9ee1b276da7ce7374804e1e392b5b60c501a52", "source": "https://huggingface.co/ibm-granite/granite-docling-258M/blob/982fe3b40f2fa73c365bdb1bcacf6c81b7184bfe/assets/new_arxiv.png", "license": "Apache-2.0" }- Rerun when
- artifact revision change; runtime version change; fixture or quality-gate change; browser/backend behavior change
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "wasm",
"status": "fail",
"quality": {
"headerTokenRecall": 0
},
"runtime": {
"loadMs": 127190.30000001192,
"inferenceMs": null,
"totalMs": 134755.59999999404,
"grade": "failed"
},
"failure": "Error: failed to call OrtRun(). ERROR_CODE: 6, ERROR_MESSAGE: std::bad_alloc"
},
{
"precision": "fp32",
"device": "webgpu",
"status": "pass",
"quality": {
"nonempty": true,
"doctags": true,
"content": true,
"repetitive": false,
"characters": 1036,
"grade": "docling-pass",
"headerTokenRecall": 1
},
"runtime": {
"loadMs": 130322.30000001192,
"inferenceMs": 8250.09999999404,
"totalMs": 139901.40000000596,
"grade": "slow",
"pagesPerSecond": 0.12121065199218464
}
},
{
"precision": "fp16",
"device": "wasm",
"status": "fail",
"quality": {
"headerTokenRecall": 0
},
"runtime": {
"loadMs": null,
"inferenceMs": null,
"totalMs": 50965,
"grade": "failed"
},
"failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/vision_model/encoder/layers.0/layer_norm1/Constant_output_0for node: /vision_model/encoder/layers.0/layer_norm1/Mul/SimplifiedLayerNormFusion/\n"
},
{
"precision": "fp16",
"device": "webgpu",
"status": "fail",
"quality": {
"nonempty": true,
"doctags": false,
"content": false,
"repetitive": true,
"characters": 256,
"grade": "failed",
"headerTokenRecall": 0
},
"runtime": {
"loadMs": 68164.79999998212,
"inferenceMs": 6669,
"totalMs": 76209.90000000596,
"grade": "slow",
"pagesPerSecond": 0.149947518368571
}
},
{
"precision": "q8",
"device": "wasm",
"status": "fail",
"quality": {
"nonempty": true,
"doctags": true,
"content": true,
"repetitive": true,
"characters": 1147,
"grade": "failed",
"headerTokenRecall": 0.2727272727272727
},
"runtime": {
"loadMs": 37610.90000000596,
"inferenceMs": 114990.09999999404,
"totalMs": 153973.80000001192,
"grade": "very-slow",
"pagesPerSecond": 0.008696400820592831
}
},
{
"precision": "q8",
"device": "webgpu",
"status": "fail",
"quality": {
"nonempty": true,
"doctags": true,
"content": false,
"repetitive": true,
"characters": 771,
"grade": "failed",
"headerTokenRecall": 0.18181818181818182
},
"runtime": {
"loadMs": 33490.19999998808,
"inferenceMs": 107011.80000001192,
"totalMs": 141849.5,
"grade": "very-slow",
"pagesPerSecond": 0.009344763848471745
}
},
{
"precision": "q4",
"device": "wasm",
"status": "pass",
"quality": {
"nonempty": true,
"doctags": true,
"content": true,
"repetitive": false,
"characters": 779,
"grade": "docling-pass",
"headerTokenRecall": 0.9090909090909091,
"streamMatches": true,
"accuracyNote": "Minor header OCR substitution; bounded prefix only."
},
"runtime": {
"loadMs": 41823.5,
"inferenceMs": 164716.39999997616,
"totalMs": 207799,
"grade": "very-slow",
"pagesPerSecond": 0.006071040892103911
}
},
{
"precision": "q4",
"device": "webgpu",
"status": "pass",
"quality": {
"nonempty": true,
"doctags": true,
"content": true,
"repetitive": false,
"characters": 779,
"grade": "docling-pass",
"headerTokenRecall": 0.9090909090909091,
"streamMatches": true,
"accuracyNote": "Minor header OCR substitution; bounded prefix only."
},
"runtime": {
"loadMs": 50243,
"inferenceMs": 6164.299999982119,
"totalMs": 57724.19999998808,
"grade": "slow",
"pagesPerSecond": 0.1622244212648477
}
},
{
"precision": "q4f16",
"device": "wasm",
"status": "fail",
"quality": {
"headerTokenRecall": 0
},
"runtime": {
"loadMs": null,
"inferenceMs": null,
"totalMs": 23969.90000000596,
"grade": "failed"
},
"failure": "Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: /mnt/vss/_work/1/s/onnxruntime/core/graph/graph_utils.cc:30 int onnxruntime::graph_utils::GetIndexFromName(const Node &, const std::string &, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/vision_model/encoder/layers.0/layer_norm1/Constant_output_0for node: /vision_model/encoder/layers.0/layer_norm1/Mul/SimplifiedLayerNormFusion/\n"
},
{
"precision": "q4f16",
"device": "webgpu",
"status": "fail",
"quality": {
"nonempty": true,
"doctags": false,
"content": false,
"repetitive": true,
"characters": 256,
"grade": "failed",
"headerTokenRecall": 0
},
"runtime": {
"loadMs": 29559,
"inferenceMs": 4497.5,
"totalMs": 35318.5,
"grade": "good",
"pagesPerSecond": 0.22234574763757642
}
}
]