PaddleOCR-VL 1.6
Read text from PNG/JPEG document regions or locally rasterized PDF pages with native PaddleOCR-VL 1.6. Returns complete raw OCR text and per-page results.
Desktop app
Run PaddleOCR-VL 1.6 on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
PaddleOCR-VL 1.6 · 0.9B — Image to Text
35 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 1736 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2235 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2305 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2313 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2322 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2337 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2349 ms / 0 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2553 ms / 711 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2654 ms / 619 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2660 ms / 654 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 2788 ms / 568 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 3066 ms / 704 ms |
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 6620 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 4305 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5407 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5484 ms / 359 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5535 ms / 357 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5563 ms / 369 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5567 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5573 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5573 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5583 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 5624 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 9835 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11078 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11238 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11259 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11263 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11268 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11271 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11280 ms / 401 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11318 ms / 388 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 11362 ms / 373 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 14197 ms / 0 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 31051 ms / 0 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-02T19:09:06.390642+00:00
- Environment
- { "hardware": "Apple M3 Max", "os": "macOS-27.2-arm64-arm-64bit-Mach-O", "python": "3.13.18 (app-owned private development venv)", "runtime": "Transformers5.22.1/Torch2.10.0/PDFium5.14.0" }
- Runtime
- transformers 5.22.1
- Artifact revision
- c5630abae1d940eafe0697512a0325494b02ab42
- Fixture
{ "file": "model-samples/desktop/docling/receipt.png", "size": 25773, "sha1": "05f77f558dde3e17f3541910abeecde08c193cd5", "sha256": "cc0123c5e9e48de091ce0d3462d267b2485295096206353c9bf4beeec594ca0e", "url": "https://assets.localai.tools/samples/v1/files/cc0123c5e9e48de091ce0d3462d267b2485295096206353c9bf4beeec594ca0e/receipt.png" }- Rerun when
- artifact revision/checksums; Transformers/Torch/PDFium/runtime/driver/backend; OCR prompt/preprocessing/PDF rendering/output config; fixtures/expected text/quality and usability gates
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 400.645000001532,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11324.802291986998,
"inferenceMs": 11279.520832991693,
"pagesPerSecond": 0.08865624832883683,
"tokensPerSecond": 4.521468664770678,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11725.44729198853,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11091.24416600389,
"inferenceMs": 11077.8712080064,
"pagesPerSecond": 0.09027005109765691,
"tokensPerSecond": 4.062152299394561,
"outputTokens": 45,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11091.24416600389,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 31083.505417002016,
"inferenceMs": 31051.173749990994,
"pagesPerSecond": 0.06440980351026442,
"tokensPerSecond": 2.18993331934899,
"outputTokens": 68,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 31083.505417002016,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 9847.740416997112,
"inferenceMs": 9834.564666991355,
"pagesPerSecond": 0.10168218257350944,
"tokensPerSecond": 0.10168218257350944,
"outputTokens": 1,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 9847.740416997112,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11250.644375002594,
"inferenceMs": 11237.856709005428,
"pagesPerSecond": 0.08898493955690435,
"tokensPerSecond": 4.538231917402122,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11250.644375002594,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11283.710542003973,
"inferenceMs": 11270.655750005972,
"pagesPerSecond": 0.08872598207069453,
"tokensPerSecond": 4.525025085605421,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11283.710542003973,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11272.106457996415,
"inferenceMs": 11258.873458005837,
"pagesPerSecond": 0.08881883287256692,
"tokensPerSecond": 4.529760476500913,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11272.106457996415,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11276.078791997861,
"inferenceMs": 11263.19120900007,
"pagesPerSecond": 0.08878478412059015,
"tokensPerSecond": 4.528023990150098,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11276.078791997861,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11281.033000006573,
"inferenceMs": 11267.897458004882,
"pagesPerSecond": 0.0887477014879635,
"tokensPerSecond": 4.526132775886139,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11281.033000006573,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 387.8166670037899,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11339.594042001409,
"inferenceMs": 11317.977541999426,
"pagesPerSecond": 0.08835500832981338,
"tokensPerSecond": 4.506105424820483,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11727.410709005198,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 373.2784169988008,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 11383.419749996392,
"inferenceMs": 11361.55804200098,
"pagesPerSecond": 0.08801609746684722,
"tokensPerSecond": 4.488820970809208,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 1,
"totalMs": 11756.698166995193,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 357.00574998918455,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5549.011916999007,
"inferenceMs": 5535.161749998224,
"pagesPerSecond": 0.1806631938082606,
"tokensPerSecond": 9.21382288422129,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5906.017666988191,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5412.052917003166,
"inferenceMs": 5406.638500004192,
"pagesPerSecond": 0.18495780696253775,
"tokensPerSecond": 8.323101313314199,
"outputTokens": 45,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5412.052917003166,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 14211.072959005833,
"inferenceMs": 14197.4435000011,
"pagesPerSecond": 0.14087043206052166,
"tokensPerSecond": 4.789594690057737,
"outputTokens": 68,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 14211.072959005833,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 4310.246666995226,
"inferenceMs": 4304.8894169915,
"pagesPerSecond": 0.2322940041277196,
"tokensPerSecond": 0.2322940041277196,
"outputTokens": 1,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 4310.246666995226,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5629.381833001389,
"inferenceMs": 5623.9088750007795,
"pagesPerSecond": 0.17781226940663425,
"tokensPerSecond": 9.068425739738347,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5629.381833001389,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5572.293749995879,
"inferenceMs": 5566.85520800238,
"pagesPerSecond": 0.17963463439151342,
"tokensPerSecond": 9.161366353967185,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5572.293749995879,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5589.021207997575,
"inferenceMs": 5583.2114160002675,
"pagesPerSecond": 0.17910838861201242,
"tokensPerSecond": 9.134527819212632,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5589.021207997575,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5578.438833996188,
"inferenceMs": 5573.049500002526,
"pagesPerSecond": 0.17943497541149542,
"tokensPerSecond": 9.151183745986266,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5578.438833996188,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5578.071500000078,
"inferenceMs": 5572.82766699791,
"pagesPerSecond": 0.1794421180331782,
"tokensPerSecond": 9.151548019692088,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5578.071500000078,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 368.9181250083493,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5576.827333003166,
"inferenceMs": 5562.751667006523,
"pagesPerSecond": 0.1797671476027131,
"tokensPerSecond": 9.168124527738367,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5945.745458011515,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 359.08891599683557,
"placement": {
"cpu/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 5497.921000001952,
"inferenceMs": 5484.249167013331,
"pagesPerSecond": 0.18234036593647154,
"tokensPerSecond": 9.299358662760048,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "cpu"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"device": "cpu",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5857.009915998788,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 703.9827080006944,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 3082.3674160055816,
"inferenceMs": 3065.5722499941476,
"pagesPerSecond": 0.32620337035015534,
"tokensPerSecond": 16.636371887857923,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9401565184,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3786.350124006276,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2243.6030829994706,
"inferenceMs": 2235.2628750086296,
"pagesPerSecond": 0.44737467399494985,
"tokensPerSecond": 20.131860329772742,
"outputTokens": 45,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9401565184,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2243.6030829994706,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 6642.127583007095,
"inferenceMs": 6620.134625991341,
"pagesPerSecond": 0.3021086598673977,
"tokensPerSecond": 10.271694435491522,
"outputTokens": 68,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "application/pdf",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13965426688,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 6642.127583007095,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 0,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 0,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 1744.832915996085,
"inferenceMs": 1735.5955000093672,
"pagesPerSecond": 0.5761711182096305,
"tokensPerSecond": 0.5761711182096305,
"outputTokens": 1,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13965426688,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 1744.832915996085,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2321.103750000475,
"inferenceMs": 2313.114874996245,
"pagesPerSecond": 0.4323174827197561,
"tokensPerSecond": 22.048191618707563,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2321.103750000475,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2358.024834000389,
"inferenceMs": 2348.791042008088,
"pagesPerSecond": 0.4257509425551345,
"tokensPerSecond": 21.71329807031186,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2358.024834000389,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2346.0327080101706,
"inferenceMs": 2336.925124996924,
"pagesPerSecond": 0.4279127257024618,
"tokensPerSecond": 21.823549010825552,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2346.0327080101706,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2313.9295000000857,
"inferenceMs": 2304.5750000019325,
"pagesPerSecond": 0.4339194862389644,
"tokensPerSecond": 22.129893798187187,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2313.9295000000857,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 0,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2330.975083998055,
"inferenceMs": 2322.0267500000773,
"pagesPerSecond": 0.43065826007386293,
"tokensPerSecond": 21.963571263767008,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 13973815296,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 2330.975083998055,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 653.638707997743,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2676.3502919930033,
"inferenceMs": 2660.101790999761,
"pagesPerSecond": 0.3759254639741303,
"tokensPerSecond": 19.172198662680643,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9401565184,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3329.9889999907464,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 710.6122920085909,
"placement": {
"mps:0/torch.float32": 608
},
"parameterDtype": "torch.float32",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2569.109625008423,
"inferenceMs": 2552.866958998493,
"pagesPerSecond": 0.39171645685457374,
"tokensPerSecond": 19.97753929958326,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 9401565184,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 3279.7219170170138,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 567.8210839978419,
"placement": {
"mps:0/torch.float16": 608
},
"parameterDtype": "torch.float16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2862.037832994247,
"inferenceMs": 2787.506000007852,
"pagesPerSecond": 0.3587436224342416,
"tokensPerSecond": 18.295924744146323,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "fp16",
"cpuThreads": 4,
"totalMs": 3429.858916992089,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"phraseRecall": 1,
"minimumPhraseRecall": 0.8,
"textTokenRecall": 1,
"minimumTextTokenRecall": 0.9,
"blankNoHallucination": true,
"pageCoverage": true,
"completeGeneration": true,
"repetitive": false,
"scope": "Independent authored content, not structured layout/table evaluation"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"pdfiumVersion": "5.14.0",
"loadMs": 618.5159159940667,
"placement": {
"mps:0/torch.bfloat16": 608
},
"parameterDtype": "torch.bfloat16",
"provider": "Metal/MPS",
"acceleration": "partial",
"attentionImplementation": "sdpa",
"accelerationDetails": "Vision/language parameters, inputs and vision activations on selected device; rendering, tokenizer and decoding on CPU",
"durationMs": 2728.1076249928446,
"inferenceMs": 2653.7129589996766,
"pagesPerSecond": 0.3768305070857974,
"tokensPerSecond": 19.218355861375667,
"outputTokens": 51,
"maxNewTokens": 512,
"pdfScale": 1.5,
"inputKind": "image",
"activationPlacement": {
"visionOutput": "mps:0"
},
"decoding": "greedy",
"outputLimitReached": false,
"allPagesReachedEos": true,
"driverAllocatedMemoryBytes": 6220185600,
"memoryMeasurementScope": "Post-inference Metal driver allocation, not peak",
"device": "mps",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3346.6235409869114,
"artifactRevision": "c5630abae1d940eafe0697512a0325494b02ab42",
"fallback": false,
"grade": "fast"
}
}
]