Granite 4.0
Local chat, drafting and text extraction using IBM Granite 4.0 Micro.
Desktop app
Run Granite 4.0 on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
Granite 4.0 Dense 1B Base
24 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 191 ms / 1453 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 191 ms / 1453 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 194 ms / 1443 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 195 ms / 1443 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 200 ms / 1453 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 207 ms / 1443 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 329 ms / 1652 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 580 ms / 1652 ms |
BF16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 797 ms / 1652 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 691 ms / 1263 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 705 ms / 1263 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 787 ms / 1263 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 837 ms / 1347 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 840 ms / 1216 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 841 ms / 1216 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 870 ms / 1216 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 914 ms / 1347 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 916 ms / 1462 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 945 ms / 1462 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1054 ms / 1347 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1155 ms / 1462 ms |
FP16 | mps | Failed | — | macos · aarch64 | 575 ms / 2381 ms |
FP16 | mps | Failed | — | macos · aarch64 | 376 ms / 2381 ms |
FP16 | mps | Failed | — | macos · aarch64 | 397 ms / 2381 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T12:43:40.917651+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 15148e99f8ca689325e2067ba282b23dec6670c7
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-1b-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-1b-base-native-geography.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"expectedText": false,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 2380.7257910011685,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 576.3418330025161,
"inferenceMs": 575.4771669962793,
"tokensPerSecond": 13.901507233998954,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323147776,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1692270592,
"cpuThreads": 4,
"totalMs": 2957.0676240036846,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"expectedText": false,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 2380.7257910011685,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 376.92220799362985,
"inferenceMs": 375.76629200339084,
"tokensPerSecond": 21.289828731971014,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323229696,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1702805504,
"cpuThreads": 4,
"totalMs": 2757.6479989947984,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "fail",
"quality": {
"expectedText": false,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 2380.7257910011685,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 398.6867089988664,
"inferenceMs": 397.0865839946782,
"tokensPerSecond": 20.146739583896938,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323328000,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1713799168,
"cpuThreads": 4,
"totalMs": 2779.412500000035,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1462.1262500004377,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 916.7027500006952,
"inferenceMs": 916.3787919969764,
"tokensPerSecond": 8.730014345450279,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6965641216,
"cpuThreads": 4,
"totalMs": 2378.829000001133,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1462.1262500004377,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1155.3825420050998,
"inferenceMs": 1155.0865829995018,
"tokensPerSecond": 6.925887736680126,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6968016896,
"cpuThreads": 4,
"totalMs": 2617.5087920055375,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1462.1262500004377,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 945.3702499959036,
"inferenceMs": 945.0021250013378,
"tokensPerSecond": 8.465589429218134,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6968918016,
"cpuThreads": 4,
"totalMs": 2407.4964999963413,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1216.2165830013691,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 869.9214169973857,
"inferenceMs": 869.663124998624,
"tokensPerSecond": 9.198964254133069,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6961905664,
"cpuThreads": 4,
"totalMs": 2086.137999998755,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1216.2165830013691,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 840.6399170053191,
"inferenceMs": 840.3959159986698,
"tokensPerSecond": 9.519322795010659,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6964297728,
"cpuThreads": 4,
"totalMs": 2056.8565000066883,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1216.2165830013691,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 841.5045000001555,
"inferenceMs": 841.2319589988329,
"tokensPerSecond": 9.509862190115745,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6965542912,
"cpuThreads": 4,
"totalMs": 2057.7210830015247,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1346.6786249991856,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1053.899457998341,
"inferenceMs": 1053.6382920035976,
"tokensPerSecond": 7.592738476491023,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6960054272,
"cpuThreads": 4,
"totalMs": 2400.5780829975265,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1346.6786249991856,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 913.8644169943291,
"inferenceMs": 913.5276250017341,
"tokensPerSecond": 8.757261172025109,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6962675712,
"cpuThreads": 4,
"totalMs": 2260.5430419935146,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1346.6786249991856,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 837.4297920017852,
"inferenceMs": 837.1525000038673,
"tokensPerSecond": 9.556203917402197,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6963068928,
"cpuThreads": 4,
"totalMs": 2184.108417000971,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1262.5136249989737,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 787.2994170029415,
"inferenceMs": 787.047125006211,
"tokensPerSecond": 10.16457559633023,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6963003392,
"cpuThreads": 1,
"totalMs": 2049.813042001915,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1262.5136249989737,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 691.472167003667,
"inferenceMs": 691.1929169946234,
"tokensPerSecond": 11.574192679497944,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6966034432,
"cpuThreads": 1,
"totalMs": 1953.9857920026407,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1262.5136249989737,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 705.4514170013135,
"inferenceMs": 705.1102089972119,
"tokensPerSecond": 11.345744109105125,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6966067200,
"cpuThreads": 1,
"totalMs": 1967.9650420002872,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1651.5233329992043,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 798.2207500026561,
"inferenceMs": 796.5757920028409,
"tokensPerSecond": 10.042986593762153,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323147776,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 516702208,
"cpuThreads": 4,
"totalMs": 2449.7440830018604,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1651.5233329992043,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 581.5407909976784,
"inferenceMs": 580.1991659973282,
"tokensPerSecond": 13.788368665178053,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323229696,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 529530880,
"cpuThreads": 4,
"totalMs": 2233.0641239968827,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1651.5233329992043,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 330.7444160018349,
"inferenceMs": 329.1062080024858,
"tokensPerSecond": 24.308262212846422,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323328000,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 539738112,
"cpuThreads": 4,
"totalMs": 1982.2677490010392,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1452.7308340038871,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 201.3932499976363,
"inferenceMs": 200.22837500437163,
"tokensPerSecond": 39.95437709478157,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323164160,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 508968960,
"cpuThreads": 4,
"totalMs": 1654.1240840015234,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1452.7308340038871,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 193.46025000413647,
"inferenceMs": 191.34533300530165,
"tokensPerSecond": 41.80922458024279,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323229696,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 519749632,
"cpuThreads": 4,
"totalMs": 1646.1910840080236,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1452.7308340038871,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 193.3153339996352,
"inferenceMs": 191.3641249993816,
"tokensPerSecond": 41.80511890630416,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323328000,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 530120704,
"cpuThreads": 4,
"totalMs": 1646.0461680035223,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1443.0119580065366,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 208.8688750009169,
"inferenceMs": 207.3262080011773,
"tokensPerSecond": 38.586535089449825,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323164160,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 511639552,
"cpuThreads": 4,
"totalMs": 1651.8808330074535,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1443.0119580065366,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 197.5936669987277,
"inferenceMs": 195.48025000403868,
"tokensPerSecond": 40.92485046358759,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323229696,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 522387456,
"cpuThreads": 4,
"totalMs": 1640.6056250052643,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "15148e99f8ca689325e2067ba282b23dec6670c7",
"torchVersion": "2.10.0",
"loadMs": 1443.0119580065366,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 322
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 195.39170800271677,
"inferenceMs": 193.92108300235122,
"tokensPerSecond": 41.25389501822761,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 4323328000,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 532725760,
"cpuThreads": 4,
"totalMs": 1638.4036660092534,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
}
]Granite 4.0 Dense 350M Base
18 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 138 ms / 1051 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 141 ms / 1051 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 141 ms / 1033 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 157 ms / 1051 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 160 ms / 1446 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 174 ms / 1033 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 224 ms / 1033 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 270 ms / 1446 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 363 ms / 1446 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 205 ms / 797 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 212 ms / 797 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 214 ms / 800 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 216 ms / 800 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 219 ms / 795 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 219 ms / 800 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 220 ms / 795 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 221 ms / 797 ms |
FP32 | cpu | Passed | Fast | macos · aarch64 | 234 ms / 795 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T12:44:37.472396+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- a50b46cef21c8a86b15f0496cb794487a78a910b
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-350m-base-native-square.json", "size": 190, "sha1": "931394b01fb0435effd4b4cd4d258fccfbbbd9a0", "sha256": "4a5c29b534e7104e5db198844851d6d783f9e861aed9d8841262e5af8182c6d8", "url": "https://assets.localai.tools/samples/v1/files/4a5c29b534e7104e5db198844851d6d783f9e861aed9d8841262e5af8182c6d8/granite-4.0-350m-base-native-square.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1445.6285419946653,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 271.85124999959953,
"inferenceMs": 269.9694999973872,
"tokensPerSecond": 29.632977058806368,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127137280,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 512196608,
"cpuThreads": 4,
"totalMs": 1717.4797919942648,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " b"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1445.6285419946653,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 161.91129199432908,
"inferenceMs": 159.7641249973094,
"tokensPerSecond": 50.073819764823476,
"inputTokens": 20,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127235584,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 522338304,
"cpuThreads": 4,
"totalMs": 1607.5398339889944,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1445.6285419946653,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 365.7944160004263,
"inferenceMs": 363.39537499588914,
"tokensPerSecond": 22.014589481471795,
"inputTokens": 15,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127317504,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 534773760,
"cpuThreads": 4,
"totalMs": 1811.4229579950916,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1050.545250000141,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 159.29970900469925,
"inferenceMs": 157.4231670019799,
"tokensPerSecond": 50.81844148072174,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127137280,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 508280832,
"cpuThreads": 4,
"totalMs": 1209.8449590048403,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " b"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1050.545250000141,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 142.95366599981207,
"inferenceMs": 140.5202919995645,
"tokensPerSecond": 56.9312793630175,
"inputTokens": 20,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127251968,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 518750208,
"cpuThreads": 4,
"totalMs": 1193.4989159999532,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1050.545250000141,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 140.79087500431342,
"inferenceMs": 138.1451669949456,
"tokensPerSecond": 57.91009684973417,
"inputTokens": 15,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127333888,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 528891904,
"cpuThreads": 4,
"totalMs": 1191.3361250044545,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1033.4577500034356,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 176.33650000061607,
"inferenceMs": 174.40383299981477,
"tokensPerSecond": 45.8705514804167,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127137280,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 504872960,
"cpuThreads": 4,
"totalMs": 1209.7942500040517,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " b"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1033.4577500034356,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 143.06125000439351,
"inferenceMs": 140.84675000049174,
"tokensPerSecond": 56.79932266787888,
"inputTokens": 20,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127251968,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 515440640,
"cpuThreads": 4,
"totalMs": 1176.5190000078292,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 1033.4577500034356,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 226
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 226.2833330023568,
"inferenceMs": 224.37091599567793,
"tokensPerSecond": 35.65524508601687,
"inputTokens": 15,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127333888,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 525664256,
"cpuThreads": 4,
"totalMs": 1259.7410830057925,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 795.2704999988782,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 220.78666699962923,
"inferenceMs": 220.4687909979839,
"tokensPerSecond": 36.28631500987891,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1827995648,
"cpuThreads": 4,
"totalMs": 1016.0571669985075,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " b"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 795.2704999988782,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 234.10525000508642,
"inferenceMs": 233.88374999922235,
"tokensPerSecond": 34.205027070185935,
"inputTokens": 20,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1829732352,
"cpuThreads": 4,
"totalMs": 1029.3757500039646,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 795.2704999988782,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 219.4556249960442,
"inferenceMs": 219.2440409999108,
"tokensPerSecond": 36.48901910179285,
"inputTokens": 15,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1834172416,
"cpuThreads": 4,
"totalMs": 1014.7261249949224,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 796.7337919981219,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 221.40679199947044,
"inferenceMs": 221.13987500051735,
"tokensPerSecond": 36.176198435407834,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1830043648,
"cpuThreads": 4,
"totalMs": 1018.1405839975923,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " b"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 796.7337919981219,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 211.8561250026687,
"inferenceMs": 211.6536250032368,
"tokensPerSecond": 37.79760445812188,
"inputTokens": 20,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1832009728,
"cpuThreads": 4,
"totalMs": 1008.5899170007906,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 796.7337919981219,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 205.2512080044835,
"inferenceMs": 205.05916700494708,
"tokensPerSecond": 39.01313029232679,
"inputTokens": 15,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1836515328,
"cpuThreads": 4,
"totalMs": 1001.9850000026054,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 799.9750829985715,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 219.75579200079665,
"inferenceMs": 219.4953330035787,
"tokensPerSecond": 36.447244187508836,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1826062336,
"cpuThreads": 4,
"totalMs": 1019.7308749993681,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " b"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 799.9750829985715,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 216.37612499762326,
"inferenceMs": 216.16479100339347,
"tokensPerSecond": 37.008802233081575,
"inputTokens": 20,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1827700736,
"cpuThreads": 4,
"totalMs": 1016.3512079961947,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "a50b46cef21c8a86b15f0496cb794487a78a910b",
"torchVersion": "2.10.0",
"loadMs": 799.9750829985715,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 226
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 213.9065000010305,
"inferenceMs": 213.70541700161994,
"tokensPerSecond": 37.43470854526518,
"inputTokens": 15,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1832271872,
"cpuThreads": 4,
"totalMs": 1013.881582999602,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
}
]Granite 4.0 Hybrid 1B Base
15 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 375 ms / 1492 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 378 ms / 1506 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 379 ms / 1492 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 381 ms / 1492 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 394 ms / 1506 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 409 ms / 1506 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 438 ms / 1873 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 493 ms / 1873 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 546 ms / 1873 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 17764 ms / 3538 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 39103 ms / 3538 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 54682 ms / 3538 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 7163 ms / 4246 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 3894 ms / 4246 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 3768 ms / 4246 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T12:40:38.588591+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 8d34631309affca6f6f8350684ddcb8994b0dc31
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-h-1b-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-1b-base-native-geography.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1873.1615420037997,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 547.3154579958646,
"inferenceMs": 545.8465839983546,
"tokensPerSecond": 14.656132756936177,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258007552,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1603469312,
"cpuThreads": 4,
"totalMs": 2420.4769999996643,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1873.1615420037997,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 494.6955839986913,
"inferenceMs": 493.1372920036665,
"tokensPerSecond": 16.22266279537529,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258138624,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1622294528,
"cpuThreads": 4,
"totalMs": 2367.857126002491,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1873.1615420037997,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 440.1028749998659,
"inferenceMs": 438.37654199887766,
"tokensPerSecond": 18.24915166199856,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258286080,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1638350848,
"cpuThreads": 4,
"totalMs": 2313.2644170036656,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1506.4057080016937,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 410.5535409980803,
"inferenceMs": 408.742166000593,
"tokensPerSecond": 19.57224056005123,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258023936,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1599504384,
"cpuThreads": 4,
"totalMs": 1916.959248999774,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1506.4057080016937,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 396.11779199913144,
"inferenceMs": 394.09499999601394,
"tokensPerSecond": 20.299673936692717,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258138624,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1617215488,
"cpuThreads": 4,
"totalMs": 1902.5235000008252,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1506.4057080016937,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 380.40933400043286,
"inferenceMs": 378.2888749992708,
"tokensPerSecond": 21.147859555783715,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258286080,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1633583104,
"cpuThreads": 4,
"totalMs": 1886.8150420021266,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1492.281416998594,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 382.5119169996469,
"inferenceMs": 381.00200000189943,
"tokensPerSecond": 20.997265106115236,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258023936,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1601404928,
"cpuThreads": 4,
"totalMs": 1874.793333998241,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1492.281416998594,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 380.5726249993313,
"inferenceMs": 378.9085829994292,
"tokensPerSecond": 21.113272063335792,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258138624,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1619640320,
"cpuThreads": 4,
"totalMs": 1872.8540419979254,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 1492.281416998594,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 377.0788330002688,
"inferenceMs": 375.4507920020842,
"tokensPerSecond": 21.30771906843012,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 3258286080,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 1637662720,
"cpuThreads": 4,
"totalMs": 1869.3602499988629,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 3537.7601660002256,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 17764.568624996173,
"inferenceMs": 17764.357082996867,
"tokensPerSecond": 0.4503399679832596,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6441222144,
"cpuThreads": 4,
"totalMs": 21302.3287909964,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 3537.7601660002256,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 39103.663957997924,
"inferenceMs": 39103.35812500125,
"tokensPerSecond": 0.20458600958072432,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6444564480,
"cpuThreads": 4,
"totalMs": 42641.42412399815,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 3537.7601660002256,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 54682.723291,
"inferenceMs": 54682.426084000326,
"tokensPerSecond": 0.1462992879597334,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6449283072,
"cpuThreads": 4,
"totalMs": 58220.48345700023,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 4246.428250000463,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 7163.7770000015735,
"inferenceMs": 7163.414041999204,
"tokensPerSecond": 1.1167859282034907,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6141591552,
"cpuThreads": 1,
"totalMs": 11410.205250002036,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius",
"fixturePassed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 4246.428250000463,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 3894.8861669996404,
"inferenceMs": 3894.0525840007467,
"tokensPerSecond": 2.0544149898923054,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6152323072,
"cpuThreads": 1,
"totalMs": 8141.314417000103,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
},
"failure": "This fixture passed but the backend did not pass the complete repeated profile."
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " x",
"fixturePassed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "8d34631309affca6f6f8350684ddcb8994b0dc31",
"torchVersion": "2.10.0",
"loadMs": 4246.428250000463,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 3768.907333003881,
"inferenceMs": 3768.443249995471,
"tokensPerSecond": 2.122892523327667,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 6159450112,
"cpuThreads": 1,
"totalMs": 8015.335583004344,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
},
"failure": "This fixture passed but the backend did not pass the complete repeated profile."
}
]Granite 4.0 Hybrid 350M Base
15 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 374 ms / 1374 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 419 ms / 1374 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 429 ms / 1252 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 465 ms / 1252 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 517 ms / 1252 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 519 ms / 1374 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 559 ms / 1304 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 694 ms / 1304 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 697 ms / 1304 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 6757 ms / 1800 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 6840 ms / 1800 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 6794 ms / 1800 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 1705 ms / 1475 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 1715 ms / 1475 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 1705 ms / 1475 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T12:28:48.153999+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- dc555b6939c863bb96034d1eae7601da36bd42e4
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-h-350m-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-350m-base-native-geography.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1304.3914590016357,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 697.7576670033159,
"inferenceMs": 696.7547079984797,
"tokensPerSecond": 11.481802574368045,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127251968,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 834699264,
"cpuThreads": 4,
"totalMs": 2002.1491260049515,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1304.3914590016357,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 695.3992080016178,
"inferenceMs": 693.9490000004298,
"tokensPerSecond": 11.528224696620422,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127366656,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 852197376,
"cpuThreads": 4,
"totalMs": 1999.7906670032535,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1304.3914590016357,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 560.810875002062,
"inferenceMs": 559.3551249985467,
"tokensPerSecond": 14.302184144680512,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127497728,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 869105664,
"cpuThreads": 4,
"totalMs": 1865.2023340036976,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1251.805959000194,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 466.8252499977825,
"inferenceMs": 465.2519589944859,
"tokensPerSecond": 17.19498401960477,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127268352,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 831782912,
"cpuThreads": 4,
"totalMs": 1718.6312089979765,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1251.805959000194,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 518.8388750029844,
"inferenceMs": 517.4960419972194,
"tokensPerSecond": 15.459055433786265,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127366656,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 849592320,
"cpuThreads": 4,
"totalMs": 1770.6448340031784,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1251.805959000194,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 431.575791000796,
"inferenceMs": 429.3407910008682,
"tokensPerSecond": 18.63321670729354,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127497728,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 866091008,
"cpuThreads": 4,
"totalMs": 1683.38175000099,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1374.3903750000754,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 519.7690419954597,
"inferenceMs": 518.6717499964288,
"tokensPerSecond": 15.424013357301765,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127251968,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 830521344,
"cpuThreads": 4,
"totalMs": 1894.1594169955351,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1374.3903750000754,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 420.21004200069,
"inferenceMs": 419.1607080065296,
"tokensPerSecond": 19.085758390968692,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127366656,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 847314944,
"cpuThreads": 4,
"totalMs": 1794.6004170007654,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1374.3903750000754,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 370
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 374.6143330063205,
"inferenceMs": 373.58783400122775,
"tokensPerSecond": 21.413973560963736,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 1127497728,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 863617024,
"cpuThreads": 4,
"totalMs": 1749.004708006396,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1799.7390419986914,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 370
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 6757.583833998069,
"inferenceMs": 6757.353875000263,
"tokensPerSecond": 1.1838953750220296,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1909538816,
"cpuThreads": 4,
"totalMs": 8557.32287599676,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1799.7390419986914,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 370
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 6839.964625003631,
"inferenceMs": 6839.769833000901,
"tokensPerSecond": 1.1696300014952485,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1909800960,
"cpuThreads": 4,
"totalMs": 8639.703667002323,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1799.7390419986914,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 370
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 6794.528875005199,
"inferenceMs": 6794.299333996605,
"tokensPerSecond": 1.177457690150688,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1911603200,
"cpuThreads": 4,
"totalMs": 8594.26791700389,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France",
"fixturePassed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1475.008749999688,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 370
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1705.6864999976824,
"inferenceMs": 1705.4741669999203,
"tokensPerSecond": 4.690777588307132,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1903198208,
"cpuThreads": 1,
"totalMs": 3180.6952499973704,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
},
"failure": "This fixture passed but the backend did not pass the complete repeated profile."
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius",
"fixturePassed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1475.008749999688,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 370
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1715.0565419942723,
"inferenceMs": 1714.8642910033232,
"tokensPerSecond": 4.665092183661604,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1907638272,
"cpuThreads": 1,
"totalMs": 3190.0652919939603,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
},
"failure": "This fixture passed but the backend did not pass the complete repeated profile."
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " x",
"fixturePassed": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "dc555b6939c863bb96034d1eae7601da36bd42e4",
"torchVersion": "2.10.0",
"loadMs": 1475.008749999688,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 370
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1705.6674159975955,
"inferenceMs": 1705.4481669983943,
"tokensPerSecond": 4.690849100433278,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 1907933184,
"cpuThreads": 1,
"totalMs": 3180.6761659972835,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
},
"failure": "This fixture passed but the backend did not pass the complete repeated profile."
}
]Granite 4.0 Hybrid Micro Base
15 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 460 ms / 2028 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 460 ms / 2635 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 462 ms / 2028 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 469 ms / 2028 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 475 ms / 2635 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 480 ms / 2635 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 484 ms / 2057 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 487 ms / 2057 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 498 ms / 2057 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 23114 ms / 6096 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 30796 ms / 6096 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 27770 ms / 6096 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 5637 ms / 7172 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 5487 ms / 7172 ms |
FP32 | cpu | Failed | — | macos · aarch64 | 5643 ms / 7172 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T12:47:48.570317+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 372ede0bc484284e08ce574e773c96cd2b89b367
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-h-micro-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-micro-base-native-geography.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2634.6227920003003,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 476.8214590003481,
"inferenceMs": 475.352458000998,
"tokensPerSecond": 16.829617403562903,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553040384,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2942648320,
"cpuThreads": 4,
"totalMs": 3111.4442510006484,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2634.6227920003003,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 461.8705000029877,
"inferenceMs": 460.3202500002226,
"tokensPerSecond": 17.379205020843926,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553204224,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2960900096,
"cpuThreads": 4,
"totalMs": 3096.493292003288,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2634.6227920003003,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 481.88800000207266,
"inferenceMs": 480.14949999924283,
"tokensPerSecond": 16.661477310738874,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7554465792,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2977841152,
"cpuThreads": 4,
"totalMs": 3116.510792002373,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2028.0584579959395,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 470.57316699647345,
"inferenceMs": 468.704582999635,
"tokensPerSecond": 17.068320409417097,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553040384,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2938142720,
"cpuThreads": 4,
"totalMs": 2498.631624992413,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2028.0584579959395,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 461.40058300079545,
"inferenceMs": 459.86562500183936,
"tokensPerSecond": 17.396386172521595,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553204224,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2956017664,
"cpuThreads": 4,
"totalMs": 2489.459040996735,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2028.0584579959395,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 463.87520900316304,
"inferenceMs": 462.3458749993006,
"tokensPerSecond": 17.303063426297687,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7554465792,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2972598272,
"cpuThreads": 4,
"totalMs": 2491.9336669991026,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2056.556499999715,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 488.31754199636634,
"inferenceMs": 487.294833001215,
"tokensPerSecond": 16.4171656627848,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553040384,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2940272640,
"cpuThreads": 4,
"totalMs": 2544.8740419960814,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2056.556499999715,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 499.15637499361765,
"inferenceMs": 497.5939579962869,
"tokensPerSecond": 16.07736563404915,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553204224,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2958376960,
"cpuThreads": 4,
"totalMs": 2555.7128749933327,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 2056.556499999715,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 466
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 485.33845800557174,
"inferenceMs": 483.688667001843,
"tokensPerSecond": 16.539564694761637,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7554465792,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2975121408,
"cpuThreads": 4,
"totalMs": 2541.8949580052868,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 6095.792624997557,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 23114.043624998885,
"inferenceMs": 23113.81395799981,
"tokensPerSecond": 0.3461133681588347,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 13419200512,
"cpuThreads": 4,
"totalMs": 29209.836249996442,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 6095.792624997557,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 30796.594999999797,
"inferenceMs": 30796.317583000928,
"tokensPerSecond": 0.2597713177375425,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 12974817280,
"cpuThreads": 4,
"totalMs": 36892.387624997355,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 6095.792624997557,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 27771.326832997147,
"inferenceMs": 27770.42245800112,
"tokensPerSecond": 0.2880762801537816,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 13113786368,
"cpuThreads": 4,
"totalMs": 33867.119457994704,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 7172.052750000148,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 5637.365375005174,
"inferenceMs": 5637.127166999562,
"tokensPerSecond": 1.4191625916890764,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 13412777984,
"cpuThreads": 1,
"totalMs": 12809.418125005323,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 7172.052750000148,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 5486.838208998961,
"inferenceMs": 5486.58629199781,
"tokensPerSecond": 1.4581015542702767,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 13417070592,
"cpuThreads": 1,
"totalMs": 12658.890958999109,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "372ede0bc484284e08ce574e773c96cd2b89b367",
"torchVersion": "2.10.0",
"loadMs": 7172.052750000148,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 466
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 5643.402708003123,
"inferenceMs": 5643.151708005462,
"tokensPerSecond": 1.4176475157758168,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 13417594880,
"cpuThreads": 1,
"totalMs": 12815.45545800327,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
}
]Granite 4.0 Hybrid Small Base
21 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 379 ms / 1452 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 381 ms / 1365 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 382 ms / 1373 ms |
Q4 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 408 ms / 1365 ms |
Q4 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 415 ms / 1373 ms |
Q4 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 415 ms / 1452 ms |
Q4 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 418 ms / 1373 ms |
Q4 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 421 ms / 1365 ms |
Q4 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 460 ms / 1452 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 738 ms / 18978 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 756 ms / 19114 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 764 ms / 18997 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 804 ms / 18978 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 809 ms / 19114 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 831 ms / 18997 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 904 ms / 19114 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 922 ms / 18978 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 956 ms / 18997 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 2276 ms / 19138 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 2580 ms / 19138 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 3015 ms / 19138 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T13:01:05.605185+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- llama.cpp 0.3.36
- Artifact revision
- d613b92b3764b8142d1a4f64017fb457505dbc66
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-h-small-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-small-base-native-geography.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1452.0784169944818,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 459.63704199675703,
"durationMs": 459.7870409997995,
"tokensPerSecond": 17.405037603684786,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19938377728,
"cpuThreads": 4,
"totalMs": 1911.8654579942813,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1452.0784169944818,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 378.67499999993015,
"durationMs": 378.8076250057202,
"tokensPerSecond": 21.126295636103453,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19940065280,
"cpuThreads": 4,
"totalMs": 1830.886042000202,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1452.0784169944818,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 414.9059170013061,
"durationMs": 415.087458997732,
"tokensPerSecond": 19.28147966126696,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19941605376,
"cpuThreads": 4,
"totalMs": 1867.1658759922138,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1364.768790997914,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 407.76229200128,
"durationMs": 408.06637499917997,
"tokensPerSecond": 19.619273672257286,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19933691904,
"cpuThreads": 4,
"totalMs": 1772.835165997094,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1364.768790997914,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 381.23891700524837,
"durationMs": 381.40541699976893,
"tokensPerSecond": 20.98421657170395,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19935248384,
"cpuThreads": 4,
"totalMs": 1746.174207997683,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1364.768790997914,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 421.3492910057539,
"durationMs": 421.6057079975144,
"tokensPerSecond": 18.9866226685801,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19936837632,
"cpuThreads": 4,
"totalMs": 1786.3744989954284,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1372.6486250016023,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 414.8267909986316,
"durationMs": 415.06904199923156,
"tokensPerSecond": 19.285157500896293,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19935756288,
"cpuThreads": 4,
"totalMs": 1787.717667000834,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1372.6486250016023,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 381.8154170003254,
"durationMs": 382.1792919989093,
"tokensPerSecond": 20.952532673643145,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19937378304,
"cpuThreads": 4,
"totalMs": 1754.8279170005117,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 1372.6486250016023,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 321.56 MiB\nload_tensors: MTL0_Mapped model buffer size = 18570.93 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 128.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 147.48 MiB\nsched_reserve: MTL0 compute buffer size = 282.28 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 418.05729199404595,
"durationMs": 418.2641669976874,
"tokensPerSecond": 19.136133140607765,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 19939016704,
"cpuThreads": 4,
"totalMs": 1790.9127919992898,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 19113.816709003004,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 755.6653749998077,
"durationMs": 756.0884159975103,
"tokensPerSecond": 10.586696525564687,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39355023360,
"cpuThreads": 4,
"totalMs": 19869.905125000514,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 19113.816709003004,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 808.9200410031481,
"durationMs": 809.127417000127,
"tokensPerSecond": 9.889729014599684,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39359135744,
"cpuThreads": 4,
"totalMs": 19922.94412600313,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 19113.816709003004,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 904.219625001133,
"durationMs": 904.4215829999303,
"tokensPerSecond": 8.847408061940678,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39364132864,
"cpuThreads": 4,
"totalMs": 20018.238292002934,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 18977.763041002618,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 737.6848749991041,
"durationMs": 737.9867909985478,
"tokensPerSecond": 10.844739089993835,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39353843712,
"cpuThreads": 4,
"totalMs": 19715.749832001165,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 18977.763041002618,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 803.7167500006035,
"durationMs": 803.8210840022657,
"tokensPerSecond": 9.953755474169219,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39356825600,
"cpuThreads": 4,
"totalMs": 19781.584125004883,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 18977.763041002618,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 922.4196249997476,
"durationMs": 922.5294590069097,
"tokensPerSecond": 8.672842362826126,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39361822720,
"cpuThreads": 4,
"totalMs": 19900.292500009527,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 18997.33595900034,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 763.7237500021001,
"durationMs": 763.8918750017183,
"tokensPerSecond": 10.474991775465934,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39352565760,
"cpuThreads": 4,
"totalMs": 19761.227834002057,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 18997.33595900034,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 830.6357090041274,
"durationMs": 830.7312500037369,
"tokensPerSecond": 9.631177558681442,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39356645376,
"cpuThreads": 4,
"totalMs": 19828.067209004075,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 18997.33595900034,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 955.6527500026277,
"durationMs": 955.8113329985645,
"tokensPerSecond": 8.371241541426008,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39361429504,
"cpuThreads": 4,
"totalMs": 19953.147291998903,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 19138.133375003235,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 2276.01987500384,
"durationMs": 2276.3410420011496,
"tokensPerSecond": 3.514907794900738,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39353434112,
"cpuThreads": 1,
"totalMs": 21414.474417004385,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 19138.133375003235,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 2579.533416996128,
"durationMs": 2580.1095420029014,
"tokensPerSecond": 3.101336058408585,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39357415424,
"cpuThreads": 1,
"totalMs": 21718.242917006137,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "d613b92b3764b8142d1a4f64017fb457505dbc66",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 19138.133375003235,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 18552.93 MiB\nload_tensors: CPU_REPACK model buffer size = 18517.69 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 3014.7489169976325,
"durationMs": 3014.860041999782,
"tokensPerSecond": 2.653620656398525,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 39362150400,
"cpuThreads": 1,
"totalMs": 22152.993417003017,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
}
]Granite 4.0 Hybrid Tiny Base
21 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 114 ms / 385 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 115 ms / 385 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 119 ms / 386 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 119 ms / 386 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 122 ms / 385 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 124 ms / 451 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 125 ms / 386 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 126 ms / 451 ms |
Q4 | mpsAccelerated | Passed | Fast | macos · aarch64 | 268 ms / 451 ms |
Q4 | cpu | Passed | Fast | macos · aarch64 | 206 ms / 4300 ms |
Q4 | cpu | Passed | Fast | macos · aarch64 | 210 ms / 4300 ms |
Q4 | cpu | Passed | Fast | macos · aarch64 | 222 ms / 4300 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 569 ms / 6245 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 658 ms / 6245 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 779 ms / 6857 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 810 ms / 6857 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 1210 ms / 6857 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 2132 ms / 6245 ms |
Q4 | cpu | Passed | Moderate | macos · aarch64 | 3805 ms / 120087 ms |
Q4 | cpu | Failed | — | macos · aarch64 | 12897 ms / 120087 ms |
Q4 | cpu | Failed | — | macos · aarch64 | 4405 ms / 120087 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T12:53:09.537151+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- llama.cpp 0.3.36
- Artifact revision
- 27f75056a8aabab6377df1d9dd4f37c9b1fbe14e
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-h-tiny-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-h-tiny-base-native-geography.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 450.72349999827566,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 268.0246250019991,
"durationMs": 268.3380840026075,
"tokensPerSecond": 29.848003704660837,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4526260224,
"cpuThreads": 4,
"totalMs": 719.0615840008832,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 450.72349999827566,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 125.9983329946408,
"durationMs": 126.19366600119974,
"tokensPerSecond": 63.49290351595581,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4527570944,
"cpuThreads": 4,
"totalMs": 576.9171659994754,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 450.72349999827566,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 123.9338330051396,
"durationMs": 124.15400000463706,
"tokensPerSecond": 64.55057352795855,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4528963584,
"cpuThreads": 4,
"totalMs": 574.8775000029127,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 385.37112500489457,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 114.76595799467759,
"durationMs": 114.93583300034516,
"tokensPerSecond": 69.70708161013225,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4525637632,
"cpuThreads": 4,
"totalMs": 500.3069580052397,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 385.37112500489457,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 114.07291599607561,
"durationMs": 114.18383399723098,
"tokensPerSecond": 70.13058209430905,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4526866432,
"cpuThreads": 4,
"totalMs": 499.55495900212554,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 385.37112500489457,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 121.5497500015772,
"durationMs": 121.70024999795714,
"tokensPerSecond": 65.81667177346061,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4528324608,
"cpuThreads": 4,
"totalMs": 507.0713750028517,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 386.3765839996631,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 118.87858400586993,
"durationMs": 119.17083300068043,
"tokensPerSecond": 67.29555257492788,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4528193536,
"cpuThreads": 4,
"totalMs": 505.54741700034356,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 386.3765839996631,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 119.46229099703487,
"durationMs": 119.69924999721115,
"tokensPerSecond": 66.96673848485432,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4529717248,
"cpuThreads": 4,
"totalMs": 506.0758339968743,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"loadMs": 386.3765839996631,
"acceleration": "partial",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: layer 0 assigned to device MTL0, is_swa = 0\nload_tensors: layer 1 assigned to device MTL0, is_swa = 0\nload_tensors: layer 2 assigned to device MTL0, is_swa = 0\nload_tensors: layer 3 assigned to device MTL0, is_swa = 0\nload_tensors: layer 4 assigned to device MTL0, is_swa = 0\nload_tensors: layer 5 assigned to device MTL0, is_swa = 0\nload_tensors: layer 6 assigned to device MTL0, is_swa = 0\nload_tensors: layer 7 assigned to device MTL0, is_swa = 0\nload_tensors: layer 8 assigned to device MTL0, is_swa = 0\nload_tensors: layer 9 assigned to device MTL0, is_swa = 0\nload_tensors: layer 10 assigned to device MTL0, is_swa = 0\nload_tensors: layer 11 assigned to device MTL0, is_swa = 0\nload_tensors: layer 12 assigned to device MTL0, is_swa = 0\nload_tensors: layer 13 assigned to device MTL0, is_swa = 0\nload_tensors: layer 14 assigned to device MTL0, is_swa = 0\nload_tensors: layer 15 assigned to device MTL0, is_swa = 0\nload_tensors: layer 16 assigned to device MTL0, is_swa = 0\nload_tensors: layer 17 assigned to device MTL0, is_swa = 0\nload_tensors: layer 18 assigned to device MTL0, is_swa = 0\nload_tensors: layer 19 assigned to device MTL0, is_swa = 0\nload_tensors: layer 20 assigned to device MTL0, is_swa = 0\nload_tensors: layer 21 assigned to device MTL0, is_swa = 0\nload_tensors: layer 22 assigned to device MTL0, is_swa = 0\nload_tensors: layer 23 assigned to device MTL0, is_swa = 0\nload_tensors: layer 24 assigned to device MTL0, is_swa = 0\nload_tensors: layer 25 assigned to device MTL0, is_swa = 0\nload_tensors: layer 26 assigned to device MTL0, is_swa = 0\nload_tensors: layer 27 assigned to device MTL0, is_swa = 0\nload_tensors: layer 28 assigned to device MTL0, is_swa = 0\nload_tensors: layer 29 assigned to device MTL0, is_swa = 0\nload_tensors: layer 30 assigned to device MTL0, is_swa = 0\nload_tensors: layer 31 assigned to device MTL0, is_swa = 0\nload_tensors: layer 32 assigned to device MTL0, is_swa = 0\nload_tensors: layer 33 assigned to device MTL0, is_swa = 0\nload_tensors: layer 34 assigned to device MTL0, is_swa = 0\nload_tensors: layer 35 assigned to device MTL0, is_swa = 0\nload_tensors: layer 36 assigned to device MTL0, is_swa = 0\nload_tensors: layer 37 assigned to device MTL0, is_swa = 0\nload_tensors: layer 38 assigned to device MTL0, is_swa = 0\nload_tensors: layer 39 assigned to device MTL0, is_swa = 0\nload_tensors: layer 40 assigned to device MTL0, is_swa = 0\nload_tensors: offloaded 41/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 120.59 MiB\nload_tensors: MTL0_Mapped model buffer size = 4031.55 MiB\nllama_kv_cache: layer 5: dev = MTL0\nllama_kv_cache: layer 15: dev = MTL0\nllama_kv_cache: layer 25: dev = MTL0\nllama_kv_cache: layer 35: dev = MTL0\nllama_kv_cache: MTL0 KV buffer size = 64.00 MiB\nllama_memory_recurrent, layer 0: dev = MTL0\nllama_memory_recurrent, layer 1: dev = MTL0\nllama_memory_recurrent, layer 2: dev = MTL0\nllama_memory_recurrent, layer 3: dev = MTL0\nllama_memory_recurrent, layer 4: dev = MTL0\nllama_memory_recurrent, layer 6: dev = MTL0\nllama_memory_recurrent, layer 7: dev = MTL0\nllama_memory_recurrent, layer 8: dev = MTL0\nllama_memory_recurrent, layer 9: dev = MTL0\nllama_memory_recurrent, layer 10: dev = MTL0\nllama_memory_recurrent, layer 11: dev = MTL0\nllama_memory_recurrent, layer 12: dev = MTL0\nllama_memory_recurrent, layer 13: dev = MTL0\nllama_memory_recurrent, layer 14: dev = MTL0\nllama_memory_recurrent, layer 16: dev = MTL0\nllama_memory_recurrent, layer 17: dev = MTL0\nllama_memory_recurrent, layer 18: dev = MTL0\nllama_memory_recurrent, layer 19: dev = MTL0\nllama_memory_recurrent, layer 20: dev = MTL0\nllama_memory_recurrent, layer 21: dev = MTL0\nllama_memory_recurrent, layer 22: dev = MTL0\nllama_memory_recurrent, layer 23: dev = MTL0\nllama_memory_recurrent, layer 24: dev = MTL0\nllama_memory_recurrent, layer 26: dev = MTL0\nllama_memory_recurrent, layer 27: dev = MTL0\nllama_memory_recurrent, layer 28: dev = MTL0\nllama_memory_recurrent, layer 29: dev = MTL0\nllama_memory_recurrent, layer 30: dev = MTL0\nllama_memory_recurrent, layer 31: dev = MTL0\nllama_memory_recurrent, layer 32: dev = MTL0\nllama_memory_recurrent, layer 33: dev = MTL0\nllama_memory_recurrent, layer 34: dev = MTL0\nllama_memory_recurrent, layer 36: dev = MTL0\nllama_memory_recurrent, layer 37: dev = MTL0\nllama_memory_recurrent, layer 38: dev = MTL0\nllama_memory_recurrent, layer 39: dev = MTL0\nllama_memory_recurrent: MTL0 RS buffer size = 55.37 MiB\nsched_reserve: MTL0 compute buffer size = 123.63 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 41,
"totalLayers": 41,
"inferenceMs": 124.71774999721674,
"durationMs": 124.92416700115427,
"tokensPerSecond": 64.14483904799863,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4531126272,
"cpuThreads": 4,
"totalMs": 511.3007510008174,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 4299.669874999381,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 210.0615840026876,
"durationMs": 210.47516699763946,
"tokensPerSecond": 38.084069669291104,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 8732213248,
"cpuThreads": 4,
"totalMs": 4510.145041997021,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 4299.669874999381,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 206.4705419979873,
"durationMs": 206.67079099803232,
"tokensPerSecond": 38.74644742337134,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 8734425088,
"cpuThreads": 4,
"totalMs": 4506.340665997413,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 4299.669874999381,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 222.43145799438935,
"durationMs": 222.71387500222772,
"tokensPerSecond": 35.96613569022145,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 8737112064,
"cpuThreads": 4,
"totalMs": 4522.383750001609,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 6856.6297910001595,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 1210.2847919959459,
"durationMs": 1211.5372080006637,
"tokensPerSecond": 6.610014479985962,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4040474624,
"cpuThreads": 4,
"totalMs": 8068.166999000823,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 6856.6297910001595,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 810.4994159948546,
"durationMs": 811.4228330014157,
"tokensPerSecond": 9.870457451447178,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 4562386944,
"cpuThreads": 4,
"totalMs": 7668.052624001575,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 6856.6297910001595,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 778.7719580010162,
"durationMs": 780.087041996012,
"tokensPerSecond": 10.27258354362775,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 5371920384,
"cpuThreads": 4,
"totalMs": 7636.716832996171,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 6245.359791995725,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 568.5241249957471,
"durationMs": 568.9494170001126,
"tokensPerSecond": 14.071522470677643,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 6858375168,
"cpuThreads": 4,
"totalMs": 6814.309208995837,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 6245.359791995725,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 658.1361670032493,
"durationMs": 658.3746670003165,
"tokensPerSecond": 12.155539235029615,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 7467794432,
"cpuThreads": 4,
"totalMs": 6903.734458996041,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 6245.359791995725,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 2131.517124995298,
"durationMs": 2131.909582996741,
"tokensPerSecond": 3.7531952740082475,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 7499988992,
"cpuThreads": 4,
"totalMs": 8377.269374992466,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": false,
"expected": " France"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 120087.31554199767,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 12897.468124996522,
"durationMs": 12919.396457997209,
"tokensPerSecond": 0.6202767800988194,
"inputTokens": 8,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 2466267136,
"cpuThreads": 1,
"totalMs": 133006.71199999488,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "fail",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": false,
"expected": " Celsius"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 120087.31554199767,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 4404.857874993468,
"durationMs": 4412.677291002183,
"tokensPerSecond": 1.8161766456566693,
"inputTokens": 11,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 3198730240,
"cpuThreads": 1,
"totalMs": 124499.99283299985,
"grade": "slow",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"artifactRevision": "27f75056a8aabab6377df1d9dd4f37c9b1fbe14e",
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"loadMs": 120087.31554199767,
"acceleration": "cpu",
"placement": "ggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nllama_prepare_model_devices: using device MTL0 (Apple M3 Max) (unknown id) - 79625 MiB free\nload_tensors: offloaded 0/41 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 4029.02 MiB\nload_tensors: CPU_REPACK model buffer size = 4013.35 MiB",
"parameterDtype": "Q4_K_M mixed tensor types",
"offloadedLayers": 0,
"totalLayers": 41,
"inferenceMs": 3804.8443329971633,
"durationMs": 3814.379124996776,
"tokensPerSecond": 2.1025827339690966,
"inputTokens": 16,
"outputTokens": 8,
"outputLimitReached": true,
"rssBytes": 3765305344,
"cpuThreads": 1,
"totalMs": 123901.69466699444,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
}
]Granite 4.0 Dense Micro Base
21 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 298 ms / 1870 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 299 ms / 1873 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 306 ms / 1873 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 306 ms / 1870 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 310 ms / 1870 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 314 ms / 3088 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 316 ms / 1873 ms |
FP16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 390 ms / 3088 ms |
FP16 | mpsAccelerated | Passed | Moderate | macos · aarch64 | 450 ms / 3088 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1390 ms / 1822 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1451 ms / 2097 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1475 ms / 2097 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1501 ms / 1716 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1559 ms / 1939 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1577 ms / 1822 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1601 ms / 2097 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1609 ms / 1939 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1622 ms / 1716 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1663 ms / 1939 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1724 ms / 1822 ms |
FP32 | cpu | Passed | Moderate | macos · aarch64 | 1779 ms / 1716 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-07T12:42:20.217541+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 98e1d641451818f6c8e6b17c1c359b5b4ac495d0
- Fixture
{ "file": "model-samples/desktop/granite4/granite-4.0-micro-base-native-geography.json", "size": 152, "sha1": "f033f82b9e5c7821b37f56df58bd3f18c26c24ef", "sha256": "48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21", "url": "https://assets.localai.tools/samples/v1/files/48f3547630afad495ce68672989f85e41ccbc27ef56df0d2805f36097901dd21/granite-4.0-micro-base-native-geography.json" }- Rerun when
- Artifact/runtime revision; Input/generation processing; Fixture/quality gates; OS/backend behavior
Per-case quality & failure details
[
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 3087.7746249971096,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 451.4157500016154,
"inferenceMs": 450.1381250011036,
"tokensPerSecond": 17.77232266202821,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7544389632,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2111815680,
"cpuThreads": 4,
"totalMs": 3539.190374998725,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 3087.7746249971096,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 392.0737910011667,
"inferenceMs": 390.0102909974521,
"tokensPerSecond": 20.512279251760212,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7544487936,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2121678848,
"cpuThreads": 4,
"totalMs": 3479.8484159982763,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 3087.7746249971096,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 315.25812500331085,
"inferenceMs": 313.62633299431764,
"tokensPerSecond": 25.508062169463766,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553024000,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2131476480,
"cpuThreads": 4,
"totalMs": 3403.0327500004205,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1869.5750420010881,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 310.65529100305866,
"inferenceMs": 309.6002500024042,
"tokensPerSecond": 25.839772416003786,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7544389632,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2105032704,
"cpuThreads": 4,
"totalMs": 2180.230333004147,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1869.5750420010881,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 299.4262920037727,
"inferenceMs": 297.85954100225354,
"tokensPerSecond": 26.85829694453022,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7544487936,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2116059136,
"cpuThreads": 4,
"totalMs": 2169.001334004861,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1869.5750420010881,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 307.621833999292,
"inferenceMs": 305.8280829936848,
"tokensPerSecond": 26.158487218340888,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553024000,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2126413824,
"cpuThreads": 4,
"totalMs": 2177.19687600038,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1873.33916699572,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 308.6056250031106,
"inferenceMs": 305.5762500007404,
"tokensPerSecond": 26.180045078701685,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7544389632,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2107883520,
"cpuThreads": 4,
"totalMs": 2181.9447919988306,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1873.33916699572,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 300.56641699775355,
"inferenceMs": 299.0010830035317,
"tokensPerSecond": 26.7557559311767,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7544487936,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2118598656,
"cpuThreads": 4,
"totalMs": 2173.9055839934736,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp16",
"device": "mps",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1873.33916699572,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float16": 322
},
"parameterDtype": "torch.float16",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 318.0984159989748,
"inferenceMs": 316.37737499841023,
"tokensPerSecond": 25.286258222605834,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"driverAllocatedMemoryBytes": 7553024000,
"memoryMeasurementScope": "Post-inference driver allocation; not peak",
"rssBytes": 2128871424,
"cpuThreads": 4,
"totalMs": 2191.437582994695,
"grade": "fast",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 2096.5990420008893,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1451.0336670064135,
"inferenceMs": 1450.7435410050675,
"tokensPerSecond": 5.514413660223944,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14042464256,
"cpuThreads": 4,
"totalMs": 3547.6327090073028,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 2096.5990420008893,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1475.6915000034496,
"inferenceMs": 1475.4095419993973,
"tokensPerSecond": 5.422223302933788,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14044594176,
"cpuThreads": 4,
"totalMs": 3572.290542004339,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 2096.5990420008893,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1601.0853330008104,
"inferenceMs": 1600.7957500041812,
"tokensPerSecond": 4.997514517376189,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14047526912,
"cpuThreads": 4,
"totalMs": 3697.6843750016997,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1939.3144589994336,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1558.9824580019922,
"inferenceMs": 1558.7354169983882,
"tokensPerSecond": 5.132365578377226,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14041481216,
"cpuThreads": 4,
"totalMs": 3498.2969170014258,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1939.3144589994336,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1663.5973749944242,
"inferenceMs": 1663.3482499964884,
"tokensPerSecond": 4.809576106517014,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14044037120,
"cpuThreads": 4,
"totalMs": 3602.9118339938577,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1939.3144589994336,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1609.4265830033692,
"inferenceMs": 1609.0771660019527,
"tokensPerSecond": 4.9717938760373235,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14048477184,
"cpuThreads": 4,
"totalMs": 3548.7410420028027,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1716.4740000007441,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1779.475708004611,
"inferenceMs": 1779.2273339946405,
"tokensPerSecond": 4.496333800155129,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14041350144,
"cpuThreads": 4,
"totalMs": 3495.949708005355,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1716.4740000007441,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1621.980750001967,
"inferenceMs": 1621.699333001743,
"tokensPerSecond": 4.933096929374763,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14042857472,
"cpuThreads": 4,
"totalMs": 3338.454750002711,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1716.4740000007441,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1501.2385829977575,
"inferenceMs": 1500.9189170013997,
"tokensPerSecond": 5.330068073219268,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14046232576,
"cpuThreads": 4,
"totalMs": 3217.7125829985016,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"plainParity": true,
"passed": true,
"expected": " France"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1821.595375004108,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1576.9694170012372,
"inferenceMs": 1576.7008750044624,
"tokensPerSecond": 5.073885685499704,
"inputTokens": 8,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14037549056,
"cpuThreads": 1,
"totalMs": 3398.564792005345,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " Celsius"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1821.595375004108,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1723.7836250060354,
"inferenceMs": 1723.5318749953876,
"tokensPerSecond": 4.641631591537237,
"inputTokens": 11,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14040924160,
"cpuThreads": 1,
"totalMs": 3545.3790000101435,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"expectedText": true,
"streamParity": true,
"withinTokenBudget": true,
"passed": true,
"expected": " x"
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"artifactRevision": "98e1d641451818f6c8e6b17c1c359b5b4ac495d0",
"torchVersion": "2.10.0",
"loadMs": 1821.595375004108,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 322
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; tokenization/decoding on CPU. MPS operator CPU fallback disabled.",
"durationMs": 1390.4377500002738,
"inferenceMs": 1390.0973749987315,
"tokensPerSecond": 5.754992523460667,
"inputTokens": 16,
"outputTokens": 8,
"maxNewTokens": 8,
"decoding": "greedy",
"outputLimitReached": true,
"rssBytes": 14045429760,
"cpuThreads": 1,
"totalMs": 3212.033125004382,
"grade": "acceptable",
"loadTimingScope": "Local snapshot load with one-token warmup. Excludes downloads."
}
}
]Release validation still required
- Verified public B2 backup and separate model/runtime source-outage recovery pending.