TranslateGemma
Translate text or text visible in still images locally. Supply source and target language codes; review OCR, names, numbers and omissions.
Desktop app
Run TranslateGemma on your computer
Download the desktop app to load this model, provide input and view results locally.
Public desktop downloads are coming soon.Device requirements
If your device falls short
MemoryAdvisory only: close other apps, reduce image/tile/token/batch settings or choose a smaller verified model/precision. CPU can need more RAM than GPU.
GPUIf the selected GPU runtime fails, existing CPU fallback may recover when enabled, but CPU has its own memory/runtime needs.
PlatformUse a compatible OS/architecture/runtime or another model. These estimates do not add hardware detection or block runs.
Benchmark evidence
Saved desktop measurements apply to the recorded hardware and runtime. Fallback cases report the device that actually ran; they do not prove GPU support on the requested device.
TranslateGemma 4B — Text & Image Translation
48 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
FP32 | mpsAccelerated | Passed | Fast | macos · aarch64 | 864 ms / 5030 ms |
BF16 | mpsAccelerated | Passed | Fast | macos · aarch64 | 955 ms / 7566 ms |
BF16 | mpsAccelerated | Passed | Slow | macos · aarch64 | 4108 ms / 7566 ms |
FP32 | mpsAccelerated | Passed | Slow | macos · aarch64 | 5165 ms / 5030 ms |
BF16 | mpsAccelerated | Passed | Very slow | macos · aarch64 | 4124 ms / 7566 ms |
FP32 | mpsAccelerated | Passed | Very slow | macos · aarch64 | 5799 ms / 5030 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 931 ms / 3917 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1052 ms / 3917 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 502 ms / 3917 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 838 ms / 3917 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1040 ms / 3917 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 651 ms / 3917 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 945 ms / 4002 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 945 ms / 4002 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 507 ms / 4002 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 730 ms / 4002 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 951 ms / 4002 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2097 ms / 3963 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1377 ms / 3963 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 825 ms / 3963 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1028 ms / 3963 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1346 ms / 3963 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 857 ms / 3963 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1190 ms / 3916 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1285 ms / 3916 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 674 ms / 3916 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 977 ms / 3916 ms |
FP32 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1335 ms / 3916 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 1285 ms / 6427 ms |
FP32 | cpu | Passed | Slow | macos · aarch64 | 7729 ms / 6427 ms |
FP32 | cpu | Passed | Very slow | macos · aarch64 | 6934 ms / 6427 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2156 ms / 2805 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 3008 ms / 2805 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 1595 ms / 2805 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2218 ms / 2805 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 3148 ms / 2805 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2098 ms / 2805 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2369 ms / 2982 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 3342 ms / 2982 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 1672 ms / 2982 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2244 ms / 2982 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 3262 ms / 2982 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2222 ms / 2982 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2154 ms / 2950 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 3063 ms / 2950 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 1713 ms / 2950 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 2299 ms / 2950 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 3097 ms / 2950 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-01T14:34:47.539510+00:00
- Environment
- { "os": "macos", "arch": "aarch64", "hardware": "Apple M3 Max", "osVersion": "27.2" }
- Runtime
- transformers 5.22.1
- Artifact revision
- 10042cb0e6e7fdce748996a71dc3dc432a4e0c89
- Fixture
{ "file": "model-samples/desktop/translategemma-translations.json", "size": 1366, "sha1": "7a7953bc58138b42b981439fff2c2b724f4cc3a3", "sha256": "fb5f28fdcd81bb7d0bed06aaa029cdde7870a8b4ae2fa999a7c019cebce06f57", "url": "https://assets.localai.tools/samples/v1/files/fb5f28fdcd81bb7d0bed06aaa029cdde7870a8b4ae2fa999a7c019cebce06f57/translategemma-translations.json" }- Rerun when
- model revision/checksums; runtime/processor/template/backend/driver; fixture or reference checksums; language codes and generation budgets; quality gates
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2805.3540000000794,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2198.0641250011104,
"inferenceMs": 2156.2477919997036,
"tokensPerSecond": 5.101454499252427,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5004.863250000199,
"performance": {
"metric": "tokensPerSecond",
"value": 5.101454499252427,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9420387649932294,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2805.3540000000794,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3008.9539579985285,
"inferenceMs": 3007.989333000296,
"tokensPerSecond": 5.651615786497314,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5815.731374999814,
"performance": {
"metric": "tokensPerSecond",
"value": 5.651615786497314,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2805.3540000000794,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1595.5337079994933,
"inferenceMs": 1594.6597500005737,
"tokensPerSecond": 5.016744167523587,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4402.23933299967,
"performance": {
"metric": "tokensPerSecond",
"value": 5.016744167523587,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2805.3540000000794,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2218.7603330003185,
"inferenceMs": 2217.9179590002605,
"tokensPerSecond": 5.410479657872048,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5025.471625000137,
"performance": {
"metric": "tokensPerSecond",
"value": 5.410479657872048,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2805.3540000000794,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3149.1119999991497,
"inferenceMs": 3148.2323330001236,
"tokensPerSecond": 5.7174941669082,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5955.8498749993305,
"performance": {
"metric": "tokensPerSecond",
"value": 5.7174941669082,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2805.3540000000794,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2099.250833998667,
"inferenceMs": 2098.437082999226,
"tokensPerSecond": 5.241996574077917,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4905.948583000281,
"performance": {
"metric": "tokensPerSecond",
"value": 5.241996574077917,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2981.542374998753,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2412.05541599993,
"inferenceMs": 2369.103625000207,
"tokensPerSecond": 4.643106314101832,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5395.135166998443,
"performance": {
"metric": "tokensPerSecond",
"value": 4.643106314101832,
"grade": "C"
},
"grade": "C"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9420387649932294,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2981.542374998753,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3342.61641699959,
"inferenceMs": 3341.629250000551,
"tokensPerSecond": 5.087338758480522,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 6325.648832998922,
"performance": {
"metric": "tokensPerSecond",
"value": 5.087338758480522,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2981.542374998753,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1673.2737499987707,
"inferenceMs": 1672.3838340003567,
"tokensPerSecond": 4.783590846405116,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4656.270999998014,
"performance": {
"metric": "tokensPerSecond",
"value": 4.783590846405116,
"grade": "C"
},
"grade": "C"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2981.542374998753,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2244.5151659994735,
"inferenceMs": 2243.5423329989135,
"tokensPerSecond": 5.348684454712187,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5227.513332998569,
"performance": {
"metric": "tokensPerSecond",
"value": 5.348684454712187,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2981.542374998753,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3263.1425839990698,
"inferenceMs": 3262.1458749999874,
"tokensPerSecond": 5.5178403081070275,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 6246.278833999895,
"performance": {
"metric": "tokensPerSecond",
"value": 5.5178403081070275,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2981.542374998753,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2222.4363340010314,
"inferenceMs": 2221.6335410012107,
"tokensPerSecond": 4.951311634880473,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5205.436541999006,
"performance": {
"metric": "tokensPerSecond",
"value": 4.951311634880473,
"grade": "C"
},
"grade": "C"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2949.97208300083,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2197.796582999217,
"inferenceMs": 2154.196708001109,
"tokensPerSecond": 5.106311767696907,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5149.17429200068,
"performance": {
"metric": "tokensPerSecond",
"value": 5.106311767696907,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9420387649932294,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2949.97208300083,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3063.5832079988177,
"inferenceMs": 3062.6570000003994,
"tokensPerSecond": 5.550735847990089,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 6015.008708000096,
"performance": {
"metric": "tokensPerSecond",
"value": 5.550735847990089,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2949.97208300083,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1713.566249998621,
"inferenceMs": 1712.6290409996727,
"tokensPerSecond": 4.671180861986521,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4664.965917001609,
"performance": {
"metric": "tokensPerSecond",
"value": 4.671180861986521,
"grade": "C"
},
"grade": "C"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2949.97208300083,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2300.364292001177,
"inferenceMs": 2299.487542000861,
"tokensPerSecond": 5.218554039026625,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5251.81883300138,
"performance": {
"metric": "tokensPerSecond",
"value": 5.218554039026625,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 2949.97208300083,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3098.0782919996273,
"inferenceMs": 3097.1428329994524,
"tokensPerSecond": 5.8118081633864325,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 6049.454458001492,
"performance": {
"metric": "tokensPerSecond",
"value": 5.8118081633864325,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3917.3670419986593,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 985.17237499982,
"inferenceMs": 930.6181669999205,
"tokensPerSecond": 11.82010021947158,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8763654144,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4904.1318759973365,
"performance": {
"metric": "tokensPerSecond",
"value": 11.82010021947158,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9420387649932294,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3917.3670419986593,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1055.1266250004119,
"inferenceMs": 1051.6604589993221,
"tokensPerSecond": 16.16491316615238,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8764719104,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4973.922957999093,
"performance": {
"metric": "tokensPerSecond",
"value": 16.16491316615238,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3917.3670419986593,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 504.15924999833805,
"inferenceMs": 501.79516699972737,
"tokensPerSecond": 15.94276016613039,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8764719104,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4423.161249998884,
"performance": {
"metric": "tokensPerSecond",
"value": 15.94276016613039,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3917.3670419986593,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 840.723582999999,
"inferenceMs": 837.8290000000561,
"tokensPerSecond": 14.32273172687887,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8765308928,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4759.513708999293,
"performance": {
"metric": "tokensPerSecond",
"value": 14.32273172687887,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3917.3670419986593,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1042.914292000205,
"inferenceMs": 1040.3773749985703,
"tokensPerSecond": 17.301414306539236,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8765898752,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4961.727416997746,
"performance": {
"metric": "tokensPerSecond",
"value": 17.301414306539236,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3917.3670419986593,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 654.4943330009119,
"inferenceMs": 650.5033749999711,
"tokensPerSecond": 16.909981443217706,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8765898752,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4573.566916998971,
"performance": {
"metric": "tokensPerSecond",
"value": 16.909981443217706,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 4002.4391669994657,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 990.976375000173,
"inferenceMs": 945.4658329996164,
"tokensPerSecond": 11.634476483513987,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8763654144,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4995.005624999976,
"performance": {
"metric": "tokensPerSecond",
"value": 11.634476483513987,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9420387649932294,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 4002.4391669994657,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 947.0597080016887,
"inferenceMs": 944.5086249997985,
"tokensPerSecond": 17.998776877239873,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8764719104,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4950.873709000007,
"performance": {
"metric": "tokensPerSecond",
"value": 17.998776877239873,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 4002.4391669994657,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 508.7640410001768,
"inferenceMs": 506.5085830010503,
"tokensPerSecond": 15.794401651794736,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8764719104,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4512.937832998432,
"performance": {
"metric": "tokensPerSecond",
"value": 15.794401651794736,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 4002.4391669994657,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 732.7146250008809,
"inferenceMs": 730.4523750008229,
"tokensPerSecond": 16.42817575887337,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8765308928,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4736.885791999157,
"performance": {
"metric": "tokensPerSecond",
"value": 16.42817575887337,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 4002.4391669994657,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 883
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 952.740000000631,
"inferenceMs": 950.6874999988213,
"tokensPerSecond": 18.933666425636517,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 8765898752,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4956.709250000131,
"performance": {
"metric": "tokensPerSecond",
"value": 18.933666425636517,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3962.649416998829,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2142.499666999356,
"inferenceMs": 2097.339500000089,
"tokensPerSecond": 5.244739823952933,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17907236864,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 6106.601916999352,
"performance": {
"metric": "tokensPerSecond",
"value": 5.244739823952933,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9420387649932294,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3962.649416998829,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1381.1138750006648,
"inferenceMs": 1377.2402079994208,
"tokensPerSecond": 12.343525770783442,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17908301824,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5345.39174999918,
"performance": {
"metric": "tokensPerSecond",
"value": 12.343525770783442,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3962.649416998829,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 831.0627080008999,
"inferenceMs": 824.9062499999127,
"tokensPerSecond": 9.698071750578743,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17908301824,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4796.62720899978,
"performance": {
"metric": "tokensPerSecond",
"value": 9.698071750578743,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3962.649416998829,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1033.1501670007128,
"inferenceMs": 1028.0356669991306,
"tokensPerSecond": 11.672746758902235,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17908891648,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4998.165041997709,
"performance": {
"metric": "tokensPerSecond",
"value": 11.672746758902235,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3962.649416998829,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1349.3314169991208,
"inferenceMs": 1345.7292500006588,
"tokensPerSecond": 13.375647441705818,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17909481472,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5314.326666999477,
"performance": {
"metric": "tokensPerSecond",
"value": 13.375647441705818,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3962.649416998829,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 860.4417919996195,
"inferenceMs": 857.0996250000462,
"tokensPerSecond": 12.833980647231536,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17909481472,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4825.681666998207,
"performance": {
"metric": "tokensPerSecond",
"value": 12.833980647231536,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3916.1630840008,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1237.134375000096,
"inferenceMs": 1190.4909589993622,
"tokensPerSecond": 9.23988537405255,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17907236864,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5155.634084001576,
"performance": {
"metric": "tokensPerSecond",
"value": 9.23988537405255,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9420387649932294,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3916.1630840008,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1287.6327499998297,
"inferenceMs": 1285.1419999988138,
"tokensPerSecond": 13.228110201063922,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17908301824,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5206.154417999642,
"performance": {
"metric": "tokensPerSecond",
"value": 13.228110201063922,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3916.1630840008,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 677.26358300024,
"inferenceMs": 673.5777499998221,
"tokensPerSecond": 11.876876871307154,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17908301824,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4595.846209000229,
"performance": {
"metric": "tokensPerSecond",
"value": 11.876876871307154,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3916.1630840008,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 980.3995000002033,
"inferenceMs": 976.9319999995787,
"tokensPerSecond": 12.283352372534807,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17908891648,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 4898.568459000671,
"performance": {
"metric": "tokensPerSecond",
"value": 12.283352372534807,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 3916.1630840008,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.float32": 883
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1338.5746670010121,
"inferenceMs": 1335.1532920005411,
"tokensPerSecond": 13.481598036604103,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"driverAllocatedMemoryBytes": 17909481472,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"totalMs": 5257.405459000438,
"performance": {
"metric": "tokensPerSecond",
"value": 13.481598036604103,
"grade": "B"
},
"grade": "B"
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 7003.267125051934,
"inferenceMs": 6933.844999992289,
"tokensPerSecond": 0.4326603781889177,
"inputTokens": 373,
"outputTokens": 3,
"maxNewTokens": 64,
"placement": {
"cpu/torch.float32": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"loadMs": 6426.95279198233,
"totalMs": 13432.36724997405,
"wallMs": 7005.117000022437,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "cpu",
"precision": "fp32",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"parameterDtype": "torch.float32",
"grade": "very-slow"
},
"failure": null
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 7733.717249997426,
"inferenceMs": 7729.107124963775,
"tokensPerSecond": 1.1644294553676768,
"inputTokens": 373,
"outputTokens": 9,
"maxNewTokens": 64,
"placement": {
"cpu/torch.float32": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"loadMs": 6426.95279198233,
"totalMs": 21167.89758298546,
"wallMs": 7735.42995803291,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "cpu",
"precision": "fp32",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"parameterDtype": "torch.float32",
"grade": "slow"
},
"failure": null
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 1285.541832970921,
"inferenceMs": 1284.7164580016397,
"tokensPerSecond": 3.89190935389544,
"inputTokens": 77,
"outputTokens": 5,
"maxNewTokens": 32,
"placement": {
"cpu/torch.float32": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"loadMs": 6426.95279198233,
"totalMs": 22454.928542021662,
"wallMs": 1286.9422499788925,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "cpu",
"precision": "fp32",
"requestedDevice": "cpu",
"requestedPrecision": "fp32",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"fallback": false,
"parameterDtype": "torch.float32",
"grade": "slow"
},
"failure": null
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 5853.161042032298,
"inferenceMs": 5798.699582985137,
"tokensPerSecond": 0.5173573759197261,
"inputTokens": 373,
"outputTokens": 3,
"maxNewTokens": 64,
"placement": {
"mps:0/torch.float32": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"driverAllocatedMemoryBytes": 22195699712,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"loadMs": 5029.8561249510385,
"totalMs": 10885.954374971334,
"wallMs": 5855.796208023094,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "mps",
"precision": "fp32",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"parameterDtype": "torch.float32",
"grade": "very-slow"
},
"failure": null
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 5174.41570898518,
"inferenceMs": 5164.625249977689,
"tokensPerSecond": 1.7426240171131253,
"inputTokens": 373,
"outputTokens": 9,
"maxNewTokens": 64,
"placement": {
"mps:0/torch.float32": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"driverAllocatedMemoryBytes": 22187606016,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"loadMs": 5029.8561249510385,
"totalMs": 16064.887749962509,
"wallMs": 5178.74716699589,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "mps",
"precision": "fp32",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"parameterDtype": "torch.float32",
"grade": "slow"
},
"failure": null
},
{
"precision": "fp32",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 868.847250007093,
"inferenceMs": 864.0725829754956,
"tokensPerSecond": 5.786550920042091,
"inputTokens": 77,
"outputTokens": 5,
"maxNewTokens": 32,
"placement": {
"mps:0/torch.float32": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 22238593024,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"loadMs": 5029.8561249510385,
"totalMs": 16938.131957955193,
"wallMs": 873.0481659877114,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "mps",
"precision": "fp32",
"requestedDevice": "mps",
"requestedPrecision": "fp32",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"parameterDtype": "torch.float32",
"grade": "good"
},
"failure": null
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 4189.44291700609,
"inferenceMs": 4123.577499995008,
"tokensPerSecond": 0.727523612689135,
"inputTokens": 373,
"outputTokens": 3,
"maxNewTokens": 64,
"placement": {
"mps:0/torch.bfloat16": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"driverAllocatedMemoryBytes": 13874200576,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"loadMs": 7565.672459022608,
"totalMs": 11758.405167027377,
"wallMs": 4192.3496669624,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "mps",
"precision": "bf16",
"requestedDevice": "mps",
"requestedPrecision": "bf16",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"parameterDtype": "torch.bfloat16",
"grade": "very-slow"
},
"failure": null
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 4118.045540992171,
"inferenceMs": 4107.648040982895,
"tokensPerSecond": 2.191034847729175,
"inputTokens": 373,
"outputTokens": 9,
"maxNewTokens": 64,
"placement": {
"mps:0/torch.bfloat16": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"driverAllocatedMemoryBytes": 14210039808,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"loadMs": 7565.672459022608,
"totalMs": 15881.529500009492,
"wallMs": 4121.432750020176,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "mps",
"precision": "bf16",
"requestedDevice": "mps",
"requestedPrecision": "bf16",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"parameterDtype": "torch.bfloat16",
"grade": "slow"
},
"failure": null
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"scope": "Predeclared fixture answer check",
"nonEmpty": true,
"repetitive": false,
"grade": "fixture-pass",
"score": 1,
"gibberish": false
},
"runtime": {
"durationMs": 959.1342080384493,
"inferenceMs": 955.3955420269631,
"tokensPerSecond": 5.2334345096398724,
"inputTokens": 77,
"outputTokens": 5,
"maxNewTokens": 32,
"placement": {
"mps:0/torch.bfloat16": 883
},
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 14219083776,
"memoryMeasurementScope": "Post-inference driver allocation, not peak",
"loadMs": 7565.672459022608,
"totalMs": 16843.326749978587,
"wallMs": 961.676582985092,
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"artifactRevision": "10042cb0e6e7fdce748996a71dc3dc432a4e0c89",
"device": "mps",
"precision": "bf16",
"requestedDevice": "mps",
"requestedPrecision": "bf16",
"provider": "Metal/MPS",
"acceleration": "partial",
"fallback": false,
"parameterDtype": "torch.bfloat16",
"grade": "good"
},
"failure": null
}
]TranslateGemma 12B — Text & Image Translation
50 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 3769 ms / 10161 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 3357 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1972 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2719 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 3616 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 4679 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 5225 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2021 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1979 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2052 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2188 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2459 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2103 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2526 ms / 5774 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2052 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1964 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1904 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1929 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1991 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 2504 ms / 5430 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1921 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1986 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1925 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1928 ms / 0 ms |
BF16 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1866 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 18701 ms / 20814 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 10917 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 5411 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8168 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 10816 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 11468 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 13791 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7774 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7879 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7691 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7779 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7679 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7669 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 17586 ms / 21513 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7994 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8162 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8221 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8278 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 8342 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 17672 ms / 16340 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7392 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7403 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7187 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7770 ms / 0 ms |
FP32 | cpu | Passed | Not measured | macos · aarch64 | 7960 ms / 0 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-05T22:29:17.343657+00:00
- Environment
- { "hardware": "Apple M3 Max", "ramBytes": 103079215104, "os": "macOS27.2", "runtime": "Transformers5.22.1/Torch2.10.0" }
- Runtime
- transformers 5.22.1
- Artifact revision
- d1b225e1caa17f1ddc7e62065d8637d0923f34e2
- Fixture
{ "file": "model-samples/desktop/translategemma-translations.json", "size": 1366, "sha1": "7a7953bc58138b42b981439fff2c2b724f4cc3a3", "sha256": "fb5f28fdcd81bb7d0bed06aaa029cdde7870a8b4ae2fa999a7c019cebce06f57", "url": "https://assets.localai.tools/samples/v1/files/fb5f28fdcd81bb7d0bed06aaa029cdde7870a8b4ae2fa999a7c019cebce06f57/translategemma-translations.json" }- Rerun when
- model revision/checksums; runtime/processor/template/backend/driver; fixture or reference checksums; language codes and generation budgets; quality gates; Concrete checkpoint/template/processor or image-translation/stream implementation changes
Per-case quality & failure details
[
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 20814.108542050235,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 18750.376000010874,
"inferenceMs": 18700.732958037406,
"tokensPerSecond": 0.6416860786647675,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 39564.48454206111,
"processAndInitWallMs": 23145.30645898776,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.6416860786647675,
"grade": "D"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 38,
"charactersPerSecond": 2.0320059157717636
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 10918.024500017054,
"inferenceMs": 10916.8608749751,
"tokensPerSecond": 1.6488256291020156,
"inputTokens": 91,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 10918.024500017054,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6488256291020156,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 51,
"charactersPerSecond": 4.671672615789044
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 5412.598291004542,
"inferenceMs": 5411.335791985039,
"tokensPerSecond": 1.4783780396421051,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 5412.598291004542,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.4783780396421051,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 34,
"charactersPerSecond": 6.283106668478947
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 8169.297959015239,
"inferenceMs": 8168.211416981649,
"tokensPerSecond": 1.5915356907845333,
"inputTokens": 83,
"outputTokens": 13,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 8169.297959015239,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5915356907845333,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 44,
"charactersPerSecond": 5.386736184193805
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 10817.268541024532,
"inferenceMs": 10816.100290976465,
"tokensPerSecond": 1.6641857523285761,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 10817.268541024532,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6641857523285761,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 61,
"charactersPerSecond": 5.639740605113508
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9726275669616796,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 11507.813916017767,
"inferenceMs": 11467.677082982846,
"tokensPerSecond": 0.34880647327746034,
"inputTokens": 373,
"outputTokens": 4,
"maxNewTokens": 64,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 11507.813916017767,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.34880647327746034,
"grade": "D"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 11,
"charactersPerSecond": 0.9592178015130158
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true,
"previousReferenceScore": 0.7241705887255625,
"referenceRevision": 2,
"referenceCorrection": "Image says arrives, not leaves; complete original output rescored without new inference or changed measurement timestamps."
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 13796.391999989282,
"inferenceMs": 13791.45849996712,
"tokensPerSecond": 0.6525778256173165,
"inputTokens": 373,
"outputTokens": 9,
"maxNewTokens": 64,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 13796.391999989282,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.6525778256173165,
"grade": "D"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 26,
"charactersPerSecond": 1.8852248295611362
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7774.467874958646,
"inferenceMs": 7773.622500011697,
"tokensPerSecond": 1.543681854885794,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7774.467874958646,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.543681854885794,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.888325873805014
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7880.37454104051,
"inferenceMs": 7879.408999986481,
"tokensPerSecond": 1.5229568613611235,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7880.37454104051,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5229568613611235,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.8226967276435575
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7692.187207983807,
"inferenceMs": 7691.100208030548,
"tokensPerSecond": 1.5602449162566336,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7692.187207983807,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5602449162566336,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.940775568146006
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7780.111416999716,
"inferenceMs": 7779.03137501562,
"tokensPerSecond": 1.5426085101727596,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7780.111416999716,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5426085101727596,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.8849269488804055
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7679.749250004534,
"inferenceMs": 7678.825040988158,
"tokensPerSecond": 1.5627390826000336,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7679.749250004534,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5627390826000336,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.948673761566773
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7669.532208004966,
"inferenceMs": 7668.669958016835,
"tokensPerSecond": 1.5648085086065269,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7669.532208004966,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5648085086065269,
"grade": "C"
},
"observedPeakRssBytes": 62635360256,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.955226943920668
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 21513.48520902684,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 17628.67908400949,
"inferenceMs": 17585.603832965717,
"tokensPerSecond": 0.6823763411242653,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 39142.16429303633,
"processAndInitWallMs": 24455.030084005557,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.6823763411242653,
"grade": "D"
},
"observedPeakRssBytes": 51748388864,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 38,
"charactersPerSecond": 2.1608584135601734
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7994.840208033565,
"inferenceMs": 7993.631374964025,
"tokensPerSecond": 1.5011950685621909,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7994.840208033565,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5011950685621909,
"grade": "C"
},
"observedPeakRssBytes": 51748388864,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.753784383780271
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 8162.989042000845,
"inferenceMs": 8162.122417008504,
"tokensPerSecond": 1.47020583457484,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 8162.989042000845,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.47020583457484,
"grade": "C"
},
"observedPeakRssBytes": 51748388864,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.655651809486994
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 8221.425041963812,
"inferenceMs": 8220.546249998733,
"tokensPerSecond": 1.4597570082404012,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 8221.425041963812,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.4597570082404012,
"grade": "C"
},
"observedPeakRssBytes": 51748388864,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.622563859427937
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 8279.021791007835,
"inferenceMs": 8277.875832980499,
"tokensPerSecond": 1.4496472575959536,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 8279.021791007835,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.4496472575959536,
"grade": "C"
},
"observedPeakRssBytes": 51748388864,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.590549649053853
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 8342.972583021037,
"inferenceMs": 8342.029083985835,
"tokensPerSecond": 1.4384989406278095,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 8342.972583021037,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.4384989406278095,
"grade": "C"
},
"observedPeakRssBytes": 51748388864,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.555246645321397
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 16340.036750014406,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 17714.265708986204,
"inferenceMs": 17671.76549998112,
"tokensPerSecond": 0.6790493004229159,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 34054.30245900061,
"processAndInitWallMs": 21317.035000014585,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.6790493004229159,
"grade": "D"
},
"observedPeakRssBytes": 62673731584,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 38,
"charactersPerSecond": 2.150322784672567
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7393.179874983616,
"inferenceMs": 7392.221041955054,
"tokensPerSecond": 1.6233280812212165,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7393.179874983616,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6233280812212165,
"grade": "C"
},
"observedPeakRssBytes": 62673731584,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 5.140538923867186
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7404.267500038259,
"inferenceMs": 7403.3715420519,
"tokensPerSecond": 1.6208831249166389,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7404.267500038259,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6208831249166389,
"grade": "C"
},
"observedPeakRssBytes": 62673731584,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 5.132796562236023
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7188.012374972459,
"inferenceMs": 7187.1529999771155,
"tokensPerSecond": 1.6696458249933193,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7188.012374972459,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6696458249933193,
"grade": "C"
},
"observedPeakRssBytes": 62673731584,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 5.287211779145511
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7771.199208975304,
"inferenceMs": 7770.266625040676,
"tokensPerSecond": 1.544348550579779,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7771.199208975304,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.544348550579779,
"grade": "C"
},
"observedPeakRssBytes": 62673731584,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.890437076835967
}
},
{
"precision": "fp32",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "cpu",
"provider": "PyTorch CPU",
"acceleration": "cpu",
"placement": {
"cpu/torch.float32": 1065
},
"parameterDtype": "torch.float32",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 7960.980458010454,
"inferenceMs": 7960.001999977976,
"tokensPerSecond": 1.5075373096681637,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"precision": "fp32",
"cpuThreads": 4,
"totalMs": 7960.980458010454,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.5075373096681637,
"grade": "C"
},
"observedPeakRssBytes": 62673731584,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 4.773868147282518
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 10161.214666964952,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3815.0306250317954,
"inferenceMs": 3768.6234999564476,
"tokensPerSecond": 3.184186480856652,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 13976.245291996747,
"processAndInitWallMs": 12546.888916986063,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 3.184186480856652,
"grade": "C"
},
"observedPeakRssBytes": 813088768,
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 10.083257189379397
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3364.3570840358734,
"inferenceMs": 3357.381042034831,
"tokensPerSecond": 5.361321748898248,
"inputTokens": 91,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25589104640,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3364.3570840358734,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 5.361321748898248,
"grade": "B"
},
"observedPeakRssBytes": 848052224,
"grade": "B",
"outputCharacters": 51,
"charactersPerSecond": 15.190411621878368
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1977.1592919714749,
"inferenceMs": 1972.1942499745637,
"tokensPerSecond": 4.0563955604794915,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25589104640,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1977.1592919714749,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 4.0563955604794915,
"grade": "C"
},
"observedPeakRssBytes": 854573056,
"grade": "C",
"outputCharacters": 34,
"charactersPerSecond": 17.239681132037838
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2725.083708995953,
"inferenceMs": 2719.0627499949187,
"tokensPerSecond": 4.781059208738119,
"inputTokens": 83,
"outputTokens": 13,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25590136832,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2725.083708995953,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 4.781059208738119,
"grade": "C"
},
"observedPeakRssBytes": 882737152,
"grade": "C",
"outputCharacters": 44,
"charactersPerSecond": 16.182046552652096
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8333179436563465,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 3621.0609159898013,
"inferenceMs": 3615.866374981124,
"tokensPerSecond": 4.978060064538188,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25591201792,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 3621.0609159898013,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 4.978060064538188,
"grade": "C"
},
"observedPeakRssBytes": 911720448,
"grade": "C",
"outputCharacters": 61,
"charactersPerSecond": 16.87009244093497
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9726275669616796,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 4703.023584035691,
"inferenceMs": 4679.290916013997,
"tokensPerSecond": 0.8548303731898244,
"inputTokens": 373,
"outputTokens": 4,
"maxNewTokens": 64,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"driverAllocatedMemoryBytes": 28896903168,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 4703.023584035691,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.8548303731898244,
"grade": "D"
},
"observedPeakRssBytes": 994590720,
"grade": "D",
"outputCharacters": 11,
"charactersPerSecond": 2.350783526272017
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true,
"previousReferenceScore": 0.7241705887255625,
"referenceRevision": 2,
"referenceCorrection": "Image says arrives, not leaves; complete original output rescored without new inference or changed measurement timestamps."
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 5232.437041995581,
"inferenceMs": 5224.547833960969,
"tokensPerSecond": 1.7226371134928795,
"inputTokens": 373,
"outputTokens": 9,
"maxNewTokens": 64,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "image",
"driverAllocatedMemoryBytes": 29434134528,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 5232.437041995581,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.7226371134928795,
"grade": "C"
},
"observedPeakRssBytes": 1014022144,
"grade": "C",
"outputCharacters": 26,
"charactersPerSecond": 4.9765072167572075
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2027.8654170106165,
"inferenceMs": 2020.978250016924,
"tokensPerSecond": 5.937718528093764,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 28874194944,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2027.8654170106165,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 5.937718528093764,
"grade": "B"
},
"observedPeakRssBytes": 1020035072,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 18.802775338963585
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1982.5050000217743,
"inferenceMs": 1979.1662080097012,
"tokensPerSecond": 6.063159299828334,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 28874194944,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1982.5050000217743,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.063159299828334,
"grade": "B"
},
"observedPeakRssBytes": 1022476288,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.200004449456394
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2056.177791033406,
"inferenceMs": 2051.8050839891657,
"tokensPerSecond": 5.848508756333389,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 28861612032,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2056.177791033406,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 5.848508756333389,
"grade": "B"
},
"observedPeakRssBytes": 1024819200,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 18.520277728389065
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2192.2904169769026,
"inferenceMs": 2188.469749991782,
"tokensPerSecond": 5.4832834678409705,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 28861612032,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2192.2904169769026,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 5.4832834678409705,
"grade": "B"
},
"observedPeakRssBytes": 1026965504,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 17.363730981496406
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2462.28074998362,
"inferenceMs": 2458.858333004173,
"tokensPerSecond": 4.880313696372533,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 28861612032,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2462.28074998362,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 4.880313696372533,
"grade": "C"
},
"observedPeakRssBytes": 1029603328,
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 15.454326705179689
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2105.3553750389256,
"inferenceMs": 2102.65158402035,
"tokensPerSecond": 5.7070796185146095,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 28861612032,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2105.3553750389256,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 5.7070796185146095,
"grade": "B"
},
"observedPeakRssBytes": 1031798784,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 18.07241879196293
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 5773.994749994017,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2570.9800419863313,
"inferenceMs": 2525.681708997581,
"tokensPerSecond": 4.751192502701651,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 8344.974791980349,
"processAndInitWallMs": 8422.221083019394,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 4.751192502701651,
"grade": "C"
},
"observedPeakRssBytes": 823869440,
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 15.045442925221893
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2058.9742090087384,
"inferenceMs": 2051.988707971759,
"tokensPerSecond": 5.847985397473811,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 2058.9742090087384,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 5.847985397473811,
"grade": "B"
},
"observedPeakRssBytes": 823869440,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 18.518620425333737
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1969.0306659904309,
"inferenceMs": 1963.9066669624299,
"tokensPerSecond": 6.1102700051221746,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1969.0306659904309,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.1102700051221746,
"grade": "B"
},
"observedPeakRssBytes": 823869440,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.349188349553554
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1907.2257080115378,
"inferenceMs": 1903.5225420375355,
"tokensPerSecond": 6.304101861150102,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1907.2257080115378,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.304101861150102,
"grade": "B"
},
"observedPeakRssBytes": 823869440,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.96298922697532
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1932.3398749693297,
"inferenceMs": 1929.2572080157697,
"tokensPerSecond": 6.220010452801124,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1932.3398749693297,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.220010452801124,
"grade": "B"
},
"observedPeakRssBytes": 823869440,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.696699767203558
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1994.610959023703,
"inferenceMs": 1990.7710829866119,
"tokensPerSecond": 6.027815102677328,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1994.610959023703,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.027815102677328,
"grade": "B"
},
"observedPeakRssBytes": 823869440,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.088081158478207
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 5429.80866698781,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 2548.2318329741247,
"inferenceMs": 2504.100916965399,
"tokensPerSecond": 4.792139134129718,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 7978.040499961935,
"processAndInitWallMs": 7114.893625024706,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 4.792139134129718,
"grade": "C"
},
"observedPeakRssBytes": 798932992,
"grade": "C",
"outputCharacters": 38,
"charactersPerSecond": 15.17510725807744
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1924.7918340261094,
"inferenceMs": 1921.4215420070104,
"tokensPerSecond": 6.24537600815356,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1924.7918340261094,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.24537600815356,
"grade": "B"
},
"observedPeakRssBytes": 804028416,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.777024025819607
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1989.8510829661973,
"inferenceMs": 1985.629250004422,
"tokensPerSecond": 6.0434242696481615,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1989.8510829661973,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.0434242696481615,
"grade": "B"
},
"observedPeakRssBytes": 806551552,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.137510187219178
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1929.9311669892631,
"inferenceMs": 1924.9550419626758,
"tokensPerSecond": 6.2339118256833945,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1929.9311669892631,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.2339118256833945,
"grade": "B"
},
"observedPeakRssBytes": 809140224,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.74072078133075
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1932.1624580188654,
"inferenceMs": 1927.6004580315202,
"tokensPerSecond": 6.225356478828859,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1932.1624580188654,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.225356478828859,
"grade": "B"
},
"observedPeakRssBytes": 811663360,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 19.71362884962472
}
},
{
"precision": "bf16",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "transformers",
"version": "5.22.1",
"torchVersion": "2.10.0",
"loadMs": 0,
"device": "mps",
"provider": "Metal/MPS",
"acceleration": "partial",
"placement": {
"mps:0/torch.bfloat16": 1065
},
"parameterDtype": "torch.bfloat16",
"accelerationDetails": "All model parameters and input tensors on selected device; text tokenization/decoding on CPU",
"durationMs": 1870.280749979429,
"inferenceMs": 1866.0280000185594,
"tokensPerSecond": 6.430771671100674,
"inputTokens": 84,
"outputTokens": 12,
"maxNewTokens": 128,
"decoding": "greedy",
"outputLimitReached": false,
"inputModality": "text",
"driverAllocatedMemoryBytes": 25562333184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"precision": "bf16",
"cpuThreads": 4,
"totalMs": 1870.280749979429,
"processAndInitWallMs": null,
"artifactRevision": "d1b225e1caa17f1ddc7e62065d8637d0923f34e2",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.430771671100674,
"grade": "B"
},
"observedPeakRssBytes": 813596672,
"grade": "B",
"outputCharacters": 38,
"charactersPerSecond": 20.3641102918188
}
}
]TranslateGemma 27B — Text & Image Translation
50 configurations testedQuality criteria
No explicit quality gate recorded. Review per-case evidence; full certification is pending.
| Precision | Device | Result | Platform | Inference / load | |
|---|---|---|---|---|---|
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1522 ms / 1308 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1914 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1260 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1543 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1970 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 4476 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 4876 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1488 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1486 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1497 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1491 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1491 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1497 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1482 ms / 1393 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1475 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1479 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1486 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1463 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1471 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1486 ms / 1350 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1476 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1471 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1464 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1480 ms / 0 ms |
Q4 | mpsAccelerated | Passed | Not measured | macos · aarch64 | 1472 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6566 ms / 15568 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 8020 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6136 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6891 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 8084 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 35062 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 35558 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6579 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6544 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6535 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6566 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6526 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6573 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6552 ms / 15610 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6537 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6556 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6534 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6553 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6544 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6581 ms / 15650 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6527 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6569 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6566 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6560 ms / 0 ms |
Q4 | cpu | Passed | Not measured | macos · aarch64 | 6613 ms / 0 ms |
Test setup & sample details
- Quality gate
- No explicit quality gate recorded. Review the per-case quality evidence; full certification is pending.
- Tested
- 2026-11-05T23:18:51.490317+00:00
- Environment
- { "hardware": "Apple M3 Max", "ramBytes": 103079215104, "os": "macOS27.2", "runtime": "llama-cpp-python0.3.36/Metal+Accelerate builtfrompinnedofficialsource" }
- Runtime
- llama.cpp 0.3.36
- Artifact revision
- 22fb50537007ccd41c41a0f0f0482e0fdb3335ab
- Fixture
{ "file": "model-samples/desktop/translategemma-translations.json", "size": 1366, "sha1": "7a7953bc58138b42b981439fff2c2b724f4cc3a3", "sha256": "fb5f28fdcd81bb7d0bed06aaa029cdde7870a8b4ae2fa999a7c019cebce06f57", "url": "https://assets.localai.tools/samples/v1/files/fb5f28fdcd81bb7d0bed06aaa029cdde7870a8b4ae2fa999a7c019cebce06f57/translategemma-translations.json" }- Rerun when
- model revision/checksums; runtime/processor/template/backend/driver; fixture or reference checksums; language codes and generation budgets; quality gates; Concrete checkpoint/template/processor or image-translation/stream implementation changes
Per-case quality & failure details
[
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 15567.559500050265,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6566.401416028384,
"inferenceMs": 6566.264791996218,
"tokensPerSecond": 1.6752294262345573,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 22133.96091607865,
"processAndInitWallMs": 15681.74083402846,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6752294262345573,
"grade": "C"
},
"observedPeakRssBytes": 35217981440,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.634862615516239
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 8020.276417024434,
"inferenceMs": 8020.1812500017695,
"tokensPerSecond": 2.119652844503514,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 8020.276417024434,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 2.119652844503514,
"grade": "C"
},
"observedPeakRssBytes": 35229515776,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 50,
"charactersPerSecond": 6.2342730720691595
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6136.322458041832,
"inferenceMs": 6136.230542033445,
"tokensPerSecond": 1.303731980928626,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6136.322458041832,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.303731980928626,
"grade": "C"
},
"observedPeakRssBytes": 35232727040,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 34,
"charactersPerSecond": 5.54086091894666
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6891.37899997877,
"inferenceMs": 6891.2759170052595,
"tokensPerSecond": 1.7413321051894317,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6891.37899997877,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.7413321051894317,
"grade": "C"
},
"observedPeakRssBytes": 35232759808,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 43,
"charactersPerSecond": 6.239773376928797
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8772489985666848,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 8084.52633296838,
"inferenceMs": 8084.430291026365,
"tokensPerSecond": 2.2265019737976854,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 8084.52633296838,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 2.2265019737976854,
"grade": "C"
},
"observedPeakRssBytes": 35239133184,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 59,
"charactersPerSecond": 7.297978691892414
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9726275669616796,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 35067.605208954774,
"inferenceMs": 35061.637542035896,
"tokensPerSecond": 0.11408480266229275,
"inputTokens": 371,
"outputTokens": 4,
"maxNewTokens": 64,
"inputModality": "image",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 35067.605208954774,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.11408480266229275,
"grade": "D"
},
"observedPeakRssBytes": 35508092928,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 11,
"charactersPerSecond": 0.31373320732130505
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 35559.80512499809,
"inferenceMs": 35558.050959021784,
"tokensPerSecond": 0.25310723611853425,
"inputTokens": 371,
"outputTokens": 9,
"maxNewTokens": 64,
"inputModality": "image",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 35559.80512499809,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.25310723611853425,
"grade": "D"
},
"observedPeakRssBytes": 35522199552,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 26,
"charactersPerSecond": 0.7311986821202101
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6578.7065839977,
"inferenceMs": 6578.569291043095,
"tokensPerSecond": 1.672096091619314,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6578.7065839977,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.672096091619314,
"grade": "C"
},
"observedPeakRssBytes": 35531505664,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.624323217264965
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6543.994582956657,
"inferenceMs": 6543.892500048969,
"tokensPerSecond": 1.6809567088575623,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6543.994582956657,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6809567088575623,
"grade": "C"
},
"observedPeakRssBytes": 35534716928,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.654127111611801
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6534.671999979764,
"inferenceMs": 6534.565916052088,
"tokensPerSecond": 1.6833558864221758,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6534.671999979764,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6833558864221758,
"grade": "C"
},
"observedPeakRssBytes": 35534716928,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.6621970725109545
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6566.134915978182,
"inferenceMs": 6566.039333003573,
"tokensPerSecond": 1.6752869488170052,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6566.134915978182,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6752869488170052,
"grade": "C"
},
"observedPeakRssBytes": 35534733312,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.63505610056629
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6525.719166966155,
"inferenceMs": 6525.6216249545105,
"tokensPerSecond": 1.6856631647068074,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6525.719166966155,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6856631647068074,
"grade": "C"
},
"observedPeakRssBytes": 35534733312,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.669957917650171
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.007 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.012 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6572.640958009288,
"inferenceMs": 6572.546250012238,
"tokensPerSecond": 1.673628390211711,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6572.640958009288,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.673628390211711,
"grade": "C"
},
"observedPeakRssBytes": 35534749696,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.629477312530301
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 15610.001625027508,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6552.596584020648,
"inferenceMs": 6552.472041978035,
"tokensPerSecond": 1.67875573211593,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 22162.598209048156,
"processAndInitWallMs": 15710.861333005596,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.67875573211593,
"grade": "C"
},
"observedPeakRssBytes": 35223830528,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.646723826208128
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6537.497500015888,
"inferenceMs": 6537.397958978545,
"tokensPerSecond": 1.6826266458036965,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6537.497500015888,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6826266458036965,
"grade": "C"
},
"observedPeakRssBytes": 35227140096,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.659744172248797
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6556.452957971487,
"inferenceMs": 6556.355791981332,
"tokensPerSecond": 1.6777612974349883,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6556.452957971487,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6777612974349883,
"grade": "C"
},
"observedPeakRssBytes": 35230334976,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.643378909554052
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6534.330042020883,
"inferenceMs": 6534.23816699069,
"tokensPerSecond": 1.6834403214087303,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6534.330042020883,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6834403214087303,
"grade": "C"
},
"observedPeakRssBytes": 35233529856,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.662481081102093
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6553.418290975969,
"inferenceMs": 6553.290333016776,
"tokensPerSecond": 1.678546110582011,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6553.418290975969,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.678546110582011,
"grade": "C"
},
"observedPeakRssBytes": 35233562624,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.646018735594037
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6544.112917035818,
"inferenceMs": 6544.00033398997,
"tokensPerSecond": 1.6809290095639624,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6544.112917035818,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6809290095639624,
"grade": "C"
},
"observedPeakRssBytes": 35233562624,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.654033941260601
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 15650.095625023823,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6581.056124996394,
"inferenceMs": 6580.926374997944,
"tokensPerSecond": 1.671497198599709,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 22231.151750020217,
"processAndInitWallMs": 15756.894916994497,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.671497198599709,
"grade": "C"
},
"observedPeakRssBytes": 35223961600,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.622308758926294
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6526.783832989167,
"inferenceMs": 6526.688374986406,
"tokensPerSecond": 1.685387652665876,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6526.783832989167,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.685387652665876,
"grade": "C"
},
"observedPeakRssBytes": 35227238400,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.669031195330674
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6568.672249966767,
"inferenceMs": 6568.572666030377,
"tokensPerSecond": 1.6746408328382996,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6568.672249966767,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6746408328382996,
"grade": "C"
},
"observedPeakRssBytes": 35230433280,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.632882801365189
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6566.041917016264,
"inferenceMs": 6565.9467499936,
"tokensPerSecond": 1.6753105711694543,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6566.041917016264,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6753105711694543,
"grade": "C"
},
"observedPeakRssBytes": 35233611776,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.635135557569982
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6559.85833297018,
"inferenceMs": 6559.759541996755,
"tokensPerSecond": 1.6768907350301534,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6559.85833297018,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6768907350301534,
"grade": "C"
},
"observedPeakRssBytes": 35233660928,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.640450654192334
}
},
{
"precision": "q4",
"device": "cpu",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "cpu",
"precision": "q4",
"provider": "llama.cpp CPU",
"acceleration": "cpu",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nload_tensors: offloading 0 repeating layers to GPU\nload_tensors: offloaded 0/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 15773.89 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nclip_ctx: CLIP using CPU backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 0,
"totalLayers": 63,
"imageEncoderUseGpu": false,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 6613.030667009298,
"inferenceMs": 6612.940749968402,
"tokensPerSecond": 1.6634051953440774,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 6613.030667009298,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.6634051953440774,
"grade": "C"
},
"observedPeakRssBytes": 35233660928,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 37,
"charactersPerSecond": 5.595090202520987
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 1308.1281250342727,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1522.6184169878252,
"inferenceMs": 1522.496499994304,
"tokensPerSecond": 7.224975558263124,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 2830.746542022098,
"processAndInitWallMs": 1409.2495419899933,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.224975558263124,
"grade": "B"
},
"observedPeakRssBytes": 18675580928,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.302190514157783
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1914.3679999979213,
"inferenceMs": 1914.2458330024965,
"tokensPerSecond": 8.880782032752546,
"inputTokens": 91,
"outputTokens": 17,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1914.3679999979213,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 8.880782032752546,
"grade": "B"
},
"observedPeakRssBytes": 18681741312,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 50,
"charactersPerSecond": 26.119947155154545
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1260.48204198014,
"inferenceMs": 1260.326666990295,
"tokensPerSecond": 6.347560683695034,
"inputTokens": 84,
"outputTokens": 8,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1260.48204198014,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 6.347560683695034,
"grade": "B"
},
"observedPeakRssBytes": 18684936192,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 34,
"charactersPerSecond": 26.977132905703893
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1542.663583997637,
"inferenceMs": 1542.5692080170847,
"tokensPerSecond": 7.779229572088732,
"inputTokens": 83,
"outputTokens": 12,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1542.663583997637,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.779229572088732,
"grade": "B"
},
"observedPeakRssBytes": 18685018112,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 43,
"charactersPerSecond": 27.875572633317955
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.8772489985666848,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1969.7164589888416,
"inferenceMs": 1969.6094170212746,
"tokensPerSecond": 9.138867759488162,
"inputTokens": 86,
"outputTokens": 18,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1969.7164589888416,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 9.138867759488162,
"grade": "B"
},
"observedPeakRssBytes": 18691391488,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 59,
"charactersPerSecond": 29.955177656100087
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 0.9726275669616796,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 4481.337374949362,
"inferenceMs": 4475.781124958303,
"tokensPerSecond": 0.8936987507487343,
"inputTokens": 371,
"outputTokens": 4,
"maxNewTokens": 64,
"inputModality": "image",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 4481.337374949362,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 0.8936987507487343,
"grade": "D"
},
"observedPeakRssBytes": 18767233024,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "D",
"outputCharacters": 11,
"charactersPerSecond": 2.4576715645590195
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 4877.424417005386,
"inferenceMs": 4875.6625420064665,
"tokensPerSecond": 1.8459029767667754,
"inputTokens": 371,
"outputTokens": 9,
"maxNewTokens": 64,
"inputModality": "image",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 4877.424417005386,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 1.8459029767667754,
"grade": "C"
},
"observedPeakRssBytes": 18789384192,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "C",
"outputCharacters": 26,
"charactersPerSecond": 5.332608599548463
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1488.4135000174865,
"inferenceMs": 1488.3152910042554,
"tokensPerSecond": 7.390907065516771,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1488.4135000174865,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.390907065516771,
"grade": "B"
},
"observedPeakRssBytes": 18794954752,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.860323765829136
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1486.2614999874495,
"inferenceMs": 1486.1637919675559,
"tokensPerSecond": 7.401606780795625,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1486.2614999874495,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.401606780795625,
"grade": "B"
},
"observedPeakRssBytes": 18798166016,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.89631371722165
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1496.6248330310918,
"inferenceMs": 1496.5173339587636,
"tokensPerSecond": 7.350399324076993,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1496.6248330310918,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.350399324076993,
"grade": "B"
},
"observedPeakRssBytes": 18798166016,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.724070453713523
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1490.628875035327,
"inferenceMs": 1490.5314169591293,
"tokensPerSecond": 7.3799182458303205,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1490.628875035327,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.3799182458303205,
"grade": "B"
},
"observedPeakRssBytes": 18798182400,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.82336137233835
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1490.8686659764498,
"inferenceMs": 1490.770792006515,
"tokensPerSecond": 7.378733242549286,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1490.8686659764498,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.378733242549286,
"grade": "B"
},
"observedPeakRssBytes": 18798182400,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.819375452211236
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1497.307334037032,
"inferenceMs": 1497.210500005167,
"tokensPerSecond": 7.346996297422465,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1497.307334037032,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.346996297422465,
"grade": "B"
},
"observedPeakRssBytes": 18798198784,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.71262390951193
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 1392.8450829698704,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.014 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1482.280374970287,
"inferenceMs": 1482.1459160302766,
"tokensPerSecond": 7.421671429937198,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 2875.1254579401575,
"processAndInitWallMs": 1504.0638750069775,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.421671429937198,
"grade": "B"
},
"observedPeakRssBytes": 18678284288,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.963803900697847
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.014 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1475.2059160382487,
"inferenceMs": 1475.105916033499,
"tokensPerSecond": 7.4570916436824835,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1475.2059160382487,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.4570916436824835,
"grade": "B"
},
"observedPeakRssBytes": 18681266176,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.082944619659262
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.014 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1479.315874981694,
"inferenceMs": 1479.2226669960655,
"tokensPerSecond": 7.436338183174459,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1479.315874981694,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.436338183174459,
"grade": "B"
},
"observedPeakRssBytes": 18684411904,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.013137525223183
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.014 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1486.4379999926314,
"inferenceMs": 1486.3432919955812,
"tokensPerSecond": 7.40071291688697,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1486.4379999926314,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.40071291688697,
"grade": "B"
},
"observedPeakRssBytes": 18687574016,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.893307084074355
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.014 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1463.5885420138948,
"inferenceMs": 1463.4912080364302,
"tokensPerSecond": 7.51627337396767,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1463.5885420138948,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.51627337396767,
"grade": "B"
},
"observedPeakRssBytes": 18687574016,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.282010439709435
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.014 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1471.5414999518543,
"inferenceMs": 1471.443333022762,
"tokensPerSecond": 7.475653158455568,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1471.5414999518543,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.475653158455568,
"grade": "B"
},
"observedPeakRssBytes": 18687590400,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.145378805714184
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 1350.2731249900535,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1486.625040997751,
"inferenceMs": 1486.4965840242803,
"tokensPerSecond": 7.399949732962405,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 2836.8981659878045,
"processAndInitWallMs": 1455.829459009692,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.399949732962405,
"grade": "B"
},
"observedPeakRssBytes": 18672779264,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 24.890740010873543
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1475.9553329786286,
"inferenceMs": 1475.8428339846432,
"tokensPerSecond": 7.453368168140903,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1475.9553329786286,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.453368168140903,
"grade": "B"
},
"observedPeakRssBytes": 18675761152,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.070420201928496
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1471.4906250010245,
"inferenceMs": 1471.3910840218887,
"tokensPerSecond": 7.475918618408838,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1471.4906250010245,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.475918618408838,
"grade": "B"
},
"observedPeakRssBytes": 18678841344,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.14627171646609
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1463.9222080004402,
"inferenceMs": 1463.8203749782406,
"tokensPerSecond": 7.514583201619606,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1463.9222080004402,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.514583201619606,
"grade": "B"
},
"observedPeakRssBytes": 18682150912,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.276325314538678
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1479.7302079969086,
"inferenceMs": 1479.6328750089742,
"tokensPerSecond": 7.434276559942806,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1479.7302079969086,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.434276559942806,
"grade": "B"
},
"observedPeakRssBytes": 18682150912,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.006202974353073
}
},
{
"precision": "q4",
"device": "mps",
"status": "pass",
"quality": {
"passed": true,
"score": 1,
"threshold": 0.55,
"metric": "character-ngram F2 (orders1–6), best authored reference",
"anchorsPassed": true,
"repetitionRejected": false,
"repeatParity": true
},
"runtime": {
"engine": "llama.cpp",
"version": "0.3.36",
"loadMs": 0,
"device": "mps",
"precision": "q4",
"provider": "llama.cpp Metal",
"acceleration": "partial",
"placement": "ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices\nggml_metal_library_init: using embedded metal library\nggml_metal_library_compile_all: compiled 'fa_aux' library in 0.002 sec\nggml_metal_library_compile_all: compiled 'fa_f16' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.003 sec\nggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.012 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'mul_mv' library in 0.006 sec\nggml_metal_library_compile_all: compiled 'mul_mm' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'quantize' library in 0.013 sec\nggml_metal_library_compile_all: compiled 'softmax' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'norm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'unary' library in 0.004 sec\nggml_metal_library_compile_all: compiled 'binbcast' library in 0.011 sec\nggml_metal_library_compile_all: compiled 'reduce' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'tri' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'ssm' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'wkv' library in 0.008 sec\nggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'solve_tri' library in 0.007 sec\nggml_metal_library_compile_all: compiled 'rope' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'conv' library in 0.010 sec\nggml_metal_library_compile_all: compiled 'upscale' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'argsort' library in 0.009 sec\nggml_metal_library_compile_all: compiled 'pool' library in 0.005 sec\nggml_metal_library_compile_all: compiled 'misc' library in 0.005 sec\nggml_metal_library_compile_all: loaded 34 libraries from embedded data in 0.013 sec (max single = 0.013 sec)\nggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)\nggml_metal_device_init: GPU name: MTL0 (Apple M3 Max)\nggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)\nggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)\nggml_metal_device_init: simdgroup reduction = true\nggml_metal_device_init: simdgroup matrix mul. = true\nggml_metal_device_init: has unified memory = true\nggml_metal_device_init: has bfloat = true\nggml_metal_device_init: has tensor = false\nggml_metal_device_init: use residency sets = true\nggml_metal_device_init: use shared buffers = true\nggml_metal_device_init: recommendedMaxWorkingSetSize = 83494.17 MB\nggml_metal_log_allocated_size: allocated buffer, size = 15773.91 MiB, (15774.36 / 79626.25)\nload_tensors: offloading output layer to GPU\nload_tensors: offloading 61 repeating layers to GPU\nload_tensors: offloaded 63/63 layers to GPU\nload_tensors: CPU_Mapped model buffer size = 1102.77 MiB\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nllama_context: enumerating backends\nllama_context: backend_ptrs.size() = 3\nggml_metal_init: allocating\nggml_metal_init: found device: Apple M3 Max\nggml_metal_init: picking default device: Apple M3 Max\nggml_metal_init: use fusion = true\nggml_metal_init: use concurrency = true\nggml_metal_init: use graph optimize = true\nclip_ctx: CLIP using MTL0 backend",
"parameterDtype": "Q4_K_M language / F16 vision projector",
"accelerationDetails": "Native layer-offload logs and strict n_gpu_layers selection; native mtmd image encoder requestedGPU only forMPS, CPU forCPU. PIL orientation/alpha and tokenization onCPU.",
"offloadedLayers": 63,
"totalLayers": 63,
"imageEncoderUseGpu": true,
"modelQuantization": {
"language": "Q4_K_M",
"vision": "F16"
},
"cpuThreads": 4,
"cpuThreadsApplied": true,
"durationMs": 1471.6865830123425,
"inferenceMs": 1471.5864580357447,
"tokensPerSecond": 7.474926083977874,
"inputTokens": 84,
"outputTokens": 11,
"maxNewTokens": 128,
"inputModality": "text",
"decoding": "greedy",
"outputLimitReached": false,
"totalMs": 1471.6865830123425,
"processAndInitWallMs": null,
"artifactRevision": "22fb50537007ccd41c41a0f0f0482e0fdb3335ab",
"fallback": false,
"performance": {
"metric": "tokensPerSecond",
"value": 7.474926083977874,
"grade": "B"
},
"observedPeakRssBytes": 18682167296,
"memoryMeasurementScope": "500ms sampled child RSS, not OS peak/minimumRAM",
"grade": "B",
"outputCharacters": 37,
"charactersPerSecond": 25.14293319156194
}
}
]